WordPress REST API – hide field for non logged users

With this example I manage to add custom post type and custom meta data in WordPress REST API.

Right now I’m trying to hide some of the data for non logged users and I’m trying like this:

add_filter( 'rest_prepare_the_event', 'add_meta_to_json', 10, 3 );
function add_meta_to_json($data, $post, $request){

$response_data = $data->get_data();

if ( $request['context'] !== 'view' || is_wp_error( $data ) ) {
    return $data;
}

$start = get_post_meta( $post->ID, '_the_event_start', true );
if(empty($start)){
    $start = '';
}

$end = get_post_meta( $post->ID, '_the_event_end', true );
if(empty($end)){
    $end = '';
}

$location = get_post_meta( $post->ID, '_the_event_location', true );
if(empty($location)){
     $location = '';
}

if($post->post_type == 'the_event'){
    $response_data['event_meta'] = array(
        'start' => $start,
        'end' => $end,
        'location' => $location,
    );

    if ( !is_user_logged_in() ) {
        unset($response_data['event_meta']['location']);
    }   
}

$data->set_data( $response_data );

return $data;
}

In above example code I’m trying to hide location field to non logged users but this is not working.
How to hide that field for non logged users?