How to conditionally hide field but still allow edit by owner

I am trying to figure out a way to conditionally hide a field from view by other members, based on a flag in a member’s profile, but still allow the member himself/herself, to edit it.

Concretely, I have a phone number field, and a “Share Phone Number” (aka share_phonenumber meta) field which will have “Yes” or “No” in it. I can use conditional logic based on a Yes or No in the share_phonenumber field to show or hide the phone number from other members. But the problem is that if the share_phonenumber field is “No”, then the actual profile owner can not see and edit their own phone number. I need a way to show or hide the phone number conditionally to other members, but the owner should always see it and be able to edit it.

Is there any way to achieve this?