Thanks for your reply!
After some testing, the issue appears to be from certain photos uploaded from mobile phones. After deactivating plugins and reverting to base theme, I tried uploading various photos and the issue appears to be effecting photos uploaded from mobile phones. The issue is not affecting all photos but, only certain ones. Once downloaded, the photos appear in the correct orientation. I assume this is an issue that effects the reading of the orientation information on the desktop version of Ultimate member. Because the photos are correct on the mobile site.
I understand this issue is hard to solve since, it does not effect all images.
Thank you in advance.