When the photo was taken (sideways) the thumbnail version was embedded into the image file.
The photographer then rotated the image, but didn’t update the embedded thumbnail.
There are many thumbnail generator programs for Windows you can use to generate a new thumbnail and embed it overtop of what’s there.
Source: superuser.com
