I fixed it.
It turns out this happens with iOS only. When photos are uploaded into your PC (which I have Windows 8) and then upload it straight to wordpress, this happens.
I downloaded two plugins, Image Rotation Fixer and iOS Images Fixer. One of them (or both) seem to fix the problem! ?? yay.
