Maybe I wasn’t clear enough – I’m just changing the address of the WP installation, not the whole address.
Example:
Wordpress Address (URL) – example.com/wordpress
Site Address (URL) – example.com
So when it is like this – HTTP Error
When I change back Site Address (URL) to example.com/wordpress – OK
Also this happens on a new, fresh site. No uploaded images/galleries before.