This one gave me a bit of a headache… This problem will also occur if your Linux server is running SELinux and you initially create the WP files somewhere else (not in /var/www/html). To fix it you need to run
restorecon -R /var/www/html/
(per https://www.linuxquestions.org/questions/programming-9/403-forbidden-when-accessing-some-.png-files-442093/)
This fixed the problem.
