I think I may have discovered the “issue”.
This particular domain was setup in Godaddy with anonymous registration, so their DNS passed through a proxy. Removing this proxy and having the A record point directly to our server has the site still displaying as well as rate limiting enabled.
Not sure why this would cause an issue but I’m able to keep rate limiting on now.