robots.txt cannot crawl after enabling maintenance mode
-
Hi,
This could be coincidence – or not. I am working on a website and have just installed the plugin (which I love btw!).
Prior to installing and activating the plugin I was able to access the site I’m working on via WayBackMachine https://web.archive.org
Now I get “Page cannot be crawled or displayed due to robots.txt.” https://web.archive.org/web/*/https://www.orcainstitute.com/
I have since disabled the plugin by still can’t get it to show up on the web.archive.org. I cleared my cache.
There is a bunch of content on the archived pages that has not be saved elsewhere :-O
Any connection to the plugin?
Thanks,
jamie
- The topic ‘robots.txt cannot crawl after enabling maintenance mode’ is closed to new replies.