The networks are /12 CIDR and the bots can appear anywhere in there. I have tried blocking the whole network before but that /12 is also used for customer applications which included some entry points into the internet for mobile clients. I have complained to AWS/Google about their bots not being behaved but you know how that goes. The good news is they just traverse all my articles one time every few days. The bad news is I can’t really block a dynamically applied IP nor can I block them by *.aws.com lookup since that also blocks the whole CIDR. It is annoying as hell. Thanks for looking at it.
