nixeens

Hosting provider: Your tariff plan does not include CRON management.
So i couldnt make cronjob for that with t hat crawler script.

But with those setting can i control how agressive crawler is? If its not using my website resource then could i make it faster do that. With these settings it runs about 4h

ok, then what about settings run duration interval between runs and delay for better performance. When crawler runs it feels like page is not usable. And cpu is high.
Just some times i think its making bootneck.