Running crawler.. too often?

  • Resolved mcdeth

    (@mcdeth)


    1 year ago

    Hello. I have around 100 products and crawler finishes crawling in 7 minutes (I use quite low settings, without rush like 30k delay etc). My question is, if browser etc cache is set at minimum 4h, is it worth to run crawler every 15 minutes?

Viewing 3 replies - 1 through 3 (of 3 total)
  • Plugin Support qtwrk

    (@qtwrk)

    browser cache is meant for static resources, it has nothing to do with crawler

    Thread Starter mcdeth

    (@mcdeth)

    Ok, thanks got it. But with any cache set to >=4h, is it worth to run crawler more often than 3h?

    Plugin Support qtwrk

    (@qtwrk)

    yes , you will need to consider the case the cache is purged in some event, like you publish/edit something , upgrade something , change some setting …etc

Viewing 3 replies - 1 through 3 (of 3 total)
  • The topic ‘Running crawler.. too often?’ is closed to new replies.