[Plugin: WP Super Cache] robots.txt can be cached and made non-format compliant

  • Resolved malcolmm

    (@malcolmm)


    16 years, 5 months ago

    I’ve just noticed that WP Super Cache adds an HTML comment <!-- --> to my robots.txt file that I generate using the KB Robots.txt plugin. This is not (AFAIK) part of the file format for robots.txt. I’m sure most browsers and spiders are smart enough to ignore this, but:

    If you are generating your robots (eg with KB Robots.txt) then WP Super Cache will add this comment, which can be removed by adding

    /robots.txt

    in the rejected URIs section of the WP Super cache config page.

  • The topic ‘[Plugin: WP Super Cache] robots.txt can be cached and made non-format compliant’ is closed to new replies.