Crawler file generation
-
Hello.
Report number: NDMIBAPI
My store have less than 700 real urls based on sitemap. This include products, categories, pages, posts, etc.
When I try to use seo sitemap index on crawler settings, if I click on button to generate it, are generated properly.
But sometime after when re-generate by itself again this file, are not including the products, only categories, maybe is any timeout while reading product sitemapt due my plugin generate by petition dinamically to provide updated. Strange that only happens when update file by itself.
So how I can’t fix this, I am trying to use the automatic crawler generation without use my sitemap, but the issue is that now is generating more than 11.800 url’s.
I checked and I found that is adding all product variations.
So if main product have this url:
/camiseta-aida-6656-roly-mujer-tirantes/Is adding too all variation inside like this:
/camiseta-aida-6656-roly-mujer-tirantes/?attribute_pa_color=turquesa&attribute_pa_talla=2xl
/camiseta-aida-6656-roly-mujer-tirantes/?attribute_pa_color=royal&attribute_pa_talla=2xl
/camiseta-aida-6656-roly-mujer-tirantes/?attribute_pa_color=roseton&attribute_pa_talla=2xl
/camiseta-aida-6656-roly-mujer-tirantes/?attribute_pa_color=rojo&attribute_pa_talla=2xlBy other hand, out of the file, on blacklist I found a lot of strange url’s, like these:
/?post_type=scheduled-action&p=25657
/?post_type=acf-field&p=24413and others that seems product id’s:
/20017/
/20018/
/20020/
/20021/
/20022/
/20023/
/20024/
/20025/I tried to use on crawler settings the exclusion content field to don’t include customized content on sitemap, and I added all that appear in the right list, except “products”. In the excluded options is included “product_variation”, but seems that don’t work for me this “exclusion list”.
Any suggestion?
Thanks!
- The topic ‘Crawler file generation’ is closed to new replies.