Duplicate Products in Feeds

I have now found some instructions on accessing the log. Here it is

debug.log

Hi Glenn,

Thanks for using our plugin and reaching out to us. I did a search on the example product you provided in your feed and it is only once in your feed. See: https://www.dropbox.com/scl/fi/k9b3yfiwjiv757bl3nfvp/Screenshot-2024-03-13-at-09.29.28.png?rlkey=yahmv32agh65inau0unu0gjge&dl=0

I suspect you might be looking at a cached version of your feed? Can you make sure to clear all caching plugins and the caching of your browser?

Thank you for your response.

In the image you provided it does show only 1 occurrence of 210473 but on mine it shows 4 of them, 2 per product. In the product the first is in the g:id and the second occurrence is in the g:link field which in your image is truncated. See below. Why is your search not at least finding 2 of 210473 in one product? There must be an issue with your search or the file you are looking at is truncated somehow.

<g:link>https://www.presentsofmind.com.au/game/everdell-2/?utm_source=Pinterest &utm_campaign=Pinterest&utm_medium=Social&utm_term=210473</g:link>

I have also investigated your suggestion and cleared my cache and even tried 2 different browsers but I can still 2 occurrences of the product Everdell with SKU 810082831054 and ID 210473.

“Why is your search not at least finding 2 of 210473 in one product?” –> because I searched on >210473< , including the < and > to make sure that it would find the value for the g:id field only.

When I now search on that product it doesn’t show in the feed at all anymore?

I am sorry to jump in, but I have same problem. My feed is for Fruugo.

Some products are duplicated in the feed. Feed is not cached. URL:

https://www.hatsfromoz.com.au/wp-content/uploads/woo-product-feed-pro/xml/hBCTYmZtIFKnloYH42aUxjFnigvm1OwE.xml

Do a search for product ID: 165764

I decreased number of products per batch to 200 from 250 and this seem to fix the problem.

Perfect, good to hear. Thanks for the heads-up.

Actually I may be jumping too fast.

Duplicates still occur, although at lower rate. Basically, the lower the batch count, the less chances of duplicates.

I think, this is a real bug that has to be dealt with. May be (as a band aid) feed file should be post-processed to remove duplicates?

I now wrote a small Python script that checks XML file for duplicates. I can confirm that duplicates exist with pretty much any batch size and their appearance is quite random. Number of duplicates is random as well

In addition to the duplicates, I am now finding some products that are not showing even though they should. I have a large inventory so I am not sure how big a problem this might be.