mapping overwrite

Thanks for the positive feedback and for your question. It’s a good chance to explain a bit more about MLA’s mapping process.

First, you probably know you can change the “Existing Text” option from “Replace” to “Keep” to avoid changes in items that already have a value set for any given field.

Second, whether the Existing Text option is Keep or Replace, MLA will only update an item if the value calculated by the mapping rule is different than the value already in the item’s field. Reading the items’ values is pretty efficient, but the logic required for an update is much more complex and avoiding it when it’s not required is a big time saver.

I am always looking for opportunities to improve efficiency and the mapping code has evolved quite a bit since it was first implemented. I welcome any further suggestions for improvement.

I hope that answers your question. I am marking this topic resolved, but please update it if I did not understand what you’re suggesting or if you have any suggestions on how to improve the mapping process. Thanks for your interest in the plugin.

Thank you for your reply.

I guess what I am asking is for MLA to flag already processed (and changed records) so that it does not overwrite a title already updated under mapping rules that it processed on the last scan. Also if any manual editing was done that record will again be overwritten if the scan is run again to update new records.

I suppose this is a feature where any of these options would be nice:
Do you want to update records already processed by MLA Y/N
Process only records uploaded since 6/7/2016

Thank you.

Thanks for your additional comments and suggestions.

I believe the “already processed” tracking would be problematic. For example, if you change a mapping rule then “already processed” may or may not still apply. It would also require recording the status of each item in the database somehow, and MLA does not add anything of its own to the standard WordPress database tables and fields. The “Existing Text: Keep” option serves a similar purpose, although it’s not exactly the same.

The “Process only records uploaded since” is an alternative to the existing MLA options to “Enable … Mapping when adding new media”. Those options automatically apply the mapping rules to each item as it is added to the Media Library and avoid the need to periodically, manually apply the rules. Automatic mapping keeps all items up to date at all times.

I hope that gives you a better sense of the choices I made in the development of MLA mapping rules. Thanks again for your comments.

Right, I had the “when uploaded” unchecked by mistake and I was using the media from ftp so I didn’t know if that would cause even more lag on that program that was already lagging alot. Meanwhile I read about bulk processing on this board somewhere you were talking with someone on how it took a long time. Mine timed out. I guess that was further motivation for me on this topic. I have 7000 pdf files that are going through on the initial implementation of this, normally it would not even approach these numbers. Now I changed the Bulk Chunk to 100, not sure if that is for the processing or not, but it’s a little further along. It timed out twice earlier about 40% through the images although it was running at full speed. Maybe it’s my host. Crossing my fingers at 60% right now.

ok, so it timed out again, but the silver lining:
I went into bulk edit and selected the last group of items and used mapping there to do a few hundred at a time. Might help someone who comes by later and reads this.

Thanks for your updates and description of your loading adventures. I know of Media from FTP but haven’t used it myself.

You wrote “Mine timed out.” Was that during a Media From FTP session, or some MLA process?

The “Bulk Chunk Size” option sets the size of the Bulk Edit and Map All processing chunks. If you want to process 100 items from the Bulk Edit area MLA will divide the work into chunks and process each chunk separately. Smaller chunks are less likely to time out, but the network delay for each chunk is slightly longer. A chunk size of 25 or 50 is plenty; the network delays won’t add much to the overall processing time. A 100 chunk size is much more likely to time out before it processes all 100 items in a single chunk.

The timeout applies to a single chunk, not the entire set of items to be processed. With the right chunk size you can map thousands of items. That will take a long time but there won’t be a server-side timeout.

On the timed out issue. It was MLA. I was doing the mapping and it just sat at like 40% forever so I just called it. I started mapping all the remaining unmapped in bulk editing instead. It works fine. I’m making the assumption that the server host cut the process. Also I am processing on https and the browser at times loses it’s connection. I know you mentioned some things, but frankly they are over my head.

Your plugin is beyond awesome and your support and amount of documented information here is incredible. I would vote it the best plugin in WordPress for it’s ability to map data from pdf that normally would cost hundreds, if not thousands of dollars and ability to process mass items. Also the templated functionality to render results exactly as desired – I couldn’t ask for more.

Thank you so much!

Thanks for your update and especially for the positive feedback.

Yes, I agree that the hang/timeout issue probably resulted from a dropped connection between the browser and he server. I will experiment with that and see if I can improve the handling of those cases.

If there’s anything else I can do to help you get the most out of MLA, let me know.