Mass import

  • Resolved sbuwp

    (@sbuwp)


    3 years, 8 months ago

    I have some strange phenomena when bulk importing photos (ZIP file) and CSV file. For example, 500 photos and a CSV file with 5000 entries.
    During the import function, the system reports several times that it is ready, but this is not the case. It continues to work in the background and cyclically presents how many CSV entries have been processed. However, these are not plausible values. I have recorded a screen record once so that you can see this (Screenrecord). After 25 minutes, I stopped the recording because the system was still running.
    I have already got used to it and look directly on the website when the photos and the CSV are available. Then I cancel the data import.
    Furthermore, at the beginning there are sometimes error messages after the first import that a file cannot be opened. A little later, the message disappears again and the system continues to convert.
    For me, troubleshooting is not urgent, but the behaviour should be analysed.

    • This topic was modified 3 years, 8 months ago by sbuwp.

    The page I need help with: [log in to see the link]

Viewing 2 replies - 1 through 2 (of 2 total)
  • Plugin Author Jacob N. Breetvelt

    (@opajaap)

    I see nothing strange happening. The .csv was almost done. This may take a long time.

    You can see the progress of csv import when you look at the depot with an ftp program.
    This is how it works:

    – it copies the my.csv to temp.csv
    – it reads from temp.csv the header, interpretes it and copies it to a new empty my.csv
    – until timeout : { read a line from temp.csv, interprete it, and, when ok: process it, when not ok copy to my.csv }
    – at almost timeout: copy the rest of temp.csv to my.csv.
    – delete temp.csv

    Now my.csv still has the header and the processed items are out of it.
    Then the system re-starts (attempt to continue message), and repeats the above until my.csv has fully been processed. You can also resart manually by presing the Import button again.

    After the csv is fully processd it will start to import the tree of folders with photos.

    I did a test with a .csv from a photo db table that contains 150.000 entries and this took over 2 hours, but worked correctly.

    Just have more patience. Custom data field imports are time consuming.

    Thread Starter sbuwp

    (@sbuwp)

    Thanks very much for you quick answer. Now its clear.

Viewing 2 replies - 1 through 2 (of 2 total)
  • The topic ‘Mass import’ is closed to new replies.