migration issues with 4.0

  • Resolved donbowman

    (@donbowman)


    7 months, 3 weeks ago

    I’m testing this on an offline setup with a mirror of my database.

    I’m seeing some errors, and have a few questions…

    wp stateless migrate 20240219175240 –progress=5 –force

    I run that, but there seems to be no progress shown, nor logs, should it?

    I end up with a large amount of duplicate key errors (I guess during the migration?)

    The migration does not finish. How may I debug this?

    WordPress database error Duplicate entry '2971901' for key 'wp_stateless_files.post_id' for query INSERT INTO wp_stateless_files (post_id, name, bucket, generation, cache_control, content_type, content_disposition, file_size, width, height, stateless_version, storage_class, file_link, self_link) VALUES (2971901, 'www/99cf42b7-authorisation.svg', 'agilicus', '1705615746660848', 'public, max-age=36000, must-revalidate', 'image/svg+xml', NULL, NULL, '0', '0', '3.3.0', NULL, 'https://www.agilicus.com/www/99cf42b7-authorisation.svg', 'https://storage.googleapis.com/storage/v1/b/agilicus/o/www%2F99cf42b7-authorisation.svg') made by require('wp-blog-header.php'), require_once('wp-load.php'), require_once('wp-config.php'), require_once('wp-settings.php'), do_action('plugins_loaded'), WP_Hook->do_action, WP_Hook->apply_filters, UsabilityDynamics\WP\Bootstrap_Plugin->pre_init, wpCloud\StatelessMedia\Bootstrap->init, wpCloud\StatelessMedia\Batch\BatchTaskManager::in…"

Viewing 5 replies - 1 through 5 (of 5 total)
  • Thread Starter donbowman

    (@donbowman)

    I’ve also observed…

    MySQL [www_database]> select count() from wp_stateless_file_sizes; 
+----------+ 
| count() |
+----------+
| 11765 |
+----------+

    that this table is larger than the number of files i have (~2900files).

    Plugin Author Vladimir Palant

    (@planvova)

    Hello @donbowman

    Thank you for your findings!

    Error 

    WordPress database error Duplicate entry '2971901'

    might be related to the action when 1 attachment is processed more than 1 time.

    In this case the Attachment is already processed and added to the new table. So it should not be an issue. Anyway thank you for reporting this – we will take a closer look!

    >that this table is larger than the number of files i have (~2900files).

    Yes, this table contains the Attachment sizes, and it depends on how many sizes are used on the site.

    The main table is wp_stateless_files – it should contain all Attachments processed with WP-Stateless.

    So it means the count lines in the table wp_stateless_files should be equal ~2900, and wp_stateless_file_sizes may contain many more entries.

    >I run that, but there seems to be no progress shown, nor logs, should it?

    Yes, if you use parameter progress=5 – it should print progress status every 5 seconds. We will check and test it again.


    Please let us know if you still have any questions/findings.

    We always appreciate your help!

    Plugin Author Oleksii Bolhunovskyi

    (@obolgun)

    Hello @donbowman,

    Thank you for your report! You collaboration is very helpful as always!
    We released a fixed version 4.0.1, please run the command
    wp stateless migrate 20240219175240 –progress=5 –force
    again.
    There should be no errors and Data Optimization should be marked as “finished”.

    Best regards!

    Thread Starter donbowman

    (@donbowman)

    thank you, will try.

    on first look it seems you just unconditionally supress error messages? Is this necesarily wise?

    Plugin Author Oleksii Bolhunovskyi

    (@obolgun)

    Hello @donbowman,

    Suppressing messages during upgrade, data migration, etc is a common practice. Checking each record and deciding if we need to add or update something requires additional resources. We made a lot of tests on hundreds of thousands attachments and such an approach gave us 25-30% performance improvement.

    The messages are for debugging purposes and as soon as it confuses our users we better hide them.

    Just to be sure, please check the number of attachments in Google Cloud Storage:

    select count(*) from wp_postmeta where meta_key = 'sm_cloud';

    and the number of migrated data:

    select count(*) from wp_stateless_files;

    If these two numbers are equal – everything worked correctly.

    Thanks!

Viewing 5 replies - 1 through 5 (of 5 total)
  • The topic ‘migration issues with 4.0’ is closed to new replies.