Broadcast Queue DB issue (string size)

  • Resolved dmayhak

    (@dmayhak)


    2 years, 9 months ago

    I am noticing a strange issue with the wp_3wp_broadcast_queue_data table: the column ‘broadcast_data’ is getting VERY large values inserted. When I step through the code in /queue/data.php, I can see the size of $data and $new_data and it looks to be as expected–one example of about 1700 chars. However, when i look at the data in the table, the length has exploded to 200K+ (sometimes 500K) characters.

    I’ve tried disabling all non-required plugins. The data looks ok at the end of the ‘compress’ call in data.php. However, INSERT statement logged by MySql has the huge string in the insert statement.

    Looking for help/ideas to help figure out where and how this string goes from 1K to 200K in length.

    Thanks!

Viewing 3 replies - 1 through 3 (of 3 total)
  • Thread Starter dmayhak

    (@dmayhak)

    I looked into the fetch from the broadcast_queue_data table and found this interesting:

    broadcasting_data field in the DB has length of 20901.

    sql query: “SELECT * FROM wp_3wp_broadcast_queue_data WHERE id IN (‘248’)”

    immediately after the $wpdb get call:

    broadcast_data length (in code) is 1025.

    guess I have more digging to do, but would welcome any ideas. ??

    Thread Starter dmayhak

    (@dmayhak)

    I think i found an issue when calling:

    ThreeWP_Broadcast()->api()->low_priority()->broadcast_children( $post_id, $broadcast_to_sites );

    this is in a handler for ‘savepost’ that will broadcast a post to zero or more sites based on some data in the post.

    When in the WP admin area, publishing one post works ok. Publishing 5 posts resulted in some issues.

    I added some simple logging to the compress() method in /queue/data.php

    $new_data = serialize( $data );
error_log( print_r( 'compress() serialized length:' . strlen( $new_data ), true ) );
$new_data = gzcompress( $new_data , 9 );
error_log( print_r( 'compress() gzcompress length:' . strlen( $new_data ), true ) );
$new_data = base64_encode( $new_data );
error_log( print_r( 'compress() base64_encode length:' . strlen( $new_data ), true ) );
return $this->set_data( $new_data );

    the publishing of 5 posts resulted in:

    [04-Jun-2022 02:54:32 UTC] compress() serialized length:<strong>5530723</strong>
[04-Jun-2022 02:54:32 UTC] compress() gzcompress length:304696
[04-Jun-2022 02:54:32 UTC] compress() base64_encode length:406264

[04-Jun-2022 02:54:32 UTC] compress() serialized length:<strong>8167436</strong>
[04-Jun-2022 02:54:32 UTC] compress() gzcompress length:1045420
[04-Jun-2022 02:54:32 UTC] compress() base64_encode length:1393896

[04-Jun-2022 02:54:32 UTC] compress() serialized length:<strong>13030598</strong>
[04-Jun-2022 02:54:32 UTC] compress() gzcompress length:3463590
[04-Jun-2022 02:54:32 UTC] compress() base64_encode length:4618120
[04-Jun-2022 02:54:33 UTC] WordPress database error MySQL server has gone away ...

[04-Jun-2022 02:54:33 UTC] compress() serialized length:<strong>31641989</strong>
[04-Jun-2022 02:54:34 UTC] compress() gzcompress length:15786512
[04-Jun-2022 02:54:34 UTC] compress() base64_encode length:21048684
[04-Jun-2022 02:54:34 UTC] WordPress database error MySQL server has gone away ...

[04-Jun-2022 02:54:34 UTC] compress() serialized length:<strong>110506248</strong>
[04-Jun-2022 02:54:38 UTC] compress() gzcompress length:71923059
[04-Jun-2022 02:54:38 UTC] compress() base64_encode length:95897412
[04-Jun-2022 02:54:40 UTC] WordPress database error MySQL server has gone away ...

    looks like some variable is not getting cleared, but rather appended to when multiple items are published in bulk.

    TIA for your help!

    Thread Starter dmayhak

    (@dmayhak)

    Update: Issue found and resolved!

    I was debugging an unrelated issue using the Query Monitor plugin. Even though it was deactivated, it was injecting the custom DB object and adding a lot of excess data to the broadcast data object. After disabling that, the broadcast performance went back to normal.

    Will probably end up adding some code to disable QM when broadcasting, but just leaving it off for now.

    Thanks!

Viewing 3 replies - 1 through 3 (of 3 total)
  • The topic ‘Broadcast Queue DB issue (string size)’ is closed to new replies.