Object Cache + Memcached Breaks website

  • Resolved dev

    (@devksec)


    4 years, 1 month ago

    Hello,

    So we have a multisite setup (2 woocommerce subsites) with total cache which works fine using DB caching + memcached. When we disabled DB caching and try enable object caching instead, it breaks the backend of the WP sites which only responds with timeouts. The frontend seems to work fine however.

    Initially when this issue occurred we performed a manual clean remove/reinstall of total cache and just enabling object cache (with nothing else enabled) caused the issue to reoccur.

    We’ve restored our old settings and all works fine but only without object cache being enabled. The memcache server connection test, from within the object cache settings, passes and the settings are default.

    Any idea what could be causing enabling object caching to break the backend or are there some extra steps within the switch over that need to be completed ?

    Many Thanks!

Viewing 15 replies - 1 through 15 (of 20 total)
  • Plugin Contributor Marko Vasiljevic

    (@vmarko)

    Hello @devksec

    I am sorry about the issue you are experiencing and I am happy to assist you with this.
    Can you please share the screenshot of the settings in Performance>Object Cache. Are you using a single Memcached instance for both DB caching and Object Caching and have you tried clearing Memcached and possibly restarting the Memcached service?
    Thanks!

    Thread Starter dev

    (@devksec)

    Hello,

    So I’ve taken this from the master.php as the I can’t upload a screenshot but can email one over if preferable. 

    "objectcache.configuration_overloaded": false,
    "objectcache.enabled": false,
    "objectcache.debug": false,
    "objectcache.debug_purge": false,
    "objectcache.enabled_for_wp_admin": false,
    "objectcache.fallback_transients": true,
    "objectcache.engine": "memcached",
    "objectcache.file.gc": 3600,
    "objectcache.file.locking": false,
    "objectcache.memcached.servers": [
        "SERVER.cache.amazonaws.com:11211"
    ],
    "objectcache.memcached.persistent": true,
    "objectcache.memcached.aws_autodiscovery": false,
    "objectcache.memcached.username": "",
    "objectcache.memcached.password": "",
    "objectcache.memcached.binary_protocol": true,
<snipped>>
objectcache.groups.global": [
        "users",
        "userlogins",
        "usermeta",
        "user_meta",
        "site-transient",
        "site-options",
        "site-lookup",
        "blog-lookup",
        "blog-details",
        "rss",
        "global-posts"
    ],
    "objectcache.groups.nonpersistent": [
        "comment",
        "counts",
        "plugins"
    ],
    "objectcache.lifetime": 180,
    "objectcache.purge.all": false,

    We have a single Memcached for DB caching, page caching etc which is very underutilised. When we disable DB caching, we clear the cache before enabling the Objecting caching.

    Thank you for the assistance

    Kind Regards

    Plugin Contributor Marko Vasiljevic

    (@vmarko)

    Hello @devksec

    Thank you for your patience and sorry for the late reply.
    I’ve been trying to replicate this with no success. Try switching to Disk instead of Memcached and see if the issue persists.
    Also, try enabling the debug in W3TC settings and also check for any PHP errors on the server and wp debug.
    Thanks!

    Thread Starter dev

    (@devksec)

    No worries, we appreciate your support.

    So before we setup a Memcached server, we had issues with PHP performance on the EC2 instance caused by object caching leading to hanging and timeouts. This lead to the same behaviour as we had now and disk DB caching was used instead.

    As the wordpress backend crashed pretty quickly once object caching is enabled, is there a way to view the W3TC logs with debug mode enable if wordpress is inaccessible ?

    Thread Starter dev

    (@devksec)

    We’ve also moved the WP instance to a new host and cleared the cache folder within WP but the issue persists.

    Plugin Contributor Marko Vasiljevic

    (@vmarko)

    Hello @devksec

    Thank you for the information.
    So to confirm, you have issues with the object caching on both Disk and Memcached?
    The Pro version of W3TC does have a purge log. So when the option is enabled and you purge the cache you can check the login/wp-content/cache/log/000000/objectcache-calls.log
    YOu can also define(‘WP_DEBUG’, true); and see if there is an error showing
    Thanks!

    Thread Starter dev

    (@devksec)

    Yes it worked for 4-6 months without issue before causing performance issues and we switched to DB disk caching. When re-enabling it with object Memcaching, it breaks the wordpress management interface.

    With the pro purge log, can the cache be purged without access to the WP management interface? E.G can it be triggered via the wp command line.

    I’ll get the WP debug setup on our test environment and see if this shows any useful info and report back.

    Thread Starter dev

    (@devksec)

    So we’ve copied the prod version of our multisite to a new dev site with no outbound internet access apart from cloudflare etc. It can be configured to use either object Memcached or disk cache without issue. It seems we cannot replicate the original issue within the Dev environment.

    No errors in wp-log and nothing out of the ordinary with the W3TC debug on the dev site.

    Tried again in the prod site and object caching seems fine with disk caching but with high PHP CPU usage (up to 70% instead of 1-15%). This slows the site down in general. When switched to Memcache it starts to slow the site before getting constant timeouts. Restart of the wordpress services and it starts up and will only serve timeouts within the access logs. Only way to stop it is to disabled object caching within the master.php and restart the wp services again.

    Have tried using ‘wp cache flush’ and clearing the Memcache directly but still wont work with Object cache + Memcache enabled.

    Plugin Contributor Marko Vasiljevic

    (@vmarko)

    Hello @devksec

    Thank you for the information.
    As your testing suggests, the problem is with the compatibility of some sort on your prod environment.
    The only way to troubleshooting this is to disable the plugins one by one with OC enabled and see if some plugin might be causing the issue.
    As I can see in provided master.php
    "objectcache.enabled_for_wp_admin": false, which means that OC is not enabled of wp-admin.
    SO the issue is not with OC+Memcached per se (it works on the dev environment) but with the current setup.
    Thanks!

    Thread Starter dev

    (@devksec)

    The prod and dev environment are exact copies of the multisites. For this test, the other difference was instead of RDS we used a local MySQL. Apart from that the load on the server from external users.

    Is there anything we can do with W3TC to diagnose this ?

    Plugin Contributor Marko Vasiljevic

    (@vmarko)

    Hello @devksec

    Thank you for the info. Possibly the issue is with the RDS Memcached support or configuration, however as I can see RDS does support Memcached MySQL.
    Please check out this article for more information.
    Thanks!

    We’re facing the same issue. We’ve replaced the whole Memcached cluster on our AWS setup, but didn’t fix the issue.

    The only way to solve the issue was disabling the Object Cache totally or enable it on the Disk method, not the Memcached one.

    Thread Starter dev

    (@devksec)

    Update: we created the custom options group in AWS for the RDS instance and DB caching started to work properly with Memcache. Seems Pagecache worked without this from reviewing the Memcache data.

    Objectcache when enabled (DB caching disabled), starting to create 1-2k objects in Memcache before the website eventually went down again. Could be resource issues when objectcache is enabled and start a heavy load on the server but seems odd.

    Plugin Contributor Marko Vasiljevic

    (@vmarko)

    Hello @devksec

    Thank you for the information.
    Interesting, so the problem of the conflict lies in the RDS configuration.
    have you tried increasing the Memcached memory in this case?
    Thanks!

    Thread Starter dev

    (@devksec)

    Definitely misleading that pagecache would input some data into Memcache without the RDS config changes but DB cache wouldn’t. I suspect both weren’t working despite this being the case.

    Is there a way for W3TC to detect improper setup of Memcache ? E.G error if it detects cached data isn’t being read/written to Memache correctly.

    DB caching & page caching seem to be working ok with Memache now just when switching to object caching we have this issue.

    The Memcache server has lots of memory free so thats not a problem.

Viewing 15 replies - 1 through 15 (of 20 total)
  • The topic ‘Object Cache + Memcached Breaks website’ is closed to new replies.