Michael Samson

I just wanted to second this. Thank you to Peter for properly identifying the issue.

This problem is still occurring in the most recent version of WP-Optimize. Please resolve this before it causes more people problems (thankfully I was very careful and didn’t delete these tables).

Here is an old DB map of BuddyPress that confirms this:
https://buddypress.org/wp-content/uploads/15/2013/11/bp_db_schema_v1.0.png

A word to the wise…

DO NOT ever delete database tables based solely on the [not installed] flag in WP-Optimize. Research every table first to be absolutely sure it is not in use!

Hi Marko,

Unfortunately this has cost me days of work, and we are now completely abandoning this effort. I’m about to dismantle our Redis cluster since it was only being used for the minification.

To be completely frank, this is a serious issue that affects one of the most used features of this plugin. I think it’s a bad sign that this has been known about for nearly a year, and is still not fixed. A year is a very long time to have a known issue.

Also, I find it strange you set these threads as Resolved simply because you wrote back. This is obviously not resolved.

In any case, I’m not going to be trying this again. It’s simply not worth all the potential problems.

We’re using W3 Total Cache for the browser caching, but that’s all. I do hope you get this fixed for your other users.

All my best,

~ Michael

For anyone wanting to fix this issue directly, simply edit options.php as follows:

Add the $ in front of homeonly and allpages as shown below:

<option value="homeonly"<?php echo selected(get_option('pagetarget'),$homeonly); ?>><?php _e('Home Only', 'adhs'); ?></option
>
<option value="allpages"<?php echo selected(get_option('pagetarget'),$allpages); ?>><?php _e('All Pages', 'adhs'); ?></option>

Add the $ in front of drop, bubble, and fade as shown below:

<option value="drop"<?php echo selected(get_option('animationin'),$drop); ?>>Drop</option>
<option value="bubble"<?php echo selected(get_option('animationin'),$bubble); ?>>Bubble</option>
<option value="fade"<?php echo selected(get_option('animationin'),$fade); ?>>Fade</option>

Please note that the drop, bubble, and fade variables are in two locations, for the Animation In and Animation Out settings. So make these corrections in both spots.

This is a simple fix for the plugin author to make btw… =)

~ Michael

Hi Andrei,

I wanted to follow up with you regarding the results of my testing.

I can confirm that Issue #1 is definitely fixed. I tested by disabling the auto-complete feature, and this time all of the meta boxes functioned properly when editing a page in the admin.

For Issue #2 it looks like I originally misunderstood what that feature was for. If you disable the editor feature then you cannot add custom body classes in the admin (by design). It simply removes the editor, but pre-exisitng entries are still there on the back-end. Please correct me if I am wrong.

I found that if you disabled the editor that it would not display even if you’re logged in as an admin (I assume this too is by design).

It looks like you added a new third setting to restrict editor access to admins only. I don’t recall that being there before. This is the only thing I couldn’t test, because we only use admin/subscriber accounts. So only admins have access to the WP admin. I don’t have an editor account to test with.

I did however test by disabling the editor feature while simultaneously limiting editor access to only admins. In that situation the editor does not display for admins (which I also assume is by design).

So it looks like everything is fixed here. Please let me know if any of my above assumptions are incorrect, and thank you again for fixing this!

All my best,

~ Michael

Hi Andrei,

Thank you for fixing these two issues. I’ll update the plugin shortly and will test the corrections for you. This way you know they are definitely fixed on my end.

Thanks again for taking care of this!

~ Michael

Hi Andrei,

Better late than never!

I’m still using your plugin and these issues are still present, so it would be great to see these fixed.

I’ll keep an eye out for the next plugin update.

Thanks!

~ Michael

Hi Raul,

Having you evaluate the site is exactly what I have in mind. I’ll reach out to you on your contact form directly.

That’s a nice trick in the Network tab there. I don’t use that tab very often and didn’t realize it gave you the document size like that. Very convenient. Our document size including all of the inlined css is 270 KB (for our home page). It’s worth noting that our pages (which are largely post/media grids) only initially load with 20 posts. We have a form of infinite scroll in place that loads the “cards” in batches of 20, and then caches any cards not currently in the viewport. That is the custom caching I referred to earlier.

I’ll contact you now from your form and will reference this thread…

Thanks for being open to talking more!

~ Michael

Hi Raul,

Thanks again for the excellent response…

There will be an invariable amount of css files that never change, so for those the cache is static and reused for all pages.
That means, it will only trigger another cache file for a smaller, unique css file/inline it finds.

I like the sound of this a lot, so that the same css is not cached over and over again. To be completely honest I really do not know how much inline/dynamic css we are dealing with. The site is so huge and complex, and this is just one aspect of it I have not examined yet. I should mention that I am the person behind all the custom css on the site. But there are likely dynamic processes going on that I am unaware of.

Inlining css is a good option for eliminating render blocking scripts but watch out for the final html size after compression (the amount of data each user downloads per request).

What is the easiest way to determine the final HTML size? I’ve never attempted that before. Would it be as simple as taking a View Source and saving it to a text file? That would include all the inlined css.

When you have external css files you can offload them to the cdn, but if you inline them, you cannot.

That is true. Even with the larger cache from the combined files, those files would all have been cached in the CDN, even though they were coming from the EFS as an origin.

Nevertheless, wordpress can do at most 1 or 2 req per cpu core uncached… so make sure you consider a good cache strategy and amount of data to transfer.

We’re not using any kind of server-side caching right now because the site has a great deal of dynamic content. It may be possible to cache our pages in some way, but it would be extremely complicated. I’ve had this on the back-burner for a long time now. Instead we focused on a client-side method of caching page content that allows our pages to run efficiently and not overload browsers. This is another topic entirely.

Our site is extremely heavy in every aspect… media, nodes, scripts… it is very complex. I have been concerned about this from day one. We’ve made the site as efficient as we can, and it does load relatively fast considering what is being loaded and the sheer amounts of data. But suffice to say this is something I think about a lot.

Let me ask you this, would you consider looking at our platform as a paid job to help us assess where/how we can improve things related to loading and efficiency? If this is something you’re open to we can connect directly via email. I rarely bring in people from the outside, but you may be the right person to look at this.

~ Michael

Hi Raul,

Your responses are truly first rate! Thank you for such detailed information.

I could speak at length about many parts of your response, but I’m going to just focus on what I need to know right now.

I am really only concerned about three things…

1. Improving page load time, efficiency, etc.
2. Not keeping an unnecessarily large cache.
3. Not breaking functionality with the optimizations.

I agree with you that I will need to closely monitor the cache after we launch to see how it is growing in size, and what is contributing to that growth. On that note I have read your comments on this and will analyze the cache as we move forward.

There is one key difference with regards to the cache when using all inline css vs. the combined files method. When the combined files were being created it was a concern because so many caches were created even for pages that should have been identical. I know there were reasons for that as we discussed. But the bottom line was it would have been a growing problem, literally.

With the inlining of all css those combined files are completely eliminated. As you have described the css is basically collected for the page, minified using the tmp directory, and then inserted inline into the header/footer of the page. This leaves me wondering about cache footprint when inlining all css. On the surface it would seem that the cache would be a lot smaller because you only need a temporary cache to minify the content. However it does seem that the files in the temp directory do stay there, so this is not truly a dynamic process. Obviously the minified content is being saved to some extent so it can be reused without having to reprocess all that css.

This is where I am curious to know if we will be better off in terms of cache size by using only inline styles. Your thoughts on this would be appreciated.

Your comments about Google ranking and how it interprets the amount of inline css were interesting. I could see how Google may not like so much non-visual code being at the top of each page. It makes sense. But I am less worried right now about the rank, and more concerned with what improves our loading times and overall page efficiency. This is after all why I chose this plugin to begin with.

About the EFS, it is better for us to stick with it then storing the cache data on the individual servers. The main reason for this is we have a truly stateless architecture for the individual servers and I want to keep it that way. I don’t like the idea of separate caches on the servers that differ from one another. This is exactly why we are using the EFS to centralize the little bit of dynamic data we do have. However, I do like you idea of taking advantage of a cache server/service such as AWS Elasticache. Our entire infrastructure is on AWS, so it wouldn’t be difficult for us to extend FVM to use something like that.

Our platform is launching sometime in February, so we are still pretty early here to truly understand how FVM will behave when the site grows. Suffice to say this is something I will have to continue to monitor. At the moment I am liking the all inline css because it eliminates all those combined cache files. But I am not certain if this will remain a practical solution.

Thank you again for your continued advice and support!

~ Michael

Hi Raul,

Are you suggesting to download the cache zip files and then compare the two sets to see if there are any differences? That’s an interesting suggestion. It would pin-point any differences. I actually have diffchecker installed, but can also use notepadd++ since we’re not talking about that many files (it has a difference checker as well).

The EFS is honestly a nightmare for speed. It caused us so many problems with our earlier stack, and also cost a lot of time to figure out it was the EFS causing it all. The new Docker/ECS stack we’re using is amazing, and I spent a lot of time and money to have it built. We use CloudFormation here for everything, which I highly recommend for creating reproducible infrastructure.

We can’t use page caching because the entire site is dynamic (we’re a new social media platform). I mean, technically it is possible, but it would be very difficult to implement.

Regarding speed, yes, it is a concern which is why I installed FVM to help streamline our css at least. But we also need absolute uptime reliability, which is why we are load balancing. There’s also the fact we need to be able to scale fast. The good thing is that we can control everything at every level. We can use larger instance sizes if we want, control how many cluster nodes (container instances) we have, and also more granular control at the container level. I’ve spent two years here engineering our stack so that we could scale quickly and reliably.

It is interesting to hear you say how amazing the C5 instance type is. Perhaps once we launch (and have a larger budget) I will consider using those. It can still be a load balanced environment using containers, and also use the C5 type at the cluster node level. I can only imagine how expensive that instance size must be. Right now everything is using t3.small instances, but this is in a pre-launch state so it doesn’t matter.

Anyway, I said I wouldn’t start talking about all this and I did anyway. I’m definitely interested in anything we can do to speed up the site. Our site is media heavy and script heavy, and I’m always looking for ways to improve it’s speed.

Ok, I’ll talk with you more tomorrow after I’ve had time to study those cache files. Thank you so much for helping me here!

~ Michael

• This reply was modified 6 years, 5 months ago by Michael Samson.

Hi Raul,

Wow, now that is a response! Thank you again.

I want to respond in detail, but that will have to wait until tomorrow as I am deep in the middle of another project here.

I’m glad I discovered these issue here today. At least now I know about them and can figure out what is going on. I did quickly take care of excluding our dynamic css files. Luckily I know our css files very well, and there were in fact two dynamic files. This is my fault as I should have excluded them earlier. I’m actually planning to eliminate the dynamic css entirely in the future and hard-code it. It’s on my to-do list. Both these dynamic css files are on the EFS.

So I excluded them but unfortunately it did not solve the problem. There is obviously something else going on here that makes the css unique. I looked at the entire list of css files, and everything left is hard-coded. I also don’t see any hash codes that would indicate inline css being cached. I wonder what is causing this.

Btw, about the EFS… you are right about how slow it is. Our old stack used to have the entire application on the EFS and we had to stop using it because it was choking the servers due to slow response time (even if we maintained burst credits). Our new stack is using Docker and ECS, and our entire application runs on the load balanced servers, with the exception of a few files that must be on the EFS to be stateless. You got me wondering about the idea of the EFS being slow and if we’re really benefiting from using FVM. Then again, once the cached files make it onto the CDN they are served from there and not directly from the EFS, so perhaps this is still an improvement in efficiency.

In any case I’ll look into this all further to determine the cause. Thank you for your offer to look at the site, but it is not yet public. I really can’t let anyone in there, even in the development site. I’m confident that with your feedback I’ll figure this out.

I’ll be in touch again tomorrow…

~ Michael

Hi Raul,

Please read my above post first…

I just discovered that something as simple as loading different member profile pages is producing duplicate sets of cache files. We use BuddyPress, and so I visited some different member profiles. I noticed that the identical set of cache files were being produced for the separate profiles even though they share the same css. If the plugin is creating duplicate file sets like this we’re in big trouble. Imagine what would happen when we have millions of users.

This is definitely going to have to be looked at more closely. I’m now thinking the duplicate cache files I saw earlier were from visiting different member profiles on the site. Do you have any idea why this is happening?

~ Michael

Hi Raul,

Thank you very much for this detailed response. It has answered many of my questions. In fact, I think your answer here is so good you should make it part of your documentation! ??

I’ve ready everything you wrote carefully and have a few remaining comments and questions...

1. Regarding having FVM enabled during development: I actually have a separate development and production environment. I have FVM enabled in both because it is important to run everything as if it were live. Otherwise we have no way of knowing if FVM is working properly and how it is affecting our systems. Our development environment dubs as a staging environment, so it must be absolutely identical to production, including having all plugins active that are active in production. We do our real development in local environments, and for that I simply disable all file processing in FVM so I can easily work on the css.

2. You have clearly answered my question about how the files are accessed prior to processing by the plugin. I checked the logs like you suggested to see if the files were being fetched locally or via URL. Luckily every single file path I see is an absolute path, and not a URL. So I assume this means all files are being fetched directly from the server. As you pointed out this means a CDN purge is technically not necessary since the new cache files will have unique URLs. Obviously all my css changes would also make it into the cache since the files are accessed directly from disk. FYI, I still need to purge our CDN after every deployment regardless. I’ll purge FVM before the CDN so the older cache files are also purged from the CDN (I know they wouldn’t be used either way).

3. While I was looking at the cache logs I noticed something odd. I’m seeing the same list of css files being cached multiple times, in separate cache files. It is the identical list of files. I thought if a page used the identical files that it would not create a duplicate cache file. You talked above this in #b of your answer. Do you think I have a potential problem here? I should mention that our site is enormous (a new social media platform), and it would be extremely bad if every page ended up having it’s own cache. My question below (#4) may be related to what is going on.

4. We’re running a load-balanced environment and I just want to make sure this won’t adversely affect FVM. I knew up-front that the cache itself had to be centralized between all servers (stateless), and have taken care of that. All the servers read/write the cache to a single location. But of course each server is running its own copy of FVM. I’m just wondering about how/when FVM generates that cache. If multiple users are accessing different severs simultaneously all running FVM, and it is writing the cache to the same location, then is it possible this could cause FVM to duplicate cache files? Is this perhaps the reason why I am seeing what looks like duplicate caches?

Just adding this thought: I can obviously monitor the cache here in our development site. It will become evident as I browse the site more (between deployments) whether or not the cache files are truly being unnecessarily duplicated.

5. Some of our css files are already minified. These are files that are using the proper min.css extension. Most of these files belong to plugins and scripts that had the css minified already (it’s not minified by another script or plugin). I haven’t noticed this causing any issues, but wanted to ask you about it. Could this pose a potential problem?

…..

I think that covers everything. I’m glad you’re the kind of developer who is willing to answer technical questions like this. I chose your plugin originally because I could tell it was designed to operate without interfering with existing systems. Frankly I am amazed by how well it is working already. Of course I still need to get this working with the JS, but that is going to be more challenging. I’ll experiment with that soon enough.

~ Michael

• This reply was modified 6 years, 5 months ago by Michael Samson.

Hi r-a-y,

Thank you so much for your quick response to this thread. Your timing is perfect as I was looking to update BuddyPress to version 3.0.0 today or tomorrow. When I read Laurent’s comment above it occurred to me it may have been from updating BP to this latest version.

I see the change you made over at GitHub and will update the plugin here so we don’t have this problem when BP is updated.

…..

While I have your attention here, I’d like to ask you two quick questions if you don’t mind…

1. Would it be difficult to modify the plugin so that it uses @username instead of username in all locations?

The reason I’m interested in this is I personally find it confusing to use a combination of @username and username within the same system. For example, to mention someone in a comment you use @username. I’d much rather see usernames in this format across the board. I think this makes it a little easier for users to understand how usernames and mentions work.

2. I’m sure you’re familiar with the small ajax window that appears when you type @username into a comment. This window shows a list of matching users as you type. Normally this window displays both the username and Full Name. After installing your plugin the window now displays the username twice, which is a little silly looking. In this one instance I’d like it to display both the username and Full Name as it normally would.

…..

My thanks for making this plugin in the first place. It saved me a lot of time on our project!

All my best,

~ Michael

Hi Laurent,

I just installed this plugin myself (despite its obvious age and lack of updates). It’s a very small plugin if you look at the files/code involved. This is a good thing, as it should be easy to correct any conflicts.

I’m still running a WP version under 4.9.6, but will be updating in the next few days/weeks possibly (it’s a development environment). At this moment the plugin is working very nicely, so I’ll know immediately if something goes wrong when updating WordPress. If I see that happen I’ll make sure to comment here.

In the meantime, if you do get to the bottom of your issue please share it here. I’d be interested in the solution just in case I also need it.

Btw, the fact this plugin still works so well shows how little of BuddyPress core has been changed over the years. Lucky for us I guess in this case.

P.S. If the plugin does break on updating I can have my own development team look at it too. I’d be happy to share our solutions with you.

~ Michael