Speed up those archives

MT?
Please dont kill me for this. :p

Well, there is a line in the Features page that may oppose itself to your project :

No rebuilding a€?? Changes you make to your templates or entries are reflected immediately on your site, with no need for regenerating static pages.

As I understand it, WP is not the best blogware to use if you want static generated pages ?

Don’t get me wrong – I think the dynamic nature of WP is great – and I will not trade it away for MT. My idea is to get the benefits of both. I’m not asking for it to become a delivered feature (which is why it’s in the “hacks” forum.) I’m planning on building it in myself.
I’m thinking that it shouldn’t be that hard to do a subset statically, and completely transparently to both the administrator/authors and the visitor.
The idea is to make WP do something that MT does well, but still retain all the flexibility of WP. Most stuff would still be 100% dynamic, but the pages that get hit the most would be pre-rendered every time a change required them to be.
The only time the entire “cache” would have to be rebuilt is if the template changed, and with the beauty of css, there’s no compelling reason to be constantly tweaking templates anymore (instead, constantly tweak the css ??
As I’ve said though – I’m not asking someone to build this for me. I’m going to build it myself – I suspect that some people would find it a valuable hack, and I was opening it up for suggestions.

EXCELLENT idea Nodecam. Choice is always better :-). It would be a great option to allow those who want static pages to still be able to use WordPress.

The problems with caching is that you still need to touch the SQL database for a number of things. But it really depends on what exactly is causing the spikes. Which queries? What kind/size of query? Can the query be changed? Can a smaller query/request/result be retrieved to lessen the load, and do larger queries only when needed?
That last Q is one I’m asking myself right now, as I only show the pre-more/excerpts on list-view pages, and as such I could in theory NOT request the body text for those pages. It’d require me to move my more text (and some other things) into the excerpt, but would cut the data retrieval to possibly less than 10% on non-single-entry pages.
d

The output cache plugin sounds like exactly what I had planned to do – too bad that link doesn’t work, I’d like to see what is being done there.
Sounds promising anyway – I’ll send him an email to see if he’s taken it off the market, or if it’s just a bad url.

I have another (theoretical) optimization I just implemented, thought I’d share with the WP world my thoughts. ??
If you’ve ever gone to my site (www.chait.net), you know I’m doing a news/reviews layout, rather than ‘a classic blog’. I show only [more] sections in ‘list pages’, not showing entire content until I get to the single-post-view pages. (Well, and I make an exception for ‘static post category’ views…)
Additionally, on my homepage, I play games with WHEN I show the [more] content, as I only want it for reviews at this time.
And to make matters worse, I upped the posts-per-page count pretty high, to make the front page useful… ??
So…. what to do?
I don’t know if this WILL make a difference in the end, but in theory it should help both performance and RAM overhead, for anyone doing a similar [more] breakdown of all list pages.
In wp-blog-header, I do a pre-query of the database to determine whether the set being requested results in more than one result. i.e., I do a “SELECT distinct(ID)…”. That gets me an array, which I can count.
IF the count is 1, the ‘normal’ query is done as a “SELECT *”. If the count is not one, I instead do a select statement that requests all fields other than post_content. In this way, I get rid of the overhead of pulling large content across the network from the server, and storing it in ram, when it isn’t being used.
Now, to make this all easier, I just wrote a script I call ‘make-excerpts’, which runs through the entire post table one-by-one, and creates an excerpt — it either grabs up to the [more] tag, or up to the first CRLF in the body. It also goes and extracts from the first page my special ‘Pullins’ tags, for inline images and amazon linkages, and tack those onto the end of the excerpt.
I can’t say for a fact the impact this will have, but it should help especially on heavy-load sites (which mine is slowly growing to be… ?? ).
Anyway, if people want to chat more about this stuff, let’s continue this thread as the ‘performance enhancements’ thread. ??
-d