how to optimize and test my site for results with lots of plugins

Hi all,
I’m hoping I can explain my issue concisely here so I can figure out the best practices for the next steps to fix my site. I greatly appreciate any help or advice from anyone in advance!

I have an ecommerce site hosted on DreamPress 2 using WooCommerce.
– Currently I’m running WP: 4.4.2 and WC: 2.5.2
– I have about 65 active plugins that I’m using.
– P3 doesn’t give me any indication that a single plugin is causing any issues. WP core is by far the top of the list.
– DreamPress uses Varnish Cache
– Using W3 for browser caching
– Autoptimize is used to help condense all CSS and JS
– I do have a BackBuddy for daily backups as well (runs in the middle of night)
– using W3 and Autoptimize has helped a lot with load times.

Now to the issue:
In one moment my site is working with no errors and TTFB is about 2-3 seconds. Actual testing shows the site loading very fast once that 2-3 passes and everything is smooth.

Then in that same day, all of a sudden my site will start issuing “internal server error, try again later” with a white page about 50% of the time a page is loaded or you click a new link. Page load time becomes very slow. after hours or half day it goes back to being fast again. This seems to happen randomly.

Traffic reports from DreamPress and a plugin do not show any unusual traffic spikes on my site during this time. Normally I’m getting about 100 visits per day per Google Analytics. DreamPress Stats shows about 1500-2500 reqs per day.

I keep reaching out to DreamHost for support but they are no help. They just tell me I should only have no more than 12 plugins and that i should delete all my plugins, then re-enable to see if there is any differences. DreamPress doesn’t allow me to see any resource utilization (at least they aren’t directly me to anything that does). They have told me I was maxing out the apache php requests and they increased those but the issues still occur. They say I’m not maxing out ram or HD space and the Server is not the problem.

So I have two primary problems with this:
1) Without having access to server utilization metrics how can I affectively optimize my sites load time and figure out why the site keeps issuing “internal server” errors. the basic idea removing a plugin and see if the site runs better won’t work for me because without changing anything the site works great one moment and horribly the next so I wouldn’t be able to conclude any success doing it that way.

2) All the load timing sites like google etc show the TTFB is 2-3 seconds which is the worst part. I currently am using Varnish Cache which I thought should be reducing that, so how can I figure out what needs to change to load the page faster? What’s the best practice for this?

overall I’m just frustrated with DreamHosts support and reading all the “How to optimize your wordpress load times” and getting no where, I’m at a lost of what I’m supposed to do now to figure out the internal server error and to get the site to load faster.

Any help would be great!