Why are certain files slower to load?

Hi everyone,

I ran my site through a speed test (averaging about 3-4 seconds). I notice that initially two JS files are loaded via my theme. 3.45 seconds for the first, 2.88 for the second, then my logo image of 0.02 seconds.

The image is 13kb and the JSS are 1.15kb and 3kb respectively. Is this because JS files have more grunt work to do?

I’m also noticing that the majority of my content on my server has excessive time to byte delays. Is that server-related? What can I do about it? Some of them are

For instance, I have about 10 images, average of 7-8kb each. Connection: 400ms, Time to byte ranges from 1571ms to 2283ms….

That seems to be the issue. I know I’m on a cheap shared host, but surely I shouldn’t be getting such horrible first time to byte.

Any thoughts are greatly appreciated!

Thanks!