Loading a WordPress Blog into a Static Site

I’m having trouble understanding a rather complicated concept – in short, I have a website I’m designing – one of the features is that it is to have a blog, but the blog is supposed to load within the actual website itself. My understanding so far is that to do that, I’d need to follow this post (Integrating WordPress with your Website) but I’m a little lost on the specifics.

Do I need to load the PHP code on every page within the site or just the page containing the blog posts? Also, with “The Loop”, where am I supposed to be putting all this code? If I’m integrating the blog within another site, why do I need to go about editing the existing code within the WordPress install? I assumed that I’d just be displaying the blog posts within the static site and then just customizing the design within the CSS of the static site. Am I wrong to assume that?

I’m just rather lost because I don’t really understand a lot of the specifics for what the guides are talking about and could use some help.