Blog is down often; porn sites linking in

I’m not sure if these two things are related or not, but I’ve found that my WordPress blog is often down (when the rest of my website is not), or it loads very slowly.

I’ve also noticed a large increase in the number of “visitors,” but when I track where they’re coming from, they’re often linking from porn sites that are hosted in LA, Texas, Russia, etc…or other bogus blogs that *aren’t* porn.

Is this influx of fake traffic responsible for my blog-loading troubles? And how can I stop this traffic? I’ve tried a plug-in that blocks specific URLs, but they keep changing.

thanks