How to shut off Not Found page?

I am in need of shutting off WordPress’s default not found page, I believe it may be potentially contributing to overusing resources and has been killing my site.

For all posts, pages, and the index I’ve setup a caching system where only optimized HTML files are delivered. If the HTML file is not found, it returns the live WP page, including cases where it is ‘Not Found’. I believe some deviant bot may possibly be constantly entering non-existent URLs, or found someway around my cache system.