Use a separate custom table (not posts) to handle file upload data

The Problem:
Website that has over 100 photos per post. Could possibly swell posts-table beyond 100K entries within a few years.

The Dilemma:
Would much rather stick to default WP UI for handling images/files with the built-in “Add Media” Uploader. I’d like to avoid using a separate File Gallery implementation if possible.

The Question:
Has anyone found a way to use your own custom table (example “files”) to store the file-data, instead of using the posts table? (without having to hack core files)