Custom templates not showing up

I am building my first WordPress site and am a bit confused, though I’ve read the codex for the past hour or so. I have a simple site with a couple static pages, including the index page. So I have index.php looking exactly how I want it. Then I attempted to create addicts.php but when I navigate there, it gives me a 404 error. I have avoided using “Pages” because each of my static pages has a different template design and I can’t get any of my custom templates to show up in the options on the page writing section.

If the basic structure of my site is the same (header, footer same across the board), would it work for me to have page.php contain my template and a center containing <div> that I would specify for each static page, created in the pages section? Does this make any sense?