Permalink Redirect Problem

I’m wanting to do the OPPOSITE of what most people want, so I’m having a hard time searching for the answer to this question. I’m creating a membership site with basic protection using OptimizePress.

I created a page with a postname permalink “/page-2”

I wanted to make it less obvious so that people with access to page-1 won’t just guess the URL for page 2.

I changed the permalink to “/page-2-0”

Now when you type it “/page-2” it still redirects to the new permalink. Can I make it so this doesn’t happen?