Installing WordPressfor preview

I have signed up for a new webhost for my existing WordPress site. I want to install WordPress on the new webhost and test it before I transfer the DNS to point to the new server. I can preview the files on the new webhost’s server though a URL that the gave me. I made the mistake of entering the actual URL for the site not the preview URL when I installed WordPress, so when I log into WordPress it takes me back to the old server.

I am thinking that what I need to do is to delete the WordPress installation and reinstall it using the preview URL the webhost supplied. Then when the domain is finally transferred I need to tell WordPress somehow to use the actual URL.
Is this the way to do this? I am sure this is easier than I think it is but worried I might mess up the site.
I appreciate any help you can give me.