I think you’re putting forward the need to use a developement environment AND a staging environment, distinct from your production site. This has become much more affordable as hosting costs have gone down. It also implies strict work methods, such as versionning (SVN, Git) and publishing methods and processes beyond the scope of that question, but worth learning – in fact they should be a requirement for any developer worth his salt – pun intended.
Also note that all plugins put forward whether they are compatible with the Wp version you wish to upgrade to – you can find this in your dashboard and on the repo, and most developers update their plugin infos for a major WP new version, so keep an eye on this.
The extent of what you wish to test is then entirely up to you. Do you simply wish to make sure there is no blank page or 500 error in the site after the upgrade, or need to check every single pixel is properly aligned under every browser and screen size ? Then again there are solutions for each situation, but they imply time and cost to logically determine depending on your project – a personal blog and a multinational company site do not have the same implications.
Failing to deploy such a very elaborate, more costly, but strict and fail-proof solution, you could choose a more basic solution, such as a staging environment with a cloned version of your site (which you maintain similar to your official/live site, there are sync plugins for that) on which you do the upgrade, and on which you then conduct basic tests with a crawler to make sure there are no empty, broken pages, 500 errors etc, – after which if things go well you do the same on your live site.
That question is very relevant and would require more explanations. Your mileage may vary…