Slow Redirect from non-www to www url

Hi there. I’m hoping somebody can help me understand and hopefully fix my slow homepage redirect.

My site is running rather slowly in general. I’m comparing speed tests that I did this evening. The non-www url of my site is taking 5.53s to load, whereas the www url is taking 3.73s to load. The difference, of course, is the redirect.

My question, though, is should a redirect take almost two seconds? I have my url set in WordPress as the www version, and I have the www url set as the ‘preferred’ url in my Webmaster tools account for both the www property (http and https) and the non-www property (http and https). I am using the Yoast SEO plugin, but don’t see where to set the canonical url…so I don’t know if that may be making a difference.

Has anybody else experienced a very slow load time between the www and non-www versions of their site, and if so…how were you able to shave time off of the redirect?

Any info or pointers you can give me would be much appreciated!
Jen