I just upgraded my site’s template. I had the same worries. Talked to an SEO about this. This is what he said,
Make sure the URL structure does not change,
The content, text and images with alt text are more or less same,
The site should load fast, (use a cache plugin),
Check speed and experience metric with PageSpeed Insights
If all these are kept in check the site’s rankings wont fluctuate. We did this and the results were what he promised.