Add "blog" before posts in URL and not CPT's

I have made a template and a page for my blog which works fine. Now I would like to preface all posts with that blog page, when I go into the permalinks section and add it in. It is being added to all of my custom post types as well.

Do I need to do it another way? Like a url rewrite?

I would like something as follows for regular posts:
https://www.mysite.com/blog/category/post

and custom post types like this:
https://www.mysite.com/cpt-name/cpt-post

Sorry, if this is an easy solution, but I can’t seem to wrap my head around how to get it working properly.