I was able to do what I wanted to do.
Instead of using the ‘Restore’ backup from the backup tab I used the Migrate / Clone tab. Towards the bottom there are migration choices. I used ‘Restore an existing backup set onto this site’.
For each of the next steps there is an option to choose the specific things you want to move. It’s tiny and easy to miss but there.
I moved the plugins first, deselecting the ones I didn’t want. Then the posts, then the uploads, then the database. For I didn’t deselect any posts or uploads, but many of the tables on the old site were junk created by old plugins and most of them I didn’t move.
These all succeeded.
WordPress/Jetpack support told me that doing this would break the site because the database would be overwritten, and it did. I have a paid plan though and they assured me that they would be able to fix it. I’m still waiting on that about 14 hours later, but I expect it will happen.
Then I can see if it actually worked.