I noticed that the “Use built-in plugin templates” option makes the audio player appear in the post, but of course the page that loads isn’t themed like the rest of the site.
EDIT: So I got it to show up by creating a single-podcast.php for the theme I’m using, and copying some code over from single-podcast.php supplied by the plugin.