[core/media-text] Restricting blocks for the text column

In the Media & Text block, I want to know in my blocks theme, either using a filter or a javascript snippet if it’s possible to restrict what can be picked in the text column.

Above is a screenshot of the Media & Text block – I clicked on the black plus button, but I only want to allow the paragraph and heading – just for this particular block. Is it possible to override in theme.json or via filters?