Gutenberg: set allowedBlocks for the core/group block

I would like to restrict the blocks that are available inside a group block, when I create a template for my innerBlocks.
I have tried the below, but this does not seem to work. Also the ALLOWE_BLOCKS does not work inside the group blocks either. All blocks are available to me inside the group block.

	const ALLOWED_BLOCKS = ['core/group', 'core/image', 'core/paragraph', 'core/heading'];
	const MY_TEMPLATE = [
		['core/group', { lock: { 'move': true, 'remove': false }, allowedBlocks: ['core/heading'] }],
	];

	return (
		<div {...useBlockProps()}>
			<InnerBlocks
				allowedBlocks={ALLOWED_BLOCKS}
				template={MY_TEMPLATE}
			/>
		</div>
	);

Is there a way to do this? If so, how can I accomplish this?