Gutenberg Gallery Block: Image scaling instead of smaller image (and giant X)

Hi.

I’m trying to get a website away from a page builder plugin and to Gutenberg. No problem for the text part, but I’m not sure what’s not working right with the Gutenberg gallery blocks.
Wordpress seems to be mostly reliable when it comes to only serving the image size that’s needed instead of scaling down the often enormous original version of an image.
But with the Gutenberg gallery, all the images, although they’re hardly bigger than a thumbnail inside the post, seem to be loaded in original size and then being scaled down by css/html to fit the space. It’s very gorgeous, but a nightmare when it comes to the loading time of this page that consists mostly of gallery posts with large (at least 20-40pics each time) image galleries.

Am I doing something wrong? The gallery is set to link each thumbnail (or rather thumbnailed original) to the file itself. When one clicks on the “thumbnail” a bigger version of the image (again scaled down original size) opens in a light box view. Screenshot of light box view with irritating giant X-Close-Button.
Unfortunately I cannot link to the page right now.

With the gallery serving to large images although there are plenty of generated thumbnails, the close-button being way too large on desktop I am not sure whether there’s something fundamentally wrong going on or me missing a very important step somewhere.