Problem solved – in an unexpected way.
What I thought was I would try to deactivate and then activate the plugin just to check if something would be “reset” in some way – you know, the classical “Have you tried turning it off and on again” solution.
When the plugin was deactivated, I tried to upload an svg just to observe what kind of error message that would give me, and then it surprisingly turned out to work!
Why it works, I have no idea. Maybe WordPress has included svg support as standard recently? For now however, we don’t care too much – it finally works again.