To add some clarity: In my widgets, I also tried setting the standard sidebars to see if they would appear, but they didn’t.
Maybe a clue: I tried every location, and I noticed it will only appear when I set it to the header location (instead of e.g. sidebar top) (Of course this is not the location I want the sidebar). I couldn’t find any relevant differences between the ‘sidebar-top.php’ and the ‘sidebar-header.php’ files.