Another installment:
Akismet or any other plugins are not at fault.
After a complete clean install of WordPress, including a new database and initially after setup everything working fine. The same problem with the Add media button came back after logging into the account from another computer.
Additionally the problem seems to get worse now, while replying to comments just worked a minute ago, after logging out and back in replying to comments does not seem to work any more either.
Can this be 1. a virus, 2. a problem with the hosting provider?