How to allow authors to upload files from the frontend?

I want to allow authors to upload images from the frontend. Problem is that I get “You don’t have permission to attach files to this post.”. If I try to upload from the backend there are no problems.
I basically added the TinyMCE editor in the frontend via the wp_editor() function and I have there the “Add Media” button.
By the way, it works fine for admins.
Is there a quick non-plugin solution for this?
It seems that I would need to make him an admin for this to work which is very bad.