Yes, I was suspecting it to be a caching problem, however I did not use a caching plugin. Thought perhaps WP 4.1 introduced some sort of caching, that the plugin wasn’t aware of since the problems seemed to start after the update.
Thanks for the reply. I will seek the solution elsewhere.