WPSuperCache an IE11 and Edge

Hi all,

Since a while my website is not showing properly on Internet Explorer 11 and Microsoft Edge (Mobile version). My website is rhcounselling.co.uk. I don’t know about my site with older IE version I only have IE11 and Edge(both newest version) to test with. I use wordpress 4.7 but the problem is for at least half a year. In the last month I tried almost everthing to solve the problem but no luck. I also use the Theme: celestrial-lite. I turned all plugin off and on (one by one). I tried different settings in all the plugins. I included the compatible line in my header (X-UA-Compatible is set to IE=edge). I contacted my host … and so on and on but no luck.
Finally, I deleted the cache and diactivated WP SuperCache and my site was working on IE and Edge but is loading very slow now and only 58% in Pagespeed. I activated the plugin again and my site did not work properly on IE11 and Edge. So, I tried different settings in the WPSuperCache plugin but again no luck.
Now, the plugin is deactivated so that people can reach my site but that is not a solution for me.

Is someone out there who may has an idea?

Thanks
Mona