Sorry, are you talking about server memory or browser memory? Your original post says your server memory shoots up to 400MB – if that was the case, your site would go down, because shared hosting sites don’t give you more than 128MB. Now it appears like you are saying that your browser memory shoots up.
I took the shortcode you provided and pasted it into the demo page. As you can see, the page is loading up just fine without any issues. On my machine I have the demo page up on Firefox (Firefox is my primary browser and has 77 tabs open, with a total memory usage of 980MB – Photonic demo pages are open on 8 tabs; this is very normal memory usage), Chrome (84MB with one tab) and IE9 (64MB). So I am not seeing any memory issues with the plugin on my site with any browser.
If you can provide your URL I can take a look and see how much memory it takes up at my end.