This thing is starting to use ridiculous amounts of memory. It used to run under PHP with a 32M limit, then I had to eventually up it to 48, then 64M. Now with 1.9.6 I get errors with 64M and had to up it to 72M.
Considering the history of needing to increase the PHP memory limit to run the plugin without errors, I am not sure there is a whole lot you can do to improve performance.
Its a great plugin with quirks that have been going on for years and increasing memory use for little return in features/functionality. For example what did 1.9.6 give me over 1.9.5 that required more memory? /boggle They don’t do a great job of updating their changelog either.