There is no reason for this plugin not to work on multisite. It can be activated either network wide or site by site, whatever you prefer.
Network wide activation means it will be active on every site (obviously) in which case you will have to be aware of the fact that this plugin adds every css / js resource it can find to the push header list. Including any browser specific (mostly IE 8/9) stylesheet or script that a theme or plugin may include on any site in the network.
For example, the default Twenty Seventeen theme includes a html5.js and a ie8.css which, when HTTP/2 Server Push is activated, are pushed to ALL browsers. In fact, they are pushed ONLY to browsers that do not need them because IE8 and 9 do not even support HTTP2…