PWA is not compatible with Cloudflare Rocket Loader.

Can you share your site URL so I can see the issue?

OK, my site is at here https://ryo.to/Edvrli with wp-rocket and cloudflare rocket loader and now i turned on the Settings > Reading > Offline browsing for you to debug. If finished do let me know, because I need to cleared the caches turn it off as wp rocket and cloudflare will cached it.

The wp.serviceworker can has the .js extension caused otherwise it will being cached like normal html page…

Actually it has problem with both plugin wp rocket and cloudflare rocket loader.
Either i need to disable the cloudflare rocket loader or the second way is to set /wp.serviceworker in the Never Cache URL(s) which inside WP rocket Advanced Rules settings in order to let PWA work.

I choose the second way to fix the issue by putting /wp.serviceworker in the Never Cache URL(s) box which inside WP rocket Advanced Rules settings because as previous version frontend service worker /?wp_service_worker=1 which will not be cached by wp rocket default as it has query parameter..

The wp.serviceworker can has the .js extension caused otherwise it will being cached like normal html page…

Unfortunately, this won’t work since many server configurations think that JS file extensions mean that WordPress shouldn’t be loaded at all. They’ll try instead to go directly to the filesystem, which here would fail since the file doesn’t exist.

We went through a pretty long journey to land at /wp.serviceworker. See https://github.com/GoogleChromeLabs/pwa-wp/issues/287#issuecomment-642838589

The wp.serviceworker can has the .js extension caused otherwise it will being cached like normal html page…

That seems like a bug with your caching layers, since /wp.serviceworker returns with these HTTP response headers:

cache-control: no-cache
content-type: text/javascript; charset=utf-8

This is because if in WP Rocket caching mode, it will cache anything except the ?query parameter or users defined NEVER CACHE URL, I had defined it /wp.serviceworker in NEVER CACHE URL otherwise it will cache the /wp.serviceworker just like the directory or post page which will caused problem that I had stated above.

How does it handle non-HTML responses from WordPress like for feeds and the REST API? It should avoid caching those as HTML since they return XML and JSON respectively, right?

In any case, the solution you came up with seems like it the right one. Going forward WP Rocket should include this in the list of URLs that are excluded, and/or avoid caching anything that returns with a non-HTML content type.