It is good that this plugin is tested compatibility with advanced WordPress and PHP versions.
It is also good that these tests are automatically done almost every day.
(are they automatically scheduled?)
(can you add criteria to the tests? and back check the criteria on earlier versions?)
Although your procedure is not public, the results of the procedure could be made public, and possibly could be automatically made available and updated.
Perhaps a “current compatibility status” message.
Perhaps more than one method of compatibility assessment, such as;
a) as tested by our systems
b) as experienced by users (no problems yet reported, and no problems currently outstanding)
c) reservations held pending confirmation of safe conditions (e.g. code in versions of WP and PHP may cause an interplay that between them could results in unreliable, uncertain performance).
Do you test a range of combinations of versions of WordPress and PHP?
The results could be presented as a table.
And this might include the information about versions of WP and PHP that are not yet main stream, so users know they will be able to remain up to date with the very latest developments.
Another important factor is ;
“in the event of incompatibility, this plugin will not behave destructively.”
type of assurance.
Note
much of the discussion here, though useful for this plugin, would also be beneficial for other plugins. So what ever gets done here, could be made available for other plugins.
i.e. auto test bed, and results reporting.
Possibly liaise with WP site to show the PHP compatibility with a PHP icon on the right hand side menu , just like the WP compatibility.