If the plugin has no documentation and does not support extending it, I’m afraid that there’s no easy option to keep things easily maintainable.
You can always copy ‘parent’ plugin code and create ‘child’ plugin and modify things in there, however that’ll require some manual work once you want to update ‘parent’ code.
There’s also another option, and that’s to create a wrapper for the plugin in ‘theme’ directory. As an example, if plugin contains function called XX() and you want to modifying it’s behaviour, you’d create a wrapper function wrapperXX(), and store it somewhere in your activated theme folder. Inside of wrapperXX() you’d call XX() and do any modifications on it you need. Also, once you update plugin, you won’t have to modify anything (most likely) in theme’s folder. This approach however is not usable in all cases.
Nonetheless, as you’re probably aware, there’s no right approach for this, if you’re stuck with a plugin that is not extendable.
