Are there any plugins to edit course archive pages or add menu to them

Existing plugins are unlikely, but with custom coding you could create your own templates to generate course archive pages which also include the content you desire. In some cases action hooks are available so you may not need a full template, rather you’d add callback code to the action hooks. In other cases the developer of the template being used has built in some easy way to override the default template. You might ask through the dedicated support channel of which ever module is responsible for the template what your options are.