create selectable custom gallery shortcode

Hello,

I build a custom gallery shortcode function::

remove_shortcode('gallery', 'gallery_shortcode');
add_shortcode('gallery', 'gallery_shortcode_custom');

function gallery_shortcode_custom($attr) {
...
}

this function overwrites the complete gallery structure which is fine but it should be a selectable option within the gallery menu where you select the order/columns in the post editor.

see gallery settings here:
https://multimedia.journalism.berkeley.edu/media/upload/tutorials/wordpress-photo-galleries/wp-gallery.png

Can anybody help me with adding this into the option popup?

* how can I add a select button at the gallery settings?
* how do I modify the function that it is only enabled when the option is selected?

Thanks a lot for your input.