Hide Toggles on Parent & Subcategories

How would I hide the toggles with CSS on all parent & subcategories that are just icons for categories (do not show any products)? I could do it manually – but i have a ton of categories and subcategories – so looking for a global solution for Main and Sub categories.

I just need the toggles to show up on pages that list products only.
thanks!