Should I be using custom post types or terms and taxonomies for this?

So for the past week or so, I’ve been developing a plugin that lets users manage a restaurant menu. I’ve learned a lot about the process of making plugins as far as adding custom post types, creating posts, customizing the admin menu, getting posts, etc. I am still stuck on terms and taxonomies and what exactly they are.

Previously, I developed this menu plugin by making custom post types for menus(breakfast, lunch, dinner, etc), categories(Appetizers, Entrees, etc), and dishes. Every time a dish was added, it would have a post_parent of the category it was under. The categories would have a post_parent of the menu they were under.

Now I am wondering… would it be better to do this by using terms and taxonomies? Or do I have the complete wrong idea about them?