First, which Theme are you using? If this Theme isn’t working, try selecting another.
Some Themes come with pre-set categories. If you have no posts within that category, and the category is visible, then you will get a page not found error. By default, empty categories should not show, but sometimes they are “forced” by the theme author. By switching to the default, classic, or any other theme, this might fix your problem.
Or make sure you have at least one post in each category visible on the sidebar.
As explained to you, on the Write Post panel in the Administration Panels, you choose the category for the post to be in from the list on the right side of the Write Post panel where it lists the categories.
If you still get errors when you click on the categories, provide us with a link to the site and we can check for errors.
It also sounds like you are new to WordPress so check out these sites for more information:
Introduction to Blogging
First Steps with WordPress
WordPress Lessons
New to WordPress – Where to Start
Finding Your CSS Styles
WordPress Blog Design and Layout
Using WordPress Themes
WordPress CSS Information and Techniques
Creating individual Pages
Stepping Into Template Tags