Show all categories and posts in it

<?php
//get all categories then display all posts in each term
$taxonomy = 'category';
$param_type = 'category__in';
$term_args=array(
  'orderby' => 'name',
  'order' => 'ASC'
);
$terms = get_terms($taxonomy,$term_args);
if ($terms) {
  foreach( $terms as $term ) {
    $args=array(
      "$param_type" => array($term->term_id),
      'post_type' => 'post',
      'post_status' => 'publish',
      'posts_per_page' => -1,
      'caller_get_posts'=> 1
      );
    $my_query = null;
    $my_query = new WP_Query($args);
    if( $my_query->have_posts() ) {  ?>
      <div class="category section">
	    <h3><?php echo 'Category '.$term->name;?></h3>
	    <ul>
	    <?php
      while ($my_query->have_posts()) : $my_query->the_post(); ?>
        <li><a href="<?php the_permalink() ?>" rel="bookmark" title="Permanent Link to <?php the_title_attribute(); ?>"><?php the_title(); ?></a></li>
       <?php
      endwhile;
      ?>
      </ul>
      </div>
 <?php
    }
  }
}
wp_reset_query();  // Restore global post data stomped by the_post().
?>

Thanks. How to sort them in that order:

most important category,
less important,
least important ?

I am thinking of using id for that, but how do I change id for existing categories? Or another field needed?

You don’t really control ‘category id’ as WordPress automatically assigns those. If you want to change the order of the categories, then review Function_Reference/get_terms. If you want to change the order of the posts within each category review Function_Reference/query_posts.