Recent Posts Offset for Home Page Only

I’d like to offset the Recent Posts (in the sidebar) by 6 on the home page, and I’d like the Recent Posts to show the most recent posts on all other pages and posts. The code below almost works, but if I click “Next” to go to Page 2 (e.g., example.com/page/2/) it still offsets the posts by 6, instead of displaying the most recent. If you go to any post, it works, just /page/2/ or /page/3/ it does not. I’ve also tried is_home () — it does the same thing as the code below.

<?php
if ( is_front_page() ) {
    query_posts('posts_per_page=5&offset=6');
} else {
    query_posts('posts_per_page=5&offset=0');
}
?>
<?php while (have_posts()) : the_post(); ?>
<dl>
 <dt class="thumb"> <a href="<?php echo get_permalink(); ?>">
 <?php the_post_thumbnail(); ?></a>
 </dt>

<dd class="title"><a href="<?php echo get_permalink(); ?>"><?php the_title(); ?></a></dd>
<dd class="readmore"><a href="<?php echo get_permalink(); ?>">READ</a></dd>
</dl>

<?php endwhile; ?>
<?php wp_reset_query(); ?>

Thank you!