Excluding posts that are in two categories

Now before anyone jumps on me for posting a question that has been asked a million times in the forum, please bear with me.

I am trying to display posts using the following logic:

I have four categories: Apples, Bananas, Oranges and Monkeys

I want to display 1 post from each: Apples(cat=1), Bananas(cat=2) and Oranges(cat=3), but NOT if any of them are also in the Monkeys(cat=4) category.

Here is the code I’m using to display 1 post from each (but also displays if in the Monkeys category:

<?php $display_categories = array(1,2,3); ?>
<?php foreach ($display_categories as $category) { ?>
  <div>
    <?php query_posts("showposts=1&cat=$category");
    $wp_query->is_category = false;
    $wp_query->is_archive = false;
    $wp_query->is_home = true;
    ?>
    <h3><a href="<?php echo get_category_link($category);?>"><?php single_cat_title(); ?></a></h3>
    <?php while (have_posts()) : the_post(); ?>
      <h2 class="title"><a href="<?php the_permalink() ?>" rel="bookmark"><?php the_title(); ?></a></h2>
      <p><?php the_content_rss('', TRUE, '', 50); ?></p>
    <?php endwhile; ?>
  </div>
<?php } ?>

I understand that to exclude categories I can just add the cat id as a negative number to the initial array. However, that excludes all posts in the Monkeys category, but because the post is also in the “included” category, it gets displayed.

Anyone have any suggestions