Well, after a little bit of playing, I managed to half fix this by changing the posts per page in the Reading pane of the dashboard. I set it to 4 and it fixes things.
My guess is that the is_404 is not set against my updated ‘posts per page’ in my query. Interesting stuff.
Anyhow, I’ve set the Reading pane back to 10 so if anyone wants to see this behaviour in action then visit the links above. Once I make my site live, I’ll have to set it back to 4.