I’m assuming WP will do the most sensible thing and ‘attach’ the images to the posts (and not vice versa). Therefore, given an image, I would think that you have to go through all posts…
But if you’re comfortable with PHP, parsing the posts shouldn’t be a problem at all. This probably isn’t something you want to do on every pageload though…