media library files

This is a general question about file handling in WordPress. Do we consider it to be a feature or a flaw that deleting files from the media library does not remove them from the folder in which they are stored?

I’m wrapping up a custom theme development and, throughout the course of the project, thumbnails were regenerated a couple of times. I was given a new set of images for all of the content and was hoping to remove all of the old files through the media library after uploading the new images. Alas, deleting them permanently does NOT remove them from the server, which would seem to be the whole point of deleting them