Problem is when re-generating other sizes, as SVG is treated as an image. This can be solved by simply checking whether the file is svg or not. Line 353 file: class-easy-media-replace-admin.php.
if ($size['mime-type'] != 'image/svg+xml') {
unlink(dirname($old_file_path) . '/' . $size['file']);
}
