Understand the concern better now. Large files are a good argument.
However, it somewhat contradicts the intent of this plugin. The block outputs a title, a description text, the file size and an icon for each file. With an externally located file, none of this is available within WordPress with which the plugin could then work. Yes, you could e.g. enter the title manually – but then you don’t need the plugin, because you can simply enter it with a list block (enter title & link title with URL + additional file info).
On the ToDo list for the plugin is still the assignment of individual file titles within the list. Maybe I can think of something in the context of this implementation to fulfill your wish as well. But I don’t see such an addition in time.