Older Post Button Broken

I suspect it might have something to do with the fact that you are using it in a widget. Does the “Older Posts” link work when you put the shortcode on a page or post?

Is there any way to simply remove it? I have not modified any of the plugin’s code within my website.

What particular shortcode would I need to put in to test your theory?

You can remove the pagination by adding the below code in your theme’s functions.php

remove_filter( 'wprss_pagination', 'wprss_pagination_links' );