delete plugin that didn’t fully install

I am trying to install the Woocommerce plugin but it fails. It then prompts me to do a manual install…which I am now trying to do but also fails with the message;

Destination folder already exists. C:/xampp/htdocs/wordpress/wp-content/plugins/woocommerce/
Plugin install failed.

I can see the directory C:\xampp\htdocs\wordpress\wp-content\plugins\woocommerce in My Computer but I cannot delete this through the WordPress dashboard because it has not installed – presumably.
How can I delete this directory?