Is there a way to retrieve just the image source of the thumbnail?

By using the dedicated function:

$thumbnail_id = get_term_thumbnail_id( $term_taxonomy_id );
$thumbnail_src = wp_get_attachment_image_src( $thumbnail_id );
$thumbnail_src = $thumbnail_src ? reset( $thumbnail_src ) : false;

After 5 months, I guess I can mark this topic as resolved.

Oh wow, yes, this worked!

Do you know how we could use this as a background image?

I’m thinking something along the lines of:

<div class="category" style="background-image: url(' . $thumbnail_src' . .')">' . get_term_thumbnail( $term->term_taxonomy_id, $attr = '' ) . '</div>

But I can’t get it working.