You will not be able to do this if posts are assigned more than one term. How would you know which one to use for the sort?
If your posts are assigned only a single term, it may be possible to do this using filters to modify the query. Here is an article that shows example filters: https://wordpress.mcdspot.com/2010/05/30/filters-to-modify-a-query/
You would set the global variables to the clauses necessary to change the query before issuing the call to WP_Query, and reset them to blank after the call.