Hi bdub02,
Yes, you will need to remove the domain mapping in wp.com.
When you bought the domain on godaddy at some point you had to point the domain to wp.com, right? Now, you need to create a NS record to your godaddy hosting account.
Just go to your godaddy account and find out what is the NS of the hosting account (if you use cPanel you can see it there, something like ns1.yourdomain.com or ns2.domaincontrol.com) you bought, and create a NS record to those from the DNS settings at godaddy domain panel. Basically you are point the domain back to your new hosting account with godaddy, so you can have control of it there.
I suggest you read this article just to get the point:
https://www.wpbeginner.com/wp-tutorials/how-to-properly-move-your-blog-from-wordpress-com-to-wordpress-org/
Cheers.