Turns out my site took much longer to load the icons. Instead of seconds it took hours to update which is weird (maybe slow internet? Who knows).
For your issues I recommend checking this response from another form. I believe this will help you answer your specific issue:
Hi,
It’s a CORS issue
https://docs.presscustomizr.com/article/233-social-icons-showing-up-as-squares
Your WordPress Address (URL) and Site Address (URL) has to be identical or browser will block the font script and you will see squares.
Thank you
