So I did some more troubleshooting and it looks like it was a postfix issue. (figured it probably would be).
I just reconfigured it using sudo dpkg-reconfigure postfix and went through setup again and setting it as Internet Site. I am still not fully sure why it was affecting certain emails and not others.
Just a heads up in case someone runs into the same thing.
