Text widgets adding characters

Almost every time I go into the widgets section and change anything .. I look at my site and see that there is suddenly random characters on the page (like an & symbol) or the code inside the text widgets have changed (like the quotes will turn into < symbol) and something doesn’t work anymore like the email to subscribe .

It’s gotten to the point I’m scared to go in there as it will take me 20 minutes to fix everything…. Am I doing something wrong or should I be adding something so this doesn’t happen?

Thanks in advance and on a side note, being very new to all this… these forums have been an absolute savior to me… I appreciate everyone that puts time in to help…