Can't change certain tags

I can add tags. I can delete tags. But if I delete a tag titled “P&G” and add a tag titled “PG”, WP shoves that ampersand back in there.

Same with changing letters from lowercase to uppercase. WP changes it back to lower case. Any idea?

Thanks in advance.