Change Color Skips to Top of Page

I’m customizing the colors I want to show in my CHART. As soon as I click on the color field the view moves to the top of the page. Things still work, it’s just annoying that I have to scroll down to my color fields every time I want to edit.