This is a fairly small tweak to the brush UI.
The brush Gradient option renders the color field invalid. The two are mutually exclusive. The trouble is that when looking at the UI, this is entirely not obvious, because the UI is currently misleading. It makes it seem as if you can enable the Gradient in addition to setting a color, which is not the case.
When enabling the Gradient option, the color just greys out, but with a simple UI change, this can become clearer, and more compact.