Cone or color opponency makes neurons much more sensitive to color modulation and less sensitive, often quite insensitive, to black/white modulation. A color-opponent neuron is devoted to color at the expense of its responsiveness to black/white. Color opponency is achieved by subtractive interaction of inputs from different cone photoreceptors. In different species, there are different varieties of color opponency, but in humans and other primates, there are two cone-opponent pathways. One receives subtractive input from the middle- and long-wavelength cones and the other subtractive inputs from the short-wavelength (S) cone and some combination of signals from the other two cones. The color-opponent pathways are distinctive anatomically as well as physiologically; the S-cone pathway is phylogenetically the older. While opponent color coding is thought to be efficient from an informational standpoint, we show that it also makes color-opponent neurons specialized for transmitting the surface characteristics of objects. Although the basic principles are well understood, there are substantial gaps in our knowledge as to the generation of opponency in the retina, how it is used in the cortex, and how it became a primate speciality.