Racial categorization consistency?

I find it very bizarre that we have the audacity to call people “White”, or even “black”, yet we do not call people “yellow”, or “red” anymore.