Solved – the most accurate way of determining an object’s color

Question

I have written a computer program that can detect coins in a static image (.jpeg, .png, etc.) using some standard techniques for computer vision (Gaussian Blur, thresholding, Hough-Transform etc.). Using the ratios of the coins picked up from a given image, I can establish with good certainty which coins are which. However, I wish to add to my confidence levels and also determine if a coin that I deduce to be of type-A (from radius ratios) is also of the correct colo[u]r. The problem is that for British coins et al. (copper, silver, gold), the respective colors (esp. of copper to gold) are very similar.

I have a routine that extracts the mean color of a given coin in terms of the RedGreenBlue (RGB) 'color-space' and routines to convert this color into HueSaturationBrightness (HSB or HSV) 'color-space'.

RGB is not very nice to work with in attempting to differentiating between the three coin colors (see attached [basic] image for an example). I have the following ranges and typical values for the colours of the different coin types:

Note: the typical value here is one selected using a 'pixel-wise' mean of a real image.

**Copper RGB/HSB:** typicalRGB = (153, 117, 89)/(26, 0.42, 0.60).

**Silver RGB/HSB:** typicalRGB = (174, 176, 180)/(220, 0.03, 0.71).

**Gold RGB/HSB:** typicalRGB = (220, 205, 160)/(45, 0.27, 0.86) 

I first tried to use the 'Euclidian distance' between a given mean coin color (using RGB) and the typical values for each coin type given above treating the RGB values as a vector; for copper we would have:

$$D_{copper} = \sqrt((R_{type} – R_{copper})^{2} + (G_{type} – G_{copper})^{2} + (B_{type} – B_{copper})^{2})$$

where the smallest value of the difference ($D$) would tell us which type the given coin is most likely to be. This method has shown itself to be very inaccurate.

I have also tried just comparing the hue of the coins with the typical values of the types provided above. Although theoretically this provides a much better 'color-space' to deal with varying brightness and saturation levels of the images, it too was not accurate enough.

Question: What is the best method to determine a coins type based on color (from a static image)?

Thanks very much for your time.

Edit 1

Note: I have tried all of the ideas discussed below and have achieved next to nothing. Variance in lighting conditions (even within the same image) make this problem very tough and should be taken into consideration.

Edit 2 (Summery of Outcome)

Thank you for your answers. Further research of my own (including your answers and comments) has highlighted just how tough this problem is to deal with in the generic case of arbitrary lighting, arbitrary camera (mobile device), fluctuation in coin colour (even for same species/type) etc. I first looked at skin colour recognition (a very active field of research) as a starting point and there are still numerous problems even with the recognition of skin colour for Caucasians alone (see this paper for a review of the current techniques), and the fact that this problem contains three distinct colour objects all of which can have continuous and varying chromacities make this topic of computer vision a very hard one to classify and deal with accordingly (in fact you could do a good Ph.D. on it!).

I looked into the Gamut Constraint Method from the very helpful post by D.W. below. This was at first sight very promising as a pre-processing step to transform the image and the separate coin objects to colours that are independent of lighting conditions. However, even this technique does not work perfectly (and involves a library of images/histograms for mappings – which I don’t want to get into) and neither does the much more complex Neural Network Architecture methodologies. In fact this paper states in the abstract that:

"current machine colour constancy algorithms are not good enough for colour-based 
 object recognition.".

That is not to say that there aren’t much more up-to-date papers on this subject out there, but I can't find them and it does not seem to be a very active research area at this time.

The answer by AVB was also helpful and I have looked into LAB* briefly.

"The nonlinear relations for L*, a*, and b* are intended to mimic the nonlinear
response of the eye. Furthermore, uniform changes of components in the L*a*b* colour
space aim to correspond to uniform changes in perceived colour, so the relative 
perceptual differences between any two colours in L*a*b* can be approximated by 
treating each colour as a point in a three dimensional space."

From what I have read, the transformation to this colour space for my device dependent images will be tricky – but I will look into this in detail (with a view to some sort of implementation) when I have a bit more time.

I am not holding my breath for a concrete solution to this problem and after the attempt with LAB* I shall be neglecting coin colour and looking to sure-up my current geometric detection algorithms (accurate Elliptic Hough Transform etc.).

Thanks you all. And as a end note to this question, here is the same image with a new geometric detection algorithm, which has no colour recognition:

Answer 1

Two things, for starters.

One, definitively do not work in RGB. Your default should be Lab (aka CIE L*a*b*) colorspace. Discard L. From your image it looks like the a coordinate gives you the most information, but you probably should do a principal component analysis on a and b and work along the first (most important) component, just to keep things simple. If this does not work, you can try switching to a 2D model.

Just to get a feeling for it, in a the three yellowish coins have STDs below 6, and means of 137 ("gold"), 154, and 162 -- should be distinguishable.

Second, the lighting issue. Here you'll have to carefully define your problem. If you want to distinguish close colors under any lighting and in any context -- you can't, not like this, anyway. If you are only worried about local variations in brightness, Lab will mostly take care of this. If you want to be able to work both under daylight and incandescent light, can you ensure uniform white background, like in your example image? Generally, what are your lighting conditions?

Also, your image was taken with a fairly cheap camera, by the looks of it. It probably has some sort of automatic white balance feature, which messes up the colors pretty bad -- turn it off if you can. It also looks like the image either was coded in YCbCr at some point (happens a lot if it's a video camera) or in a similar variant of JPG; the color information is severely undersampled. In your case it might actually be good -- it means the camera has done some denoising for you in the color channels. On the other hand, it probably means that at some point the color information was also quantized stronger than brightness -- that's not so good. The main thing here is -- camera matters, and what you do should depend on the camera you are going to use.

If anything here does not make sense -- leave a comment.