k-means is usually not a good way to do color segmentation. It may be okay IF you are 100% certain that there are exactly k color clusters in the image. I attach a demo of using it in RGB color space.
Like you said, because of the wrap around nature of hue in the red, it's not good if there are red/purple colors in the image. To get around that you have to do the clustering in LAB color space. There is a function rgb2lab that you can use. Think of LAB like a cartesian coordinate system whereas HSV is a cylindrical coordinate system. Thus there will be no wrap around problem with LAB color space.
Not sure what you mean about the low saturation pixels. If you want, you can compute chroma = sqrt(A.^2 + b.^2) (which is like saturation) and say that chroma values below some value are to be considered as one "neutral colored" class. Then remove those pixels from the pixels you send into kmeans() so that they will not be classified.
Best Answer