Yes, the mixing is happening in the eyes & brain; no, an RGB mix of yellow isn't the same as a pure yellow frequency; but our eyes will see it as the same.
The eyes have 3 (or 2, if you're colour-blind) types of colour sensors, each of which responds with a different signal profile - each peaks at a particular frequency, and trails off for frequencies that differ from that. The brain merges the signals from those 3 (or 2) different sensors, to make sense of the colour signals to create a single colour signal, and it can't tell whether that was a balanced combination of red and green, or a pure yellow frequency.
See also this answer to a previous, related question.
That explains most colours we see. Except for when we see a combination of red and blue, with no signals in between. There isn't a colour in the spectrum for that - the colours in between red and blue all feature higher signals in the middle, around green. To have signals from red and blue but not green, doesn't map to the spectrum. And our brain won't show a combination of two or more colours for a single point, it always maps a single point to a single colour.
So our brain creates a new colour, not on the spectrum, for a combination of red and blue. Hence, purple pigments aren't real, in that sense - purple is the brain's interpolation of red + blue + no green. Purple is just a pigment of our imagination.
Best Answer
I noticed this too. My conclusion was that it's not because of the polarizers in the screen but the viewing angle. I am not sure about LCD screens but LEDs are focused in a direction and if you view it from a different direction, that particular color could now have lower intensity as compared to other colors in the vicinity and you'll different color. Or some internal component could be blocking the view of those particular crystals if viewed from a particular direction.
When you view a screen in a window, you are not viewing it from the front as intended but from an angle. This creates the color distortion.