visible light - Why does the sun always have some whiteness?

So the atmosphere scatters light on its way to earth, making the color of our sky. For example, when the sky is blue on a clear, sunny day, the sunlight appears somewhat yellow because the blue light has been scattered away. However, there is something about this that I do not understand. If certain colors are scattered away, why is there 99.5% of the time some white when you look at the sun (e.g. in pictures)? Even at red sunsets with reddish skies, the sun seems to be somewhat white - how can this happen if almost all the original colors/photons have been scattered away and thus the rest should not be able to constitute a whitish color?

Example: