Coulours are different frequencies of light; and light is the visible part of the electro-magnetic radiation that we receive from very hot sources, like the Sun or light bulbs. Although all radiation is fundamentally similar, different frequencies interact with matter in quite different ways, for example: radio waves can pass through matter relatively freely, whereas [higher frequency] radioactivity damages molecules and kills life.
What makes light radiation special is that it interacts strongly with the different atoms of matter without destroying them. Just like radios can tune-in to radio waves, atoms resonate to specific frequencies of light. This is especially true of carbon-based molecules; the prime example is the conversion of light energy into edible energy by the green chlorophyll in plants.
We see "colour" because cells in our eyes are sensitive to visible radiation. Different types of cells respond to different light frequencies and our brain interprets the relative activation of these groups as the different colours of the rainbow. Light which contains all colours - like sunlight - is seen as White. [Reference]
Note that paint does not CREATE colour; rather, it works as a filter by REMOVING the some frequencies from the light it receives. For example, a red patch on a painting looks Red because the pigment in the paint absorbs the Green part of the ambient light; and the light reflected from the patch contains Red as well as some neighbouring colours. This mixture is perceived as Red. The important fact here is that adding more paint does not add more colour, it just reduces the brightness of the mix.
This explains the downside of creating all colours by mixing from the primaries. Each pigment removes part of the ambient light spectrum. The more pigments are used the more light disappears. This means that a Green mixed from 2 pigments (Yellow and Blue) will have lower intensity than pure Green paint with a single pigment.
Because each pigment used in a paint reduces the intensity of colour, the best quality paints use a single pigment. Cheaper paints use mixes of pigments to imitate the colour of traditional paints. Generally those mixes include the term HUE in the name, like: Cadmium Red Hue.
The wide variety of sources means that pigments can differ quite widely in their properties like cost, transparency, permanence, ability to mix with others, etc... Inclusion of a given paint in a set may depend on attributes other than just colour. A typical example is Cadmium Red which is widely used in oil painting but problematic in watercolour because of its opacity.
Phthalo Blue & Green: These modern pigments are very intense and tend to overwhelm any other colour they are mixed with; but, with experience, they are quite useful. [Advice from Steve Mitchell]
Another example is the pair Ultramarine Blue and its complementary, Burnt Sienna (Orange). These colours combine perfectly to produce a whole range of warm to cool Greys. Sometimes called the "dynamic Duo," they are usually the most used pigments in any palette.
Other colour Models
The Colour Wheel is a useful colour model which explains colour mixing starting from 3 basic pigments (Red, Yellow and Blue). Because paint works by removing parts of the illuminating light, it is called a "Subtractive Model"; other models which specify colour as the sum of component light sources are called "Additive Models". Note that each model uses different colours as their fundamental primaries.
Colour printing is also a subtractive model; it uses 4 colours of ink. Three are specific shades of the Red, Yellow and Blue of the Colour Wheel. The fourth is plain Black. It's called the CMYK Model for the colours used: Cyan (a shade of Blue), Magenta (type of Red), Yellow and blacK.
Digital colour for TV and computer screens is an additive Model. Each colour is specified (and generated) by the addition of Red, Blue and Green LIGHT from 3 different sources. Digital colour has been made possible by modern technology which allows each PIXEL to have 3 microscopic LEDs producing the required colours. Note that one primary is different in this model: Green being used instead of Yellow.
Human Model: The digital model is closer to the way we see colour [Reference]. Each eye contains about 6 million colour-sensitive cone cells; about 2/3 of which respond to RED, 1/3 to GREEN and only 2% to Blue. The interpretation of various level of stimulation as colours is left to the brain.