Basically, although all people with normal hearing capacity can hear sounds over a wide range of frequencies (about 60 to 20,000 hertz as I recall), people have widely different listening skills. Yes, listening is a skill. Musicians, for example, undergo a course of formal ear-training that involves learning to identify intervals, harmonies and rhythms. When I say "identify" I mean be able to tell the difference between a major second and a perfect fifth and so on. Be able to recognize major, minor, seventh and diminished chords. Be able to write down rhythmic patterns and distinguish different metrical groupings. All this is essentially about musical literacy, or being able to go back and forth between heard music and written music and convert one to the other. A typical test in ear-training involves sight-singing. You are handed a short, written out melody and have to sing it without reference to an instrument--they usually give you the first note.
People with so-called "perfect pitch" are able to recognize not only all the above, but also know exactly what notes are being sounded. They have an absolute memory of pitch. This can sometimes be a drawback! A string player I know with perfect pitch went through an agonizing time rehearsing for a baroque performance because the ensemble was using the old concert 'A' of 415 hertz as opposed to the modern one of 440. Every 'A' sounded to her like a G#!
So when a trained musician/listener listens to music, they hear something rather different from an ordinary listener who, instead of hearing something in a major key, in 4/4 meter, with lots of modulation, rather hears a fluctuating mass of sound. As a teacher, I train people to develop their listening skills because if you can't hear it, you can't play it. The interesting thing is the relationship between listening skills and the way you enjoy music. Listeners are aware of the expressive qualities of the music they are hearing, whether or not they are aware of any of the details. To pick a simple example, the opening of Richard Strauss' Thus Spake Zarathustra which was used in the film 2001: A Space Odyssey sounds like this:
This kind of large brush musical gesture is pretty easily heard whether you know anything at all about music. Everyone can hear a bold, brassy, stark, rising theme answered with some harmonies. But if you have a trained year you hear this: a very low C pedal, followed by trumpets playing C G C, outlining the boundaries of C but without specifying major or minor. Then the whole orchestra comes in strongly giving us E, which completes the C major harmony, but immediately crashing down to E flat, which is C minor. The next phrase reverses this, going from E flat to E natural. The question is, how much does hearing all this stuff add to your musical enjoyment? Well, some I'm pretty sure! But if we ask how much does it add to your musical understanding, the answer is hugely.
Some music only becomes clear--expressively as well as technically--if you know how to listen to it which answers the question why some famously 'difficult' composers are highly regarded in some circles and scarcely known in others. Have a listen to this piece for an example:
Even highly trained listeners are not going to pick up what is going on there right away! This is a twelve-tone composition by Anton Webern that is full of symmetrical interval palindromes. The piece mirrors itself in a couple of dimensions. What does a naive listener hear? I think that quite a few hearings would be needed to pick up on much of what is going on there, even on the level of expression. Pieces like this challenge our aesthetic perceptions...