Blind Audition Statistics

When I was a graduate performance major I shared a required chamber music concert with a very fine flute player. He later auditioned for principal flute in one of Canada's finest orchestras. That audition was blind, that is, each candidate played behind a screen so their identity was unknown to the auditors. At that time the procedure in Canada was to hold national auditions, open only to Canadians, and then, if no suitable candidate was found, to hold international auditions. After the first round, no-one was chosen though my friend was certainly the leading candidate. Then he went on to win the international auditions as well and became the principal flute in the orchestra. This pretty much shows up the process as being a mere gesture to political correctness as however good the national candidates were, they were going to be rejected and an international audition held anyway.

The principle of blind auditions, which here concealed the fact that the same Canadian flute player participated in both rounds, is widely used in orchestral auditions. It has many justifications. For example, it conceals which candidates are the special favorites of the conductor or administration (or vice versa), it conceals which candidates are members of a visible minority, or which are women and so on. Screens are not used as much in European auditions and some orchestras ask a short list of candidates to play in concerts with the orchestra in order to see how good the fit is.

One of the main stimuli to the use of screens and blind auditions was a survey done twenty years ago, though blind auditions were common long before. Christina Hoff Sommers has just written a very interesting critique of that study in today's Wall Street Journal, Blind Spots in the ‘Blind Audition’ Study: A lauded 2000 article claiming to find sexism in American orchestras looks increasingly spurious. That's probably behind a paywall, so let me excerpt part of it.

It is one of the most famous social-science papers of all time. Carried out in the 1990s, the “blind audition” study attempted to document sexist bias in orchestra hiring. Lionized by Malcolm Gladwell, extolled by Harvard thought leaders, and even cited in a dissent by Justice Ruth Bader Ginsburg, the study showed that when orchestras auditioned musicians “blindly,” behind a screen, women’s success rates soared. Or did they? 

They collected four decades of data from eight leading American orchestras. But the data were inconclusive: The paper includes multiple warnings about small sample sizes, contradictory results and failures to pass standard tests of statistical significance. But few readers seem to have noticed. What caught everyone’s attention was a big claim in the final paragraph: “We find that the screen increases—by 50 percent—the probability that a woman will be advanced from certain preliminary rounds and increases by severalfold the likelihood that a woman will be selected in the final round.”

For all the details, you need to read the whole article. But the results were finally put into question by a new study:

In 2017 a team of behavioral economists in the Australian government published the results of a large, randomized controlled study entitled “Going Blind to See More Clearly.” It was directly inspired by the blind-audition study. Iris Bohnet, a Harvard Kennedy School dean and Goldin-Rouse enthusiast, served as an adviser.

For the study, more than 2,000 managers in the Australian Public Service were asked to select recruits from randomly assigned résumés—some disguising the applicant’s sex, others not. The research team fully expected to find far more female candidates shortlisted when sex was disguised. But, as the stunned team leader told the local media: “We found the opposite, that de-identifying candidates reduced the likelihood of women being selected for the shortlist.” It turned out that many senior managers, aware that sexist assumptions had once kept women out of upper-level positions, already practiced a mild form of affirmative action.

I think that the lesson to be learned here is that all studies that seem to deliver results that support or favor current progressive ideas should be regarded with considerable skepticism. Does this apply to the "climate crisis" as well? You bet.

And now, chosen completely randomly, a major Canadian orchestra playing the Prélude à l'après-midi d'un faune by Claude Debussy: