If you follow the link from the word "problematic" in the above paragraph you will be taken to another interesting paper on the concept of "statistically significant" and why it often means very little.
Now let me offer a little free-wheeling philosophical speculation. Science, in its hunt for facts, depends on two things: the ability to turn witnessed phenomena into numerical statements, and the ability to understand these numerical statements as being an index of causality. In other words, we have to be able to measure a phenomenon and record it in numbers. Then we have to be able to analyze these numbers in terms of causality.
The first idea explains why scientific study of music so often falls short or tells us things that we already know with no new insights. The aesthetic experience of music is not measurable with numbers and every time we see a study that attempts this, say, by asking questions of a group of listeners, we see the inadequacies. They tend to start by designing a question that can be answered by means of a psychological survey--this supposedly captures an aesthetic experience in numbers. I talked about this a bit in this post: A Theory of Scientism. I quote Roger Scruton as follows:
This is the sure sign of scientism — that the science precedes the question, and is used to redefine it as a question that the science can solve.The second idea is the notion of causality understood as statistically probable, which has its own inherent problems, largely with false positives. Whenever I hear about a study or diagnosis that results in a false positive I think that there is a real, if seldom acknowledged, problem with the understanding of causality. As David Colquhoun writes in this paper:
The aim of science is to establish facts, as accurately as possible. It is therefore crucially important to determine whether an observed phenomenon is real, or whether it’s the result of pure chance. If you declare that you’ve discovered something when in fact it’s just random, that’s called a false discovery or a false positive. And false positives are alarmingly common in some areas of medical science.Aristotle had a quite different notion of causality and while it is not applicable to the problems of medical research, it is certainly a refreshingly different concept of what causality is. For Aristotle you cannot understand a phenomenon until you know why it occurs, in other words you look for an explanation. He looked largely at things that result from human action and distinguished four kinds of cause. The Stanford Encyclopedia of Philosophy describes them as follows:
In Physics II 3 and Metaphysics V 2, Aristotle offers his general account of the four causes. This account is general in the sense that it applies to everything that requires an explanation, including artistic production and human action. Here Aristotle recognizes four types of things that can be given in answer to a why-question:
The material cause: “that out of which”, e.g., the bronze of a statue.
The formal cause: “the form”, “the account of what-it-is-to-be”, e.g., the shape of a statue.
The efficient cause: “the primary source of the change or rest”, e.g., the artisan, the art of bronze-casting the statue, the man who gives advice, the father of the child.
The final cause: “the end, that for the sake of which a thing is done”, e.g., health is the end of walking, losing weight, purging, drugs, and surgical tools.I wonder if any of this might be of use to modern science. Certainly it encourages one to look deeper than a merely statistical result. If you give a new drug to 100 people and 95 of them feel better, this is called "statistically significant" but we certainly lack an explanation of why or how the drug worked. It is possible that the 95 people felt better for some other reason than the drug. In other words, statistics, when used properly (as often they are not) offers no explanation other than a probability. Unfortunately, with the kinds of phenomena that are usually observed in modern science, Aristotle is of little help and we turn to the statisticians like Thomas Bayes.
For our envoi, let's listen to some music by a near-contemporary of Bayes, C. P. E. Bach. This is his Symphony No. 1 in D major conducted by Ton Koopman: