I
would like to start this post asking my readers to take 3 min to watch this
short TED clip:
Arthur
Benjamin: Teach statistics before calculus!
I
watched this video few days ago and I found it interesting not only because of
the undoubtedly courageous proposal of Prof Arthur Benjamin about teaching statistics
before calculus, but because it made me think (again) on how important is improving
the way I deal with “uncertainty” and probability in my research.
The “invention”
of the scientific method (whose framework was firstly established by René
Descartes in his treatise, Discourse on Method Then, in 1637, and then further
developed by Alhazen, Bacon and John Stuart Mill) is, in my view, one of the
greatest accomplishments we have achieved as human beings. However, to be worth its
power, this “tool” has to be “handled” with care. The scientific method, used
in the process of “expanding” our knowledge of nature, has to be translated
into well designed and then well interpreted experiments. To
support this assumption, I would like to quote the work of Fisher (1935) “The Design of Experiment”,
(a milestone of modern science), which presents the problem of design and
interpretation in scientific experiments:
“WHEN
any scientific conclusion is supposed to be proved on experimental evidence,
critics who still refuse to accept the conclusion are accustomed to take one of
two lines of attack. They may claim that the interpretation of the experiment
is faulty, that the results reported are not in fact those which should have
been expected had the conclusion drawn been justified, or that they might
equally well have arisen had the conclusion drawn been false. Such criticisms
of interpretation are usually treated as falling within the domain of
statistics…
The
other type of criticism to which experimental results are exposed is that the
experiment itself was ill designed, or, of course, badly executed. If we
suppose that the experimenter did what he intended to do, both of these points
come down to the question of the design, or the logical structure of the
experiment…
Now
the essential point is that the two sorts of criticism I have mentioned come
logically to the same thing, although they are usually delivered by different
sorts of people and in very different language. If the design of an experiment
is faulty, any method of interpretation which makes it out to be decisive must
be faulty too. It is true that there are a great many experimental procedures
which are well designed in that they may lead to decisive conclusions, but on
other occasions may fall to do so; in such cases, if decisive conclusions are
in fact drawn when they are unjustified, we may say that the fault is wholly in
the interpretation, not in the design. But the fault of interpretation, even in
these cases, lies in overlooking the characteristic features of the design
which lead to the result being sometimes inconclusive, or conclusive on some
questions but not on all. To understand correctly the one aspect of the problem
is to understand the other. Statistical procedure and experimental design are
only two different aspects of the same whole, and that whole is the logical
requirements of the complete process of adding to natural knowledge by
experimentation."
Now,
let’s put this in practice. Let’s say we have been good (or lucky) enough to
have our experiment within the ones considered, as mentioned by Fisher, being so
well designed to have the potential to lead to decisive conclusions. Now it
comes to interpretation. Faulty interpretations may cause well designed studies
to be faulty in their conclusions.
Experimental
interpretation is based on the so called “Null hypothesis significance testing”.
By applying statistical procedure, we support experimental hypotheses, we make
choices about data and we estimate whether our results have occurred by chance
alone, or if a different explanation can be found, which can be potentially extended
into more general conclusions.
However,
developing a mature and correct use of statistical testing to make
interpretations is one of the most difficult challenges I am facing as a young
scientist.
How
do we improve the power of our interpretation? In my view, by a better
understanding of the “instruments” we have, in other words (and coming back to
the opening video), by a better understanding of statistics. For
these reasons, I would like to bring to your attention these two very interesting
pieces of work, which deal with interpretation and its implications.
Enjoy
the reading!
Misinterpretations
of significance: A problem students share with their teachers.
Haller & Krauss, 2002
Abstract
The
use of significance tests in science has been debated from the invention of
these tests until the present time. Apart from theoretical critiques on their
appropriateness for evaluating scientific hypotheses, significance tests also
receive criticism for inviting misinterpretations. We presented six common
misinterpretations to psychologists who work in German universities and found
out that they are still surprisingly widespread – even among instructors who
teach statistics to psychology students. Although these misinterpretations are
well documented among students, until now there has been little research on
pedagogical methods to remove them. Rather, they are considered “hard facts”
that are impervious to correction. We discuss the roots of these
misinterpretations and propose a pedagogical concept to teach significance
tests, which involves explaining the meaning of statistical significance in an
appropriate way.
On
being sane in insane places
Rosenhan, 1972
Abstract
It
is clear that we cannot distinguish the sane from the insane in psychiatric
hospitals. The hospital itself imposes a special environment in which the meanings
of behavior can easily be misunderstood. The consequences to patients
hospitalized in such an environment-the powerlessness, depersonalization,
segregation, mortification, and self-labeling-seem undoubtedly
countertherapeutic. I do not, even now, understand this problem well enough to
perceive solutions. But two matters seem to have some promise. The first
concerns the proliferation of community mental health facilities, of crisis
intervention centers, of the human potential movement, and of behavior
therapies that, for all of their own problems, tend to avoid psychiatric
labels, to focus on specific problems and behaviors, and to retain the
individual in a relatively non-pejorative environment. Clearly, to the extent
that we refrain from sending the distressed to insane places, our impressions
of them are less likely to be distorted. (The risk of distorted perceptions, it
seems to me, is always present, since we are much more sensitive to an
individual's behaviors and verbalizations than we are to the subtle contextual
stimuli that often promote them. At issue here is a matter of magnitude. And,
as I have shown, the magnitude of distortion is exceedingly high in the extreme
context that is a psychiatric hospital.) The second matter that might prove promising
speaks to the need to increase the sensitivity of mental health workers and
researchers to the Catch 22 position of psychiatric patients. Simply reading
materials in this area will be of help to some such workers and researchers.
For others, directly experiencing the impact of psychiatric hospitalization
will be of enormous use. Clearly, further research into the social psychology
of such total institutions will both facilitate treatment and deepen
understanding. I and the other pseudopatients in the psychiatric setting had
distinctly negative reactions. We do not pretend to describe the subjective
experiences of true patients. Theirs may be different from ours, particularly
with the passage of time and the necessary process of adaptation to one's environment.
But we can and do speak to the relatively more objective indices of treatment
within the hospital. It could be a mistake, and a very unfortunate one, to
consider that what happened to us derived from malice or stupidity on the part
of the staff. Quite the contrary, our overwhelming impression of them was of
people who really cared, who were committed and who were uncommonly
intelligent. Where they failed, as they sometimes did painfully, it would be
more accurate to attribute those failures to the environment in which they,
too, found themselves than to personal callousness. Their perceptions and
behavior were controlled by the situation, rather than being motivated by a
malicious disposition. In a more benign environment, one that was less attached
to global diagnosis, their behaviors and judgments might have been more benign
and effective.
References
Fisher, R. (1935). The design of experiments. Retrieved
from http://psycnet.apa.org/psycinfo/1939-04964-000
Haller, H.,
& Krauss, S. (2002). Misinterpretations of significance: A problem students
share with their teachers. Methods of Psychological Research Online, 7(1),
1-20. Retrieved from
http://www2.uni-jena.de/svw/metheval/lehre/0405-ws/evaluationuebung/haller.pdf
Rosenhan,
D. (1972). On being sane in insane places. Santa Clara Lawyer, 237-256.
Retrieved from http://heinonlinebackup.com/hol-cgi-bin/get_pdf.cgi?handle=hein.journals/saclr13§ion=30
No comments:
Post a Comment