Dec 21, 2004

the eternal discussion... sciences versus social sciences

Everyone in social sciences at one point (or many :-) comes across the kind of discussion that originated in Lilia's post concerning reliability and validity. Every time this comes up i always turn to a story told by Jerome Kirk and Mark Miller on their great and very sintetic book: Reliability and validity in qualitative research. The story goes like this:

"A standart physical example of reliability and validity involves the use of thermometers to measure temperature. A thermometer that shows the same reading of 82 degrees each time it is plunged into boiling water gives a reliable measurement. A second thermometer might give readings over a series of measurements that vary from around 100 degrees. The second thermometer would be unreliable but relatively valid, whereas the first would be invalid but perfectly reliable" (p.19)

The question in here is to interpret data readings and relating to previous knowledge. In this case theory: one as to know that water boils at 100º and that varies under different ambient pressures, plus many other things that affect and condition interpretation (find out reading on p.21).

What we are dealing in social sciences in not an utopic laboratory experience were we can control the variables we want to account for, regardless of the context they inhabit. Because of that, the amount of variables that we are dealing with, in real life contexts, can not be desagregated for a laboratory test. They have to be observed in their context by «us», the observers. At the same time we have to acknowledge that we influence what we are observing and that we only observe what our mental grids allows us to see.

That's why it is so important to begin by designing the case we are studying. The case study protocol becames more than an instrument, it guides the researcher during all the work to be conducted, how it is going to be carried, what is going to be observed and how, norms and procedures during research, so that reliability becames higher (Robert Yin, 1994, Case Study Research: design and Methods). On pp. 63-74 you will find guidance for the design:

- overview of the case study project

- field procedures

- case study questions

- guide for the case study report

To have a research notebook (or blog) helps remind us of possible bias one might have regarding: what's being observed, under what conditions, other things we didn't previously considered, something that went wrong that originated new prepositions, and so on... The «ah ah» factor that´s been moving science forward and the process of discovery.

1 comment: