"A standart physical example of reliability and validity involves the use of thermometers to measure temperature. A thermometer that shows the same reading of 82 degrees each time it is plunged into boiling water gives a reliable measurement. A second thermometer might give readings over a series of measurements that vary from around 100 degrees. The second thermometer would be unreliable but relatively valid, whereas the first would be invalid but perfectly reliable" (p.19)
What we are dealing in social sciences in not an utopic laboratory experience were we can control the variables we want to account for, regardless of the context they inhabit. Because of that, the amount of variables that we are dealing with, in real life contexts, can not be desagregated for a laboratory test. They have to be observed in their context by «us», the observers. At the same time we have to acknowledge that we influence what we are observing and that we only observe what our mental grids allows us to see.
That's why it is so important to begin by designing the case we are studying. The case study protocol becames more than an instrument, it guides the researcher during all the work to be conducted, how it is going to be carried, what is going to be observed and how, norms and procedures during research, so that reliability becames higher (Robert Yin, 1994, Case Study Research: design and Methods). On pp. 63-74 you will find guidance for the design:
- overview of the case study project
- field procedures
- case study questions
- guide for the case study report
To have a research notebook (or blog) helps remind us of possible bias one might have regarding: what's being observed, under what conditions, other things we didn't previously considered, something that went wrong that originated new prepositions, and so on... The «ah ah» factor that´s been moving science forward and the process of discovery.