Laboratory v. field experiments
The tradeoff between internal and external validity
Natural sciences
In natural sciences, most experiments have been undertaken in carefully controlled laboratory conditionsSeeking laws of physics, etc.Inanimate objects are unlikely to react differently in controlled and uncontrolled conditionsThe main goal is to avoid contaminating the experiment via the introduction of third variablesFar more exacting and finer measurement is possible under controlled conditions
Laboratory experiments in social science
Closely controlled social science experiments do have the advantage of limiting the impact of third variables, but the unnatural situation presents problems for applying the findings outside the laboratory“Artificiality” of laboratory setting
Human behavior is sensitive to the environment within which it occursPeople act differently in a laboratory than in the natural worldSeveral characteristics of laboratories are thought to influence behaviorThe very third variables controlled in the lab may be the ones that determine behavior in the real worldSo, findings from laboratory experiments may only be valid for laboratory environments
An example
New commercials are tested in controlled conditionsEye trackingLiking for commercialsInfluence on purchase interestMay try to provide less artificial conditions for studySimulated living roomCommercials that test high in lab experiments often do not work very well when used in real marketing campaigns
So, experiments move out of the lab:
Researchers want to retain some of the advantages of the experiment:Ability to manipulate/introduce the independent variable and to control how much of it is presentedTime order—which comes firstWhile sacrificing some of their ability to control third variablesThe goal is to improve our ability to generalize our findings to the real world
The field experiment
One way to do so is to carry out a ‘field experiment’The researcher still manipulates the independent variable, but she does so within the natural worldFor example, police try randomly stopping motorists and testing for alcohol to see if alcohol-related crashes decline
Information Resources, Inc.
BehaviorScansystem
What problems do we encounter?
Greatly reduced ability to prevent third variable contamination of resultsThe crusading doctor in MHHPHistory effectsCyclesWhat can you do?Measure/monitor likely alternative explanatory variablesQuestion subjects about sources of influenceMultiple manipulations of the independent variable over timeMultiple measurement wavesMultiple dependent measuresNone will be perfect
ExpenseTradeoff between extensive and intensive studyBudget constraints on number of sites, etc.Access/permissionSome research may present concerns to authorities, citizens, etc.Gain authorization/support prior to entering the fieldMaintain good relationships with community leaders, etc. throughout the intervention/research
Capturing ‘natural experiments’
Sometimes unusual or unique events occurALAR scareService disruptionsPolitical campaignsSchool shootingInstitution of Wi-Fi in anarea
“Natural experiments”
Because most such events are unplanned, the ability to prepare for them is limitedMay keep a research group, materials and resources ready for certain types of eventsMust engage in ‘firehouse research’ gathering as much data as possible in a short timeInefficient, and may miss important dataHowever, real-world events, etc. may provide very valuable data—may have both internal and external validity
Use statistics, research analysis models that are appropriate to experimental studies
0
Embed
Upload