Research Methods in Crime and Justice
Chapter 9Experimental Design Research Methods
Experimental Design Research Methods
Experimental design research methodsare oneof the ‘purest’ forms of social science inquiry.Resultsprovideinsightinto the actual causes of social phenomena.A useful method in inexplanatory research.
Experimental Research Design Basics
Anexperimentis a research method that measures how much, if any, an independent variable causes a change to a dependent variable.Thereare various types of experimental design models.Allexperimental design models have three essential characteristics
Experimental Research Design Basics
All experimental design models have;Anexperimental groupof subjects that are exposed to the independent variable (i.e. treatment) that the researcher believes will cause change in the dependent variable.Treatment(the independent variable) that the researcher alleges will cause change to the dependent variable, andAposttest, during which the dependent variable is measured after the treatment has been applied.
Experimental Research Design Basics
More sophisticated experimental design models may have;Apretestduring which the dependent variable is measured beforethe treatment is applied.Acontrol groupof researchsubjects that are similar to theexperimental group in every way except that they are not exposed to the treatment.
Experimental Research Design Basics(Basic Steps)
Select a sample of research subjects.Randomly divide the sample into two equivalent groups.Measure the dependent variable in both groups to confirm they are equivalent.Expose only the experimental group to the treatment.Measure the dependent variable in both groups to see of the treatment (independent variable) had its intended effect on the dependent variable.
Types of Experimental Research Designs
The most basic experimental design model is often called aone group no pretest experimental design model.Thistype of design only has the basic elements of the experimental designmodel;anexperimental group,a treatment,andaposttest
Types of Experimental Research Designs
Theone group, no pretest experimental designdoes not include a pretest or a control group.Because ofthis;theeffect of the treatment cannot be accuratelymeasured,andtheinfluence of other factors on the dependent variable cannot be identified.
Types of Experimental Research Designs
Theone group pretest/posttest experimental design modelalso includes the basic elements of an experimental designmodel;anexperimental group,a treatment,andaposttest.Thismodelalso includea pretest that measures the dependent variable prior to administering the treatment.
Types of Experimental Research Designs
Because theone group pretest/posttest experimental design modelincludes a pretest the researcher can measure the actual effect of the treatment (independent variable) on the dependent variable.This design does not include a control group so there is really no way for the researcher to know whether something other than the treatment caused a change to the dependent variable.
Types of Experimental Research Designs
Thetwo group no pretest experimental design modelalso includes the basic elements of the experimental designmodel;anexperimental group,a treatment,andaposttest.Thisdesign includes a controlgroup.
Types of Experimental Research Designs
Because thetwo group no pretest experimental design modelincludes a control group the researcher would be able to determine that the independent variable (by itself) had some effect on the dependent variable.Becausethere is no pretest the researcher cannot measure how much effect the independent variable had on the dependent variable.
Types of Experimental Research Designs
Thetwo grouppretest/posttest experimental design modelalso includes the basic elements of the experimental designmodel;anexperimental group,a treatment,andaposttest.Inaddition this model has both a pretest and a control group.
Types of Experimental Research Designs
Because thetwo group pretest/posttest experimental design modelcontains both a pretest and a control group, the researcher canMeasure the actual effect of the treatment on the dependent variable, andRemove all other factors that might also have an effect on the dependent variable.
Types of Experimental Research Designs
TheSolomonFour Group experimental designmodelincludesthe basic elements of the experimental designmodel;anexperimental group,a treatment,andaposttest.Thismodel alsocontains;a pretest,andacontrol group.
Types of Experimental Research Designs
More significantly, this model containsan extra experimental group, andan extra control group.This results in an experimental model with four groups of research subjects.
Types of Experimental Research Designs
Because theSolomon Four Group experimental design modelcontains an extra experimental and control group the researcher is able to determine the potential for a pretest effect.
Validity
Previously, validity was defined as the accuracy of a measure.In experimental research the term, validity, has different meanings.Internal validityrefers to the ability of an experimental design to document the causal relationship between an independent and dependent variable.External validityrefers to the capacity of an experimental research finding to be applicable to other settings.
Threats to Internal Validity
History-Major events happen during an experiment that affect the research subjects and thus the dependent variable.Maturation-Natural developmental changes in the research subjects affect the outcome of the experiment.Mortality-A loss of research subjects can occur over the course of an experiment and affect the outcome of the experiment.
Threats to Internal Validity
Testing-Exposing research subjects to a pretest prior to the treatment can change the outcome of the posttest.Instrumentation-Differences between the pretest and posttest instruments cause a change in the dependent variable.
Threats to Internal Validity
Regression-Although there may be an initial treatment effect, the effect diminishes over time, indicating that the independent variable has no long-term effect.Selection bias– When assigning members to the experimental and control groups the researcher erred so that so that the groups are not equivalent.
Threats to External Validity
Reactivity-An awareness that they are being measured causes a change in the behavior of research subjects.Interaction between selection bias and the dependent variable-There is a failure to ensure that the subjects assigned to the experimental and control groups are more or less equivalent with respect to the variables that might influence the dependent variable.
The Benefits of Experimental Research
Experimental research is useful for;Isolating the effect of an independent variable on a dependent variable.Measuring how much of an effect a treatment has on an outcome.Demonstrating causality, or cause and effect relationships.
The Limitations of Experimental Research
Experimental research is not useful when;Resources (time and money) are limited.The researcher cannot control the behavior of the research subjects.There are serious ethical concerns associated with the control of the research subjects or the treatment.
The Experimental Research Process
ACase Study in ExperimentalResearch(The Minneapolis Domestic Violence Experiment)Lawrence W. Sherman and Richard A.Berk(1984)Applied an experimental model to determine the most effective way to reduce domestic violence recidivism.Research continues to have a profound effect on contemporary policing practice.
The Experimental Research Process
Askinga Research Question in ExperimentalResearchBecause of their ability to isolate the effect of a single variable, experimental designs are most often used for explanatory research.“If the purpose of police responses to domestic violence calls is to reduce the likelihood of that violence recurring, the question is which of these approaches(no action, active mediation, or arrest) ismore effective than the others?” (1984b: 1).
The Experimental Research Process
Conductinga Literature Review in ExperimentalResearchA review of the prior experimental research may reveal useful testing instruments or innovative ways of measuring variables.Thisliterature reviewfocused on;classicresearch on police decision-making,researchon mediation and arbitration in domestic violence prevention, andmorecontemporary research on police responses to domestic violence (Sherman andBerk1994a and 1984b).
The Experimental Research Process
Refiningthe Research Question in ExperimentalResearchBecause experimental models are best suited to explanatory research, it is important for the researcher to create solid hypotheses.This research involved one nominal independent variable with three attributes and two interval dependent variables.The result was six sets of alternative and null hypotheses.
The Experimental Research Process
Conceptualizationand Measurement in ExperimentalResearchExperimental researchers should be concerned about;The measures used to determine equivalency between the experimental and control groups.The conceptualization and measurement of the independent variable, andThe conceptualization and operational measurement of the dependent variable.
The Experimental Research Process
Conceptualization and Measurement in Experimental Research – cont’dSherman andBerkdefined‘domestic abuse’ broadly to include numerous forms of abusivebehaviors (ShermanandBerk, 1984a: 263).Sherman andBerkconceptualized the frequency of reoffending as the number of repeat offenses that occurred within a six-month period following the first domestic violence incident.
The Experimental Research Process
DesigningExperimentalResearchThe choice of experimental design depends on a number of factors,including;howconfident you want to be in your results,whatis feasible and ethical in a given researchsetting,andwhatyou want to know about the research subjects.Sherman andBerkused three one group pretest/posttest experimental designs, one for each strategy (arrest, separate, mediate)
The Experimental Research Process
Designing Experimental Research – cont’dIt was critical to insure that the three groups (arrest, separate, mediate) were equivalent with respect to all factors (e.g. severity of injury) that could affect the outcome.Because domestic violence incidents occurred over time it was not possible assign cases at one time.Sherman andBerkdeveloped a lottery system to help police officers randomly assign cases to the three groups(Sherman andBerk, 1984b: 3).
The Experimental Research Process
CollectingData in ExperimentalResearchMost data collection during an experiment occurs during the pretest and the posttest.Sherman andBerkspend considerable time training the police officers that participated in the experiment.They pretested their design and made appropriate changes.They identified a need to train data entry assistants to be sure they adhered to the experiment’s measurement strategy(1984b: 5).
The Experimental Research Process
Analyzingand Interpreting Data from ExperimentalResearchQuantitative results are the most common type of data gathered through an experimental design.In most cases the analysis requires comparing pre- and posttest results.Sherman andBerkused three statistical models.Overall they found that arrest had the highest deterrent effect(Sherman andBerk, 1984a: 268).
The Experimental Research Process
Analyzing and Interpreting Data from Experimental Research – cont’dBecause their research had potentially profound effects on public policy, Sherman andBerkspent considerable time discussing its weaknesses.Officers that did not adhere to the experiment’s procedures for assigning cases.The incapacitation effect of arrests.Small sample sizes.
The Experimental Research Process
Communicatingthe Findings from Experimental ResearchHow, when and where the results of an experiment are reported typically depends on who is interested in the results of the experiment.Because this research would be of interest to criminal justice scholars and policing professionals it was published in;A scholarly journalA trade publication that is distributed to policing leaders.
The Experimental Research Process
Askinganother Research Question in ExperimentalResearchGood research tends to produce as many questions as itanswers.One would think that with results this convincing, the controversy over whether a mandatory arrest policy will reduce domestic assaults would have ended. But this is not exactly what happened.Additional questions continue to this day on the external validity of these results.
Getting to the Point
Anexperimentis a research method that measures the effect of an independent variable on a dependent variable.Allexperimental design modelsfeature;anexperimental group,a treatment,andaposttest.Moresophisticated experimental design models also include a pretest and a control group.
Getting to the Point
Theone group, no pretest experimentaldesign modelonlyincludes the basic elements of an experimental design model – the experimental group, a treatment and a posttest.Thisdesign does not include a pretest or a control group.Becauseof this the effect of the treatment cannot be accurately measured and the influence of other factors on the dependent variable cannot be identified.
Getting to the Point
Theone group pretest/posttest experimental design modelincludes the basic elements of an experimental design model – an experimental group, a treatment and a posttest.Inaddition, this model includes a pretest that allows the researcher to measure the actual effect of the treatment (independent variable) on the dependent variable.Thisdesign does not include a control group so there is really no way for the researcher to know whether something other than the treatment caused a change to the dependent variable.
Getting to the Point
Thetwo group no pretest experimental design modelincludes the basic elements of the experimental design model – an experimental group, a treatment and a posttest.Thisdesign includes a control group so the researcher would be able to determine that the independent variable, by itself, had some effect on the dependent variable.Becausethere is no pretest the researcher cannot measure how much effect the independent variable had on the dependent variable.
Getting to the Point
Thetwo group pretest/posttest experimental design modelincludes the basic elements of the experimental design model – an experimental group, a treatment and a posttest.Inaddition this model has both a pretest and a control group.Thesetwo features enable the researcher to measure the actual effect of the treatment on the dependent variable and to determine whether or not other factors might have caused a change in the dependent variable.
Getting to the Point
TheSolomon Four Group experimental design modelincludes the basic elements of the experimental design model – an experimental group, a treatment and a posttest.Thismodel also contains a pretest and a control group.Moresignificantly, this model contains an extra experimental group and an extra control group.Theseadditional features enable the researcher to determine how much, if any, the research subjects’ exposure to the pretest affected their performance on the posttest.
Getting to the Point
Internal validityrefers to the ability of an experimental design to document the causal relationship between an independent variable and a dependent variable.Therearesevencommon threats to the internal validity of an experiment: history, maturation, mortality, testing,instrumentation, regression and regression
Getting to the Point
External validityrefers to the generalizability of an experiment’s results to other settings and situations.Thereare two common threats to external validity: Reactivity and interaction between selection bias and the dependent variable.
Getting to the Point
Experimental research is effective at isolating and measuring the effect of a single independent variable on a dependent variable.Experimentalresearch is also effective at demonstrating a causal relationship between two variables.
Getting to the Point
Experimental research requires considerable resources (e.g. time and money).Oftenexperimental research is not feasible because of the amount of control the researcher must exert over the research subjects.Inexperiments involving human subjects there is a potential for ethical violations.
Getting to the Point
Because of their ability to isolate the effect of a single variable on an outcome, experimental designs are most often used in explanatory research. Experiments are appropriate for both pure and applied research purposes.
Getting to the Point
In preparing to conduct an experiment, researchers should review the previous literature, paying particular attention to how past researchers measured the dependent variable and what independent variables have been found to affect the dependent variable.
Getting to the Point
Because experimental models are used in explanatory research, creating specific hypotheses is an essential step in the experimental research process.
Getting to the Point
Experimental researchers should be concerned about conceptualization and measurement of the independent and dependent variables, as well as the variables used to determine equivalency between the experimental and control groups.
Getting to the Point
Experimental researchers should determine which experimental design model they want to use early in the research process.Thechoice of experimental design depends on a number of factors,including;howconfident the researchers want to be in their results,whatis feasible and ethical in a given researchsetting,andwhatresearchers want to know about the research subjects.
Getting to the Point
The number and nature of data collection strategies used by experimental researchers depend on the experimental design used by the researcher.
Getting to the Point
Experimental designs tend to include variables that are measured at the interval or ratio levels.Assuch, t-tests or analyses of variance are the most common statistical techniques used to analyze data in experimental research.Wheninterpreting the findings from these and other analyses, experimental researchers should be up front about the possible threats to internal and external validity within the experimental design.
Getting to the Point
How, when and where the results of an experiment are reported typically depends on who is interested in the results of the experiment.
Getting to the Point
Good research tends to produce as many questions as it answers. These new questions are opportunities to continue the research process.
Research Methods in Crime and Justice
Chapter 9Experimental Design Research Methods
0
Embed
Upload