Publish on 14th November 2019
Category: Birds
0

The schedule moving forward…

TodayEvaluation ResearchRemaining time = start on quantitative data analysisThursdayMethodology section dueQuantitative analysis continued (start Exercise 10)Next TuesdayExercise 10 DueMaahsreturns Exercise #9Research proposal (everything is now online) is due on Dec 21st(Monday)

Evaluation Research

PURPOSErather than specific methodMuch more popular in last 20 yearsA form of “applied” researchResults, though designed to impact decision-making, often have no impact

Types of Evaluation Research

Needs assessmentCost-benefit studyMonitoring studyProgram evaluation

Doing evaluation (program) research

Research questions / measurement as Job #1Political context how do measureOperationalize “outcome” of interest“Response” variableNEED AGREEMENT: what is the program trying to do?Operationalize “processes”Intermediate objectivesContext of”

Research designs for evaluation

ExperimentalProblems, ethics (placebo)The “black box” issueQuasi-experimental designsTime Series (multiple time series)Qualitative evaluationsLow birth weight study

Evaluation Research ISSUES

Evaluation research as “MESSY”Administrative control, context of “real life”Importance of looking at “black box”EthicsThe intervention itself can raise issuesControl group membersViolating Ethics 101 (Tuskegee)Evaluation impacts people’s lives

The (non) use of findings

Purely rational/scientific world vs. the one we inhabitNixon panel on pornScared Straight3 Strikes and You’re Out legislationStudies gone wrong…Fire the evaluator (or dismiss as pointy headed idiot) and keep the programWhy not trust/use the results?Scientist/practitioner gap“True believers”Vested interests

Bivariate Analysis

Backed up with alittle inferentialstatistics……yeah baby!

Review

Descriptive statisticsPurpose?Types?These are “univariate” statisticsExplanatory researchAttempt to demonstrate cause-effectNecessary Criteria?

Demonstrating Associations

Bivariate (2 variables) analysisThere are a number of ways to do thisThe method you choose depends largely on characteristics of the two variablesLevels of Measurement?

Contingency Tables (cross tabs)

Some find these very intuitive…others struggleIt is very easy to misinterpret these crittersConvention: the independent variable is on the top of the table (dictates columns) and the dependent variable is on the side (dictates rows).What is in the individual “cells?”Frequencies (number of cases that fit criteria)Convert to Percentages: a way to standardize cells and make relationships more apparent

Example

A survey of 10,000 U.S. residentsResearch question: is one’s political view related to attitudes towards police?What is the DV and IV?In constructing a table, what goes where?

An Example

The Percentages of Interest

Inferential Statistics

Researchers are typically not interested in whether there is a relationship in thesampleWant to know about the populationWhy might there be a difference between what you find in the sample and what actually exists in the pop?Even with a probability sample, there is always sampling errorWithout a probability sample = error + bias

Inferential Statistics II

Cannot assume that the relationship in your sample is true in populationBUT:probability theoryallows us to estimate:“The likelihood of obtaining a particular finding if in the population, there was no relationship” OR“The likelihood of obtaining a particular finding assuming the null hypothesis”

Test Statistics and Significance Tests

What does “statistically significant” mean?Not due to sampling errorWhat do you need to do to test for statistical significance?A “test statistic”An indicator of how different the sample is from “no relationship”The level of error that is acceptable (.05, .01)Degrees of Freedom

The Test Statistic for Contingency Tables

Chi Square, orχ2CalculationObserved frequenciesExpected frequenciesCan get confusing calculating these critters by handIntuitive: how different are the observed cell frequencies from the expected (under null hypothesis) cell frequenciesDegrees of Freedom:(# of Rows -1) (# of Columns -1)

The Chi-Square Sampling Distribution(Assuming Null is True)

Conventional Significance Testing

Calculate the test statisticSet your “Type II” error levelThe risk of being wrong that you are willing to live withFind the “critical region” within the distribution for your sample statisticHow far out on the curve does your test statistic have to get before you can reject the null hypothesisDecide whether to reject, or fail to reject the null hypothesis

Significance Testing in SPSS

Within your crosstabs, select “Chi-Square”SPSS gives you the χ2 valueWhat you would calculate based on the observed and expected cell frequenciesSPSS also gives you “p”The exact probability of obtaining this χ2, if indeed the null hypothesis was correctIn newer versions of SPSS, the p values are listed under “sig” or “significance” or something similar

Review of Contingency Tables

Level of MeasurementBoth IV and DV are Nominal or Ordinal (Categorical data)Constructing a tableIV on top (columns), DV on bottom (rows)With this format, select “column percentages”

Review of Inferential Statistics

Purpose of inferential statisticsFigure out the odds that a finding from sample is due to “chance” or “sampling error”ProcessAssume null hypothesis is correctCalculate the odds of obtaining your particular finding under this assumptionIf the odds are very low, you may be suspicious that the null hypothesis is incorrectAt some point (research sets level), you say that the odds are so low that you are going to go ahead and reject the null hypothesis

Review of Chi-Square

A measure of how different observed (from your sample data) cell frequencies are from what would be expected under the null hypothesis (e.g., no relationship)The Chi-square distribution changes shape with different degrees of freedomThis is because, by definition, the more cells you have, the higher χ2 getsdf = (R-1)(C-1)

Interpreting Chi-Square

Chi-square has no intuitive meaning, it can range from zero to very largeThe real interest is the “p value” associated with the calculated chi-square valueThis is the exact probability of obtaining the chi-square if in the population there was no relationshipIn other words, the exact probability of finding that chi-square, under the null hypothesis

0

Embed

Upload

The schedule moving forward…