Slide 1:  Title slide – Session 3 Stats 101: What Can You Do with the Data You Have

Slide 2:  Agenda – 1) The basic concepts you need to know (What is alpha, significance, power, error rate, generalizability and other confusing terms?), 2) Descriptive Statistics (Frequencies, crosstabs, means, standard deviations, etc.), 3) Group comparisons (t-tests, ANOVA), 4) Tests of Association (chi-square, correlation).

Slide 3:  Discusses Evidence for Evaluation: As discussed by Dr. Pi in the first presentation in this series. “. . .the purpose of an evaluation is to provide useful information for decision-making to a variety of stakeholders.”  This information can be provided by statistical techniques, which can be used to: summarize the data – describe, model the data – examine the differences or relationships.

Slide 4:  Type of Information Statistical Analysis Can Provide: 1) Describe the participants in the program; 2) Describe the nature of the outcome variables (e.g. satisfaction, achievement, success or not on a criteria); 3) Determine if groups of participants differ on an outcome variable; 4) Determine if there is a relationship between participant characteristics and an outcome variable; 5) Determine if the participants changed from the beginning to the end of the program.

Slide 5:  Discusses a model for producing good information.  Model includes Goal, Level, Match, and Choose.  Goal – do you want to summarize or model?  Level – what is the measurement level of the variable/s?  Match – which statistical techniques are appropriate for the measurement level of your variable/s?  Choose – which technique best communicates the information and works best with the data at hand?

Slide 6:  Information Types: Summary, Model.  Summary: What is the age distribution of the clients? What services do the clients receive?  Model: Are younger clients over-represented? Did the clients receiving one service benefit more than clients receiving another service?

Slide 7:  How Do You Pick the Right Method?  -Statistical techniques are chosen on the basis of the measurement level of your variable/s (regardless of whether you’re summarizing or modeling).  -Categorical variable: Measures that are made by placing observations into mutually exclusive and exhaustive categories – ordinal variable: ordered categories (e.g. number of services); nominal variable: unordered categories (e.g. race).  –Continuous Variable: Measures that are made by positioning observations on a linear continuum (e.g. annual income).

Slide 8:  Operationalization Method – In the second presentation of this series, Dr. Thielsen talked about the importance of operationalization.  The third component, Measurement Procedures, has implications for the choice of statistical techniques.  A small number of transformations are possible: Continuous variables can be transformed to ordinal variables; Categories of nominal or ordinal variables can be combined.

Slide 9:  Summaries for Categorical Data: Graphical summaries – pie chart, bar chart; Numerical summaries – frequency table, contingency table; Numerical summaries limited to ordinal variables (because the order of the values must be meaningful): Mean, Mode, Median (measures of Central Tendency), Range (minimum, maximum values) – Measure of Variability.

Slide 10:  Summaries for Continuous Data: Graphical summaries – histogram, boxplot, scatterplot (relationship between two variables); Numerical summaries – Mean, Mode, Median (measures of Central Tendency), Range (minimum, maximum values), Standard deviation, Variance, Quartiles (Measures of Variability), Skew, Kurtosis (Measures of the shape of the value distribution.

Slide 11:  Case in point: Summary: Suppose you are interested in summarizing the racial characteristics of your client load.  You decide to operationalize ‘Race’ according to clients’ self-reported responses to an item asking them to indicate which of seven racial groups they best identify with.  The measurement level of this variable is categorical :  Measures that are made by placing observations into mutually exclusive and exhaustive categories.  Furthermore, the variable is nominal because the categories are unordered categories.  The nature of the variable suggests several different options for summarizing the data. 

Slide 12:  Slide shows an example of a pie chart.

Slide 13:  Slide shows an example of a bar chart.

Slide 14:  Slide shows an example of a frequency table.

Slide 15:  Discusses Modeling Data.  This technique involves hypothesis testing or creating confidence intervals.  Modeling data involves the ideas of: generalizability, alpha, power, significance.

Slide 16:  More Modeling Data.  Determine whether you need to generalize beyond the sample at hand to a population – sampling becomes very important in this instance; population must be specifically defined.  Determine how many people you’ll need in the study – you need to specify power level (usually .80), alpha (usually two-tailed .05), and effect size to compute required sample size; Specifying an effect size can be tricky (you haven’t done the study yet!) but you can rely on previous research or at the very least Cohen’s rules of thumb.

Slide 17:  Slide discusses Generalizability: Ability to extend findings to a population; Random sampling and other related approaches promote generalizability; Oftentimes, the goal of the evaluation doesn’t require generalizability, but you need to know if this is the case; The interpretation of your findings needs to match whether they are generalizable.  Generalizability dictates sampling design.

Slide 18:  Slide discusses Alpha: Also known as – the level of significance, Type I error rate; this is the error rate of rejecting the null hypothesis when you shouldn’t reject, that is, finding statistical significance when it really isn’t there.  The researcher chooses alpha, which dictates required sample size, given a level of power and an effect size.

Slide 19:  Slide discusses Power: This is the probability of rejecting the null hypothesis when you should reject, that is, finding statistical significance when it really is there; Power is a function of sample size, the effect you expect to get, and alpha; Your goal is to maximize power.  The researcher chooses power, which dictates sample size, given an expected effect size and alpha.

Slide 20:  Slide discusses Significance: This is also known as the p-value; The probability of observing a test statistic equal to or more extreme than the one you computed, if the null hypothesis were true; If p-value < alpha, reject the null; Your goal is to maximize power.  Significance is calculated according to the statistical model chosen, and is compared to alpha; if something is statistically significant, that implies that the significance was less than alpha.

Slide 21:  Slide discusses Effect Size: This is a standardized index that is independent of sample size; The specific computation of ES depends on what statistical model you’ll use to analyze the data; ES is used to calculate required sample size for a study.  ES is calculated according to the statistical model chosen, and dictates power.

Slide 22:  Slide shows chart of what tests to use (one-sample t-test, independent samples t-test, etc.) when you want to know specific information.

Slide 23:  Slide continues with the chart of what tests to use for what information you want.

Slide 24:  Slide shows a list of useful Print resources: Creswell, J. (2008).  Research Design: Qualitative, Quantitative, and Mixed Methods Approaches.  Sage.
Field, Andy (2009).  Discovering Statistics.  Thousand Oaks, CA:  Sage. 
Huck, S. (2011).  Reading Statistics and Research (6th ed.). Addison Wesley.
Nicol, A. and Pexman, P. (2010).  Presenting Your Findings: A Practical Guide (6th ed.).  American Psychological Association.
Shadish, W. Cook, T., and Campbell, D. (2001).  Experimental and Quasi-Experimental Design.  Wadsworth Publishing.

Slide 25:  Slide gives list of useful Web resources: Garson’s Statnotes website:
  http://faculty.chass.ncsu.edu/garson/PA765/statnote.htm
Rice Virtual Lab in Statistics (see Hyperstat Online):
  http://onlinestatbook.com/rvls.html
UCLA Statistical Computing:
  http://www.ats.ucla.edu/stat/
Online Statistics Education:
  http://onlinestatbook.com/2/index.html