Samples, Power Analysis, and Design Sensitivity

Warm-up Activity

Download G*Power and play around with it. See how changes in assumptions and parameters affect sample size estimates.

Part 1

1. Compare and contrast internal and external validity. Describe and give examples of research questions for which external validity is a primary concern. Describe and give examples of research questions in which internal validity is a primary concern. Discuss strategies researchers use in order to make strong claims about the applicability of their findings to a target population.

2. Compare and contrast random selection and random assignment. Be sure to include a discussion of when you would want to do one or the other and the possible consequences of failing to do random selection or random assignment in particular situations.

3. Explain the relationship between sample size and the likelihood of a statistically significant difference between measured values of two groups. In other words, explain why, all else being equal, as sample size increases the likelihood of finding a statistically significant relationship increases.

4. Compare and contrast probability and non-probability sampling. What are the advantages and disadvantages of each?

Part 2

If you do a quantitative study for your dissertation, you must estimate the sample size you will need in order to have a reasonable chance of finding a relationship among the variables stated in your research hypotheses (should one exist), given your statistical analysis(es) and assumptions/calculations of factors 2-4 above. You must do this, even if you plan to use a convenience sample (see below). There are a number of sample size calculators available. Northcentral uses G*Power, which is required in this Activity. You will use G*Power's "a priori power analysis" function to calculate a sample size. If it yields an unrealistically large size sample, you will rethink your design and assumptions and, perhaps, use G*Power's "compromise power analysis" to estimate a workable sample size that makes sense. If you plan on using a convenience sample, you would use both analyses as part of your argument that your convenience sample is large enough.

Submit the Following

1. Calculate the sample size needed given these factors:

• one-tailed t-test with two independent groups of equal size
• small effect size (see Piasta, S.B., & Justice, L.M., 2010)
• alpha =.05
• beta = .2
• Assume that the result is a sample size beyond what you can obtain. Use the compromise function to compute alpha and beta for a sample half the size. Indicate the resulting alpha and beta. Present an argument that your study is worth doing with the smaller sample.

2. Calculate the sample size needed given these factors:

• ANOVA (fixed effects, omnibus, one-way)
• small effect size
• alpha =.05
• beta = .2
• 3 groups
• Assume that the result is a sample size beyond what you can obtain. Use the compromise function to compute alpha and beta for a sample approximately half the size. Give your rationale for your selected beta/alpha ratio. Indicate the resulting alpha and beta. Give an argument that your study is worth doing with the smaller sample.

3. In a few sentences, describe two designs that can address your research question. The designs must involve two different statistical analyses. For each design, specify and justify each of the four factors and calculate the estimated sample size you'll need. Give reasons for any parameters you need to specify for G*Power.