Logic of Hypothesis Testing

In Week 4, you explored how the confidence interval helps to estimate a population mean. Hypothesis testing is an approach that allows us to make some determination about whether a hypothesis should be rejected, based upon sample statistics. This approach is integral to the scientific method and provides us with a measurable level of certainty when making inferences back to the population.

Consider the following:

A researcher is conducting work on social inequality and wants to know whether there are marked differences between socioeconomic status of Caucasians and Non-Caucasians. Since the researcher cannot measure the entire population, a sample is drawn and a hypothesis can be constructed and evaluated as to whether any noticeable differences in the sample also likely appear in the population.

As a scholar-practitioner, it will be important for you to develop your knowledge and skillset in hypothesis testing. As evident in the scenario provided, hypothesis testing establishes a process to determine the probability of observing similar scores noted in the sample under the null hypothesis.

For this week, you will examine hypothesis testing and determine the statistical significance and meaningfulness in the data. You also will explore the results of data to determine implications for social change.

Learning Objectives

Students will:

Evaluate statements related to null hypothesis

Evaluate p-values

Evaluate type I and type II errors

Evaluate for meaningfulness

Evaluate statistical significance

Evaluate sample size

Analyze implications for social change

Photo Credit: [Hero Images]/[Hero Images]/Getty Images

Discussion: Statistical Significance and Meaningfulness

Once you start to understand how exciting the world of statistics can be, it is tempting to fall into the trap of chasing statistical significance. That is, you may be tempted always to look for relationships that are statistically significant and believe they are valuable solely because of their significance. Although statistical hypothesis testing does help you evaluate claims, it is important to understand the limitations of statistical significance and to interpret the results within the context of the research and its pragmatic, "real world" application.

As a scholar-practitioner, it is important for you to understand that just because a hypothesis test indicates a relationship exists between an intervention and an outcome, there is a difference between groups, or there is a correlation between two constructs, it does not always provide a default measure for its importance. Although relationships are significant, they can be very minute relationships, very small differences, or very weak correlations. In the end, we need to ask whether the relationships or differences observed are large enough that we should make some practical change in policy or practice.
For this Discussion, you will explore statistical significance and meaningfulness.

To prepare for this Discussion:

Review the Learning Resources related to hypothesis testing, meaningfulness, and statistical significance.

Review Magnusson's web blog found in the Learning Resources to further your visualization and understanding of statistical power and significance testing.

Review the American Statistical Association's press release and consider the misconceptions and misuse of p-values.

Consider the scenario:
A research paper claims a meaningful contribution to the literature based on finding statistically significant relationships between predictor and response variables. In the footnotes, you see the following statement, "given this research was exploratory in nature, traditional levels of significance to reject the null hypotheses were relaxed to the .10 level."

Assignment: Evaluating Significance of Findings

Part of your task as a scholar-practitioner is to act as a critical consumer of research and ask informed questions of published material. Sometimes, claims are made that do not match the results of the analysis. Unfortunately, this is why statistics is sometimes unfairly associated with telling lies. These misalignments might not be solely attributable to statistical nonsense, but also "user error." One of the greatest areas of user error is within the practice of hypothesis testing and interpreting statistical significance. As you continue to consume research, be sure and read everything with a critical eye and call out statements that do not match the results.
For this Assignment, you will examine statistical significance and meaningfulness based on sample statements.

To prepare for this Assignment:

Review the Week 5 Scenarios found in this week's Learning Resources and select two of the four scenarios for this Assignment.

For this Assignment:

Critically evaluate the two scenarios you selected based upon the following points:

Critically evaluate the sample size.

Critically evaluate the statements for meaningfulness.

Critically evaluate the statements for statistical significance.

Based on your evaluation, provide an explanation of the implications for social change.

Use proper APA format and citations, and referencing.