Exercise for Test Validity
You are an I/O psychologist. You have been hired by XYZ Cola Company to develop and assess the validity of a clerical selection test. The intent of the clerical selection test will be to test applicants who want a job as a secretary.
The XYZ Cola Company currently has 2000 secretaries on payroll. The company has learned from experience that it is very important to have qualified individuals in their secretarial positions, and that top clerical employees seem to require less training time, produce higher quality work, and get higher performance ratings. Yet they have never had a formal test to select new secretaries.
You complete a careful job analysis and find that there are four major components to the work done by the clerical workers. These components are:
1. Filing
2. Recording and checking
3. Written communication skills
4. Typing
You develop four tests to assess these skills. For example, below is an example of an item from your "Written communication skills" test. You decide to calculate a total score for each person who takes your test by adding his or her four test scores together.
Example item:
The following are several sentences. The underlined part of each sentence may contain an error in spelling, punctuation, or capitalization. If the underlined part of the sentence contains an error, then mark the box in front of the change that should be implemented.
The Federal Express package should arrive at 3:00 pm Wenesday.
- 3:00 pm Wenesday
- 3:00 p.m. Wenesday
- 3:00 p.m. Wednesday
- NO change needed
Based on the information presented, answer the following questions.
1. What does face validity refer to? Is the example item face valid?
2. What does content validity refer to? How might you determine whether the clerical test you have developed has sufficient content validity?
3. Give an example of how you might assess the concurrent criterion-related validity of your clerical test. Be specific.
4. How would you assess the predictive criterion-related validity of your clerical test?
5. What would be an acceptable validity coefficient for your predictive criterion-related validity study?
6. How might you assess the convergent validity of your clerical selection test?
7. How might you assess the divergent validity of your test?