Questions: 1. Which of the following are examples of counseling research?

• An experiment where individuals were randomly assigned to CBT vs. Motivational Interviewing to determine if one approach yields better results

• A peer review of a clinic to determine if it is maintaining legally mandated standards

• A longitudinal study of a foster children who entered the system at the same age in the same time period to document their mental and emotional well-being throughout their lives

• A needs assessment to determine the extent to which individuals in the community are able to effectively cope with mental illness.

2. Which of the following are examples of things you might do when evaluating a program?

• Generalize your results to a broader population than the sample served at the agency in question

• Collect data on outcomes such as symptom checklists or data on psychiatric hospitalizations

• Interview stakeholders such as referral sources or community members.

• Separate clients into control and experimental groups

• Make determinations on possible changes to the program as a result of the data

3. Which of the following would be the LEAST appropriate form of evidence to include in a program evaluation?

• Surveys about clients' satisfaction with services they've received

• Records of staff attendance at mandatory trainings

• Aggregated results of psychometric instruments

• Direct measures of client outcomes

4. Match each example with the model of program evaluation it represents

Objectives-oriented

Management-oriented

Consumer-oriented

Expertise-oriented

5. We say that a needs assessment can serve as a diagnostic evaluation of the status of a program. What does this mean? Choose the best answer.

A needs assessment determines the variance of different DSM diagnoses that are treated at the site

A needs assessment uses diagnostic criteria from the field of psychiatry

A needs assessment includes biopsychosocial information

A needs assessment determines the status, seriousness, and extent of the problems clients utilizing the program face

The rest of these questions are about the program that you chose to evaluate. While you were not expected to actually evaluate the program, these questions ask for information about how you would conduct an evaluation if you were in a position to do so. Please answer all questions as thoroughly as possible, so that someone could adequately evaluate the program you identified by following what you wrote.

6. What specific program have you familiarized yourself with for the sake of conducting a program evaluation? What is the name of the program and what agency or organization is responsible for implementing it?

7. What is the target population that this program serves? Are there specific presenting problems that it addresses? What are the defining characteristics of the clients that utilize this program (e.g. certain age range, certain diagnoses)? Are there any people or groups of people it purposely does not serve?

8. What are the specific outcomes that this program aims to meet? What exactly is this program trying to achieve in terms of treatment goals or community change?

9. If you were to conduct an evaluation of this program, which stakeholders would you want input from? Why? How would you engage them to get them invested in helping you to evaluate the program? What kind of input would you want from each stakeholder group, and why?

10. What kinds of data would you collect to evaluate the program? Which instruments would you use to assess the effectiveness of the program? Does the program already use particular instruments for this purpose? If so, do you support continuing to use these? Why or why not? Are there any additional instruments you would want to use? How would you determine how well the program is working? What evidence would motivate you to make changes to the program?