Evaluation in Nonprofits-Associations

Directions:  Answers to the exam should demonstrate knowledge and application of materials covered in the class, clearly reflect excellent graduate-level writing abilities, and adhere to the APA Manual referencing and citation requirements.  Your answers should reflect and reference specific course materials. 

There are 10 questions on the exam; each question is worth 10 points, for a total of 100 points on the exam.  I am reticent to answer the ever-present question, "How long should my answer be?"  Oh, well, one or two paragraphs should do it, though I promise I will look for quality and accuracy and references, and NOT length.

1. Define the key concepts of validity and reliability.  How are they related and not related?  Support your answer with examples.  What are some potential ethical "mine fields" in this area?

2. Knowing the critical importance of obtaining valid and reliable findings, suggest three technical approaches an evaluator can take to enhance validity and reliability. Briefly say why each of these approaches will enhance V and R.  What should the evaluator do if these findings become extremely political?

3. Fitzpatrick et al. maintain that one approach to evaluation will not fit every organization or program. Familiarity with the various approaches is a more powerful tool for an evaluator and for their organization.  Take a position for or against this statement, and support your answer with application to a program within your organization.

4. The Guiding Principles of the AEA (Shadish, Newman, Scheirer, & Wye, 1995) maintain that evaluators must ensure the honesty and integrity of the entire evaluation process.  Apply this principle to three areas of the evaluation process.

5. We discussed that the object of evaluation or what is being evaluated is often, at first, fairly obscure, probably not intentionally, but rather because important concepts that will be under evaluation are quite abstract and complex.  One tool that is quite useful in answering the "what will be evaluated" question is the ability of the evaluator to provide operational definitions of the object of evaluation.  Using your own organization, identify an abstract concept (object) that might lend itself to evaluation and provide an operational definition for this concept.  (HINT:  If low employee morale were a problem, what is an operational definition of the concept "employee morale.")

6. What is the "best" approach to sampling to produce a representative sample given that you have access to the entire population of interest? And, when subjects (people) in your evaluation are hard to find, what might be the best approach to sampling?

7. When you are gathering data about your constituents/members/clients etc. and the data are quite sensitive, and you are not sure they will be comfortable in providing an accurate/honest answer, how might you approach collecting this sensitive data so that you increase the chances of getting accurate information and decrease making people uncomfortable.  Provide a rationale for your answer and support it with examples.

8. Assume you have completed a needs assessments for new programs/services for your organization.  Your needs assessment survey included 20 questions, all of which could be answered on five-point Likert scales (this is the typical rating scale like you see on the course evaluation form with five benchmarks ranging from strongly agree to strongly disagree).  Thus you have a set of data.  Identify three statistics that would assist you in getting a general picture of the data. Say why you would use these three statistics over other options.

9. Using the data described above in Question 8, say that you now divide the data into two samples, i.e., you want to compare how women vs. men responded to your needs assessment.  What steps would you take to determine if there are important or "significant" differences between the responses of your two samples?

10. Evaluating Nonprofits and Associations-If you were requested to develop a 10-page paper on this topic, provide a list of no fewer than seven references for your literature review on this topic. (NOTE:  At least three references must be books and two must be professional journals dealing with issues of evaluating nonprofits. References may not include any class readings.  Provide complete reference information.)