Impact Evaluation - Assignment

Problem Set

Evaluation Task in R or Stata

• You have been assigned a subsample of a data set from a real development project on Moodle
• Conduct an impact evaluation using appropriate methods in R or Stata
• Write a short report (~1.250 words, 4 pages) presenting and interpreting the results in a MS Word or Open office format
• Please include the script of all commands used for your respective software along with the results as an appendix to your report

Variables and Dataset
• In the assignment two datasets are provided to you
- Midline Survey (ml.dta)
- Combined Dataset for baseline and midline survey (mlbl.dta)
• Both the datasets have been uploaded on Moodle
• Please use the midline survery for all questions except for question number 3.
- Each one of you will use your matriculation number
- You are required to draw a unique random 30% sample from the dataset given to you using your matriculation number (see next slide)

Select your 30% sample
• Please use the command below in R or Stata to draw your random sample for the assignment
• Your random number will be given to you via moodle

• Stata:

set seed sample 30 if n_pup_id

• R:
set.seed(enter your matriculation number here) test_IDs=sample(ML$n_pup_id,size=0.3*nrow(ML),replace=FALSE) test_data=subset(ML, n_pup_id %in% test_IDs)
head(ML)

Introduction to the Evaluation

• The EQUIP-Tanzania program, funded by DFID and implemented in cooperation with the Government of Tanzania, aims to improve the quality of education, especially for girls, in seven relatively educationally- disadvantaged regions of Tanzania, with the eventual goal of replication to the whole of Tanzania.

• Covering one quarter of the primary education system, with 3,680 schools, approximately 49,000 teachers, EQUIP- Tanzania will support more than 2 million girls and boys across 48 districts to have an equal opportunity to access and benefit from quality education.

Description of Dataset

• This evaluation project measures the impact of educational quality improvement programme in Tanzania on pupil learning. Baseline survey was carried out in year 2014,
• Midline survey was carried in year 2016.
• The final survey will be carried out in year 2018.
- The total number of students surveyed are 2200,000
- The total number of schools included in survey are 3680.
- 48 districts covered in the study
• For this assignment, you will be using a subset of only 23 variables
• Each student will have approximately 892 unique oberservations in their dataset based on their unique code
• We will be carrying out the evaluation of reading comprehension of the students using the variable n_p_rctotal as the outcome variable

1.) Randomization

In this problem, assume pupils were randomly selected in control and treatment groups. Using dataset for pupil EQUIP, consider the outcome variable (n_p_rctotal). Please compare the outcome between pupils in the treated and control groups. For this question, use five most relevant independent variables from the dataset.

a) Is the difference in outcome between treated and control pupils significant? Use t-test for this purpose.

b) Which of the independent variables influence the difference in the outcome? Use regression estimation.

c) In addition, chose any three variables and run t-test to see if there is significant difference between treatment and control groups. Based on your results what can you say about randomization?

2.) Instrumental Variable Regression

Suppose that program participation was non-random and the treatment variable is endogenous. Run an instrumental variable regression to rectify for the problem of endogeneity.

a) Use sc_asvehicles as an instrument for treatment and run two stage least squares regression. Interpret the results.

b) Test for endogeneity (Durbin or Wu-Hausman test) and Sargan-test in order to conclude whether the instrument is appropriate.

c) Discuss, which conditions need to be fulfilled before choosing a specific variable as an instrument?

3.) Difference-in-Differences

In order to eliminate time invariant unobserved heterogeneity, the difference-in-differences approach should be used.

a) Use the "mlbl.dta"-dataset to measure the impact of improvement of primary education in Tanzania. Take the difference in outcomes between treatment and control group and compare it for both surveys (baseline and endline).

b) From the data set take the variable (n_p_rctotal) as outcome variable and run a simple regression by interacting treatment and year dummy variable. The estimate of the interaction variable gives the impact of treatment on outcome Y. Interpret the estimate.

c) Now, estimate the same equation by controlling for other covariates that you think might vary over time and interpret the interaction variable.

4.) Propensity score matching
In this problem, use the "ml.dta"-dataset.

a) Estimate the Propensity Score
- Start with treatment variable and estimate probit regression equation.
- Predict propensity scores and discuss it using summary statistics.
- define the area of common support.

b) Match participants and non-participants
- Use nearest neighbor matching for treatment and control group participants and check the balancing property.
- In your equation (used in a) identify the variables that cause the balancing property not to be satisfied
in STATA: variables are directly reported
in R: look at the graphs of 5 independent variables and conclude

c) In order to rectify the problem discovered in b), remove the variables that are unbalanced from the set of independent variables and rerun the task a).

Attachment:- Data.rar