RE: st: Significance test with -pscore- and -attnd-

["Ariel Linden, DrPH" <ariel.linden@gmail.com>]

Lucas,

You are asking some very basic questions that would suggest you need to
learn more about propensity score matching (or matching in a more broad
sense). I suggest you read Stuart (2010) and Caliendo & Kopeinig (2008) for
general guidance in this area.:

Here are some more specific responses to your questions, but as you can see,
they require you to know what you're asking (and thus, I suggest you read
the papers)
 
- keep only matched observations? 

* it depends on what you plan on doing next, and how you limit your
observations for analysis

- keep only observations within the common support?

* it depends on what you plan on doing next, and how you limit your
observations for analysis

- estimate the ATT with -attnd- using both panel rounds or only the post
round?

* This question suggests that you need to learn about the estimators and how
you use them. Briefly, the baseline characteristics are used as a
pre-processing step for matching. Once matches are derived (and you have
checked for covariate balance), you move on to the analysis stage. In an
analysis, you evaluate the effect of the "treatment" on the outcome,
controlling for baseline characteristics. In this case, you have controlled
for baseline characteristics via the matching process, so the analysis is on
the outcome (in your terminology, the post round).

- Do I have to convert my dataset from long to wide to estimate the ATT?

* If you want to use this procedure, then you should have the data in wide
format. If you have multiple waves, you have other matching and outcomes
analysis approaches to consider.

- How can I see whether the ATT is significant (as, for example, shown but
not explained in Khandker et al., 2010)?

*First off, you didn't provide us with the complete reference. Should we
guess, or are you assuming that we all know exactly what you are
referencing? What if Khandker wrote 10 papers in 2010? 

*Second, you have (at least) two ways of finding the level of significance:
first, you know that p = 0.05 is approximately equal to t = 1.96, so in this
case with a t score of > 1.96, you can be reasonably assured that the value
is statistically significant. A more specific approach to use, following
-attnd- (a user written program -findit attnd-) looks something like this
(after running some example code): 

. return list

scalars:
               r(ncnd) =  12144
               r(ntnd) =  185
            r(tsattnd) =  -12.52377338537646
            r(seattnd) =  587.1491980157692
              r(attnd) =  -7353.32349935502

* estimate t(df,t)
. di t(r(ncnd)+r(ntnd)-2,r(tsattnd))
4.567e-36

I hope this helps

Ariel

Stuart, E.A. (2010) Matching methods for causal inference: a review and a
look forward. Statistical Science, 25(1), 1?21.

Caliendo, M. Kopeinig, S. (2008) Some practical guidance for the
implementation of propensity score matching. Journal of Economic Surveys,
22, 31-72. 

________________________________________
From
  Lukas Borkowski <570722@soas.ac.uk>
To
  "statalist@hsphsun2.harvard.edu" <statalist@hsphsun2.harvard.edu>
Subject
  st: Significance test with -pscore- and -attnd-
Date
  Fri, 9 Aug 2013 17:32:23 +0200
________________________________________
Dear Statalist,

I am using Stata 12.1 and run an impact evaluation on a two-period panel
dataset using -pscore- and related commands (written by Sascha O. Becker and
Andrea Ichino). I got myself a bit confused and would really appreciate your
help:

After running -pscore- to estimate the propensity score using the first
panel round (baseline data) only, should I 

- keep only matched observations?
- keep only observations within the common support?
- estimate the ATT with -attnd- using both panel rounds or only the post
round?
- Do I have to convert my dataset from long to wide to estimate the ATT?

Moreover, the -attnd- command provides the standard error and a t-statistic.
How can I see whether the ATT is significant (as, for example, shown but not
explained in Khandker et al., 2010)?

Thank your very much for your help!

Best,

Lukas

#
Lukas Borkowski
University of London, School of Oriental and African Studies (SOAS)



*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/