Re: st: RE: RE: RE: psmatch2 vs attk

[Nick Cox <njcoxstata@gmail.com>]

The wonderful thing about standards is that there are so many of them
to choose from. Attributed to several leading computer scientists and
applicable to standard methods too.

Nick
njcoxstata@gmail.com


On 26 March 2014 14:52, Joe Canner <jcanner1@jhmi.edu> wrote:
> Rossella,
>
> My glib answer would be that these are two different programs written by two different people who may have used different methods to implement kernel matching.
>
> That aside, it is apparent that the two programs are not on the same page with respect to common support.  There are 154 observations total (79 treated and 75 untreated), but -psmatch2- thinks there are 137 (62 trt +75 untrt) on common support and -attk- thinks there are 149 (79 trt  + 70 untrt).  So, that seems like it might be a good place to start.  Try both commands without common support and see if they at least agree on the denominators.  Then, see if you can figure out on your own what the common support should be and see if you can reverse engineer how common support is determined for each.
>
> Maybe someone else who has "looked under the hood" of each of these programs can tell us what other differences there are.
>
> Regards,
> Joe
>
> -----Original Message-----
> From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Rossella Verzulli
> Sent: Wednesday, March 26, 2014 10:17 AM
> To: statalist@hsphsun2.harvard.edu
> Subject: st: RE: RE: psmatch2 vs attk
>
> Joe
>
> Many thanks for your advice. However adding the common option in psmatch2 does not help to obtain the same results. I'm reporting the Stata output below.
>
> Best
>
> Rossella
>
>  psmatch2 T, out(performance) kernel kerneltype(normal) pscore(mypscore) common
> ----------------------------------------------------------------------------------------
>         Variable     Sample |    Treated     Controls   Difference         S.E.   T-stat
> ----------------------------+-----------------------------------------------------------
>      performance  Unmatched | 99.7088608 100.653333  -.944472574    1.5097252    -0.63
>                         ATT | 99.6129032   99.3059791   .306924102     1.993187     0.15
> ----------------------------+-----------------------------------------------------------
> Note: S.E. does not take into account that the propensity score is estimated.
>
>  psmatch2: |   psmatch2: Common
>  Treatment |        support
> assignment | Off suppo  On suppor |     Total
> -----------+----------------------+----------
>  Untreated |         0         75 |        75
>    Treated |        17         62 |        79
> -----------+----------------------+----------
>      Total |        17        137 |       154
>
>
> . attk performance T, normal pscore(mypscore) comsup boot dots
>
>
>  The program is searching for matches of each treated unit.
>  This operation may take a while.
>
>
>
> ATT estimation with the Kernel Matching method
>
> ---------------------------------------------------------
> n. treat.   n. contr.         ATT   Std. Err.           t
> ---------------------------------------------------------
>
>        79          70      -1.568           .           .
>
> ---------------------------------------------------------
> Note: Analytical standard errors cannot be computed. Use
> the bootstrap option to get bootstrapped standard errors.
>
>
>
>
>
> Bootstrapping of standard errors
>
> command:      attk performance T , pscore(mypscore) comsup bwidth(.06)
> statistic:    attk       = r(attk)
> ..................................................
>
>
> Bootstrap statistics                              Number of obs    =       154
>                                                   Replications     =        50
>
> ------------------------------------------------------------------------------
> Variable     |  Reps  Observed      Bias  Std. Err. [95% Conf. Interval]
> -------------+----------------------------------------------------------------
>         attk |    50 -1.567504 -.2310363   2.32807  -6.245935   3.110928   (N)
>              |                                      -6.104194   3.185225   (P)
>              |                                      -5.293512   4.606492  (BC)
> ------------------------------------------------------------------------------
> Note:  N   = normal
>        P   = percentile
>        BC  = bias-corrected
>
>
>
> ATT estimation with the Kernel Matching method
> Bootstrapped standard errors
>
> ---------------------------------------------------------
> n. treat.   n. contr.         ATT   Std. Err.           t
> ---------------------------------------------------------
>
>        79          70      -1.568       2.328      -0.673
>
> ---------------------------------------------------------
>
>
>
>
>
>
> ________________________________________
> Da: owner-statalist@hsphsun2.harvard.edu [owner-statalist@hsphsun2.harvard.edu] per conto di Joe Canner [jcanner1@jhmi.edu]
> Inviato: mercoledě 26 marzo 2014 15.11
> A: statalist@hsphsun2.harvard.edu
> Oggetto: st: RE: psmatch2 vs attk
>
> Rossella,
>
> It's hard to say for sure without seeing more details, but I see that you are using the -comsup- option for -attk-, but you are not using the corresponding -common- option for -psmatch2-.  Try adding that and see if it helps.
>
> Regards,
> Joe Canner
> Johns Hopkins University School of Medicine
>
> -----Original Message-----
> From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Rossella Verzulli
> Sent: Wednesday, March 26, 2014 9:58 AM
> To: statalist@hsphsun2.harvard.edu
> Subject: st: psmatch2 vs attk
>
> Dear Statalist
>
> I'm running the following commands to estimate the average treatment effects with the Kernel Matching method:
>
> psmatch2 T, out(performance) kernel kerneltype(normal) pscore(mypscore)
> attk performance T, normal pscore(mypscore) comsup
>
> Does anybody know why psmatch2 and attk produce slightly different output?
>
> I will very grateful for your advice.
>
> Best,
>
> Rossella
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/