Claire--
I'm sure I did not read your post carefully enough, but are you
comparing the means of outcomes across matched obs for both treatment
and control groups? The ATT looks only at treatment cases and their
matches.
On Jan 11, 2008 12:29 PM, Claire Kamp Dush <[email protected]> wrote:
> Dear Statalist,
> I have a question regarding the interpretation of the psmatch2 output. I am trying to compare the change in mental health before and after a divorce as compared to a cohabitation dissolution (separating when you are living together). First, I do a probit predicting a dichotomous indicator of whether one dissolves a cohabitation or a marriage among those that dissolve a union (marcohdis23) from race, education, etcetera. I save the predicted probability of dissolution (ppmarcohdis23). Then, I do a change score of depressive symptoms over the two waves in between which the dissolutions occur (changemdepc). I then set the seed and use this pscore in the psmatch2 command. So, the command I run is as follows:
>
> psmatch2 marcohdis23, out(changemdepc) pscore(ppmarcohdis23) neighbor(1) caliper(0.03) common
>
> I then run the pstest command to calculate the t-tests for equality of means in the treated and non-treated groups after matching. So the second command I run is:
> pstest changemdepc, treated(_treated) support(_support)
>
> My question regards the output. As you can see below (my commands and output are pasted below), the output for the psmatch2 command includes a T-stat. This T-stat is reported as -1.28 for the unmatched sample and -0.85 for the matched sample. On the other hand, the t-test reported by the pstest command is the same for the unmatched sample, -1.28, but different for the matched sample, now -2.14.
>
> I hand calculated the t-statistic using the psmatch2 generated variable _changemdepc which is the value of changemdepc for the on matches of the on support, treated group. I assumed 268 * 2 people in my sample, and use the mean and variance from each group (as obtained via the sum, detail command) to calculate the t-statistic using ttesti. Doing this, I come up with a t-test value of -2.14 as well. Therefore, I know where the -2.14 comes from, as I reproduced this number myself. However, I do not know where the -0.85 number comes from. Can anyone shed any light on this? I would like to report my results, but I am worried that the -0.85 number may take into account some kind of bias that the pstest command does not. So, I do not know which t-statistic to report. Any help would be much appreciated.
>
> Best,
> Claire Kamp Dush
>
>
> Results
>
> . psmatch2 marcohdis23, out(changemdepc) pscore(ppmarcohdis23) neighbor(1) caliper(0.03) common
>
> There are observations with identical propensity score values.
> The sort order of the data could affect your results.
> Make sure that the sort order is random before calling psmatch2.
>
> ----------------------------------------------------------------------------Variable Sample |Treated Controls Difference S.E. T-stat
> -----------------------+----------------------------------------------------
> changemdepc Unmatched | .46129 .87179 -.41050 .31977 -1.28
> ATT | .45149 .95896 -.50746 .59478 -0.85
> -----------------------+----------------------------------------------------Note: S.E. for ATT does not take into account that the propensity score is estimated.
>
> psmatch2: | psmatch2: Common
> Treatment | support
> assignment | Off suppo On suppor | Total
> -----------+----------------------+----------
> Untreated | 0 78 | 78
> Treated | 42 268 | 310
> -----------+----------------------+----------
> Total | 42 346 | 388
>
>
> . pstest changemdepc, treated(_treated) support(_support)
>
> ----------------------------------------------------------------------------
> | Mean %reduct | t-test
> Variable Sample | Treated Control %bias |bias| | t p>|t|
> ------------------------+----------------------------------+----------------
> changemdepc Unmatched | .46129 .87179 -15.4 | -1.28 0.200
> Matched | .45149 .95896 -19.0 -23.6 | -2.14 0.033
> | |
> ----------------------------------------------------------------------------
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
