Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: st: rbounds Hodges-Lehmann point estimates and ATT estimates
From
Richard Palmer-Jones <[email protected]>
To
[email protected]
Subject
Re: st: rbounds Hodges-Lehmann point estimates and ATT estimates
Date
Fri, 2 Jul 2010 09:15:21 +0100
Steve (and any others interested)
Reading about sensitivity analysis of PSM estimates suggests that it
is a common view that the Hodges-Lehmann point estimates of difference
which are based on medians are expected to be smaller that
differences in means which is what the AT is based on. Are you
convinced? Of course we know that medians are less sensitive to
outliers, but why whould that make differences in medians between
treatments smaller?
I have noted that the HL estimates vary considerably with the
algorithm used to compute pscores even though it does not appear that
there are big differences in the actual matches, or the ATTs. Again,
an insight into this would be appreciated.
Richard
On Thu, Jul 1, 2010 at 10:39 PM, Steve Samuels <[email protected]> wrote:
> Quite right, Richard. I should really read the command documentation!
> The difference you note does not appear to be a difference of
> consequence, because the CI's for ATT from -psmatch2- and
> from-rbounds- overlap substantially and are very wide.
>
> Steve
>
>
>
>
> On Thu, Jul 1, 2010 at 4:52 PM, Richard Palmer-Jones
> <[email protected]> wrote:
>> Thanks, Steve
>>
>> But, unfortunately this is not the problem. he ATT is calculated for
>> the treatment observations - 185, and these are used in the rbounds
>> estimate of the Hodges-Lehman point estimates.
>>
>> Any other ideas?
>>
>> Richard
>> .
>>
>>
>>
>>
>> On Thu, Jul 1, 2010 at 5:45 PM, Steve Samuels <[email protected]> wrote:
>>> The N's are different. -psmatch2- has n= 2675. -bounds reports 185
>>> matched pairs.
>>> Your "diff" variable was set to missing for 2490 observations.
>>>
>>> Steve
>>>
>>> On Thu, Jul 1, 2010 at 12:06 PM, Richard Palmer-Jones
>>> <[email protected]> wrote:
>>>> Dear Readers
>>>>
>>>> When I run rbounds after psmatch2 I find that the Hodges-Lehman
>>>> minimum and maximum point estimates of impact are (generally)
>>>> substantially different to the estimated ATT when the Gamma =1. Could
>>>> someone explain this?
>>>>
>>>> . use lalonde.dta
>>>>
>>>>
>>>>
>>>> . psmatch2 t age age2 educ educ2 black hisp marr re74 u74 re74 re75 ,
>>>> outcome(re78)
>>>> note: re74 dropped because of collinearity
>>>>
>>>> Probit regression Number of obs = 2675
>>>> LR chi2(10) = 882.99
>>>> Prob > chi2 = 0.0000
>>>> Log likelihood = -231.1534 Pseudo R2 = 0.6564
>>>>
>>>> ------------------------------------------------------------------------------
>>>> t | Coef. Std. Err. z P>|z| [95% Conf. Interval]
>>>> -------------+----------------------------------------------------------------
>>>> age | .165835 .0597947 2.77 0.006 .0486395 .2830305
>>>> age2 | -.0031243 .0009159 -3.41 0.001 -.0049195 -.001329
>>>> educ | .4034187 .1637615 2.46 0.014 .082452 .7243854
>>>> educ2 | -.0233423 .0081187 -2.88 0.004 -.0392546 -.0074299
>>>> black | 1.179666 .1692016 6.97 0.000 .8480369 1.511295
>>>> hisp | 1.200164 .3092023 3.88 0.000 .5941388 1.80619
>>>> marr | -1.000713 .1456276 -6.87 0.000 -1.286138 -.715288
>>>> re74 | -.0000509 .0000141 -3.62 0.000 -.0000784 -.0000233
>>>> u74 | .326188 .1842919 1.77 0.077 -.0350175 .6873935
>>>> re75 | -.000103 .0000205 -5.03 0.000 -.0001431 -.0000629
>>>> _cons | -4.082649 1.221929 -3.34 0.001 -6.477585 -1.687713
>>>> ------------------------------------------------------------------------------
>>>> Note: 659 failures and 0 successes completely determined.
>>>> There are observations with identical propensity score values.
>>>> The sort order of the data could affect your results.
>>>> Make sure that the sort order is random before calling psmatch2.
>>>> ----------------------------------------------------------------------------------------
>>>> Variable Sample | Treated Controls Difference
>>>> S.E. T-stat
>>>> ----------------------------+-----------------------------------------------------------
>>>> re78 Unmatched | 6349.14537 21553.9213 -15204.7759
>>>> 1154.61435 -13.17
>>>> ATT | 6349.14537 5387.78028 961.365096
>>>> 1420.27513 0.68
>>>> ----------------------------+-----------------------------------------------------------
>>>> Note: S.E. for ATT does not take into account that the propensity
>>>> score is estimated.
>>>>
>>>> | psmatch2:
>>>> psmatch2: | Common
>>>> Treatment | support
>>>> assignment | On suppor | Total
>>>> -----------+-----------+----------
>>>> Untreated | 2,490 | 2,490
>>>> Treated | 185 | 185
>>>> -----------+-----------+----------
>>>> Total | 2,675 | 2,675
>>>>
>>>>
>>>> . gen diff = re78- _re78
>>>> (2490 missing values generated)
>>>>
>>>> . rbounds diff, gamma(1(.2)3)
>>>>
>>>> Rosenbaum bounds for diff (N = 185 matched pairs)
>>>>
>>>> Gamma sig+ sig- t-hat+ t-hat- CI+ CI-
>>>> ----------------------------------------------------------------------
>>>> 1 .203558 .203558 498.13 498.13 -618.815 1671.48
>>>> 1.2 .594682 .028425 -109.352 1108.27 -1261.98 2361.57
>>>> 1.4 .873746 .002358 -622.61 1680.15 -1795.44 2936.95
>>>> 1.6 .973397 .000138 -1113.6 2160.85 -2305.18 3440.96
>>>> 1.8 .995808 6.3e-06 -1507.8 2567.3 -2722.57 3920.69
>>>> 2 .999467 2.4e-07 -1811.98 2961.7 -3098.75 4317.7
>>>> 2.2 .999942 8.3e-09 -2171.23 3265 -3463.81 4741.49
>>>> 2.4 .999995 2.6e-10 -2410.04 3622.82 -3749.03 5057.69
>>>> 2.6 1 7.3e-12 -2717.38 3919.67 -4063.75 5380.66
>>>> 2.8 1 2.0e-13 -2960.24 4167 -4308.32 5657.37
>>>> 3 1 5.1e-15 -3195.96 4413.54 -4566.85 5982.9
>>>>
>>>> * gamma - log odds of differential assignment due to unobserved factors
>>>> sig+ - upper bound significance level
>>>> sig- - lower bound significance level
>>>> t-hat+ - upper bound Hodges-Lehmann point estimate
>>>> t-hat- - lower bound Hodges-Lehmann point estimate
>>>> CI+ - upper bound confidence interval (a= .95)
>>>> CI- - lower bound confidence interval (a= .95)
>>>>
>>>> .
>>>> end of do-file
>>>>
>>>> My problem is:
>>>>
>>>> ATT = 961
>>>> Gammat-hat+ t-hat-
>>>> 1 498.13 498.13
>>>> *
>>>> * For searches and help try:
>>>> * http://www.stata.com/help.cgi?search
>>>> * http://www.stata.com/support/statalist/faq
>>>> * http://www.ats.ucla.edu/stat/stata/
>>>>
>>>
>>>
>>>
>>> --
>>> Steven Samuels
>>> [email protected]
>>> 18 Cantine's Island
>>> Saugerties NY 12477
>>> USA
>>> Voice: 845-246-0774
>>> Fax: 206-202-4783
>>>
>>> *
>>> * For searches and help try:
>>> * http://www.stata.com/help.cgi?search
>>> * http://www.stata.com/support/statalist/faq
>>> * http://www.ats.ucla.edu/stat/stata/
>>>
>>
>> *
>> * For searches and help try:
>> * http://www.stata.com/help.cgi?search
>> * http://www.stata.com/support/statalist/faq
>> * http://www.ats.ucla.edu/stat/stata/
>>
>
>
>
> --
> Steven Samuels
> [email protected]
> 18 Cantine's Island
> Saugerties NY 12477
> USA
> Voice: 845-246-0774
> Fax: 206-202-4783
>
> *
> * For searches and help try:
> * http://www.stata.com/help.cgi?search
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
>
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/