Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
st: Output problem attnd , attr, atts
From
"Jason Zarmulski" <[email protected]>
To
[email protected]
Subject
st: Output problem attnd , attr, atts
Date
Tue, 28 Jun 2011 11:10:17 +0200
Hi Statalists,
>
>
> Now I have the problem with interpreting the different outcomes.
My Questions:
>
> -How can I Interpret the different Outcomes?
> -Why isnt the .pstest method working with these outcomes?
> -Are these Outcomes my the final ones, or do I have to do some robustness
> tests as well? (If yes, which one?)
>
>
> My new Propensity Score and the different outcomes are:
>
>
> The balancing property is satisfied
>
>
> This table shows the inferior bound, the number of treated
> and the number of controls for each block
>
> Inferior |
> of block | REL(Dummy)
> of pscore | 0 1 | Total
> -----------+----------------------+----------
> .2 | 5 6 | 11
> .4 | 16 21 | 37
> .6 | 20 17 | 37
> .7 | 14 47 | 61
> .8 | 19 110 | 129
> .9 | 9 223 | 232
> -----------+----------------------+----------
> Total | 83 424 | 507
>
>
>
> . attnd stockprice2010 reldummy, pscore(mypscore)comsup
>
>
> The program is searching the nearest neighbor of each treated unit.
> This operation may take a while.
>
>
>
> ATT estimation with Nearest Neighbor Matching method
> (random draw version)
> Analytical standard errors
>
> ---------------------------------------------------------
> n. treat. n. contr. ATT Std. Err. t
> ---------------------------------------------------------
>
> 424 65 -0.572 6.828 -0.084
>
> ---------------------------------------------------------
> Note: the numbers of treated and controls refer to actual
> nearest neighbour matches
>
>
>
> . attr stockprice2010 reldummy, pscore(mypscore) radius(0.1) comsup
>
>
> The program is searching for matches of treated units within radius.
> This operation may take a while.
>
>
>
> ATT estimation with the Radius Matching method
> Analytical standard errors
>
> ---------------------------------------------------------
> n. treat. n. contr. ATT Std. Err. t
> ---------------------------------------------------------
>
> 424 83 1.354 3.588 0.377
>
> ---------------------------------------------------------
> Note: the numbers of treated and controls refer to actual
> matches within radius
>
>
> . atts stockprice2010 reldummy, pscore(mypscore) comsup blockid(myblock)
>
>
>
>
> ATT estimation with the Stratification method
> Analytical standard errors
>
> ---------------------------------------------------------
> n. treat. n. contr. ATT Std. Err. t
> ---------------------------------------------------------
>
> 424 83 0.273 2.520 0.109
>
> ---------------------------------------------------------
>
>Thanks
Jason
>
>
>
> --- Ursprüngliche Nachrsht ---
> Von: Jan Bryla <[email protected]>
> Datum: 21.06.2011 15:04:02
> An: "[email protected]" <[email protected]>
>
> Betreff: RE: st: RE: Ousout problem psmatch2
>
> > Ah, yes. You are right - my mistake.
> >
> > Regarding -attnd- I see a few reasons why you can get different results:
>
> >
> > - using -psmatch2- you "restrict" to 10 neighbours. What is
> the
> > number og nearest neighbours selected by -attnd-?
> > - you set caliper to 0.1 using psmatch2. But I don't think a similar
> "restriction"
> > is imposed in attnd?
> >
> > Maybe this is a start...
> >
> > /Jan
> >
> >
> > -----Original Message-----
> > From: [email protected] [mailto:[email protected]]
>
> > On Behalf Of Jason Zarmulski
> > Sent: 21. juni 2011 12:23
> > To: [email protected]
> > Subject: Re: st: RE: Ousout problem psmatch2
> >
> > Hi Jan,
> >
> > thanks a lot for your support, I changed it and my new output is:
> > :-)
> >
> > . psmatch2 reldummy , outcome( stockprice2010) pscore(mypscore) neighbor(10)
>
> > caliper(0.1) ate
> > ----------------------------------------------------------------------------------------
>
> >
> > Variable Sample | Treated Controls Difference
> > S.E. T-stat
> > ----------------------------+-----------------------------------------------------------
>
> >
> > stockprice2010 Unmatched | 20.6236969 21.7278571 -1.10416018
> 2.08916252
> > -0.53
> > ATT | 20.6236969 20.4730466 .150650303
> 4.16715274
> > 0.04
> > ATU | 21.7278571 17.8625509 -3.86530621
> > . .
> > ATE | -.513405498
>
> > . .
> > ----------------------------+-----------------------------------------------------------
>
> >
> > Note: S.E. does not take into account that the propensity score is estimated.
>
> >
> >
> > | psmatch2:
> > psmatch2: | Common
> > Treatment | support
> > assignment | On suppor | Total
> > -----------+-----------+----------
> > Untreated | 84 | 84
> > Treated | 424 | 424
> > -----------+-----------+----------
> > Total | 508 | 508
> >
> >
> > To the attnd problem : I didnt install a package for attnd. Maybe it
> was
> > with the"nnmatch ado" package
> >
> > The differences between the psmatch2 and the attnd are still there....
>
> >
> > This is the help attnd file:
> >
> >
> > Calculate the average treatment effect on the treated using nearest
> neighbor
> > matching
> >
> > attnd outcome treatment [varlist] [weight] [if exp] [in
> range]
> > [ ,
> > pscore(scorevar) logit index comsup detail bootstrap
> reps(#)
> >
> > noisily dots ]
> >
> > fweights, iweights, and pweights are allowed; see help weights.
>
> >
> >
> > Description
> >
> > attnd estimates the average treatment effect on the treated (ATT)
> using
> > nearest
> > neighbor matching. attnd should be run after the correct propensity
> score
> >
> > specification; i.e., the one satisfying the balancing property has
> been
> > found
> > using, for example, pscore. If users do not provide a variable name
> for
> > the
> > propensity score, the propensity score is estimated based on the
> specification
> >
> > in varlist. Note that in this case the balancing property is not
> tested.
> >
> >
> > It is left under the responsibility of the user to select the comsup
>
> > option if
> > the user provided propensity score has been estimated on a common
> support
> > for
> > treated and controls. Otherwise, the ATT is estimated using also
> the
> >
> > observations outside the common support for which the propensity
> score
> > may not
> > be balanced.
> >
> > To save on computing time, nearest neighbors are not determined
> by comparing
> >
> > treated observations to every single control, but by first sorting
> all
> > records
> > by the estimated propensity score and then searching forward and
> backward
> > for
> > the closest control unit(s). If a treated unit forward and backward
> matches
> >
> > happen to be equally good, this program randomly draws (hence the
> letters
> > "nd"
> > for Nearest neighbor and random Draw) either the forward or backward
>
> > matches.
> > This approach is one of two computationally feasible options to
> obtain
> >
> > analytical standard errors while at the same time exploiting the
> very
> > fast
> > forward and backward search strategy. The second possibility is
> based
> > on giving
> > equal weight to the groups of forward and backward matches in case
> of
> > equally
> > good forward and backward matches and is performed by attnw. In
> practice,
> > the
> > case of multiple nearest neighbors should be very rare. In particular,
>
> > if the
> > set of X's contains continuous variables, in which case, both attnd
> and
> > attnw
> > should give equal results (except for bootstrapped standard errors).
>
> > The
> > likelihood of multiple nearest neighbors is further reduced if the
> propensity
> >
> > score is estimated and saved in double precision, which is what
> pscore
> > does by
> > default.
> >
> > The ATT is computed by averaging over the unit-level treatment effects
>
> > of the
> > treated where the control(s) matched to a treated observation is/are
>
> > those
> > observations in the control group that have the closest propensity
> score.
> > If
> > there are multiple nearest neighbors, the average outcome of those
> controls
> > is
> > used.
> >
> >
> > Options
> >
> > pscore(scorevar) specifies the name of the user-provided variable
> name
> > for the
> > estimated propensity score. If no name is provided the propensity
>
> > score is
> > estimated based on the specification in varlist.
> >
> > logit uses a logit model to estimate the propensity score instead
> of
> > the default
> > probit model when the option pscore(scorevar) is not specified
> by
> > the user.
> > Otherwise, no effect is produced.
> >
> > index requires the use of the linear index as the propensity score
> when
> > the
> > option pscore(scorevar) is not specified by the user. Otherwise,
>
> > no effect
> > is produced.
> >
> > comsup restricts the computation of the ATT to the region of common
> support.
> >
> >
> > detail displays more detailed output documenting the steps performed
>
> > to obtain
> > the final results.
> >
> > bootstrap bootstraps the standard error of the treatment effect.
>
> >
> > reps(#) specifies the number of bootstrap replications to be performed.
>
> > The
> > default is 50. This option produces an effect only if the bootstrap
>
> > option
> > is specified.
> >
> > noisily requests that any output from the replications be displayed.
>
> > This
> > option produces an effect only if the bootstrap option is specified.
>
> >
> >
> > dots requests that a dot be placed on the screen at the beginning
> of
> > each
> > replication. This option produces an effect only if the bootstrap
>
> > option is
> > specified.
> >
> >
> > Remarks
> >
> > Please remember to use the update query command before running this
> program
> > to
> > make sure you have an up-to-date version of Stata installed. Otherwise,
>
> > this
> > program may not run properly.
> >
> > The treatment has to be binary.
> >
> > When users do not specify their own previously estimated propensity
> score,
> > the
> > bootstrap encompasses the estimation of the propensity score based
> on
> > the
> > specification given by varlist. This procedure is actually recommended
>
> > to
> > account for the uncertainty associated with the estimation of the
> propensity
> >
> > score. Even more so when the comsup option is specified because
> in this
> > case
> > the region of common support changes with every bootstrap sample,
> and
> >
> > bootstrapped standard errors pick up this uncertainty as well. So,
> typically
> >
> > users would first identify a specification satisfying the balancing
> property
> > --
> > using pscore -- and then provide exactly this specification in varlist
>
> > and use
> > bootstrapped standard errors.
> >
> >
> > Saved results
> >
> > The program stores the estimated treatment effect, its standard
> error,
> > and the t
> > statistic respectively in the scalars r(attnd), r(seattnd), and
> r(tsattnd).
> >
> >
> > The number of treated and the number of controls are stored respectively
>
> > in the
> > scalars r(ntnd) and r(ncnd).
> >
> > The bootstrapped standard error and t statistic are stored respectively
>
> > in the
> > scalars r(bseattnd) and r(btsattnd).
> >
> >
> > Examples
> >
> > . attnd wage training age age2 exp exp2
> >
> > . attnd wage training age age2 exp exp2, boot reps(100) dots
> >
> > . attnd wage training age age2 exp exp2, logit boot reps(100)
> >
> > . attnd wage training age age2 exp exp2, comsup boot reps(100)
> >
> >
> > Authors
> >
> > Sascha O. Becker
> > Center for Economic Studies, University of Munich
> >
> > Andrea Ichino
> > Department of Economics, European University Institute, Florence
>
> >
> >
> > Email [email protected] or [email protected] if you observe any problems.
>
> >
> >
> >
> > Acknowledgments
> >
> > The way to implement the propensity score estimation in the bootstrap
>
> > procedure
> > has been adapted from the psmatch program written by Barbara Sianesi
>
> > (University
> > College London and Institute for Fiscal Studies) Email: [email protected].
>
> >
> >
> >
> > Also see
> >
> > Online: help for pscore, atts, attr, attk, attnw (if installed),
> and
> > bs.
> >
> > Further details on the analytical formulas and on the
> algorithms
> > used
> > in these programs can be found under http://www.sobecker.de
>
> > or
> > http://www.iue.it/Personal/Ichino.
> >
> > Thanks
> > Jason
> >
> >
> > --- Ursprüngliche Nachricht ---
> > Von: Jan Bryla <[email protected]>
> > Datum: 20.06.2011 20:14:03
> > An: "[email protected]" <[email protected]>
>
> >
> > Betreff: st: RE: Ousout problem psmatch2
> >
> > > Jason, your first question seems easy to solve: I think your treatment
>
> > variable
> > > and the outcome variable are identical. Recall the syntax for psmatch2,
>
> > see
> > > -help psmatch2-.
> > >
> > > The second point left me a bit confused. Searching for -attnd-
> using
> > -findit
> > > attnd- didn't really turn out any hints. Is -attnd- available from
> SSC?
> > Maybe
> > > you can clarify your steps there? Maybe differences are due to
> the issue
> >
> > > with the first question.
> > >
> > > Hope it helps
> > > Jan Bryla
> > >
> > >
> > > -----Original Message-----
> > > From: [email protected] [mailto:[email protected]]
>
> >
> > > On Behalf Of Jason Zarmulski
> > > Sent: 20. juni 2011 16:45
> > > To: [email protected]
> > > Subject: st: Ousout problem psmatch2
> > >
> > > Dear Statalists,
> > > I got two problems, first one:
> > >
> > > I have a problem understanding the Output of my psmatch2 results.
> I'm
> > new
> > > to
> > > this so it could be a trivial error, but I'm not sure. I wanted
> to do
> > a
> > > nearest neighbor matching with replacement.
> > > My results are:
> > >
> > > . psmatch2 stockprice2010, outcome( stockprice2010) pscore(mypscore)
>
> >
> > > neighbor(10) caliper(0.1) ate
> > > ----------------------------------------------------------------------------------------
>
> >
> > >
> > > Variable Sample | Treated Controls Difference
>
> > > S.E. T-stat
> > > ----------------------------+-----------------------------------------------------------
>
> >
> > >
> > > stockprice2010 Unmatched | 1.628 .479999997 .99964329
>
> > > .000849677 1176.50
> > > ATT | 1.62 .689999992 .930000013
>
> >
> > > .399478823 2.33
> > > ATU | .479999997 1.79500002 1.31500002
>
> > > . .
> > > ATE | 1.05833335
>
> >
> > > . .
> > > ----------------------------+-----------------------------------------------------------
>
> >
> > >
> > > Note: S.E. does not take into account that the propensity score
> is
> > > estimated.
> > >
> > > psmatch2: | psmatch2: Common
> > > Treatment | support
> > > assignment | Off suppo On suppor | Total
> > > -----------+----------------------+----------
> > > Untreated | 0 2 | 2
> > > Treated | 1 4 | 5
> > > 2 | 0 11 | 11
> > > 3 | 0 12 | 12
> > > 4 | 0 17 | 17
> > > 5 | 0 34 | 34
> > > 6 | 0 15 | 15
> > > 7 | 0 19 | 19
> > > 8 | 0 18 | 18
> > > 9 | 0 17 | 17
> > > 10 | 0 18 | 18
> > > 11 | 0 13 | 13
> > > 12 | 0 13 | 13
> > > 13 | 0 20 | 20
> > > 14 | 0 23 | 23
> > > 15 | 0 23 | 23
> > > 16 | 0 10 | 10
> > > 17 | 0 13 | 13
> > > 18 | 0 10 | 10
> > > 19 | 0 11 | 11
> > > 20 | 0 11 | 11
> > > 21 | 0 10 | 10
> > > 22 | 0 10 | 10
> > > 23 | 0 11 | 11
> > > 24 | 0 10 | 10
> > > 25 | 0 7 | 7
> > > 26 | 0 9 | 9
> > > 27 | 0 6 | 6
> > > 28 | 0 7 | 7
> > > 29 | 0 4 | 4
> > > 30 | 0 7 | 7
> > > 31 | 0 6 | 6
> > > 32 | 0 7 | 7
> > > 33 | 0 7 | 7
> > > 34 | 0 5 | 5
> > > 35 | 0 6 | 6
> > > 36 | 0 6 | 6
> > > 37 | 0 6 | 6
> > > 38 | 0 5 | 5
> > > 39 | 0 4 | 4
> > > 40 | 0 4 | 4
> > > 41 | 0 9 | 9
> > > 42 | 0 7 | 7
> > > 43 | 0 5 | 5
> > > 44 | 0 1 | 1
> > > 45 | 0 1 | 1
> > > 46 | 0 1 | 1
> > > 47 | 0 1 | 1
> > > 48 | 0 3 | 3
> > > 49 | 0 4 | 4
> > > 50 | 0 1 | 1
> > > 51 | 0 1 | 1
> > > 52 | 0 1 | 1
> > > 53 | 0 1 | 1
> > > 55 | 0 1 | 1
> > > 56 | 0 3 | 3
> > > 57 | 0 1 | 1
> > > 59 | 0 1 | 1
> > > 60 | 0 2 | 2
> > > 62 | 0 1 | 1
> > > 63 | 0 2 | 2
> > > 69 | 0 1 | 1
> > > 70 | 0 1 | 1
> > > 78 | 0 1 | 1
> > > 83 | 0 1 | 1
> > > 84 | 0 2 | 2
> > > 92 | 0 1 | 1
> > > -----------+----------------------+----------
> > > Total | 1 505 | 506
> > >
> > > Question: Why arent there only the untreated and treated in the
> table
> > and
> > >
> > > what ist the meaning of the numbers 2-92?
> > >
> > >
> > > Second problem:
> > >
> > > I've done the nearest neighbor matching of the same data sample
> with
> > the
> > >
> > > attnd function.
> > > My results are:
> > >
> > > . attnd stockprice2010 reldummy, pscore(mypscore)
> > >
> > >
> > > The program is searching the nearest neighbor of each treated
> unit.
> >
> > > This operation may take a while.
> > >
> > >
> > >
> > > ATT estimation with Nearest Neighbor Matching method
> > > (random draw version)
> > > Analytical standard errors
> > >
> > > ---------------------------------------------------------
> > > n. treat. n. contr. ATT Std. Err. t
> > > ---------------------------------------------------------
> > >
> > > 424 63 -2.255 6.070 -0.371
> > >
> > > ---------------------------------------------------------
> > > Note: the numbers of treated and controls refer to actual
> > > nearest neighbour matches
> > >
> > > Why is this result for ATT so different to the one of psmatch2?
>
> > > Is the attnd matching with replacement?
> > >
> > > Any suggestions would be much appreciated
> > >
> > > Thanks
> > >
> > > Jason
> > >
> > > --
> > > View this message in context: http://statalist.1588530.n2.nabble.com/Ousout-problem-psmatch2-tp6496035p6496035.html
>
> >
> > >
> > > Sent from the Statalist mailing list archive at Nabble.com.
> > > *
> > > * For searches and help try:
> > > * http://www.stata.com/help.cgi?search
> > > * http://www.stata.com/support/statalist/faq
> > > * http://www.ats.ucla.edu/stat/stata/
> > >
> > > *
> > > * For searches and help try:
> > > * http://www.stata.com/help.cgi?search
> > > * http://www.stata.com/support/statalist/faq
> > > * http://www.ats.ucla.edu/stat/stata/
> >
> > *
> > * For searches and help try:
> > * http://www.stata.com/help.cgi?search
> > * http://www.stata.com/support/statalist/faq
> > * http://www.ats.ucla.edu/stat/stata/
> >
> > *
> > * For searches and help try:
> > * http://www.stata.com/help.cgi?search
> > * http://www.stata.com/support/statalist/faq
> > * http://www.ats.ucla.edu/stat/stata/
>
> *
> * For searches and help try:
> * http://www.stata.com/help.cgi?search
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/