RE: st: Pstest and Sensatt

["Radwin, David" <dradwin@rti.org>]

Isobel,

Regarding t-tests as an indication of balance, please see
http://www.stata.com/statalist/archive/2013-08/msg00629.html and the
article linked therein.

David
--
David Radwin, Senior Research Associate
Education and Workforce Development
RTI International
2150 Shattuck Ave. Suite 800, Berkeley, CA 94704
Phone: 510-665-8274

www.rti.org/education


> -----Original Message-----
> From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-
> statalist@hsphsun2.harvard.edu] On Behalf Of Fernando Rios Avila
> Sent: Tuesday, March 04, 2014 5:24 AM
> To: statalist@hsphsun2.harvard.edu
> Subject: Re: st: Pstest and Sensatt
> 
> Hi Isobel,
> So, Im not familiar with the sensatt, but regarding the pstest, keep
> in mind that what you want is either small absolute biases (after
> matching) and/or non significant t-stats.
> The t stats reported are the test for whether or not any particular
> variable is statistically different between control and treatment
> sample, after matching the information. The rule of thumb should be
> the same as when you read the significance of a coefficient on a
> regression (they should be smaller than 2).
> Regarding the bias, i would not pay attention on the bias change or
> bias reduction, but the absolute bias after matching. The rule of
> thumb here is usually to have all biases after match lower than 20%.
> Ideally all your variables should have small biases, or small t-stats,
> to "accept" the model. If not, you need to modify the specification of
> the propensity score model.
> HTH
> Fernando
> 
> On Tue, Mar 4, 2014 at 4:54 AM, Isobel Williams
<iwilliams24@hotmail.com>
> wrote:
> > Dear All,
> >
> > I am having some trouble interpreting my -pstest- (a user-written
SSC
> programme) results. Most of my t-statistics are insignificant, and the
> most absolute biases are at less than 10% after matching. The mean
> absolute bias is 3.8% after matching. Is this permissible in the
model? I
> am using 80+ parameters so I thought that a small bias of 3.8% and 24
> significant t-statistics would be tolerable in the model. However, of
all
> of these 24 significant t-statistics, the highest absolute bias level
is
> 14%
> >
> > I am also trying to carry out the sensatt test (also a user-written
SSC
> programme by Becker and Ichino, 2002) on the model, however I always
get
> the error message 'matsize too small'. I have the IC13.1; I set the
> matsize to 800 and I am still getting that error message. Is there any
> other way I can carry out the test?
> >
> > Many thanks,
> > Isobel Williams



*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/