Re: st: RE: Propensity score balancing property

[Adam Olszewski <adam.olszewski@gmail.com>]

I believe that -pscore- performs the propensity score analysis using
the stratification method. It will estimate the score and then
subdivide the population into "blocks" (typically 5 quintiles). It
will then require that balance of covariates be achieved within each
stratum. Unless you have a large number of observations, this may not
be achieveable at all, although the p-value of 0.01 is actually quite
relaxed (and you could lower it even further, but it is picky). It is
very difficult to adjust the PS model with this method, because you
have to keep track of balance in each stratum separately.
This is altogether not a very popular way of doing the propensity
score matching nowadays. There are various matching and weighting
methods that may be more attractive, although they have their own
weaknesses: matching may discard a number of observations unless
performed carefully, and weighting may produce unrealistic, distorted
pseudopopulation if the PS model is misspecified or there are major
outliers. The -pstest- command can accomodate an alternative test of
balance that is not stratified, which will likely get rid of your
problem. You should however use the balance check that is appropriate
for your PS methodology. In any case, as widely discussed in the
relevant literature, balance tests that rely on sample-size dependent
statistics (t-tests, chi2-tests etc.) are not really the best
approach. Using standardized differences of means and proportions (see
e.g. the user-written -pbalchk- command) and particularly a thorough
assessment of cumulative distributions may be more appropriate, even
though it requires more work then just running a series of t-tests.
I hope that might help you design your study better.
Best,
AO

On Tue, Dec 31, 2013 at 3:19 PM, Joe Canner <jcanner1@jhmi.edu> wrote:
> Orrin,
>
> I have had this same problem with -pscore-.  My gut feeling is that it is using an overly-conservative definition of "balance", although I'm not sure how to prove such an accusation.  In any case, you may want to try -psmatch2- (also available from SSC) which seems to have a more realistic view of balance.
>
> Regards,
> Joe Canner
> Johns Hopkins University School of Medicine
> ________________________________________
> From: owner-statalist@hsphsun2.harvard.edu [owner-statalist@hsphsun2.harvard.edu] on behalf of Orrin Pail [orrinpail@gmail.com]
> Sent: Tuesday, December 31, 2013 2:46 PM
> To: statalist@hsphsun2.harvard.edu
> Subject: st: Propensity score balancing property
>
> Hello,
>
> I am trying to use pscore for propensity matching analysis and I am
> having difficulties in satisfying the balancing property.
>
> My covariates (xlist below) include 8 different variables. I have been
> trying different combinations of them, sometimes removing some, adding
> others. I also tried including interaction terms into my xlist but
> stata says that they are not allowed. No matter what I do, my
> balancing property is not being met.
>
> The closest I get to satisfying the balancing property is when I use
> five of the variables. In this case, the output says that the final
> number of blocks is 8 and that three of the variables (they are listed
> separately) are not balanced in block 7.
>
> Does anyone have any recommendations on how I can satisfy the
> balancing property besides adding more variables? From what I
> understand, my propensity score analysis would be useless without the
> balancing property being met, so I would appreciate all the help!
>
> The command I am using is below (I defined treatment and xlist prior
> to this command):
>
> pscore $treatment $xlist, pscore(myscore) blockid(myblock) comsup
>
> Thanks!
>
> Orrin
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/