Re: st: RE: Propensity score balancing property

[Orrin Pail <orrinpail@gmail.com>]

Thank you both for your replies!

I'm a novice using stata and with statistics in general. I was
wondering if you would be able to give me a little more guidance as to
how to perform the actual test?

With the pscore, I was using the initial command (above) to generate
the propensity scores, and then I wanted to use the nearest neighbor
method using this command:

attnd $ylist $treatment $xlist, pscore(myscore) comsup boot reps($breps) dots

With psmatch2, how would I go about doing the entire analysis? Does it
take two steps like in pscore?

And with pstest, I guess I have the same question. Unless a different
method would be better for nearest neighbor.

FYI: my data set contains 14,000 untreated observations and 800
treated observations. It seamed like nearest neighbor would be most
appropriate, but I'm open to other ideas as well...

Thanks again!

Orrin


On Tue, Dec 31, 2013 at 3:43 PM, Adam Olszewski
<adam.olszewski@gmail.com> wrote:
> I believe that -pscore- performs the propensity score analysis using
> the stratification method. It will estimate the score and then
> subdivide the population into "blocks" (typically 5 quintiles). It
> will then require that balance of covariates be achieved within each
> stratum. Unless you have a large number of observations, this may not
> be achieveable at all, although the p-value of 0.01 is actually quite
> relaxed (and you could lower it even further, but it is picky). It is
> very difficult to adjust the PS model with this method, because you
> have to keep track of balance in each stratum separately.
> This is altogether not a very popular way of doing the propensity
> score matching nowadays. There are various matching and weighting
> methods that may be more attractive, although they have their own
> weaknesses: matching may discard a number of observations unless
> performed carefully, and weighting may produce unrealistic, distorted
> pseudopopulation if the PS model is misspecified or there are major
> outliers. The -pstest- command can accomodate an alternative test of
> balance that is not stratified, which will likely get rid of your
> problem. You should however use the balance check that is appropriate
> for your PS methodology. In any case, as widely discussed in the
> relevant literature, balance tests that rely on sample-size dependent
> statistics (t-tests, chi2-tests etc.) are not really the best
> approach. Using standardized differences of means and proportions (see
> e.g. the user-written -pbalchk- command) and particularly a thorough
> assessment of cumulative distributions may be more appropriate, even
> though it requires more work then just running a series of t-tests.
> I hope that might help you design your study better.
> Best,
> AO
>
> On Tue, Dec 31, 2013 at 3:19 PM, Joe Canner <jcanner1@jhmi.edu> wrote:
>> Orrin,
>>
>> I have had this same problem with -pscore-.  My gut feeling is that it is using an overly-conservative definition of "balance", although I'm not sure how to prove such an accusation.  In any case, you may want to try -psmatch2- (also available from SSC) which seems to have a more realistic view of balance.
>>
>> Regards,
>> Joe Canner
>> Johns Hopkins University School of Medicine
>> ________________________________________
>> From: owner-statalist@hsphsun2.harvard.edu [owner-statalist@hsphsun2.harvard.edu] on behalf of Orrin Pail [orrinpail@gmail.com]
>> Sent: Tuesday, December 31, 2013 2:46 PM
>> To: statalist@hsphsun2.harvard.edu
>> Subject: st: Propensity score balancing property
>>
>> Hello,
>>
>> I am trying to use pscore for propensity matching analysis and I am
>> having difficulties in satisfying the balancing property.
>>
>> My covariates (xlist below) include 8 different variables. I have been
>> trying different combinations of them, sometimes removing some, adding
>> others. I also tried including interaction terms into my xlist but
>> stata says that they are not allowed. No matter what I do, my
>> balancing property is not being met.
>>
>> The closest I get to satisfying the balancing property is when I use
>> five of the variables. In this case, the output says that the final
>> number of blocks is 8 and that three of the variables (they are listed
>> separately) are not balanced in block 7.
>>
>> Does anyone have any recommendations on how I can satisfy the
>> balancing property besides adding more variables? From what I
>> understand, my propensity score analysis would be useless without the
>> balancing property being met, so I would appreciate all the help!
>>
>> The command I am using is below (I defined treatment and xlist prior
>> to this command):
>>
>> pscore $treatment $xlist, pscore(myscore) blockid(myblock) comsup
>>
>> Thanks!
>>
>> Orrin
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/faqs/resources/statalist-faq/
>> *   http://www.ats.ucla.edu/stat/stata/
>>
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/faqs/resources/statalist-faq/
>> *   http://www.ats.ucla.edu/stat/stata/
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/