Re: st: RE: MCNEMAR test or Average treatment effects in matched data.

[mccali mccalister <mccali85@hotmail.com>]

Dear Ariel,
About the first question. I got it clear. As I said in my first thread I checked the balance of the covariates using ps test, I meant using standarised mean differences. About the variables included, I only included variables known to be related to both treatment assignment and the outcome(RubinandThomas,1996;Heckman,IchimuraandTodd,1998;Glazerman,LevyandMyers,2003; Hill,ReiterandZanutto,2004). But  i guess this issue still in debate. 
So i still have don't know how to do, to show all these variables that are not related to the outcome nor  to the treatment assignment. These may be interesting to the readers to know. How should should I show these results? Should i include them in table 1 before and after the match? But without showing the p values? How can they know that there are not statistical differences?

I may be new in this issue but that does not mean that  I just run stata. I am aware that  it's me who does the research and stata only calculate. I only want to make sure that what I do is correct.

Best regards


Enviado desde mi iPad

> El 08/01/2014, a las 17:02, "Ariel Linden" <ariel.linden@gmail.com> escribió:
> 
> Let me weigh in here, since I was the poster of that previous thread on this
> subject (http://www.stata.com/statalist/archive/2012-08/msg00985.html)
> 
> The questions that are being asked here are somewhat difficult to follow, so
> I'll attempt to answer them, given what I understand:
> 
> " So if I understood well, you simply would use the log.regression to see
> the effect of the my treatment. What about the average treatment effects?
> Are they usefull? In my case?(the incidence of mediastinitis is the outcome
> ) Maybe the interpretaion in medical issues is not so clear?So should I
> avoid it?"
> 
> If your outcome is binary, then yes, you can use a logistic regression.
> Which type of treatment effect you'll be estimating depends on how you do
> your matching/weighting. In general, matching untreated individuals to
> treated individuals provides an average treatment effect on the treated
> (ATT). Why, because you are choosing untreated units that have the same
> characteristics as the treated. You can use weighting strategies instead of
> matching, and choose your treatment effect estimator, if you are not happy
> with the matching approach.
> 
> " It is very frequent that many paper using propensity score (not necessary
> should  be right)  they show the new propensity score groups that they are
> balanaced using parametric tests or non parametric test(even then they use
> standarizes mean differences). What would you advice me to use if I want to
> compare baseline caracteristic of my population  before and after the
> propensity score ? Note that I want to show all the baseline variables and
> many of them are not included in the propensity score model. Should. I
> simply use parametric tests weighted by psmatch2?"
> 
> As common practice, you should show a "table 1" before and after matching.
> This will provide "evidence" that matching resulted in balance on observed
> pre-intervention characteristics. As per my previous thread, you should NOT
> be using t-tests to assess balance. This is not inferential statistics, and
> thus we are not attempting to draw assumptions to a population. Moreover,
> sample size will influence the statistics, which we hope to avoid. You
> should use standardized differences for calculating balance, with the intent
> of reducing the value as far as possible (down to zero). You should also
> review Q-Q plots, box plots, density plots, etc. to visually inspect the
> distributions of each covariate.
> 
> Lastly, you write that you want to show baseline variables that are not
> included in the propensity score model? Why would you not include them in
> the model? If you read the Elizabeth Stuart article that Joe referenced, you
> will see that the rule of thumb is to include as many covariates as
> available into the propensity score. This increases the likelihood that
> you've balanced all observable characteristics, thereby reducing the
> possibility of bias from omitted variables.
> 
> In summary, you should spend some time reading about these issues. Blindly
> running statistics without a firm understanding of the underlying principles
> is not a recommended practice for researchers.
> 
> I hope this helps
> 
> Ariel
> 
> 
> Date: Tue, 7 Jan 2014 17:28:37 +0000
> From: Joe Canner <jcanner1@jhmi.edu>
> Subject: RE: st: RE: MCNEMAR test or Average treatment effects in matched
> data.
> 
> Dr. Ayaon,
> 
> I am not an expert on propensity score matching; I have used it occasionally
> and was sharing a few insights and experiences since no one else had
> replied.  I would recommend that you familiarize yourself with the recent
> literature on the subject.  A good start would be:  Stuart, E.A. (2010).
> Matching Methods for Causal Inference: A review and a look forward.
> Statistical Science 25(1): 1-21.
> 
> In a recent thread on this subject, someone recommended against checking for
> balance using methods like t-tests which are dependent on sample size.  This
> is certainly a useful caution, depending on the size of your study.  If
> balance is determined by not rejecting the null hypothesis of no difference,
> you need to be careful that this result is due to the lack of difference
> between the groups rather than to insufficient sample size.  If sample size
> is an issue for you in this regard, then perhaps you can try a
> non-parametric test.
> 
> Regards,
> Joe
> ________________________________________
> From: owner-statalist@hsphsun2.harvard.edu
> [owner-statalist@hsphsun2.harvard.edu] on behalf of mccali mccalister
> [mccali85@hotmail.com]
> Sent: Tuesday, January 07, 2014 8:17 AM
> To: statalist@hsphsun2.harvard.edu
> Subject: Re: st: RE: MCNEMAR test or Average treatment effects in matched
> data.
> 
> Dear Dr.Canner,
> 
> I really apreciate your replay. So if I understood well, you simply would
> use the log.regression to see the effect of the my treatment. What about the
> average treatment effects? Are they usefull? In my case?(the incidence of
> mediastinitis is the outcome ) Maybe the interpretaion in medical issues is
> not so clear?So should I avoid it?
> 
> It is very frequent that many paper using propensity score (not necessary
> should  be right)  they show the new propensity score groups that they are
> balanaced using parametric tests or non parametric test(even then they use
> standarizes mean differences). What would you advice me to use if I want to
> compare baseline caracteristic of my population  before and after the
> propensity score ? Note that I want to show all the baseline variables and
> many of them are not included in the propensity score model. Should. I
> simply use parametric tests weighted by psmatch2?
> 
> Best regards
> Dr. Ayaon
> Cardiovascular resident
> 
> Enviado desde mi iPad
> 
>> El 06/01/2014, a las 23:12, "Joe Canner" <jcanner1@jhmi.edu> escribió:
>> 
>> Dear Dr. Ayaon,
>> 
>> According to a previous thread on this subject
> (http://www.stata.com/statalist/archive/2012-08/msg00985.html) it is not
> necessary to used a matched analysis in a 1:k propensity score matched
> analysis.  In fact, I'm not even sure how one would do a McNemar test for
> 1:5 matching (although with a little work it might be possible for a 1:1
> match).  More to the point, as mentioned in the previous thread, I would
> question whether propensity score matching truly qualifies as a matched
> analysis for McNemar purposes.  Two people can have the same (or similar)
> propensity score even if they have a quite different set of characteristics.
>> 
>> As noted in the previous thread, it should be sufficient (if not
> preferable) to do a logistic regression of your outcome variable versus the
> treatment group, weighted using the _weight variable provided by -psmatch2-,
> and including all of the matching variables (and any other relevant
> variables) as covariates.
>> 
>> Regards,
>> Joe Canner
>> Johns Hopkins University School of Medicine
> 
> 
> 
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/