Re: st: RE: MCNEMAR test or Average treatment effects in matched data.

[mccali mccalister <mccali85@hotmail.com>]

Thank you. i will read it carrefully.

Best regards

Enviado desde mi iPad

> El 08/01/2014, a las 21:07, "Ariel Linden" <ariel.linden@gmail.com> escribió:
> 
> First off, I suggest you adhere to a couple of important items noted on the
> Statalist FAQ, (1) use your real name so that I know who I am actually
> addressing, and (2) if you cite a paper, provide the entire reference, not
> something incoherent as "RubinandThomas, 1996"
> 
> As for where you got this quote from, I see it was taken directly from
> Elizabeth Stuart's paper. However, if you would have read the following few
> sentences in that paper, you would have seen where my point was made: 
> 
> " When matching using propensity scores, detailed below, there is little
> cost to including variables that are
> actually unassociated with treatment assignment, as they will be of little
> influence in the propensity score
> model. Including variables that are actually unassociated with the outcome
> can yield slight increases in variance.
> However, excluding a potentially important confounder can be very costly in
> terms of increased bias.
> Researchers should thus be liberal in terms of including variables that may
> be associated with treatment assignment and/or the outcomes. Some examples
> of matching have 50 or even 100 covariates included in the procedure (e.g.,
> Rubin, 2001)."
> 
> So, no, the issue is not under debate as you claim.
> 
> More importantly it addresses the crux of the matter here, and that is that
> you should utilize all the variables you have in generating the propensity
> score, and then present them, as I have suggested, in a typical "table 1"
> format. 
> 
> I am not sure where you are having difficulty in understanding the point
> about not using P values for determining covariate balance? I gave the
> reasoning in my prior post, yet you are asking the same question again: 
> 
> "How should should I show these results? Should i include them in table 1
> before and after the match? But without showing the p values? How can they
> know that there are not statistical differences?"
> 
> At this point, I suggest that you spend time reading Elizabeth Stuart's
> paper, as well as the following references. I don't think there is anything
> else for me to add to this thread.
> 
> Ariel
> 
> References:
> 
> Stuart EA. Matching methods for causal inference: a review and a look
> forward. Statistical Science 2010;25(1):1–21.
> 
> Caliendo M, Kopeinig S. Some practical guidance for the implementation of
> propensity score matching. Journal of Economic Surveys 2008;22:31-72.
> 
> Austin PC. Balance diagnostics for comparing the distribution of baseline
> covariates between treatment groups in propensity-score matched samples.
> Statistics in Medicine 2009;28:3083-3107.
> 
> Linden A, Samuels SJ. Using balance statistics to determine the optimal
> number of controls in matching studies. Journal of Evaluation in Clinical
> Practice 2013;19(5):968–975.
> 
> Linden A. Identifying spin in health management evaluations. Journal of
> Evaluation in Clinical Practice. 2011;17:1223-1230.
> 
> From: mccali mccalister <mccali85@hotmail.com>
> Date: 8 January 2014 18:01
> Subject: Re: st: RE: MCNEMAR test or Average treatment effects in matched
> data.
> To: "statalist@hsphsun2.harvard.edu" <statalist@hsphsun2.harvard.edu>
> 
> 
> Dear Ariel,
> About the first question. I got it clear. As I said in my first thread I
> checked the balance of the covariates using ps test, I meant using
> standarised mean differences. About the variables included, I only included
> variables known to be related to both treatment assignment and the
> outcome(RubinandThomas,1996;Heckman,IchimuraandTodd,1998;Glazerman,LevyandMy
> ers,2003;
> Hill,ReiterandZanutto,2004). But  i guess this issue still in debate.
> So i still have don't know how to do, to show all these variables that are
> not related to the outcome nor  to the treatment assignment. These may be
> interesting to the readers to know. How should should I show these results?
> Should i include them in table 1 before and after the match? But without
> showing the p values? How can they know that there are not statistical
> differences?
> 
> I may be new in this issue but that does not mean that  I just run stata. I
> am aware that  it's me who does the research and stata only calculate. I
> only want to make sure that what I do is correct.
> 
> Best regards
> 
> 
> Enviado desde mi iPad
> 
>>> El 08/01/2014, a las 17:02, "Ariel Linden" <ariel.linden@gmail.com>
>> escribió:
>> 
>> Let me weigh in here, since I was the poster of that previous thread 
>> on this subject 
>> (http://www.stata.com/statalist/archive/2012-08/msg00985.html)
>> 
>> The questions that are being asked here are somewhat difficult to 
>> follow, so I'll attempt to answer them, given what I understand:
>> 
>> " So if I understood well, you simply would use the log.regression to 
>> see the effect of the my treatment. What about the average treatment
> effects?
>> Are they usefull? In my case?(the incidence of mediastinitis is the 
>> outcome
>> ) Maybe the interpretaion in medical issues is not so clear?So should 
>> I avoid it?"
>> 
>> If your outcome is binary, then yes, you can use a logistic regression.
>> Which type of treatment effect you'll be estimating depends on how you 
>> do your matching/weighting. In general, matching untreated individuals 
>> to treated individuals provides an average treatment effect on the 
>> treated (ATT). Why, because you are choosing untreated units that have 
>> the same characteristics as the treated. You can use weighting 
>> strategies instead of matching, and choose your treatment effect 
>> estimator, if you are not happy with the matching approach.
>> 
>> " It is very frequent that many paper using propensity score (not 
>> necessary should  be right)  they show the new propensity score groups 
>> that they are balanaced using parametric tests or non parametric 
>> test(even then they use standarizes mean differences). What would you 
>> advice me to use if I want to compare baseline caracteristic of my 
>> population  before and after the propensity score ? Note that I want 
>> to show all the baseline variables and many of them are not included 
>> in the propensity score model. Should. I simply use parametric tests
> weighted by psmatch2?"
>> 
>> As common practice, you should show a "table 1" before and after matching.
>> This will provide "evidence" that matching resulted in balance on 
>> observed pre-intervention characteristics. As per my previous thread, 
>> you should NOT be using t-tests to assess balance. This is not 
>> inferential statistics, and thus we are not attempting to draw 
>> assumptions to a population. Moreover, sample size will influence the 
>> statistics, which we hope to avoid. You should use standardized 
>> differences for calculating balance, with the intent of reducing the 
>> value as far as possible (down to zero). You should also review Q-Q 
>> plots, box plots, density plots, etc. to visually inspect the
> distributions of each covariate.
>> 
>> Lastly, you write that you want to show baseline variables that are 
>> not included in the propensity score model? Why would you not include 
>> them in the model? If you read the Elizabeth Stuart article that Joe 
>> referenced, you will see that the rule of thumb is to include as many 
>> covariates as available into the propensity score. This increases the 
>> likelihood that you've balanced all observable characteristics, 
>> thereby reducing the possibility of bias from omitted variables.
>> 
>> In summary, you should spend some time reading about these issues. 
>> Blindly running statistics without a firm understanding of the 
>> underlying principles is not a recommended practice for researchers.
>> 
>> I hope this helps
>> 
>> Ariel
>> 
>> 
>> Date: Tue, 7 Jan 2014 17:28:37 +0000
>> From: Joe Canner <jcanner1@jhmi.edu>
>> Subject: RE: st: RE: MCNEMAR test or Average treatment effects in 
>> matched data.
>> 
>> Dr. Ayaon,
>> 
>> I am not an expert on propensity score matching; I have used it 
>> occasionally and was sharing a few insights and experiences since no 
>> one else had replied.  I would recommend that you familiarize yourself 
>> with the recent literature on the subject.  A good start would be:
> Stuart, E.A. (2010).
>> Matching Methods for Causal Inference: A review and a look forward.
>> Statistical Science 25(1): 1-21.
>> 
>> In a recent thread on this subject, someone recommended against 
>> checking for balance using methods like t-tests which are dependent on 
>> sample size.  This is certainly a useful caution, depending on the 
>> size of your study.  If balance is determined by not rejecting the 
>> null hypothesis of no difference, you need to be careful that this 
>> result is due to the lack of difference between the groups rather than 
>> to insufficient sample size.  If sample size is an issue for you in 
>> this regard, then perhaps you can try a non-parametric test.
>> 
>> Regards,
>> Joe
>> ________________________________________
>> From: owner-statalist@hsphsun2.harvard.edu
>> [owner-statalist@hsphsun2.harvard.edu] on behalf of mccali mccalister 
>> [mccali85@hotmail.com]
>> Sent: Tuesday, January 07, 2014 8:17 AM
>> To: statalist@hsphsun2.harvard.edu
>> Subject: Re: st: RE: MCNEMAR test or Average treatment effects in 
>> matched data.
>> 
>> Dear Dr.Canner,
>> 
>> I really apreciate your replay. So if I understood well, you simply 
>> would use the log.regression to see the effect of the my treatment. 
>> What about the average treatment effects? Are they usefull? In my 
>> case?(the incidence of mediastinitis is the outcome ) Maybe the 
>> interpretaion in medical issues is not so clear?So should I avoid it?
>> 
>> It is very frequent that many paper using propensity score (not 
>> necessary should  be right)  they show the new propensity score groups 
>> that they are balanaced using parametric tests or non parametric 
>> test(even then they use standarizes mean differences). What would you 
>> advice me to use if I want to compare baseline caracteristic of my 
>> population  before and after the propensity score ? Note that I want 
>> to show all the baseline variables and many of them are not included 
>> in the propensity score model. Should. I simply use parametric tests
> weighted by psmatch2?
>> 
>> Best regards
>> Dr. Ayaon
>> Cardiovascular resident
>> 
>> Enviado desde mi iPad
>> 
>>> El 06/01/2014, a las 23:12, "Joe Canner" <jcanner1@jhmi.edu> escribió:
>>> 
>>> Dear Dr. Ayaon,
>>> 
>>> According to a previous thread on this subject
>> (http://www.stata.com/statalist/archive/2012-08/msg00985.html) it is 
>> not necessary to used a matched analysis in a 1:k propensity score 
>> matched analysis.  In fact, I'm not even sure how one would do a 
>> McNemar test for
>> 1:5 matching (although with a little work it might be possible for a 
>> 1:1 match).  More to the point, as mentioned in the previous thread, I 
>> would question whether propensity score matching truly qualifies as a 
>> matched analysis for McNemar purposes.  Two people can have the same 
>> (or similar) propensity score even if they have a quite different set of
> characteristics.
>>> 
>>> As noted in the previous thread, it should be sufficient (if not
>> preferable) to do a logistic regression of your outcome variable 
>> versus the treatment group, weighted using the _weight variable 
>> provided by -psmatch2-, and including all of the matching variables 
>> (and any other relevant
>> variables) as covariates.
>>> 
>>> Regards,
>>> Joe Canner
>>> Johns Hopkins University School of Medicine
> 
> 
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/