Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: st: RE: re: St : matching estimator
From
Shikha Sinha <[email protected]>
To
[email protected]
Subject
Re: st: RE: re: St : matching estimator
Date
Wed, 8 Feb 2012 12:55:53 -0800
Dear Daniel,
A related question is what variables should be included in the
estimation of propensity score model. I believe that covariates that
are likely to affect the probability of being into treatment should be
included for sure; but I am not sure about variables that would affect
the outcomes other than the selection variables should also be
included.
I read an article that suggest that PSM model should include variables
that are likely to affect outcomes should also be included in addition
to observed selection variables.
Thanks,
Shikha
On Tue, Jan 3, 2012 at 9:53 AM, Millimet, Daniel <[email protected]> wrote:
> 1. My rough opinion with respect to the first question is that the metric - say Mahalanobis distance versus propensity score difference - is much less significant than the choice over (a) what variables are used to construct the distance measure (including the decision over higher order terms and interaction terms involved the matched on variables) and (b) what estimator is actually used once distance measures are chosen (e.g., nearest neighbor, kernel matching, etc.)
>
> 2. No one can judge whether is unobserved heterogeneity without knowing the details of the application and the covariates included in the model. If you are worried, there are methods designed to assess sensitivity to the unconfoundedness assumption, such as Rosenbaum bounds or the method developed in the user-written -sensatt- command or the methods discussed in Altonji et al. (J Political Economy, 2005).
>
> Best,
> Dann
>
> ****************************************************
> Daniel L. Millimet
> Research Fellow, IZA
> Professor, Department of Economics
> Box 0496
> SMU
> Dallas, TX 75275-0496
> phone: 214.768.3269
> fax: 214.768.1821
> web: http://faculty.smu.edu/millimet
> ****************************************************
>
>
> -----Original Message-----
> From: [email protected] [mailto:[email protected]] On Behalf Of Ayman Farahat
> Sent: Tuesday, January 03, 2012 11:32 AM
> To: [email protected]
> Subject: st: re: St : matching estimator
>
>
>
>
>
>
> Thanks for the list. To give more context i am trying to evaluate a treatment effect (users self selecting to download an app. If they don't download the app, they can still access the service through the web interface). I am trying to estimate the causal impact of the treatment (downloading the app) on the user engagement (number of times they access the service, churn rate ... etc).
>
> I had two concerns about the matching estimator :
> 1) The sensitivity to the metric used to find matching subjects.
>
> 2) The risk of unobserved heterogeneity specially given the self selection.
>
> Thanks
> Ayman
>
>
>
> ________________________________
> From: Cameron McIntosh <[email protected]>
> To: STATA LIST <[email protected]>
> Sent: Thursday, December 29, 2011 12:52 PM
> Subject: RE: st: re: St : matching estimator
>
> I will add a list of references that you can cull from, many are directly relevant to your case, I am sure (others may be interested as well):
>
> Glynn, A.N., & Quinn, K.M. (2010). An Introduction to the Augmented Inverse Propensity Weighted Estimator. Political Analysis. 18(1):36-56.
>
> Glynn, A.N., & Quinn, K.M. (January 23, 2008). Choosing an Identifying Set of Matching or
> Conditioning Variables. http://scholar.harvard.edu/sites/scholar.iq.harvard.edu/files/aglynn/files/Choosing_an_Identifying_Set.pdf
>
> Steiner, P.M., Cook, T.D., & Shadish, W.R. (2011). On the Importance of Reliable Covariate Measurement in Selection Bias Adjustments Using Propensity Scores. Journal of Educational and Behavioral Statistics, 36(2), 213-236.
> Trojano, M., Pellegrini, F., Paolicelli, D., Fuiani, A., & Di Renzo, V. (2009). Observational Studies: Propensity Score Analysis of Non-randomized Data. The International MS Journal, 16, 90–97.http://www.msforum.net/journal/download/20091690.pdf
>
> Li, L. (May 13, 2011). Propensity Score Analysis with Matching Weights. COBRA Preprint Series, Article 79. http://biostats.bepress.com/cobra/ps/art79
> Fan, X., & Nowell, D.A. (2011). Using Propensity Score Matching in Educational Research. Gifted Child Quarterly, 55(1), 74-79.
>
> Austin, P.C. (2008a). A critical appraisal of propensity-score matching in the medical literature between 1996 and 2003. Statistics in Medicine, 27(12), 2037–2049.http://www.epi.msu.edu/janthony/requests/propensity/Austin_A%20critical%20appraisal%20of%20propensity%20score.pdf
> *** the above paper is followed by numerous commentaries
> Williamson, E., Morley, R., Lucas, A., & Carpenter, J. (2011). Propensity scores: From naive enthusiasm to intuitive understanding. Statistical Methods in Medical Research, Online First.
>
> Pruzek, R.M. (2011).
> Introduction to the Special Issue on Propensity Score Methods in Behavioral Research. Multivariate Behavioral Research, 46(3), 389-398.
>
> Austin, P.C. (2011). An Introduction to Propensity Score Methods for Reducing the Effects of Confounding in Observational Studies. Multivariate Behavioral Research, 46(3), 399-424.http://www.tandfonline.com/doi/pdf/10.1080/00273171.2011.568786
>
> Kelcey, B. (2011). Covariate Selection in Propensity Scores Using Outcome Proxies. Multivariate Behavioral Research, 46(3), 453-476.
>
> Hill, J., Weiss, C., & Zhai, F. (2011). Challenges With Propensity Score Strategies in a High-Dimensional Setting and a Potential Alternative. Multivariate Behavioral Research, 46(3), 477-513.
>
> Thoemmes, F.J., & West, S.G. (2011). The Use of Propensity Scores for Nonrandomized Designs With Clustered Data. Multivariate
> Behavioral Research, 46(3), 514-543.
>
> Lottridge, S.M., Nicewander, W.A., & Mitzel, H.C. (2011). A Comparison of Paper and Online Tests Using a Within-Subjects Design and Propensity Score Matching Study. Multivariate Behavioral Research, 46(3), 544-566.
>
> Coffman, D.L. (2011). Estimating Causal Effects in Mediation Analysis Using Propensity Scores. Structural Equation Modeling, 18(3), 357-369.
>
> Jo, B., Stuart, E.A., MacKinnon, D.P., & Vinokur, A.D. (2011). The Use of Propensity Scores in Mediation Analysis. Multivariate Behavioral Research, 46(3), 425-452.
> Stuart, E.A. (2010). Matching Methods for Causal Inference: A Review and a Look Forward. Statistical Science, 25(1), 1-21.
> http://arxiv.org/PS_cache/arxiv/pdf/1010/1010.5586v1.pdf
>
> Spector, P.E., & Brannick, M.T. (2011). Methodological Urban Legends: The Misuse of Statistical
> Control Variables. Organizational Research Methods, 14(2), 287-305.
>
> Stürmer, T., Rothman, K.J., Avorn, J., & Glynn, R.J. (2010). Treatment Effects in the Presence of Unmeasured Confounding: Dealing With Observations in the Tails of the Propensity Score Distribution—A Simulation Study. American Journal of Epidemiology, 172(7), 843-854.
>
> Westreich, D., Cole, S.R., Funk, M.J., Brookhart, M.A., & Stürmer, T. (2011). The role of the c-statistic in variable selection for propensity score models. Pharmacoepidemiology and Drug Safety, 20(3), 317–320.
> Waernbaum, I. (2010). Propensity score model specification for estimation of average treatment effects. Journal of Statistical Planning and Inference, 140(7), 1948-1956.
>
> Thoemmes, F.J., & Kim, E.S. (2011). A Systematic Review of Propensity Score Methods in the Social Sciences. Multivariate Behavioral Research, 46(1), 90-118,
>
> Austin, P.C. (2011). A Tutorial and
> Case Study in Propensity Score Analysis: An Application to Estimating the Effect of In-Hospital Smoking Cessation Counseling on Mortality. Multivariate Behavioral Research, 46(1), 119-151.
> Heinrich, C., Maffioli, A., & Vázquez, G. (2010). A primer for applying propensity-score matching. Impact-Evaluation Guidelines, Technical Notes No. IDB-TN-161. Office of Strategic Planning and Development Effectiveness, Inter-American Development Bank.http://idbdocs.iadb.org/wsdocs/getdocument.aspx?docnum=35320229
>
> Harder, V.S., Stuart, E.A., & Anthony, J.C. (2010). Propensity Score Techniques and the Assessment of Measured Covariate Balance to Test Causal Associations in Psychological Research. Psychological Methods, 15(3), 234-249.
> Steiner, P.M., Cook, T.D., Shadish, W.R., & Clark, M.H. (2010). The Importance of Covariate Selection in
> Controlling for Selection Bias in Observational Studies. Psychological Methods, 15(3), 250-267.
> Sekhon, J.S. (2011). Multivariate and propensity score matching software with automated balance optimization: the Matching package for R. Journal of Statistical Software, 42(7).http://sekhon.berkeley.edu/papers/MatchingJSS.pdf
>
> Sekhon, J.S. (May 20, 2011). Multivariate and Propensity Score Matching with Balance Optimization. Package ‘Matching’, Version 4.7-14.http://cran.r-project.org/web/packages/Matching/Matching.pdfhttp://cran.r-project.org/web/packages/Matching/index.html
>
> Cheers,
> Cam
>> From: [email protected]
>> To: [email protected]
>> Subject: st: re: St : matching estimator
>> Date: Thu, 29 Dec 2011 11:58:02 -0500
>>
>> Hi Ayman,
>>
>> It is not clear from your post exactly what you are asking for, or trying to
>> do with your data?
>>
>> The propensity score is typically used as the matching variable when you
>> have a very large number of variables. You would find the appropriate
>> approach that minimizes the distance of the propensity score within matches.
>> Note that just because you have close matches on the propensity score, does
>> not ensure you have balance on the underlying covariates. Thus, it is more
>> important that you check for covariate balance than have balance on the
>>
> propensity score.
>>
>> If you prefer to match on the covariates themselves, you have several
>> options. The most ubiquitous approach is mahalanobis distance matching. This
>> can be done within an existing user written program such as -psmatch2- or
>> -optmatch2-, or - mahapick-.
>>
>> You could also consider -cem- which is another user written program that
>> uses stratification to bucket units.
>>
>> My personal preference is to use a propensity score weighting approach
>> (IPTW, ATT, or ATE weights), and then model the outcome using that weight.
>> You could also consider kernel weighting, which is one of the options within
>> -psmatch2-.
>>
>> So basically you have a lot of options, but if you'd like a more specific
>> response, you'll have to ask your question in a more specific manner...
>>
>> I hope this helps
>>
>> Ariel
>>
>>
>> Date: Wed, 28 Dec 2011 12:22:58 -0800
>> From: Ayman Farahat <[email protected]>
>> Subject: st: St : matching estimator
>>
>> Hello
>> I am looking into using nmatch to evaluate treatment effect.
>> However, i am not sure if there is a natural meteric to find closest match.
>> Could the choice of meteric impact résult?
>> Any references to work comparing matching estimator to propensity or Heckman
>> greatly appreciated.
>> Thanks
>> Ayman
>> Sent from my iPad
>>
>>
>> *
>> * For searches and help try:
>> * http://www.stata.com/help.cgi?search
>> * http://www.stata.com/support/statalist/faq
>> * http://www.ats.ucla.edu/stat/stata/
>
> *
> * For searches and help try:
> * http://www.stata.com/help.cgi?search
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
>
> *
> * For searches and help try:
> * http://www.stata.com/help.cgi?search
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
>
> *
> * For searches and help try:
> * http://www.stata.com/help.cgi?search
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/