Re: Re: st: Regression Discontinuity Design

[Nyasha Tirivayi <ntirivayi@gmail.com>]

Dear Ariel

Thanks for your reply. I will try looking for a good IV!

Regards

Nyasha Tirivayi
Maastricht University
Netherlands

On Sun, Oct 9, 2011 at 6:23 PM, Ariel Linden, DrPH
<ariel.linden@gmail.com> wrote:
> First off, I apologize that my initial response was more-or-less the same as
> Austin's. I get the digest, so my responses are always a day "after the
> fact".
>
> On the other hand, Austin and I both had identical concerns, so that should
> give you some comfort (at least it does for me) :-)
>
> A couple of additional points:
>
> Austin is correct in suggesting that if your reviewers are concerned with
> the impact of "unobservables", you may want to consider the IV approach.
> However, the main problem with the IV approach (at least in my opinion), is
> actually finding a good IV. Austin suggested the "distance to clinic", which
> may be a good IV if you have it. Under this approach, you may have
> sufficient sample size if the unit of measure is the individual. You'd have
> to test other potential IVs accordingly...
>
> My initial suggested approach using a multi-level approach was to allow for
> clustering at the various levels. However, this will not adjust for
> unobservables, and so the reviewers' concerns are not addressed.
>
> Propensity score matching (or weighting) would only ensure balance on
> observables, and so the reviewers' concerns about the confounding trend in
> income, etc. remains a viable threat to validity.
>
> Perhaps a viable alternative approach would be to model the data
> longitudinally as either a time series (at the community level), or
> longitudinally using individual level data. I am not clear what your data
> look like, but if you have multiple time points, you could perhaps account
> for the differing "trends" you spoke of.
>
> Good luck!
>
> Ariel
>
> Date: Fri, 7 Oct 2011 16:57:54 +0200
> From: Nyasha Tirivayi <ntirivayi@gmail.com>
> Subject: Re: st: Regression Discontinuity Design
>
> Dear Ariel
>
> Quick answers to your questions:
> a) X variable for assignment is HIV prevalence rate at community level
> b) cut off is 22%
> c) yes I am also worried n=8 at community level is not sufficient
>
> My outcome is labour supply for individuals in 200 treated households
> (residing in 4 chosen communities) and 200 control households
> (residing in 4 control communities). However does this still seem as
> randomization at community level if program placement was non-random
> i.e. they specifically targeted communities with higher HIV rates
> (above 22%). Household recruitment was not randomized either.
>
> In that case can I use multilevel modelling? I had done propensity
> score matching, but reviewers feel there are unobservables I am
> overlooking. So with cross sectional data, what other methods can I
> plausibly use?
>
> Kindly advise
>
> Regards
>
> Nyasha Tirivayi
> Maastricht University
> Netherlands
>
> On Fri, Oct 7, 2011 at 4:33 PM, Ariel Linden, DrPH
> <ariel.linden@gmail.com> wrote:
>> Hi Nyasha,
>>
>> It seems like you've got several different things going on here at once.
> The
>> RD design can be thought of as an observational study equivalent of an RCT
>> (where the cutoff represents the randomization). If we think about it in
>> those simple terms, then I'd ask you this: (a) what is the X variable that
>> you'd be using for assignment, (b) what would be the cutoff, and (c) do
> you
>> think that a N=8 is reasonable?
>>
>> It is not clear from your description what either (a) or (b) is, but I can
>> certainly say without any hesitation that N=8 is not sufficient.
>>
>> An excellent recent article for you to read on the RD design is: Lee,
> D.S.,
>> Lemieux, T. (2010) Regression discontinuity designs in econometrics.
> Journal
>> of Economic Literature 48, 281-355.
>>
>> Without getting into too deep of a methodological discussion, it seems to
> me
>> that if you already have randomization at the community level, you should
>> consider hierarchical/multi-level modeling to tease out whatever effect
> you
>> are looking for.
>>
>> Ariel
>>
>>
>>
>> Date: Fri, 7 Oct 2011 00:39:17 +0200
>> From: Nyasha Tirivayi <ntirivayi@gmail.com>
>> Subject: st: Regression Discontinuity Design
>>
>> Hello
>>
>> I have questions about implementing a regression discontinuity
>> approach. I have cross sectional data from 200 households on a social
>> program and 200 control households. The program was targeted at two
>> levels- geographically and at household level.
>>
>> The geographic placement of the social program in communities appears
>> to have been done based on HIV prevalence rates of more than 20.5% for
>> 3 "treated" communities and less than 20.5% for 3 "control
>> communities". Two clinics do not follow this cutoff making it a fuzzy
>> discontinuity design at community level. After geographic placement,
>> households were then selected based on a means tested score. However
>> we do not have access to this data. We have data from 200 randomly
>> sampled households who are actually in the social program and residing
>> in the treated communities and from 200 control households with
>> similar household characteristics to the treated households but
>> residing in the control communities.
>>
>> My questions are as follows:
>> 1. Would it be valid to use the community level discontinuity for
>> impact evaluation? What software can I use in Stata?
>> 2. If so would an RD approach based on 8 communities be valid? Is the
>> sample of communities too small?
>> 3. If RD is no appropriate what other methods besides propensity score
>> matching can I use, that can also take care of unobservables even with
>> cross sectional data?
>>
>> Kindly advise
>>
>> Regards
>>
>> N.Tirivayi
>> Maastricht University
>> Netherlands
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/