Re: Re: st: Propensity score matching: Must all treated samples have a counterfactual?

[Ricky Lim <ricky.lim12345@gmail.com>]

Hi Ariel,

Thank you very much for your reply.
& thanks for sharing the doubly robust approach.
It is the first time I hear about it (I'm still quite new to
propensity score matching), and will do some readings before I use it.
Just downloaded several articles online.

Yea...the sample size that I have is really small.
Actually the organisations are my units of sampling,
And I'm using the staff level data as the unit of analysis.
I have about 100 staff data from each organisation in my sampling frame.
I was worried about getting the right caliper size because if too few
organisations are included in the analysis, the results might not be
generalisable as well.

My current plan is:
after generating the propensity score using -pscore- with common support,
I'll select the staff level data from both treated organisations and
their counterfactuals and run a regression with the model:

y = a + BiXi + pT + qM + rMT + c
where
T = time period before or after the merger
M = selected for merger or not selected
r = difference-in-difference term

In running the above regression I will cluster the standard error
according to the organisations.

I don't think I can use commands such as -psmatch2- (and maybe
-teffects- as well I'm not sure) to calculate the impact of merger,
because the outcome variables need to be specified in the same
datasheet where propensity scores are calculated. In my analysis, the
propensity scores are generated using organisation level data while
the outcome variable is at the staff level.

I realise I might have strayed slightly away from my original question.
But I thought I should explain this to make my original questions clearer.

Hope this is making sense.
Comments from all Statalisters are welcomed.

Thank you very much & have a great day!! =D


Regards,
Ricky

On 20 August 2013 15:13, Ariel Linden, DrPH <ariel.linden@gmail.com> wrote:
> Ricky,
>
> You've got a very small sample size here, and by using matching within a
> caliper, you're likely to lose some "non-treated" units, simply because
> there are not enough similar non-treated units to choose from.
>
> If it were me, I would probably consider using a weighting mechanism on all
> units within a doubly robust approach. You won't lose non-treated units.
> Moreover, if either the outcome model or the propensity model is correct,
> the doubly robust estimate is unbiased. Again, I will reiterate that you
> have a small sample size...
>
> It is up to you to do your due diligence to determine if there is overlap in
> covariates used to generate the propensity score (and used in the outcome
> model). If there is little or no overlap, it means that you'll be
> extrapolating beyond the data. This not a situation you want to find
> yourself. You may ultimately decide to use regression with no fanciness, and
> call it a day.
>
> In Stata 13 there are doubly robust commands available: -teffects aipw-, and
> -teffects iprwa-.
>
> If you are using older versions of Stata, you can implement doubly robust
> methods using -dr- (http://personalpages.manchester.ac.uk/staff/mark.lunt)
> or -drglm- (http://www.stata-journal.com/software/sj13-1)
>
> Be aware that these two programs provide an ATE, not an ATT estimator as you
> would get from matching.
>
> Ariel
>
>
>
> Date: Mon, 19 Aug 2013 18:49:15 +0100
> From: Ricky Lim <ricky.lim12345@gmail.com>
> Subject: Re: st: Propensity score matching: Must all treated samples have a
> counterfactual?
>
> Dear Ariel & Sebastian,
>
> Thank you very much for your replies!
> They were very helpful.
>
> And thanks for the two articles, Ariel.
> I have read one of them today and will read another one tomorrow.
> And yes, I meant caliper, not radius.
> Sorry for the confusion.
>
> ***
>
> Dear Ariel, Sebastian & all Statalisters,
>
> My study is about organisational merger actually.
> So there are 2 organisations pre-treatment (pre-merger) and 1
> organisation post-treatment (post-merger).
>
> I'm generating the propensity scores based on the per-treatment observables.
> The standard deviation for my PS is around 0.06.
> 20 - 25% of that would be 0.012 - 0.015.
>
> I only have 20 organisations for 9 mergers.
> Applying this caliper on my samples would mean that 5 organisations
> (one each from a different merger, hence 5 mergers) will have no
> counterfactuals.
> This leaves me with only 4 mergers which are too small a sample.
> I think if one of the constituent organisations do not have a
> couunterfactuals pre-merger, I would have to give up the whole merger
> case.
>
> Also, each of the constituent organisation for the remaining 4 mergers
> have 1 to >10 counterfactuals within the caliper +-0.015.
> Is it necessary for each of the treated organisation to have the same
> number of counterfactuals?
> Can one of them have one, and another one have say, five?
>
> At the moment,
> I am more keen to just use 3 nearest neighbour without caliper, just
> so that I can still use all 20 organisations / 9 mergers.
> But that will involve the assumption that the matches are good enough.
>
> Would be grateful to hear your thoughts / advices.
>
> Thank you very much in advanced.
>
> Regards,
> Ricky
>
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/