Re: st: Pscores: Interpretation of results

[Duru <duru80@gmail.com>]

Thanks very much Ariel. Actually, I ran the PS models separately in
each country and used slightly different specifications since some
covariates did not differ significantly between mobile and landline
samples in some countries.The sampling method is the same for all
countries, RDD (LL+cell numbers), but you end up with different
proportions of respondents reached on a mobile and landline because
the frequency of mobile use, number of landline/cell only population,
call times etc. change by country.

I am trying to test whether respondents reached on a mobile differ
substantially from those reached on a landline with respect to my
survey outcomes -after we control for socio-demographic variables.

Yes, I was expecting propensity scores to differ by country. However,
I am not sure how to explain low pscores for treatment groups in some
countries. How do I explain if all cases in a treatment group to have
propensity scores below .4? Sample size of the treatment group is
lower in those countries, this seemed to be a potential reason since
in a small sample I imagine there will be high number of different
combination of covariates relative to the sample size. Then, number of
covariates might be another factor, I thought! Of course I maybe
totally misled. I havent come accross any article discussing these
things. They simply look at the overlap and proceed. Any ideas,
references are appreciated.

PS: Using one data base with all countries is a jolie good idea for
comparison. Although I couldnt satisfy the balancing property using
the same model in each country. Still, might be different in an
combined version. Plus, need to figure out how to exact match on
country.

Thanks very much again,

Duru



On Fri, Nov 12, 2010 at 7:22 PM, Ariel Linden, DrPH
<ariel.linden@gmail.com> wrote:
> Duru,
>
> I am somewhat confused with the information you have provided as well as the
> questions you are raising?
>
> Did you run the propensity score modesl separately for each country or did
> you put them altogether in one data-base and then run the logistic
> regression?
>
> If you ran the models separately for each country, you will very likely get
> different propensity score ranges. After all, the outcome is the probability
> of "being reached on a mobile phone". Would you expect this outcome to be
> the same in each country? Were the same sampling methods applied in each
> country?
>
> You also ask how we'd explain different sample sizes and numbe of
> covariates? I honestly don't know what this means. You, as the researcher,
> should be telling us where you drew your sample sizes from and which
> covariates you used in your modeling process.
>
>
> So now to a possible solution:
>
> Put data from all countries in one database. Generate the propensity score
> for the entire set. One of the matching variables could be "country", so you
> would ensure matches were country specific. In doing this aggregate, the
> "scale" of the propensity score will be uniform. Whatever covariates you use
> for the propensity score (and outcome models) should be encompassing for all
> countries.
>
> As for the outcome model, I would suggest you choose a propensity
> score-based weighting model rather than matching. You'll have a much larger
> sample size to work with (basically the entire population as opposed to
> being limited to only the matches)...
>
> I hope this helps
>
> Ariel
>
>
> Date: Thu, 11 Nov 2010 15:18:35 +0100
> From: Duru <duru80@gmail.com>
> Subject: st: Pscores: Interpretation of results
>
> Hi all,
>
> Sorry for the previous mail.
>
> I have estimated propensity scores for being reached on a mobile phone
> in a telephone survey for five countries (using pscore in Stata).
> There was good overlap in each country. Yet, pscores for some
> countries ranged between .1-.4 while some other countries it was
> .1-.8, for both control (landline respondents) and the treatment
> (mobile respondents) groups. I used the same number of covariates in
> each country, and the sample size of the treatment group was smaller
> in those countries with lower propensity scores. Any ideas, how we
> explain lower/higher propensity scores across countries? Sample size
> and number of covariates?
>
> Thanks,
>
> Duru
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/