Re: st: Pscores: Interpretation of results

["Ariel Linden, DrPH" <ariel.linden@gmail.com>]

Duru,

Here are some thoughts/comments/questions, and they are a bit out of order
relative to your statements below:

You state "The sampling method is the same for all countries, RDD (LL+cell
numbers), but you end up with different proportions of respondents reached
on a mobile and landline because the frequency of mobile use, number of
landline/cell only population, call times etc. change by country"

Well, this straight out tells you that countries will differ in their use of
mobiles. So if I were building a logistic regression model, by country, with
the outcome being "the probability of using a mobile" conditional on
observed covariates, then given the difference in proportions of users in
each country (either due to your sampling methodology or other reason -
bias, confounding, or reality), I would somewhat expect to see different
propensity score ranges. Also, if the covariates you are using in fact do
differentiate between mobile users and landline users, then that is telling
you something as well about the what are the observed explanatory variables
that help explain this difference.

Which brings me to the next point. I am not clear if the use of mobile
phones is, in fact, the outcome variable in addition to being used as the
propensity score estimation model? You state "I am trying to test whether
respondents reached on a mobile differ substantially from those reached on a
landline with respect to my survey outcomes -after we control for
socio-demographic variables."

It seems to me that you are asking the question backwards: "what are the
characteristics of those using mobiles vs landlines". If that is the case,
it doesn't make sense to use a propensity scoring methodology at all. You're
making everyone conditionally look alike (as you would in an RCT), which
means that you won't be able to see what makes them different? It could be
that I am just not clear on what you are trying to achieve here.

It almost seems to be that you should be running some basic stratification
tables comparing characteristics of those using mobiles versus landlines.
Then you can compare these characteristics by country to see if they are
different. Perhaps you should look at discrimminant analyses procedures.

I hope this helps

Ariel    



Date: Fri, 12 Nov 2010 21:26:55 +0100
From: Duru <duru80@gmail.com>
Subject: Re: st: Pscores: Interpretation of results

Thanks very much Ariel. Actually, I ran the PS models separately in
each country and used slightly different specifications since some
covariates did not differ significantly between mobile and landline
samples in some countries.The sampling method is the same for all
countries, RDD (LL+cell numbers), but you end up with different
proportions of respondents reached on a mobile and landline because
the frequency of mobile use, number of landline/cell only population,
call times etc. change by country.

I am trying to test whether respondents reached on a mobile differ
substantially from those reached on a landline with respect to my
survey outcomes -after we control for socio-demographic variables.

Yes, I was expecting propensity scores to differ by country. However,
I am not sure how to explain low pscores for treatment groups in some
countries. How do I explain if all cases in a treatment group to have
propensity scores below .4? Sample size of the treatment group is
lower in those countries, this seemed to be a potential reason since
in a small sample I imagine there will be high number of different
combination of covariates relative to the sample size. Then, number of
covariates might be another factor, I thought! Of course I maybe
totally misled. I havent come accross any article discussing these
things. They simply look at the overlap and proceed. Any ideas,
references are appreciated.

PS: Using one data base with all countries is a jolie good idea for
comparison. Although I couldnt satisfy the balancing property using
the same model in each country. Still, might be different in an
combined version. Plus, need to figure out how to exact match on
country.

Thanks very much again,

Duru

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/