Re: st: Pscore and balancing

["Gauri Khanna" <gwkhanna@hotmail.com>]

Dear Ricardo,

Thank you for your prompt response:

1. So that I know what you mean by "more parsimonious",  you are suggesting 
to start off with fewer variables than my current set of 73 to generate the 
pscore? Am I right?

The reason I say this is because in Becker and Ichino's article in Stata 
Journal, Vol 2, No 4 the exact opposite is stated... if the balancing 
property is not satisfied a "less parsimonious" specification is needed.

2. A related question, if I start off with fewer variables is there a 
criteria that one uses to pick and choose?

(To be on the safe side I looked  up the dictionary meaning of parsimonious 
and it means frugal.)


I repeat the other two questions that perhaps you or anyone else might know

1. Firstly, why does the number of observations get reduced to

> > 13821 from 24,672 in the logit estimation?
4.  browsed at the pscore and blockid variables generated "pscorepump" and 
"myblock" and found  pscorepump==. for myblock==0. I don't understand what 
this means. There are atleast 10,000 missing values for pscorepump.

I appreciate your continued advice.


Regards,

Gauri

PS. Thanks for the reference of Deheja and Wahba (i had their working paper 
of this and I will see the final version also)



> From: "ricardo sierra" <ricardo.sierra@gmail.com>
> Reply-To: statalist@hsphsun2.harvard.edu
> To: statalist@hsphsun2.harvard.edu
> Subject: Re: st: Pscore and balancing
> Date: Wed, 11 Apr 2007 11:39:29 -0400
>
> Is there a way you can start with a more parsimonious logit
> specification to estimate de score ??
>
> ** This is the 1st step proposed in Dehejia and Wahba (2002)
>
>
> On 4/11/07, Gauri Khanna <gwkhanna@hotmail.com> wrote:
> > Dear Stata List,
> >
> > I am using Stata Version 9.2. I am generating propensity scores using the
> > pscore module (authors Becker & Ichino 2002) and it is updated. I am stuck
> > with the balancing property which remains unsatisfied. I have 73 
> > variables,
> > mostly categorical or dummy, for generating pscore. I have copied the 
> > output
> > below for reference. I have also copied a description of my variables, in
> > case you would like to scan through them. I looked at the statalist 
> > archives
> > but did not find a response to my question.
> >
> > This is the command that I use: "waterpump, a binary variable, is my
> > treatment"
> >
> > -pscore waterpump v136 v151 v152 hv217 v012 v109 v110 v112 v115 v119 sh39
> > v131 v137 ///
> > v208 v218 v404 v416 husbwifelive v512 v627 v628 v629 ///
> > v715 v705 v717 v718 v714 v731 v730 b9 s509 s510 s511a s511b s511d s512a 
> > ///
> > s512b s513 sh29 sh32a sh32b sh32c sh32d sh32e sh32f sh32x sh34 sh35 ///
> > sh36 sh37 sh42 sh43 sh46 sh49 sh47a sh47b sh47c sh47d sh47e sh47f ///
> > sh47g sh47h sh47j sh47k sh47l sh47p sh47q sh47r sh47s sh47t ownerTV ///
> > hv211 v024, pscore(pscorepump) blockid(myblock) numblo(5) level(0.005)
> > logit-
> >
> > 1. Firstly, why does the number of observations get reduced to 13821 from
> > 24,672 in the logit estimation?
> > (When I used the -comsup-option, the no. of observations in the table
> > "inferior of block pscore" then the no. of observations gets reduced to
> > 13000 odd from 24,762 which is expected. But in the output below, I have 
> > not
> > run -comsup- so the no. of observations remain 24,762)
> >
> > 2. Is there an efficienty way of trying to meet the balancing property?
> > Since I have so many variables this might be a near impossible task.
> >
> > 3. I understand that one of the ways to meet the balancing property is to
> > use higher order terms and interactions. Given that I have many dummy
> > variables or categorical variables this suggestion is not very useful.
> >
> > 4. I browsed at the pscore and blockid variables generated "pscorepump" 
> > and
> > "myblock" and found  pscorepump==. for myblock==0. I don't understand what
> > this means. There are atleast 10,000 missing values for pscorepump.
> >
> > Given that I have generate propensity scores several times over for
> > different treatment groups I would be extremely grateful if someone could
> > please advise me.
> >
> > Sincerely,
> >
> > Gauri
> >
> > ****************************************************
> > Algorithm to estimate the propensity score
> > ****************************************************
> >
> >
> > The treatment is waterpump
> >
> >  waterpump |      Freq.     Percent        Cum.
> > ------------+-----------------------------------
> >          0 |     15,885       64.15       64.15
> >          1 |      8,877       35.85      100.00
> > ------------+-----------------------------------
> >      Total |     24,762      100.00
> >
> >
> >
> > Estimation of the propensity score
> >
> > Iteration 0:   log likelihood =  -9109.675
> > Iteration 1:   log likelihood = -8131.2148
> > Iteration 2:   log likelihood = -8090.9179
> > Iteration 3:   log likelihood = -8089.9872
> > Iteration 4:   log likelihood = -8089.9861
> > Iteration 5:   log likelihood = -8089.9861
> >
> > Logistic regression                               Number of obs   =
> > 13821
> >                                                  LR chi2(73)     =
> > 2039.38
> >                                                  Prob > chi2     =
> > 0.0000
> > Log likelihood = -8089.9861                       Pseudo R2       =
> > 0.1119
> >
> > ------------------------------------------------------------------------------
> >   waterpump |      Coef.   Std. Err.      z    P>|z|     [95% Conf.
> > Interval]
> > -------------+----------------------------------------------------------------
> >        v136 |   .0165368   .0096416     1.72   0.086    -.0023604
> > .035434
> >        v151 |  -.0437896   .0882641    -0.50   0.620    -.2167841
> > .129205
> >        v152 |   .0003202   .0016804     0.19   0.849    -.0029733
> > .0036138
> >       hv217 |  -.0211372    .024753    -0.85   0.393    -.0696522
> > .0273777
> >        v012 |  -.0370832   .0079621    -4.66   0.000    -.0526886
> > -.0214777
> >        v109 |   .0142181   .0726133     0.20   0.845    -.1281014
> > .1565376
> >        v110 |  -.1876761   .0590738    -3.18   0.001    -.3034587
> > -.0718935
> >        v112 |  -.0562225   .0476427    -1.18   0.238    -.1496003
> > .0371554
> >        v115 |  -.0058888   .0009844    -5.98   0.000    -.0078182
> > -.0039594
> >        v119 |  -.9156391   .1534767    -5.97   0.000    -1.216448
> > -.6148303
> >        sh39 |  -.0093317   .0046107    -2.02   0.043    -.0183686
> > -.0002948
> >        v131 |  -.0449353    .017664    -2.54   0.011    -.0795562
> > -.0103145
> >        v137 |  -.0812052   .0256932    -3.16   0.002     -.131563
> > -.0308473
> >        v208 |   .0150714   .0351368     0.43   0.668    -.0537955
> > .0839383
> >        v218 |  -.0535865   .0225205    -2.38   0.017    -.0977259
> > -.0094471
> >        v404 |   .2572917   .0560506     4.59   0.000     .1474345
> > .3671488
> >        v416 |  -.0089503   .0161706    -0.55   0.580    -.0406442
> > .0227436
> > husbwifelive |   .1250677   .0961814     1.30   0.193    -.0634444
> > .3135797
> >        v512 |   .0439059   .0087173     5.04   0.000     .0268204
> > .0609914
> >        v627 |  -.0036405   .0259638    -0.14   0.888    -.0545286
> > .0472475
> >        v628 |   .0055433   .0290203     0.19   0.849    -.0513354
> > .062422
> >        v629 |  -.0025496   .0170102    -0.15   0.881    -.0358891
> > .0307898
> >        v715 |   .0156505   .0050324     3.11   0.002     .0057872
> > .0255139
> >        v705 |   .0009952   .0026297     0.38   0.705    -.0041589
> > .0061492
> >        v717 |   .0157639   .0075627     2.08   0.037     .0009413
> > .0305866
> >        v718 |   -.054141   .0170187    -3.18   0.001    -.0874971
> > -.020785
> >        v714 |   .5117575   .2638649     1.94   0.052    -.0054081
> > 1.028923
> >        v731 |  -.2636456   .1454615    -1.81   0.070    -.5487448
> > .0214537
> >        v730 |   .0071485   .0034344     2.08   0.037     .0004172
> > .0138798
> >          b9 |  -.1381251   .1456387    -0.95   0.343    -.4235717
> > .1473215
> >        s509 |  -.0037555   .0020511    -1.83   0.067    -.0077756
> > .0002645
> >        s510 |   .0039359   .0022827     1.72   0.085    -.0005381
> > .0084099
> >       s511a |   .0046628   .0172106     0.27   0.786    -.0290693
> > .038395
> >       s511b |   .0244404   .0220627     1.11   0.268    -.0188018
> > .0676825
> >       s511d |  -.0662242   .0226893    -2.92   0.004    -.1106944
> > -.021754
> >       s512a |   .0551096   .0352604     1.56   0.118    -.0139995
> > .1242188
> >       s512b |   .1820999   .0517648     3.52   0.000     .0806427
> > .283557
> >        s513 |  -.1124802   .0398112    -2.83   0.005    -.1905087
> > -.0344516
> >        sh29 |   .0133232   .0017694     7.53   0.000     .0098552
> > .0167912
> >       sh32a |  -.0457174   .1929909    -0.24   0.813    -.4239726
> > .3325379
> >       sh32b |  -.4238643   .2675577    -1.58   0.113    -.9482677
> > .1005391
> >       sh32c |   .0914454   .2024985     0.45   0.652    -.3054444
> > .4883352
> >       sh32d |  -.7686991   .1828472    -4.20   0.000    -1.127073
> > -.4103252
> >       sh32e |  -.3027793   .6182508    -0.49   0.624    -1.514529
> > .9089701
> >       sh32f |   .3425503   .1947313     1.76   0.079     -.039116
> > .7242167
> >       sh32x |   .0599032   .3102354     0.19   0.847     -.548147
> > .6679534
> >        sh34 |  -.4306421   .1426025    -3.02   0.003    -.7101378
> > -.1511464
> >        sh35 |  -.0315757   .0168604    -1.87   0.061    -.0646214
> > .00147
> >        sh36 |  -.1901594   .0456417    -4.17   0.000    -.2796155
> > -.1007033
> >        sh37 |   .0214189   .0073974     2.90   0.004     .0069202
> > .0359176
> >        sh42 |   .2409778   .0744005     3.24   0.001     .0951556
> > .3868001
> >        sh43 |  -.1217373   .0462229    -2.63   0.008    -.2123325
> > -.0311421
> >        sh46 |     .02258   .0465304     0.49   0.627    -.0686178
> > .1137779
> >        sh49 |   .0316219   .0306231     1.03   0.302    -.0283983
> > .0916421
> >       sh47a |   -.281183   .0458982    -6.13   0.000    -.3711419
> > -.1912241
> >       sh47b |  -.3105555    .067574    -4.60   0.000    -.4429981
> > -.1781129
> >       sh47c |  -.2774826   .0570406    -4.86   0.000    -.3892801
> > -.1656852
> >       sh47d |   .2857967   .0507951     5.63   0.000     .1862401
> > .3853534
> >       sh47e |  -.0465287   .0600335    -0.78   0.438    -.1641923
> > .0711348
> >       sh47f |  -.0569313   .0474101    -1.20   0.230    -.1498534
> > .0359908
> >       sh47g |   .1779822   .0602837     2.95   0.003     .0598282
> > .2961362
> >       sh47h |    .453847   .0426894    10.63   0.000     .3701772
> > .5375167
> >       sh47j |  -.0186251   .0684948    -0.27   0.786    -.1528723
> > .1156222
> >       sh47k |   .2616956   .2048625     1.28   0.201    -.1398276
> > .6632188
> >       sh47l |   .0638542   .1556463     0.41   0.682     -.241207
> > .3689154
> >       sh47p |   .1524665   .3714589     0.41   0.681    -.5755795
> > .8805126
> >       sh47q |   .4475498   .0971597     4.61   0.000     .2571203
> > .6379794
> >       sh47r |    .382921   .0751051     5.10   0.000     .2357178
> > .5301243
> >       sh47s |    .666872   .1650351     4.04   0.000     .3434092
> > .9903348
> >       sh47t |  -.3674771   .1922123    -1.91   0.056    -.7442063
> > .0092521
> >     ownerTV |    .126107   .0726295     1.74   0.083    -.0162441
> > .2684582
> >       hv211 |   .2028699   .1075005     1.89   0.059    -.0078272
> > .413567
> >        v024 |   .0062517   .0025395     2.46   0.014     .0012744
> > .0112291
> >       _cons |   .0984652   .4290789     0.23   0.818     -.742514
> > .9394444
> > ------------------------------------------------------------------------------
> >
> >
> >
> > Description of the estimated propensity score
> >
> >                 Estimated propensity score
> > -------------------------------------------------------------
> >      Percentiles      Smallest
> > 1%     .0435489       .0014547
> > 5%     .0808509       .0104286
> > 10%     .1245135       .0129766       Obs               13821
> > 25%     .2320692       .0131112       Sum of Wgt.       13821
> >
> > 50%     .3675371                      Mean           .3703061
> >                        Largest       Std. Dev.      .1787873
> > 75%     .5033935       .9397132
> > 90%     .6070773       .9507915       Variance       .0319649
> > 95%     .6681031       .9670365       Skewness       .1283541
> > 99%     .7641454       .9676308       Kurtosis       2.319066
> >
> >
> >
> > ******************************************************
> > Step 1: Identification of the optimal number of blocks
> > Use option detail if you want more detailed output
> > ******************************************************
> >
> >
> > The final number of blocks is 14
> >
> > This number of blocks ensures that the mean propensity score
> > is not different for treated and controls in each blocks
> >
> >
> >
> > **********************************************************
> > Step 2: Test of balancing property of the propensity score
> > Use option detail if you want more detailed output
> > **********************************************************
> >
> > Variable v717 is not balanced in block 1
> >
> > Variable sh47h is not balanced in block 1
> >
> > Variable v115 is not balanced in block 2
> >
> > Variable v705 is not balanced in block 2
> >
> > Variable s511a is not balanced in block 2
> >
> > Variable sh32x is not balanced in block 2
> >
> > Variable sh43 is not balanced in block 2
> >
> > Variable sh47r is not balanced in block 2
> >
> > Variable sh42 is not balanced in block 3
> >
> > Variable v119 is not balanced in block 4
> >
> > Variable sh32a is not balanced in block 4
> >
> > Variable sh34 is not balanced in block 4
> >
> > Variable sh47d is not balanced in block 4
> >
> > Variable sh47g is not balanced in block 4
> >
> > Variable sh32f is not balanced in block 5
> >
> > Variable sh32a is not balanced in block 6
> >
> > Variable sh32f is not balanced in block 6
> >
> > Variable sh47h is not balanced in block 6
> >
> > Variable sh32a is not balanced in block 9
> >
> > Variable sh32f is not balanced in block 9
> >
> > Variable sh47g is not balanced in block 11
> >
> > Variable sh47j is not balanced in block 11
> >
> > Variable v115 is not balanced in block 12
> >
> > Variable sh47q is not balanced in block 12
> >
> > Variable s512a is not balanced in block 13
> >
> > Variable s512b is not balanced in block 13
> >
> > Variable sh29 is not balanced in block 13
> >
> > Variable sh47c is not balanced in block 13
> >
> > The balancing property is not satisfied
> >
> > Try a different specification of the propensity score
> >
> >  Inferior |
> >  of block |       waterpump
> > of pscore  |         0          1 |     Total
> > -----------+----------------------+----------
> >         0 |     7,398      3,765 |    11,163
> >       .05 |       370         18 |       388
> >      .075 |       340         34 |       374
> >        .1 |       696        107 |       803
> >       .15 |       754        174 |       928
> >        .2 |       912        251 |     1,163
> >       .25 |       895        339 |     1,234
> >        .3 |     1,745        890 |     2,635
> >        .4 |     1,383      1,160 |     2,543
> >        .5 |       517        610 |     1,127
> >       .55 |       359        546 |       905
> >        .6 |       391        668 |     1,059
> >        .7 |       109        268 |       377
> >        .8 |        16         47 |        63
> > -----------+----------------------+----------
> >     Total |    15,885      8,877 |    24,762
> >
> >
> >
> > *******************************************
> > End of the algorithm to estimate the pscore
> > *******************************************
> >
> > DESCRIPTION OF VARIABLES:
> >
> > des v136 v151 v152 hv217 v012 v109 v110 v112 v115 v119 sh39 v131 v137 v208
> > v218 v404 v416 husbwifelive v512 v627 v628 v629 v715 v705 v717 v718 v714
> > v731 v730 b9 s509 s510 s511a s511b s511d s512a s512b s513 sh29 sh32a sh32b
> > sh32c sh32d sh32e sh32f sh32x sh34 sh35 sh36 sh37 sh42 sh43 sh46 sh49 
> > sh47a
> > sh47b sh47c sh47d sh47e sh47f sh47g sh47h sh47j sh47k sh47l sh47p sh47q
> > sh47r sh47s sh47t ownerTV hv211 v024
> >
> >              storage  display     value
> > variable name   type   format      label      variable label
> > -------------------------------------------------------------------------------
> > v136            byte   %8.0g                  number of household members
> > v151            byte   %8.0g       v151       sex of household head
> > v152            byte   %8.0g                  age of household head
> > hv217           byte   %8.0g       hv217      relationship structure
> > v012            byte   %8.0g                  current age - respondent
> > v109            byte   %8.0g       v109       reads newspaper once a week
> > v110            byte   %8.0g       v110       watches tv every week
> > v112            byte   %8.0g       v112       listens to radio every week
> > v115            int    %8.0g       v115       time to get to water source
> > v119            byte   %8.0g       v119       has electricity
> > sh39            byte   %8.0g       sh39       religion of household head
> > v131            byte   %8.0g       v131       ethnicity (scheduled caste 
> > or
> >                                                tribe)
> > v137            byte   %8.0g                  number of children 5 and 
> > under
> > v208            byte   %8.0g                  births in last five years
> > v218            byte   %8.0g                  number of living children
> > v404            byte   %8.0g       v404       currently breastfeeding
> > v416            byte   %8.0g       v416       heard of oral rehydration
> > husbwifelive    float  %9.0g
> > v512            byte   %8.0g                  years since first marriage
> > v627            byte   %8.0g       v627       ideal number of boys
> > v628            byte   %8.0g       v628       ideal number of girls
> > v629            byte   %8.0g       v629       ideal number of either sex
> > v715            byte   %8.0g       v715       husbands education-single 
> > yrs
> > v705            byte   %8.0g       v705       partner's occupation
> > v717            byte   %8.0g       v717       respondent's occupation
> > v718            byte   %8.0g       v718       current type of employment
> > v714            byte   %8.0g       v714       respondent currently working
> > v731            byte   %8.0g       v731       worked in last 12 months
> > v730            byte   %8.0g       v730       partners age
> > b9              byte   %8.0g       b9         who child lives with
> > s509            byte   %8.0g       s509       how much education should be
> >                                                given to girls
> > s510            byte   %8.0g       s510       how much education should be
> >                                                given to boys
> > s511a           byte   %8.0g       s511a      who decides what to cook
> > s511b           byte   %8.0g       s511b      who decides on obtaining
> > health
> >                                                care
> > s511d           byte   %8.0g       s511d      who decides about respondent
> >                                                staying with family
> > s512a           byte   %8.0g       s512a      permission needed to go to
> >                                                market
> > s512b           byte   %8.0g       s512b      permission needed to visit
> >                                                relatives or friends
> > s513            byte   %8.0g       s513       allowed to have money set
> > aside
> > sh29            byte   %8.0g       sh29       where do household members 
> > go
> >                                                for treatment
> > sh32a           byte   %8.0g       sh32a      strain by cloth to purify
> > water
> > sh32b           byte   %8.0g       sh32b      use alum to purify water
> > sh32c           byte   %8.0g       sh32c      use water filter to purify
> > water
> > sh32d           byte   %8.0g       sh32d      boil water to purify
> > sh32e           byte   %8.0g       sh32e      use electronic purifier to
> >                                                purify water
> > sh32f           byte   %8.0g       sh32f      use nothing to purify water
> > sh32x           byte   %8.0g       sh32x      use other method to purify
> > water
> > sh34            byte   %8.0g       sh34       main source of lighting
> > sh35            byte   %8.0g       sh35       number of rooms
> > sh36            byte   %8.0g       sh36       separate room used as a
> > kitchen
> > sh37            byte   %8.0g       sh37       main cooking fuel
> > sh42            byte   %8.0g       sh42       does this household own this
> >                                                house or any other house?
> > sh43            byte   %8.0g       sh43       does this household own any
> >                                                agricultural land?
> > sh46            byte   %8.0g       sh46       household owns livestock
> > sh49            byte   %8.0g       sh49       type of house
> > sh47a           byte   %8.0g       sh47a      mattress
> > sh47b           byte   %8.0g       sh47b      pressure cooker
> > sh47c           byte   %8.0g       sh47c      chair
> > sh47d           byte   %8.0g       sh47d      cot or bed
> > sh47e           byte   %8.0g       sh47e      table
> > sh47f           byte   %8.0g       sh47f      clock or watch
> > sh47g           byte   %8.0g       sh47g      fan
> > sh47h           byte   %8.0g       sh47h      bicycle
> > sh47j           byte   %8.0g       sh47j      sewing maching
> > sh47k           byte   %8.0g       sh47k      telephone
> > sh47l           byte   %8.0g       sh47l      refrigerator
> > sh47p           byte   %8.0g       sh47p      car
> > sh47q           byte   %8.0g       sh47q      water pump
> > sh47r           byte   %8.0g       sh47r      bullock cart
> > sh47s           byte   %8.0g       sh47s      thresher
> > sh47t           byte   %8.0g       sh47t      tractor
> > ownerTV         float  %9.0g
> > hv211           byte   %8.0g       hv211      has motorcycle
> > v024            byte   %8.0g       v024       region
> >
> > .
> >
> > _________________________________________________________________
> > Express yourself instantly with MSN Messenger! Download today it's FREE!
> > http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/
> >
> > *
> > *   For searches and help try:
> > *   http://www.stata.com/support/faqs/res/findit.html
> > *   http://www.stata.com/support/statalist/faq
> > *   http://www.ats.ucla.edu/stat/stata/
> *
> *   For searches and help try:
> *   http://www.stata.com/support/faqs/res/findit.html
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE! 
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/