Re: st: marginal effects for simultaneous changes in interacted variables

[jpitblado@stata.com (Jeff Pitblado, StataCorp LP)]

David Quinn <dxquinnx@gmail.com> had a follow-up:

> I have a quick follow-up question.  Does -lincom- in Stata 11 always
> calculate the discrete change and associated reliability estimates
> (z-scores, p-values, and confidence intervals) the same way as
> -contrast- would in Stata 12?  I notice that in the example that you
> use, the predictive margin of x1=0, x2=0 is statistically significant
> at the 95% level but x1=1, x2=1 is not, yet the discrete change
> between the two is statistically significant.  In my data, I am
> encountering a similar case where one value is significant at the 95%
> level (p=.001) and the other is not (p=.125), and -lincom- is telling
> me that the .46 discrete change between the two values is only
> approaching significance (p=.08).  This may be a function of my data,
> but I'm kind of surprised by the result because when I use King et
> al.'s Clarify program to calculate the same exact discrete change, it
> is also .46 but highly significant at the 95% level.

This kind of result may seem paradoxical, but we need to be mindful that it is
too easy to build the wrong intuition on a direct comparison when that
intuition is based on two separate test results.  The covariance is not
accounted for in the separate tests.

To be more specific, in my previous example we wanted to compare the
predictive margin when x1 and x2 are both equal 0 to the predictive margin
when x1 and x2 are both equal 1.  Here is a replay of the individual
predictive margins:

***** BEGIN:
. margins, at(x1=0 x2=0) at(x1=1 x2=1)

Predictive margins                                Number of obs   =         74
Model VCE    : OIM

Expression   : Pr(foreign), predict()

1._at        : x1              =           0
               x2              =           0

2._at        : x1              =           1
               x2              =           1

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         _at |
          1  |   .4344516   .0756514     5.74   0.000     .2861776    .5827255
          2  |   .0519108   .0556121     0.93   0.351    -.0570869    .1609085
------------------------------------------------------------------------------
***** END:

As David noted, the margin for 1._at is significantly different from zero at
the 5% level since the p-value is less than 0.001 < 0.05, but the margin for
2._at is not significantly different from zero at the 5% level since the
p-value is 0.351 > 0.05.  The estimated covarinace matrix for the predictive
margins is stored in -r(V)-.

***** BEGIN:
. matrix list r(V)

symmetric r(V)[2,2]
                1.          2.
              _at         _at
1._at   .00572313
2._at  -.00003049   .00309271
***** END:

The estimated covariance is pretty small relative to the variances, so it
doesn't expose as misguided the intuition that the predictive margins should
be significantly differenct from each other at the 5% level.

What's more, to answer David's follow-up question, we can verify that -lincom-
does indeed calculte the discrete change the same way as -margins- with the
-contrast(at(r))- option.  Recall the -margins- with -contrast- results:

***** BEGIN:
. margins, at(x1=0 x2=0) at(x1=1 x2=1) contrast(at(r) effects)

Contrasts of predictive margins
Model VCE    : OIM

Expression   : Pr(foreign), predict()

1._at        : x1              =           0
               x2              =           0

2._at        : x1              =           1
               x2              =           1

------------------------------------------------
             |         df        chi2     P>chi2
-------------+----------------------------------
         _at |          1       16.49     0.0000
------------------------------------------------

------------------------------------------------------------------------------
             |            Delta-method
             |   Contrast   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         _at |
   (2 vs 1)  |  -.3825407   .0942169    -4.06   0.000    -.5672024   -.1978791
------------------------------------------------------------------------------
***** END:

I added the -effects- sub-option to the -contrast()- option so that -margins-
would report the test statistic and p-value for the test against zero;
otherwise, only the confidence limits are reported by default.

Now here are the results from using -lincom- aftert posting the -margins-
results to -e()-:

***** BEGIN:
. margins, at(x1=0 x2=0) at(x1=1 x2=1) post
(output omitted)

. lincom _b[2._at] - _b[1._at]

 ( 1)  - 1bn._at + 2._at = 0

------------------------------------------------------------------------------
             |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         (1) |  -.3825407   .0942169    -4.06   0.000    -.5672024   -.1978791
------------------------------------------------------------------------------
***** END:

Notice that the standard error, test statistic, p-value, and confidence limits
are the same.

For David's case, I assume the estimated covariance between the two predictive
margins is playing a bigger role in the calculation of the standard error of
the difference between the two predictive margins.

David also mentioned a Clarify program, but I am not familiar with the Clarify
program, so I'll leave it for someone else comment.

--Jeff
jpitblado@stata.com
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/