Re: st: Bias: Monte Carlo

[Maarten Buis <maartenlbuis@gmail.com>]

On Mon, May 6, 2013 at 9:49 AM, John Antonakis wrote:
> I am running some Monte Carlos where I am interested in observing the bias
> in parameter estimates across manipulated conditions. By bias I mean the
> absolute percentage difference of the simulated value from the true value.
>
> I was wondering whether there has been another written about how much bias
> is "acceptable"--I know that this is like asking how long is a piece of
> string and that there is no statistical fiat that can give a definitive
> answer, because it also is a very field specific issue.

It is probably not quite the answer you are looking for (and I think
you are right by wondering whether such an answer can exist), but one
thing you can do is take into account that a Monte Carlo experiment
contains a random component, so if you repeat the experiment (with a
different seed) you will get a slightly different estimate of your
bias. The logic behind this variation between Monte Carlo experiments
is pretty much the same as the logic behind statistical testing: so
you can compute standard errors and confidence intervals. This is the
idea behind: Ian R. White (2010) "simsum: Analyses of simulation
studies including Monte Carlo error" The Stata Journal, 10(3):369--385
and <http://www.maartenbuis.nl/software/simpplot.html>. It is not very
useful as a definition of what amount of bias is "acceptable" as you
can arbitrarily make the bounds around your estimate of the bias
smaller by increasing the number of iterations, but at least this type
of bounds prevents you from over-interpretting the result from your
simulation, as happend here:
<http://stats.stackexchange.com/questions/55676#55676>.

Hope this helps,
Maarten

---------------------------------
Maarten L. Buis
WZB
Reichpietschufer 50
10785 Berlin
Germany

http://www.maartenbuis.nl
---------------------------------
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/