Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: st: "Can Your Results be Replicated?" (Stata error?)
From
Joerg Luedicke <[email protected]>
To
[email protected]
Subject
Re: st: "Can Your Results be Replicated?" (Stata error?)
Date
Fri, 13 Sep 2013 10:01:36 -0400
After having a quick glance at their paper
(http://jcr.sagepub.com/content/early/2013/08/19/0022002713499718.abstract?papetoc)
it seems that they are talking about a problem with Stata's -xtgee-
command which, in the case of separation in a logit model, provides
nonsense results as opposed to omitting predictors or the like. Below
is a toy example showing what seems to be the problem. However,
finding an effect of something like "x is 3 million times less likely
than y" and not getting suspicious rather looks like sloppy research
to me in the first place.
Joerg
*------------------------
clear
set obs 100
set seed 123
gen id = _n
gen ui = rnormal(0,0.5)
expand 10
bys id : gen year = _n
gen x = cond(mod(_n-1, 3) == 1, 1, cond(mod(_n-1, 3) == 0, 2, 3))
tab x, gen(x_)
gen xb = 1 / (1 + exp(-(0.3*x_2 + 0.3*x_3 + ui)))
gen y = rbinomial(1,xb)
replace y = 0 if x_1 == 1
tab y x
xtset id year
xtgee y i.x, fam(binomial) link(logit)
melogit y i.x || id:
logit y i.x
*------------------------
On Fri, Sep 13, 2013 at 10:41 AM, Richard Williams
<[email protected]> wrote:
> At 08:14 AM 9/13/2013, Anders Alexandersson wrote:
>>
>> I just made this reply on the blog:
>>
>> "Where is the error in Stata? The author’s so called “Do-File for
>> Analyses.txt” is actually not a Stata do file but it does refer to
>> Stata’s user-written command -firthlogit- from SSC. Please provide a
>> reproducible do-file in Stata.The claim that results and conclusions
>> were due to an error in Stata is not supported."
>> See
>> http://politicalsciencereplication.wordpress.com/2013/09/11/guest-blog-how-to-persuade-journals-to-accept-your-replication-paper/comment-page-1/#comment-653
>
>
> Even if -firthlogit- did get it wrong, it is a bit of a stretch to imply
> that Stata has some terrible flaw. Stata Corp can hardly be held responsible
> for flaws in programs it did not write.
>
> When I first sent a program to SSC, I thought there might be some sort of
> exhaustive review process before it was released to the public. I got the
> feeling that wasn't the case when I got a message less than an hour later
> saying the program had been posted. Most user-written routines are fine but
> people should realize they haven't undergone the kind of testing that
> official programs have. And even in this case, we don't have any proof yet
> that firthlogit did get it wrong.
>
>
>> Anders Alexandersson
>> [email protected]
>>
>> On Fri, Sep 13, 2013 at 8:10 AM, Philip Jones
>> <[email protected]> wrote:
>> > Hi all,
>> >
>> > I found a link on my Twitter feed this AM, purporting to show how
>> > Stata "made a mistake" that R did not make:
>> >
>> > http://www.r-bloggers.com/can-your-results-be-replicated/
>> >
>> > which actually points to:
>> >
>> >
>> > http://politicalsciencereplication.wordpress.com/2013/09/11/guest-blog-how-to-persuade-journals-to-accept-your-replication-paper/
>> >
>> > I realize that "r-bloggers" is likely not the most bias-free site when
>> > it comes to reviewing/rating stats packages, but has anyone got an
>> > idea as to what is actually going on here? Is Stata really at fault?
>> >
>> > Regards,
>> >
>> > Phil
>> > @pmgjones on Twitter
>> > *
>> > * For searches and help try:
>> > * http://www.stata.com/help.cgi?search
>> > * http://www.stata.com/support/faqs/resources/statalist-faq/
>> > * http://www.ats.ucla.edu/stat/stata/
>>
>> *
>> * For searches and help try:
>> * http://www.stata.com/help.cgi?search
>> * http://www.stata.com/support/faqs/resources/statalist-faq/
>> * http://www.ats.ucla.edu/stat/stata/
>
>
> -------------------------------------------
> Richard Williams, Notre Dame Dept of Sociology
> OFFICE: (574)631-6668, (574)631-6463
> HOME: (574)289-5227
> EMAIL: [email protected]
> WWW: http://www.nd.edu/~rwilliam
>
>
>
> *
> * For searches and help try:
> * http://www.stata.com/help.cgi?search
> * http://www.stata.com/support/faqs/resources/statalist-faq/
> * http://www.ats.ucla.edu/stat/stata/
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/faqs/resources/statalist-faq/
* http://www.ats.ucla.edu/stat/stata/