Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
RE: st: RE: Quantile regression runtimes
From
"Martin Weiss" <[email protected]>
To
<[email protected]>
Subject
RE: st: RE: Quantile regression runtimes
Date
Sat, 5 Jun 2010 22:38:24 +0200
<>
So, Jacob, I bet the answer is somewhere in the "Methods and Formulas"
section in [R], page 1463. It mentions, for instance, that for the median,
there is no need to weight observations, so that would be consistent with my
observation of the lowest timing for the 50% quantile. I am not enough of an
expert on this subject, but the function being optimized in the iterations
for -qreg-, in conjunction with the idiosyncracies of your data, must hold
the key to this riddle.
HTH
Martin
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Martin Weiss
Sent: Samstag, 5. Juni 2010 22:20
To: [email protected]
Subject: RE: st: RE: Quantile regression runtimes
<>
The -set mem 1G- line, of course, is not necessary. It is a remnant from a
couple of minutes ago, when I imagined Jacob`s sample to be much, much
bigger...
HTH
Martin
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Martin Weiss
Sent: Samstag, 5. Juni 2010 22:12
To: [email protected]
Subject: RE: st: RE: Quantile regression runtimes
<>
So, for anyone who wants to replicate Jacob`s problem:
***********
vers 11.1
clear*
set mem 1G
set obs 673721
gen x1= rnormal()
gen x2= runiform()
gen x3=rchi2(3)
gen y=1*2*x1-3*x2+2*x3+rnormal()
timer clear
forv q=1/9{
timer on `q'
qreg y x?, q(`=`q'/10')
timer off `q'
}
timer list
**********
I end up with a -remarkably- U-shaped list of times:
. timer list
1: 24.24 / 1 = 24.2400
2: 18.06 / 1 = 18.0600
3: 13.92 / 1 = 13.9200
4: 8.20 / 1 = 8.2000
5: 5.24 / 1 = 5.2400
6: 9.40 / 1 = 9.4000
7: 14.46 / 1 = 14.4600
8: 19.04 / 1 = 19.0400
9: 29.87 / 1 = 29.8700
This may have much to do with my setup of the problem. How many covariates
are there in your -qreg- model, Jacob?
HTH
Martin
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Jacob Felson
Sent: Samstag, 5. Juni 2010 21:50
To: [email protected]
Subject: Re: st: RE: Quantile regression runtimes
Martin,
Sorry I was vague. The analytical sample was 673,721.
Jacob Felson
On Sat, Jun 5, 2010 at 3:28 PM, Martin Weiss <[email protected]> wrote:
>
> <>
>
> " on a very
> large dataset (the Census' 2008 American Community Survey 1% samples)."
>
>
> How large is the dataset exactly, Jacob? Remember, you cannot presume
every
> listmember is familiar with this dataset, even though in your profession
it
> may well be famous...
>
>
> HTH
> Martin
>
> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]] On Behalf Of Jacob Felson
> Sent: Samstag, 5. Juni 2010 21:04
> To: [email protected]
> Subject: st: Quantile regression runtimes
>
> I'm curious about the runtimes for quantile regression. I am running
> decile regressions (.1, .2, .3, .4, .5, .6, .7, .8, and .9) on a very
> large dataset (the Census' 2008 American Community Survey 1% samples).
> Runtimes generally decrease as deciles increase:
>
>
> Runtimes are in minutes
>
> .1 74.27413
> .2 34.95253
> .3 13.1072
> .4 8.738133
> .5 8.738133
> .6 4.369067
> .7 6.5536
> .8 8.738133
>
>
> I'm very curious -- why is the runtime for .1 regression so much
> higher than for .2? And what might explain the general pattern of
> these runtimes?
>
>
> Thanks,
>
> Jacob Felson
> Assistant Professor
> Department of Sociology
> William Paterson University
> *
> * For searches and help try:
> * http://www.stata.com/help.cgi?search
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
>
> *
> * For searches and help try:
> * http://www.stata.com/help.cgi?search
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
>
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/