Thanks for that information Sergiy, very helpful.
Paul
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Sergiy Radyakin
Sent: Tuesday, 17 November 2009 6:58 AM
To: [email protected]
Subject: Re: st: Large Datasets Panel Data Logit Limits
Well, it is what it is - no room for more variables. Stata creates
temporary variables during it's work.
I have requested the number-of-temps-per-Stata-command from
developers, but this info is not readily available.
The rule of thumb is for most commands: 5vars. But obviously not for
all commands.
For more info, see the reply from Bill Gould to my question 5 here:
http://www.stata.com/statalist/archive/2008-01/msg00876.html
I would urge the developers of Stata to provide as much as possible of
such rules, e.g. how much memory will -reshape- need?
Your case looks like something I mention here:
http://www.stata.com/statalist/archive/2008-02/msg00468.html
You have a very long dataset, lots of observations, few variables. The
overhead created by Stata (pointers) becomes non-trivial. Type
-memory- and see how much is used for that purpose and how much is
left available. multiply _N*40 to get the number of bytes needed for
the 5 vars that are required according to the rule of thumb.
Recommendation: get a 64-bit machine.
Best wishes, Sergiy Radyakin
On Mon, Nov 16, 2009 at 5:37 PM, Paul GERRANS <[email protected]> wrote:
> Hi,
> I have an unbalanced dataset with approx 2.4 million individuals with up to 10 observations over time (quarters). I have six regressors. The file size is 630mb.
> In trying to run a panel data logistic regression with group (individuals) and time (quarter) random effects, I am receiving a r(902) error:
>
> no room to add more variables because of width
> An attempt was made to add a variable that would have increased the memory required to store an observation beyond what is currently possible. You have the following alternatives:
> 1. Store existing variables more efficiently; see help compress.
> 2. Drop some variables or observations; see help drop. (Think of Stata's data area as the area of a rectangle; Stata can trade off width and length.)
> 3. Increase the amount of memory allocated to the data area using the set memory command; see help m
> I have increased the mem to 1500mb which is the largest I can allocate. I have dropped all variables I can and have compressed. I continue to get the error.
> I am using version 11 and I am using Windows Vista (32-bit).
> Any suggestions as to what may be preventing the regression running or anything I can do?
> Thanks
> Paul
>
> This e-mail is confidential. If you are not the intended recipient you must not disclose or use the information contained within. If you have received it in error please return it to the sender via reply e-mail and delete any record of it from your system. The information contained within is not the opinion of Edith Cowan University in general and the University accepts no liability for the accuracy of the information provided.
>
> CRICOS IPC 00279B
>
> *
> * For searches and help try:
> * http://www.stata.com/help.cgi?search
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
>
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
This e-mail is confidential. If you are not the intended recipient you must not disclose or use the information contained within. If you have received it in error please return it to the sender via reply e-mail and delete any record of it from your system. The information contained within is not the opinion of Edith Cowan University in general and the University accepts no liability for the accuracy of the information provided.
CRICOS IPC 00279B
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
