Here is a demonstration illustrating ~2048 files limit (Windows):
// ----------------------------------------------------------
set more off
forval i=1/2050 {
tempname file`i'
display "Opening file `i'"
file open `file`i'' using c:\temp\delme`i', write text replace
}
file close _all
// ----------------------------------------------------------
After several runs, I am getting ~2045 as the failure value (2044
successful open files). Since the .do file remains open while running,
this brings the total open files to 2045 (if I run the loop from the
Stata's prompt, I get precisely one more open file), plus
Stata-executable remains opened during the whole session, so total
reaches 2046. There is one more file created in the Windows\Temp\
directory for each Stata session (STXXXX), so we have 2047 opened
files. Does Stata open anything else? (to bring the total to declared
2048) Note that all commands in the benchmark are built-in, so no .ado
files remain open during execution.
Could somebody please try it on other OSes and say how many files can
be simultaneousely opened by Stata there?
Best regards,
Sergiy Radyakin
On 4/21/08, Sergiy Radyakin <[email protected]> wrote:
> Dear Dan,
>
> one of the possible explanations could be that you are opening a file
> for output ("file open ....") and don't close it within iterations.
> Than Stata will ultimately run out of handlers (perhaps somebody from
> the Stata, Corp could tell more precisely how many files can be opened
> simultaneously).
>
> The following article from the M$ knowledgebase gives some support and
> an approximate value of how big (small) this limit can be depending on
> the version of OS: http://support.microsoft.com/kb/163202
> Though they mention quite old OSes, there probably is a limit in the
> new ones too. There could also be a limit imposed by the compiler,
> used to compile Stata. Microsoft's compiler has it's restrictions
> described here:
> http://msdn2.microsoft.com/en-us/library/6e3b887c(VS.80).aspx
>
> ************* Quote *****************************************
> Up to 2,048 files can be open simultaneously at the lowio level (that
> is, opened and accessed by means of the _open, _read, _write, and so
> forth family of I/O functions). Up to 512 files can be open
> simultaneously at the stdio level (that is, opened and accessed by
> means of the fopen, fgetc, fputc, and so forth family of functions).
> The limit of 512 open files at the stdio level can be increased to a
> maximum of 2,048 by means of the _setmaxstdio function.
>
> Because stdio-level functions, such as fopen, are built on top of the
> lowio functions, the maximum of 2,048 is a hard upper limit for the
> number of simultaneously open files accessed through the C run-time
> library.
>
> ************* End of quote *****************************************
>
> Hope this helps,
> Sergiy Radyakin
>
>
>
>
> On 4/21/08, Dan Weitzenfeld <[email protected]> wrote:
> > Hi Folks,
> > I'm working on a dofile that loops through a process thousands of
> > times. Part of that process includes the following:
> >
> > ----begin code
> >
> > use "$directory/REACH_RESULTS.dta", clear
> > * random stuff
> > save "$directory/REACH_RESULTS.dta", replace
> >
> > ----end code
> >
> > I am repeatedly getting the following error
> >
> > file C:\Documents and Settings\Dan
> > Weitzenfeld\Desktop\REACH_RESULTS.dta could not be opened
> > r(603);
> >
> > ...but when this error occurs, Stata has that very file open!
> > Moreover, the error is occuring at seemingly random times. That is,
> > sometimes it will occur on the 124th run through the loop, while
> > others it won't occur until the 865th run through the loop.
> > I have had similarly random errors with a different
> > computationally-intensive do file. It's as if Stata starts going so
> > fast it trips over itself.
> > Having done a good deal of coding, I am well aware of how suspicious
> > "randomly occurring error" sounds. But trust me, I am changing
> > nothing in the do file, there is no randomness within the file, but
> > the error occurs at random loop iterations.
> > I am using Stata 10 on a Windows XP Professional SP2 on a Dell
> > Dimension DXP061 Core2, [email protected], 2GB ram
> >
> > Has anyone else had this problem?
> >
> > Thanks in advance,
> > Dan
> > *
> > * For searches and help try:
> > * http://www.stata.com/support/faqs/res/findit.html
> > * http://www.stata.com/support/statalist/faq
> > * http://www.ats.ucla.edu/stat/stata/
> >
>
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
