st: file cannot be opened in loop appending many datasets

[adrien bouguen <abouguen@povertyactionlab.org>]

Dear all,

I am trying to append a large number of datasets using 2 loops and I am
getting a strange error. After a while, the loop stops with a r(603) error
saying that the data cannot be opened whereas the data is there and was not
opened before.

The error seems to be affected datasets randomly : it is never the same
dataset that can't be opened and it is never the same loop number (here
count) that poses the problem.

I suspect a problem with the memory but I have tried this command in
different computers without success. See below for the code.

Any ideas?

Thanks so much for your help


clear
set mem 2g
set virtual on
global dta "Q:/numerique/dta"
global text "Q:/numerique/text"

clear
set mem 1000
set virtual on
local files : dir "$dta/enfants" files "*"
dis `files'
global count=0
foreach file in `files' {
local z=subinstr("`file'",".dta","",.)
use `"$var/`z'_v1.dta"', clear
global count=$count+1
forval j=3(2)31 {
dis "`j'"
cap append using `"$var/`z'_v`j'.dta"'
if _rc==0 {
local y=`j'+1
drop if v1=="" & v`j'==""
replace v2=v`y' if v1==""
replace v1=v`j' if v1==""
drop v`j' v`y'
}
}
save `"$col/`z'_colonne.dta"', replace
clear
}
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/