st: Re: problems with insheet and variables with "" , $ characters

["Joseph Coveney" <jcoveney@bigplanet.com>]

Ben Hoen wrote:

I have a large set of csv files that have roughly 60 variables (and as many
as 15,000 observations), two of variables are ad hoc descriptions (string
length 244) that users inputted and contain a whole host of characters
including: (, ), "", \, /, $, commas and many spaces.  Using insheet on the
csv file causes problems (because of the commas in these two fields I
suspect), but if I first open the file in excel and save it as a tab
delineated txt file, insheet seems to work perfectly (at least based on the
first 3 files).  

Because I have over 50 files to import I was hoping to find a way to
automate those steps.  

Any ideas for how to do this via Stata?  (I also have stat transfer)

--------------------------------------------------------------------------------

StatTransfer's probably your better bet, and that's what I'd try first, along
the lines of the following three steps.

1. If they're not already in a common directory (folder), then copy all of your
.CSV files into a source directory (it can be a temporary directory created for
the purpose).  

2. Write a one- or two-line script file (a simple ASCII text file having a
.STCMD extension to its name) to direct StatTransfer's "command processor" to
COPY all of the .CSV files into corresponding .DTA files.  The script file will
contain a line something like:

COPY <source directory>*.csv <desired target directory>*.dta

3. Exceute the .STCMD command file by double clicking on it.  It will go through
all 50 .CSV files, converting them into Stata files with the same name (the
extension will of course be different).

See StatTransfer' help-file or user's manual for "command processor" for
details.  There are several settable options that you might like to set in your
.STCMD script file.

Joseph Coveney

P.S. You can actually call StatTransfer from within Stata to do all of this, via
the user-written -stcmd- command, immediately opening the resulting Stata file
as the active dataset, but that's more of interest when working with one file
conversion at at time, right when you want to work with it, instead of the batch
conversion approach above.


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/