Re: st: insheet and dropping cases

[Phil Schumm <pschumm@uchicago.edu>]

On Feb 19, 2014, at 9:08 PM, Ben Hoen <bhoen@lbl.gov> wrote:
> I have 70 files to import and insheet (via a loop with append) is the simpliest so I would like to use it but cannot figure out why it is not importing all of the cases.  It does the same thing for all states, always importing less than what are transferred via stattransfer, regardless of the total file size or number of cases..
> 
> Any ideas?


As Ronnie pointed out, -insheet- has been superseded by -import delimited- in Stata 13.  Based on my experience, I believe this was more than simply renaming -insheet-, but reflects a substantial change under the hood.  So, if you have access to Stata 13, the first thing would be to try this to see if you get a different result.  You might, especially if Stat/Transfer reads the file properly.

If you don't have Stata 13 or if you get the same result with -import delimited-, then take one file and find out where it stopped (e.g., count how many observations were read successfully).  Open the file with a text editor (ideally one capable of showing non-printing characters), go to that line, and start looking (either on that line or the line after).  I'll bet you'll find either some unexpected character or something different from the lines above it.  This will at least tell you what is causing the problem, and then perhaps someone here can suggest a fix.


-- Phil

P.S. You might find http://www.stata.com/support/faqs/data-management/malformed-end-of-line-sequences/ helpful, which shows how to use -hexdump- to troubleshoot a similar problem.


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/