Re: st: Verify that all values of a variable are mapped after -label values-

[Robert Picard <picard@netbox.com>]

The -decode- solution is simple and quick, which is what you asked
for. If your dataset is large and your labels are long, use the
maxlength(#) option. Since you don't mind collapsing your original
data, you can also use -decode- on the reduced set of values and the
size of the data won't matter.

* -------------- begin example ----------------------------

clear
set obs 50
gen s = "this is a lame way to create long labels"
replace s = s + s + s + string(_n)

encode s, gen(n)
drop s

* make a large dataset with 1,000,000 observations

expand 20000

* spike the first observations with values that are not
* mapped to a variable label
replace n = _n + 1000 in 1/5

* first method

preserve

decode n, gen(s) maxlen(4)

tab n if mi(s)

* second method, collapsing to unique values of n

restore
sort n
by n: keep if _n == 1
count

decode n, gen(s)

keep if mi(s)
list, noobs

* -------------- end example ------------------------------



On Thu, Jun 20, 2013 at 5:38 PM, Toby Robertson
<toby.robertson@hotmail.com> wrote:
> In the absence of an answer I've come up with this solution to identify numeric values in the data file that are not present in the lookup file from numeric values to strings that I want to use as labels:
>
> use datafile
> collapse (count)N=somevar, by(myvar)
> merge myvar using lookupfile, nokeep keep() unique sort
>
> Which then permits checks like:
>
> tab _m
> list myvar N if _merge==1
> assert _merge==3
>
> etc. etc.
>
> In other words, do an empty left join of the lookups onto a table of the values in the data, and see which if any values are not in the lookups.
>
> Of the suggestions offered:
>
> - 'tab myvar, missing' and then eyeballing the data is OK for one-off interactive work but is not what I need in this context
>
> - 'decode myvar' defeats the whole purpose of not introducing long string variables into a very large dataset
>
> - 'labelbook mylabel, problem' seems to detect labels that are not used in the data, but not values in the data that are not present in the labels (and doesn't report what the values are anyway)
>
>
>> From: toby.robertson@hotmail.com
>> To: statalist@hsphsun2.harvard.edu
>> Subject: st: Verify that all values of a variable are mapped after -label values-
>> Date: Thu, 20 Jun 2013 19:48:57 +0000
>>
>> What is the easiest way to check whether, having applied a value label to a variable...
>>
>> label values myvar mylabel
>>
>> ...every value of myvar in the dataset is mapped to mylabel?
>>
>> I am using Stata 10, creating value labels from lookup files from numeric code variables to string descriptor variables using -labmask- followed by -label save-, and then applying them to the values of the numeric variables in very large datasets.
>>
>> The obvious solution might be to merge the string variable itself from the lookup file into the target file (after which I could check the integrity of the merge, use -labmask-, and drop the string variable again) because I want to avoid that because of file size and read time considerations.
>>
>> Thanks in advance to anyone who knows the answer offhand!
>>
>> Toby
>> *
>> * For searches and help try:
>> * http://www.stata.com/help.cgi?search
>> * http://www.stata.com/support/faqs/resources/statalist-faq/
>> * http://www.ats.ucla.edu/stat/stata/
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/