st: Identifying the best scale without a "gold standard"

["Seed, Paul" <paul.seed@kcl.ac.uk>]

Dear Statalist, 

I have six scales, all of which are supposed to measure the same thing (breathlessness)
Factor analysis confirms a single large factor, with different weightings by the different scales.
I can now say in general terms, which scales are best & which worst, but I am 
would like to confirm that the observed differences are not due to chance. 

Method 1: Extract the main factor & use Richard Goldstein's -corcor- to compare 
correlations between the scales & factors 

Method 2 : take the simple average of the scales & use -corcor- as before.

This gives me very different answers:

******** example code ****************
version 11.2
webuse bg2
factor bg2cost1-bg2cost6

* Method 1
predict f1
foreach v in varlist bg2cost1- bg2cost5 { 
	corcor f1 `v' bg2cost6
}

* Method 2 
gen mean_bgcost = ( bg2cost1+ bg2cost2+ bg2cost3+ bg2cost4+ bg2cost5+ bg2cost6)/6
foreach v in varlist bg2cost1- bg2cost5 { 
	corcor  mean_bgcost `v' bg2cost6
}

******** end example ****************

I am fairly sure that method 2 is better, as 
in method 1 there is a circularity about 
using the weighted average; and then showing that 
the variable with the biggest weighting also has 
the biggest correlation.

However, is there also a flaw in method 2?
(apart from the multiple testing issues)
Is there a better approach?
Any thoughts, references,  programs appreciated.


Paul T Seed, Senior Lecturer in Medical Statistics, 
Division of Women's Health, King's College London
Women's Health Academic Centre KHP
020 7188 3642.

"I see no reason to address the comments of your anonymous expert ... I prefer to publish the paper elsewhere" - Albert Einstein








*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/