Re: st: a stat question

[Steven Samuels <sjhsamuels@earthlink.net>]

Walt,

I think that you are right to be concerned.  I would like to know  
more about comparability of sampled populations and of key  
measurements between old and new surveys.

1. Was there multi-stage probability sampling for both old and new  
surveys (cities as PSU's for example? If so, were the target  
populations the same?

2. Are the questions on the key variables are identical in both  
questionnaries?   Do the questions differ in their placement and  
order? Did the questionnaires differ in length?  Was there any  
calibration of old and new questions?

3.  Were previous analyses appropriate to finite-population designs  
(e.g. standard errors based primarily on the number of PSU's?)

-Steve


On Jul 3, 2008, at 9:42 PM, Data Analytics Corp. wrote:


> Hi,
>
> I have a client who has been conducting a survey for several years,  
> each year being a "wave".  This year, the whole survey was changed:  
> most of the questionnaire, sample design and size, locations (prior  
> waves had only four cities, the new one has 8 cities).  I was asked  
> about doing some significance testing on a few variables.  I was  
> very uneasy about this because everything about the surveys is  
> different.  I feel that any comparisons would be shaky because so  
> much has changed.  Stat theory doesn't seem to say anything about  
> comparing across different waves such as this.  Any thoughts on  
> doing significance tests on waves of surveys where everything about  
> the surveys have changed from one wave to the next?  Any literature  
> I can be pointed to?  I've run into this before with other clients,  
> so it's a common occurrence.
>
> Thanks,
>
> Walt
>
> --
> ________________________
>
> Walter R. Paczkowski, Ph.D.
> Data Analytics Corp.
> 44 Hamilton Lane
> Plainsboro, NJ 08536
> ________________________
> (V) 609-936-8999
> (F) 609-936-3733
> dataanalytics@earthlink.net
> www.dataanalyticscorp.com
>
> *
> *   For searches and help try:
> *   http://www.stata.com/support/faqs/res/findit.html
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/