RE: st: Interesting numerical accuracy/collinearity issue

["Schaffer, Mark E" <M.E.Schaffer@hw.ac.uk>]

More good info ... Thanks!

--Mark 

> -----Original Message-----
> From: owner-statalist@hsphsun2.harvard.edu 
> [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of 
> Rodrigo A. Alfaro
> Sent: 13 April 2006 02:08
> To: statalist@hsphsun2.harvard.edu
> Subject: Re: st: Interesting numerical accuracy/collinearity issue
> 
> Just in case: Numerical Recipes book is available on-line
> 
> http://www.numerical-recipes.com/nronline_switcher.html
> 
> Rodrigo.
> 
> 
> ----- Original Message -----
> From: "Stas Kolenikov" <skolenik@gmail.com>
> To: <statalist@hsphsun2.harvard.edu>
> Sent: Wednesday, April 12, 2006 5:45 PM
> Subject: Re: st: Interesting numerical accuracy/collinearity issue
> 
> 
> At your leisure time (probably a few years after retirement???...),
> you might want to check a reference like Numerical Recipes or Demmel's
> Applied Numerical Linear Algebra book from SIAM. The finite
> arithemtics space can be formally set up, and it is really strange;
> the series \sum 1/n converges in that space, for instance, according
> to the difference in partial sums criterion. So yes, you can think
> about linear algebra augmented by finite precision. Stata is generally
> aware of strange properties of that space, so commands like -_rmcoll-
> or  -issymmetric()- or -diag0cnt()- count the differences of matrix
> entries or eigenvalues from zero up to that finite precision. If there
> is a roundoff error introduced at some steps in shifting and scaling,
> -_rmcoll- would be still able to tell if there is collinearity, up to
> numerical accuracy of the X'X matrix.
> 
> The general principles of finite precision arithmetics are generally
> based on condition numbers, which is, roughly, the largest possible
> change in the answer due to infinitesimal change in the inputs of the
> procedure. Invoking appropriate infinitesimal (mathematical rather
> than computer!) calculus, the condition numbers for many matrix
> operations like matrix inversions or determinants or linear systems
> can be shown to be the ratios of the largest to the smallest
> eigenvalues. Suppose this ratio for a your particular matrix is 10^4
> (which is not that huge; in -reg pri wei for trunk disp- with auto
> data, the condition number of the covariance matrix is 4e8). Then by
> taking your variables to the fourth power, you'll get that number to
> 10^16 (not quite so, but you can think of this as the worst case
> scenario), and that is already beyond the double arithmetics routinely
> employed (may be in the guts of Stata there is also quad arithmetics;
> I was coming across it a couple of times in Mata): epsdouble() =
> 2e-16, so a single blurp of that order leads to the change
> epsdouble()*condition# approx= order of one: you cannot trust even the
> first digit of your answer. That's, roughly, why the unscaled
> variables are bad; and why you should center your variables; and why
> integrated processes lead to weird distributions... oops, that's
> another story, sorry :))
> 
> On 4/12/06, Schaffer, Mark E <M.E.Schaffer@hw.ac.uk> wrote:
> > My follow-up question is simple: why does the shifting and 
> scaling used by 
> > Stata's
> > ‑ovtest‑ introduce greater accuracy rather than, say, 
> greater rounding 
> > error?  (Either
> > accuracy or error would remove the numerical collinearity.) 
>  The algebra 
> > doesn't help
> > me here, since all three methods are algebraically equivalent.  I'm 
> > guessing that there's
> > probably a general principle about how best to maintain numerical 
> > precision, but I don't
> > know what it might be.
> 
> 
> --
> Stas Kolenikov
> http://stas.kolenikov.name
> 
> *
> *   For searches and help try:
> *   http://www.stata.com/support/faqs/res/findit.html
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
> *
> *   For searches and help try:
> *   http://www.stata.com/support/faqs/res/findit.html
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
> 
> 

*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/