Epigraph xvii
Preface to the Second Edition xix
Preface to the First Edition xxi
Acknowledgments from First Edition xxv
Notation, Abbreviations, and Key Ideas xxvii
1 Introduction
1.1 Objects and Variables
1.2 Some Multivariate Problems and Techniques
1.2.1 Generalizations of Univariate Techniques
1.2.2 Dependence and Regression
1.2.3 Linear Combinations
1.2.4 Assignment and Dissection
1.2.5 Building Configurations
1.3 The Data Matrix
1.4 Summary Statistics
1.4.1 The Mean Vector and Covariance Matrix
1.4.2 Measures of Multivariate Scatter
1.5 Linear Combinations
1.5.1 The Scaling Transformation
1.5.2 Mahalanobis Transformation
1.5.3 Principal Component Transformation
1.6 Geometrical Ideas
1.7 Graphical Representation
1.7.1 Univariate Scatters
1.7.2 Bivariate Scatters
1.7.3 Harmonic Curves
1.7.4 Parallel Coordinates Plot
1.8 Measures of Multivariate Skewness and Kurtosis
Exercises and Complements
2 Basic Properties of Random Vectors
2.1 Cumulative Distribution Functions and Probability Density Functions
2.2 Population Moments
2.2.1 Expectation and Correlation
2.2.2 Population Mean Vector and Covariance Matrix
2.2.3 Mahalanobis Space
2.2.4 Higher Moments
2.2.5 Conditional Moments
2.3 Characteristic Functions
2.4 Transformations
2.5 The Multivariate Normal Distribution
2.5.1 Definition
2.5.2 Geometry
2.5.3 Properties
2.5.4 Singular Multivariate Normal Distribution
2.5.5 The Matrix Normal Distribution
2.6 Random Samples
2.7 Limit Theorems
Exercises and Complements
3 Nonnormal Distributions
3.1 Introduction
3.2 Some Multivariate Generalizations of Univariate Distributions
3.2.1 Direct Generalization
3.2.2 Common Components
3.2.3 Stochastic Generalizations
3.3 Families of Distributions
3.3.1 The Exponential Family
3.3.2 The Spherical Family
3.3.3 Elliptical Distributions
3.3.4 Stable Distributions
3.4 Insights into Skewness and Kurtosis
3.5 Copulas
3.5.1 The Gaussian Copula
3.5.2 The Clayton–Mardia Copula
3.5.3 Archimedean Copulas
3.5.4 Fréchet-Höffding Bounds
Exercises and Complements
4 Normal Distribution Theory
4.1 Introduction and Characterization
4.1.1 Introduction
4.1.2 A Defintion by Characterization
4.2 Linear Forms
4.3 Transformations of Normal Data Matrices
4.4 The Wishart Distribution
4.4.1 Introduction
4.4.2 Properties of Wishart Matrices
4.4.3 Partitioned Wishart Matrices
4.5 The Hotelling
T² Distribution
4.6 Mahalanobis Distance
4.6.1 The Two-Sample Hotelling T² Statistic
4.6.2 A Decomposition of Mahalanobis Distance
4.7 Statistics Based on the Wishart Distribution
4.8 Other Distributions Related to the Multivariate Normal
Exercises and Complements
5 Estimation
5.1 Likelihood and Sufficiency
5.1.1 The Likelihood Function
5.1.2 Efficient Scores and Fisher's Information
5.1.3 The Cramér–Rao Lower Bound
5.1.4 Sufficiency
5.2 Maximum-likelihood Estimation
5.2.1 General Case
5.2.2 Multivariate Normal Case
5.2.3 Matrix Normal Distribution
5.3 Robust Estimation of Location and Dispersion for Multivariate Distributions
5.3.1 M-Estimates of Location
5.3.2 Minimum Covariance Determinant
5.3.3 Multivariate Trimming
5.3.4 Stahel–Donoho Estimator
5.3.5 Minimum Volume Estimator
5.3.6 Tyler's Estimate of Scatter
5.4 Bayesian Inference
Exercises and Complements
6 Hypothesis Testing
6.1 Introduction
6.2 The Techniques Introduced
6.2.1 The Likelihood Ratio Test (LRT)
6.2.2 The Union Interesection Test (UIT)
6.3 The Techniques Further Illustrated
6.3.1 One-sample hypotheses on µ
6.3.2 One-sample hypotheses on ∑
6.3.3 Multisample Hypotheses
6.4 Simultaneous Confidence Intervals
6.4.1 The one-sample Hotelling T² case
6.4.2 The two-sample Hotelling T² case
6.4.3 Other examples
6.5 The Behrens–Fisher Problem
6.6 Multivariate Hypothesis Testing: Some General Points
6.7 Nonnormal Data
6.8 Mardia’s Nonparametric Test for the Bivariate Two-sample Problem
Exercises and Complements
7 Multivariate Regression Analysis
7.1 Introduction
7.2 Maximum-likelihood Estimation
7.2.1 Maximum-likelihood Estimators for B and ∑
7.2.2 The distribution of B and ∑
7.3 The General Linear Hypothesis
7.3.1 The Likelihood Ratio Test (LRT)
7.3.2 The Union Intersection Test (UIT)
7.3.3 Simultaneous Confidence Intervals
7.4 Design Matrices of Degenerate Rank
7.5 Multiple Correlation
7.5.1 The Effect of the Mean
7.5.2 Multiple Correlation Coefficient
7.5.3 Partial Correlation Coefficient
7.5.4 Measures of Correlation Between Vectors
7.6 Least-squares Estimation
7.6.1 Ordinary Least-squares (OLS) Estimation
7.6.2 Generalized Least Squares
7.6.3 Application to Multivariate Regression
7.6.4 Asymptotic Consistency of Least-squares Estimators
7.7 Discarding of Variables
7.7.1 Dependence Analysis
7.7.2 Interdependance Analysis
Exercises and Complements
8 Graphical Models
8.1 Introduction
8.2 Graphs and Conditional Independence
8.3 Gaussian Graphical Models
8.3.1 Estimation
8.3.2 Model Selection
8.4 Log-linear Graphical Models
8.4.1 Notation
8.4.2 Log-linear Models
8.4.3 Log-linear Models with a Graphical Interpretation
8.5 Directed and Mixed Graphs
Exercises and Complements
9 Principal Component Analysis
9.1 Introduction
9.2 Definition and Properties of Principal Components
9.2.1 Population Principal Components
9.2.2 Sample Principal Components
9.2.3 Further Properties of Principal Components
9.2.4 Correlation Structure
9.2.5 The Effect of Ignoring Some Components
9.2.6 Graphical Representation of Principal Components
9.2.7 Biplots
9.3 Sampling Properties of Principal Components
9.3.1 Maximum-likelihood Estimation for Normal Data
9.3.2 Asymptotic Distributions for Normal Data
9.4 Testing Hypotheses About Principal Components
9.4.1 Introduction
9.4.2 The Hypothesis that (γ1 + ⋯ γk/γ1 + ⋯ γp) = ψ
9.4.3 The Hypothesis that (p – k) Eigenvalues of ∑ are Equal
9.4.4 Hypotheses Concerning Correlation Matrices
9.5 Correspondence Analysis
9.5.1 Contingency Tables
9.5.2 Gradient Analysis
9.6 Allometry – Measurement of Size and Shape
9.7 Discarding of Variables
9.8 Principal Component Regression
9.9 Projection Pursuit and Independent Component Analysis
9.9.1 Projection Pursuit
9.9.2 Independent Component Analysis
9.10 PCA in High Dimensions
Exercises and Complements
10 Factor Analysis
10.1 Introduction
10.2 The Factor Model
10.2.1 Definition
10.2.2 Scale Invariance
10.2.3 Nonuniqueness of Factor Loadings
10.2.4 Estimation of the Parameters in Factor Analysis
10.2.5 Use of the Correlation Matrix R in Estimation
10.3 Principal Factor Analysis
10.4 Maximum-likelihood Factor Analysis
10.5 Goodness-of-fit Test
10.6 Rotation of Factors
10.6.1 Interpretation of Factors
10.6.2 Varimax Rotation
10.7 Factor Scores
10.8 Relationships Between Factor Analysis and Principal Component Analysis
10.9 Analysis of Covariance Structures
Exercises and Complements
11 Canonical Correlation Analysis
11.1 Introduction
11.2 Mathematical Development
11.2.1 Population Canonical Correlation Analysis
11.2.2 Sample Canonical Correlation Analysis
11.2.3 Sample Properties and Tests
11.2.4 Scoring and Prediction
11.3 Qualitative Data and Dummy Variables
11.4 Qualitative and Quantitative Data
Exercises and Complements
12 Discriminant Analysis and Statistical Learning
12.1 Introduction
12.2 Bayes’ Discriminant Rule
12.3 The Error Rate
12.3.1 Probabilities of Misclassification
12.3.2 Estimation of Error Rate
12.3.3 Confusion Matrix
12.4 Discrimination Using the Normal Distribution
12.4.1 Population Discriminant Rules
12.4.2 The Sample Discriminant Rules
12.4.3 Is Discrimination Worthwhile?
12.5 Discarding of Variables
12.6 Fisher’s Linear Discriminant Function
12.7 Nonparametric Distance-based Methods
12.7.1 Nearest-neighbor Classifier
12.7.2 Large Sample Behavior of the Nearest-neighbor Classifier
12.7.3 Kernel Classifiers
12.8 Classification Trees
12.8.1 Splitting Criteria
12.8.2 Pruning
12.9 Logistic Discrimination
12.9.1 Logistic Regression Model
12.9.2 Estimation and Inference
12.9.3 Interpretation of the Parameter Estimates
12.9.4 Extensions
12.10 Neural Networks
12.10.1 Motivation
12.10.2 Multilayer Perceptron
12.10.3 Radial Basis Functions
12.10.4 Support Vector Machines
Exercises and Complements
13 Multivariate Analysis of Variance
13.1 Introduction
13.2 Formulation of Multivariate One-way Classification
13.3 The Likelihood Ratio Principle
13.4 Testing Fixed Contrasts
13.5 Canonical Variables and A Test of Dimensionality
13.5.1 The Problem
13.5.2 The LR Test (∑ Known)
13.5.3 Asymptotic Distribution of the Likelihood Ratio Criterion
13.5.4 The Estimated Plane
13.5.5 The LR Test (Unknown ∑)
13.5.6 The Estimated Plane (Unknown ∑)
13.5.7 Profile Analysis
13.6 The Union Intersection Approach
13.7 Two-way Classification
13.7.1 Tests for Interactions
13.7.2 Tests for Main Effects
13.7.3 Simultaneous Confidence Regions
13.7.4 Extension to Higher Designs
Exercises and Complements
14 Cluster Analysis and Unsupervised Learning
14.1 Introduction
14.2 Probabilistic Membership Models
14.3 Parametric Mixture Models
14.4 Partitioning Methods
14.5 Hierarchical Methods
14.5.1 Agglomerative Algorithms
14.5.2 Minimum Spanning Tree and Single Linkage
14.5.3 Properties of Different Agglomerative Algorithms
14.6 Distances and Similarities
14.6.1 Distances
14.6.2 Similarity Coefficients
14.7 Grouped Data
14.8 Mode Seeking
14.9 Measures of Agreement
Exercises and Complements
15 Multidimensional Scaling
15.1 Introduction
15.2 Classical Solution
15.2.1 Some Theoretical Results
15.2.2 An Algorithm for the Classical MDS Solution
15.2.3 Similarities
15.3 Duality Between Principal Coordinate Analysis and Principal Component Analysis
15.4 Optimal Properties of the Classical Solution and Goodness of Fit
15.5 Seriation
15.5.1 Description
15.5.2 Horseshoe Effect
15.6 Nonmetric Methods
15.7 Goodness of Fit Measure: Procrustes Rotation
15.8 Multisample Problem and Canonical Variates
Exercises and Complements
16 High-dimensional Data
16.1 Introduction
16.2 Shrinkage Methods in Regression
16.2.1 The Multiple Linear Regression Model
16.2.2 Ridge Regression
16.2.3 Least Absolute Selection and Shrinkage Operator (LASSO)
16.3 Principal Component Regression
16.4 Partial Least Squares Regression
16.4.1 Overview
16.4.2 The PLS1 Algorithm to Construct the PLS Loading Matrix for p = 1 Response Variable
16.4.3 The PLS2 Algorithm to Construct the PLS Loading Matric for p > 1 Response Variables
16.4.4 The Predictor Envelope Model
16.4.5 PLS Regression
16.4.6 Joint Envelope Models
16.5 Functional Data
16.5.1 Functional Principal Component Analysis
16.5.2 Functional Linear Regression Models
Exercises and Complements
A Matrix Algebra
A.1 Introduction
A.2 Matrix Operations
A.2.1 Transpose
A.2.2 Trace
A.2.3 Determinants and Cofactors
A.2.4 Inverse
A.2.5 Kronecker Products
A.3 Further Particular Matrices and Types of Matrices 
