Methods of Multivariate Analysis

Second Edition

Methods of Multivariate Analysis

Second Edition

ALVIN C. RENCHER

Brigham Young University

A JOHN WILEY & SONS, INC. PUBLICATION

This book is printed on acid-free paper.

∞

Copyright c 2002 by John Wiley & Sons, Inc. All rights reserved.

Published simultaneously in Canada.

No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form

or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as

permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior

written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to

the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978)

750-4744. Requests to the Publisher for permission should be addressed to the Permissions Department,

John Wiley & Sons, Inc., 605 Third Avenue, New York, NY 10158-0012, (212) 850-6011, fax (212)

850-6008. E-Mail: PERMREQ@WILEY.COM.

For ordering and customer service, call 1-800-CALL-WILEY.

Library of Congress Cataloging-in-Publication Data

Rencher, Alvin C., 1934–

Methods of multivariate analysis / Alvin C. Rencher.—2nd ed.

p. cm. — (Wiley series in probability and mathematical statistics)

“A Wiley-Interscience publication.”

Includes bibliographical references and index.

ISBN 0-471-41889-7 (cloth)

1. Multivariate analysis. I. Title. II. Series.

QA278 .R45 2001

519.5 35—dc21

2001046735

Printed in the United States of America

10 9 8 7 6 5 4 3 2 1

Contents

1. Introduction

1.1

1.2

1.3

1.4

1

Why Multivariate Analysis?, 1

Prerequisites, 3

Objectives, 3

Basic Types of Data and Analysis, 3

2. Matrix Algebra

5

2.1 Introduction, 5

2.2 Notation and Basic Definitions, 5

2.2.1 Matrices, Vectors, and Scalars, 5

2.2.2 Equality of Vectors and Matrices, 7

2.2.3 Transpose and Symmetric Matrices, 7

2.2.4 Special Matrices, 8

2.3 Operations, 9

2.3.1 Summation and Product Notation, 9

2.3.2 Addition of Matrices and Vectors, 10

2.3.3 Multiplication of Matrices and Vectors, 11

2.4 Partitioned Matrices, 20

2.5 Rank, 22

2.6 Inverse, 23

2.7 Positive Definite Matrices, 25

2.8 Determinants, 26

2.9 Trace, 30

2.10 Orthogonal Vectors and Matrices, 31

2.11 Eigenvalues and Eigenvectors, 32

2.11.1 Definition, 32

2.11.2 I + A and I − A, 33

2.11.3 tr(A) and |A|, 34

2.11.4 Positive Definite and Semidefinite Matrices, 34

2.11.5 The Product AB, 35

2.11.6 Symmetric Matrix, 35

v

vi

CONTENTS

2.11.7

2.11.8

2.11.9

2.11.10

Spectral Decomposition, 35

Square Root Matrix, 36

Square Matrices and Inverse Matrices, 36

Singular Value Decomposition, 36

3. Characterizing and Displaying Multivariate Data

43

3.1 Mean and Variance of a Univariate Random Variable, 43

3.2 Covariance and Correlation of Bivariate Random Variables, 45

3.2.1 Covariance, 45

3.2.2 Correlation, 49

3.3 Scatter Plots of Bivariate Samples, 50

3.4 Graphical Displays for Multivariate Samples, 52

3.5 Mean Vectors, 53

3.6 Covariance Matrices, 57

3.7 Correlation Matrices, 60

3.8 Mean Vectors and Covariance Matrices for Subsets of

Variables, 62

3.8.1 Two Subsets, 62

3.8.2 Three or More Subsets, 64

3.9 Linear Combinations of Variables, 66

3.9.1 Sample Properties, 66

3.9.2 Population Properties, 72

3.10 Measures of Overall Variability, 73

3.11 Estimation of Missing Values, 74

3.12 Distance between Vectors, 76

4. The Multivariate Normal Distribution

4.1 Multivariate Normal Density Function, 82

4.1.1 Univariate Normal Density, 82

4.1.2 Multivariate Normal Density, 83

4.1.3 Generalized Population Variance, 83

4.1.4 Diversity of Applications of the Multivariate Normal, 85

4.2 Properties of Multivariate Normal Random Variables, 85

4.3 Estimation in the Multivariate Normal, 90

4.3.1 Maximum Likelihood Estimation, 90

4.3.2 Distribution of y and S, 91

4.4 Assessing Multivariate Normality, 92

4.4.1 Investigating Univariate Normality, 92

4.4.2 Investigating Multivariate Normality, 96

82

vii

CONTENTS

4.5 Outliers, 99

4.5.1 Outliers in Univariate Samples, 100

4.5.2 Outliers in Multivariate Samples, 101

5. Tests on One or Two Mean Vectors

112

5.1 Multivariate versus Univariate Tests, 112

5.2 Tests on with ⌺ Known, 113

5.2.1 Review of Univariate Test for H0 : µ = µ0

with σ Known, 113

5.2.2 Multivariate Test for H0 : = 0 with ⌺ Known, 114

5.3 Tests on When ⌺ Is Unknown, 117

5.3.1 Review of Univariate t-Test for H0 : µ = µ0 with σ

Unknown, 117

5.3.2 Hotelling’s T 2 -Test for H0 : = 0 with ⌺ Unknown, 117

5.4 Comparing Two Mean Vectors, 121

5.4.1 Review of Univariate Two-Sample t-Test, 121

5.4.2 Multivariate Two-Sample T 2 -Test, 122

5.4.3 Likelihood Ratio Tests, 126

5.5 Tests on Individual Variables Conditional on Rejection of H0 by

the T 2 -Test, 126

5.6 Computation of T 2 , 130

5.6.1 Obtaining T 2 from a MANOVA Program, 130

5.6.2 Obtaining T 2 from Multiple Regression, 130

5.7 Paired Observations Test, 132

5.7.1 Univariate Case, 132

5.7.2 Multivariate Case, 134

5.8 Test for Additional Information, 136

5.9 Profile Analysis, 139

5.9.1 One-Sample Profile Analysis, 139

5.9.2 Two-Sample Profile Analysis, 141

6. Multivariate Analysis of Variance

156

6.1 One-Way Models, 156

6.1.1 Univariate One-Way Analysis of Variance (ANOVA), 156

6.1.2 Multivariate One-Way Analysis of Variance Model

(MANOVA), 158

6.1.3 Wilks’ Test Statistic, 161

6.1.4 Roy’s Test, 164

6.1.5 Pillai and Lawley–Hotelling Tests, 166

viii

CONTENTS

6.2

6.3

6.4

6.5

6.6

6.7

6.8

6.9

6.10

6.11

6.1.6 Unbalanced One-Way MANOVA, 168

6.1.7 Summary of the Four Tests and Relationship to T 2 , 168

6.1.8 Measures of Multivariate Association, 173

Comparison of the Four Manova Test Statistics, 176

Contrasts, 178

6.3.1 Univariate Contrasts, 178

6.3.2 Multivariate Contrasts, 180

Tests on Individual Variables Following Rejection of H0 by the

Overall MANOVA Test, 183

Two-Way Classification, 186

6.5.1 Review of Univariate Two-Way ANOVA, 186

6.5.2 Multivariate Two-Way MANOVA, 188

Other Models, 195

6.6.1 Higher Order Fixed Effects, 195

6.6.2 Mixed Models, 196

Checking on the Assumptions, 198

Profile Analysis, 199

Repeated Measures Designs, 204

6.9.1 Multivariate vs. Univariate Approach, 204

6.9.2 One-Sample Repeated Measures Model, 208

6.9.3 k-Sample Repeated Measures Model, 211

6.9.4 Computation of Repeated Measures Tests, 212

6.9.5 Repeated Measures with Two Within-Subjects

Factors and One Between-Subjects Factor, 213

6.9.6 Repeated Measures with Two Within-Subjects

Factors and Two Between-Subjects Factors, 219

6.9.7 Additional Topics, 221

Growth Curves, 221

6.10.1 Growth Curve for One Sample, 221

6.10.2 Growth Curves for Several Samples, 229

6.10.3 Additional Topics, 230

Tests on a Subvector, 231

6.11.1 Test for Additional Information, 231

6.11.2 Stepwise Selection of Variables, 233

7. Tests on Covariance Matrices

7.1 Introduction, 248

7.2 Testing a Specified Pattern for ⌺, 248

7.2.1 Testing H0 : ⌺ = ⌺0 , 248

248

CONTENTS

ix

7.2.2 Testing Sphericity, 250

7.2.3 Testing H0 : ⌺ = σ 2 [(1 − ρ)I + ρJ], 252

7.3 Tests Comparing Covariance Matrices, 254

7.3.1 Univariate Tests of Equality of Variances, 254

7.3.2 Multivariate Tests of Equality of Covariance Matrices, 255

7.4 Tests of Independence, 259

7.4.1 Independence of Two Subvectors, 259

7.4.2 Independence of Several Subvectors, 261

7.4.3 Test for Independence of All Variables, 265

8. Discriminant Analysis: Description of Group Separation

270

8.1 Introduction, 270

8.2 The Discriminant Function for Two Groups, 271

8.3 Relationship between Two-Group Discriminant Analysis and

Multiple Regression, 275

8.4 Discriminant Analysis for Several Groups, 277

8.4.1 Discriminant Functions, 277

8.4.2 A Measure of Association for Discriminant Functions, 282

8.5 Standardized Discriminant Functions, 282

8.6 Tests of Significance, 284

8.6.1 Tests for the Two-Group Case, 284

8.6.2 Tests for the Several-Group Case, 285

8.7 Interpretation of Discriminant Functions, 288

8.7.1 Standardized Coefficients, 289

8.7.2 Partial F-Values, 290

8.7.3 Correlations between Variables and Discriminant

Functions, 291

8.7.4 Rotation, 291

8.8 Scatter Plots, 291

8.9 Stepwise Selection of Variables, 293

9. Classification Analysis: Allocation of Observations to Groups

9.1 Introduction, 299

9.2 Classification into Two Groups, 300

9.3 Classification into Several Groups, 304

9.3.1 Equal Population Covariance Matrices: Linear

Classification Functions, 304

9.3.2 Unequal Population Covariance Matrices: Quadratic

Classification Functions, 306

299

x

CONTENTS

9.4 Estimating Misclassification Rates, 307

9.5 Improved Estimates of Error Rates, 309

9.5.1 Partitioning the Sample, 310

9.5.2 Holdout Method, 310

9.6 Subset Selection, 311

9.7 Nonparametric Procedures, 314

9.7.1 Multinomial Data, 314

9.7.2 Classification Based on Density Estimators, 315

9.7.3 Nearest Neighbor Classification Rule, 318

10. Multivariate Regression

322

10.1 Introduction, 322

10.2 Multiple Regression: Fixed x’s, 323

10.2.1 Model for Fixed x’s, 323

10.2.2 Least Squares Estimation in the Fixed-x Model, 324

10.2.3 An Estimator for σ 2 , 326

10.2.4 The Model Corrected for Means, 327

10.2.5 Hypothesis Tests, 329

10.2.6 R 2 in Fixed-x Regression, 332

10.2.7 Subset Selection, 333

10.3 Multiple Regression: Random x’s, 337

10.4 Multivariate Multiple Regression: Estimation, 337

10.4.1 The Multivariate Linear Model, 337

10.4.2 Least Squares Estimation in the Multivariate Model, 339

ˆ 341

10.4.3 Properties of Least Squares Estimators B,

10.4.4 An Estimator for ⌺, 342

10.4.5 Model Corrected for Means, 342

10.5 Multivariate Multiple Regression: Hypothesis Tests, 343

10.5.1 Test of Overall Regression, 343

10.5.2 Test on a Subset of the x’s, 347

10.6 Measures of Association between the y’s and the x’s, 349

10.7 Subset Selection, 351

10.7.1 Stepwise Procedures, 351

10.7.2 All Possible Subsets, 355

10.8 Multivariate Regression: Random x’s, 358

11. Canonical Correlation

11.1 Introduction, 361

11.2 Canonical Correlations and Canonical Variates, 361

361

CONTENTS

xi

11.3 Properties of Canonical Correlations, 366

11.4 Tests of Significance, 367

11.4.1 Tests of No Relationship between the y’s and the x’s, 367

11.4.2 Test of Significance of Succeeding Canonical

Correlations after the First, 369

11.5 Interpretation, 371

11.5.1 Standardized Coefficients, 371

11.5.2 Correlations between Variables and Canonical Variates, 373

11.5.3 Rotation, 373

11.5.4 Redundancy Analysis, 373

11.6 Relationships of Canonical Correlation Analysis to Other

Multivariate Techniques, 374

11.6.1 Regression, 374

11.6.2 MANOVA and Discriminant Analysis, 376

12. Principal Component Analysis

380

12.1 Introduction, 380

12.2 Geometric and Algebraic Bases of Principal Components, 381

12.2.1 Geometric Approach, 381

12.2.2 Algebraic Approach, 385

12.3 Principal Components and Perpendicular Regression, 387

12.4 Plotting of Principal Components, 389

12.5 Principal Components from the Correlation Matrix, 393

12.6 Deciding How Many Components to Retain, 397

12.7 Information in the Last Few Principal Components, 401

12.8 Interpretation of Principal Components, 401

12.8.1 Special Patterns in S or R, 402

12.8.2 Rotation, 403

12.8.3 Correlations between Variables and Principal

Components, 403

12.9 Selection of Variables, 404

13. Factor Analysis

13.1 Introduction, 408

13.2 Orthogonal Factor Model, 409

13.2.1 Model Definition and Assumptions, 409

13.2.2 Nonuniqueness of Factor Loadings, 414

13.3 Estimation of Loadings and Communalities, 415

13.3.1 Principal Component Method, 415

13.3.2 Principal Factor Method, 421

408

xii

CONTENTS

13.4

13.5

13.6

13.7

13.8

13.3.3 Iterated Principal Factor Method, 424

13.3.4 Maximum Likelihood Method, 425

Choosing the Number of Factors, m, 426

Rotation, 430

13.5.1 Introduction, 430

13.5.2 Orthogonal Rotation, 431

13.5.3 Oblique Rotation, 435

13.5.4 Interpretation, 438

Factor Scores, 438

Validity of the Factor Analysis Model, 443

The Relationship of Factor Analysis to Principal Component

Analysis, 447

14. Cluster Analysis

451

14.1 Introduction, 451

14.2 Measures of Similarity or Dissimilarity, 452

14.3 Hierarchical Clustering, 455

14.3.1 Introduction, 455

14.3.2 Single Linkage (Nearest Neighbor), 456

14.3.3 Complete Linkage (Farthest Neighbor), 459

14.3.4 Average Linkage, 463

14.3.5 Centroid, 463

14.3.6 Median, 466

14.3.7 Ward’s Method, 466

14.3.8 Flexible Beta Method, 468

14.3.9 Properties of Hierarchical Methods, 471

14.3.10 Divisive Methods, 479

14.4 Nonhierarchical Methods, 481

14.4.1 Partitioning, 481

14.4.2 Other Methods, 490

14.5 Choosing the Number of Clusters, 494

14.6 Cluster Validity, 496

14.7 Clustering Variables, 497

15. Graphical Procedures

15.1 Multidimensional Scaling, 504

15.1.1 Introduction, 504

15.1.2 Metric Multidimensional Scaling, 505

15.1.3 Nonmetric Multidimensional Scaling, 508

504

CONTENTS

xiii

15.2 Correspondence Analysis, 514

15.2.1 Introduction, 514

15.2.2 Row and Column Profiles, 515

15.2.3 Testing Independence, 519

15.2.4 Coordinates for Plotting Row and Column Profiles, 521

15.2.5 Multiple Correspondence Analysis, 526

15.3 Biplots, 531

15.3.1 Introduction, 531

15.3.2 Principal Component Plots, 531

15.3.3 Singular Value Decomposition Plots, 532

15.3.4 Coordinates, 533

15.3.5 Other Methods, 535

A. Tables

549

B. Answers and Hints to Problems

591

C. Data Sets and SAS Files

679

References

681

Index

695

Preface

I have long been fascinated by the interplay of variables in multivariate data and by

the challenge of unraveling the effect of each variable. My continuing objective in

the second edition has been to present the power and utility of multivariate analysis

in a highly readable format.

Practitioners and researchers in all applied disciplines often measure several variables on each subject or experimental unit. In some cases, it may be productive to

isolate each variable in a system and study it separately. Typically, however, the variables are not only correlated with each other, but each variable is influenced by the

other variables as it affects a test statistic or descriptive statistic. Thus, in many

instances, the variables are intertwined in such a way that when analyzed individually they yield little information about the system. Using multivariate analysis, the

variables can be examined simultaneously in order to access the key features of the

process that produced them. The multivariate approach enables us to (1) explore

the joint performance of the variables and (2) determine the effect of each variable

in the presence of the others.

Multivariate analysis provides both descriptive and inferential procedures—we

can search for patterns in the data or test hypotheses about patterns of a priori interest. With multivariate descriptive techniques, we can peer beneath the tangled web of

variables on the surface and extract the essence of the system. Multivariate inferential

procedures include hypothesis tests that (1) process any number of variables without

inflating the Type I error rate and (2) allow for whatever intercorrelations the variables possess. A wide variety of multivariate descriptive and inferential procedures

is readily accessible in statistical software packages.

My selection of topics for this volume reflects many years of consulting with

researchers in many fields of inquiry. A brief overview of multivariate analysis is

given in Chapter 1. Chapter 2 reviews the fundamentals of matrix algebra. Chapters

3 and 4 give an introduction to sampling from multivariate populations. Chapters 5,

6, 7, 10, and 11 extend univariate procedures with one dependent variable (including

t-tests, analysis of variance, tests on variances, multiple regression, and multiple correlation) to analogous multivariate techniques involving several dependent variables.

A review of each univariate procedure is presented before covering the multivariate

counterpart. These reviews may provide key insights the student missed in previous

courses.

Chapters 8, 9, 12, 13, 14, and 15 describe multivariate techniques that are not

extensions of univariate procedures. In Chapters 8 and 9, we find functions of the

variables that discriminate among groups in the data. In Chapters 12 and 13, we

xv

xvi

PREFACE

find functions of the variables that reveal the basic dimensionality and characteristic

patterns of the data, and we discuss procedures for finding the underlying latent

variables of a system. In Chapters 14 and 15 (new in the second edition), we give

methods for searching for groups in the data, and we provide plotting techniques that

show relationships in a reduced dimensionality for various kinds of data.

In Appendix A, tables are provided for many multivariate distributions and tests.

These enable the reader to conduct an exact test in many cases for which software

packages provide only approximate tests. Appendix B gives answers and hints for

most of the problems in the book.

Appendix C describes an ftp site that contains (1) all data sets and (2) SAS command files for all examples in the text. These command files can be adapted for use

in working problems or in analyzing data sets encountered in applications.

To illustrate multivariate applications, I have provided many examples and exercises based on 59 real data sets from a wide variety of disciplines. A practitioner

or consultant in multivariate analysis gains insights and acumen from long experience in working with data. It is not expected that a student can achieve this kind of

seasoning in a one-semester class. However, the examples provide a good start, and

further development is gained by working problems with the data sets. For example,

in Chapters 12 and 13, the exercises cover several typical patterns in the covariance

or correlation matrix. The student’s intuition is expanded by associating these covariance patterns with the resulting configuration of the principal components or factors.

Although this is a methods book, I have included a few derivations. For some

readers, an occasional proof provides insights obtainable in no other way. I hope that

instructors who do not wish to use proofs will not be deterred by their presence. The

proofs can be disregarded easily when reading the book.

My objective has been to make the book accessible to readers who have taken as

few as two statistical methods courses. The students in my classes in multivariate

analysis include majors in statistics and majors from other departments. With the

applied researcher in mind, I have provided careful intuitive explanations of the concepts and have included many insights typically available only in journal articles or

in the minds of practitioners.

My overriding goal in preparation of this book has been clarity of exposition. I

hope that students and instructors alike will find this multivariate text more comfortable than most. In the final stages of development of both the first and second

editions, I asked my students for written reports on their initial reaction as they read

each day’s assignment. They made many comments that led to improvements in the

manuscript. I will be very grateful if readers will take the time to notify me of errors

or of other suggestions they might have for improvements.

I have tried to use standard mathematical and statistical notation as far as possible and to maintain consistency of notation throughout the book. I have refrained

from the use of abbreviations and mnemonic devices. These save space when one

is reading a book page by page, but they are annoying to those using a book as a

reference.

Equations are numbered sequentially throughout a chapter; for example, (3.75)

indicates the 75th numbered equation in Chapter 3. Tables and figures are also num-

PREFACE

xvii

bered sequentially throughout a chapter in the form “Table 3.8” or “Figure 3.1.”

Examples are not numbered sequentially; each example is identified by the same

number as the section in which it appears and is placed at the end of the section.

When citing references in the text, I have used the standard format involving the

year of publication. For a journal article, the year alone suffices, for example, Fisher

(1936). But for books, I have usually included a page number, as in Seber (1984,

p. 216).

This is the first volume of a two-volume set on multivariate analysis. The second

volume is entitled Multivariate Statistical Inference and Applications (Wiley, 1998).

The two volumes are not necessarily sequential; they can be read independently. I

adopted the two-volume format in order to (1) provide broader coverage than would

be possible in a single volume and (2) offer the reader a choice of approach.

The second volume includes proofs of many techniques covered in the first 13

chapters of the present volume and also introduces additional topics. The present

volume includes many examples and problems using actual data sets, and there are

fewer algebraic problems. The second volume emphasizes derivations of the results

and contains fewer examples and problems with real data. The present volume has

fewer references to the literature than the other volume, which includes a careful

review of the latest developments and a more comprehensive bibliography. In this

second edition, I have occasionally referred the reader to Rencher (1998) to note that

added coverage of a certain subject is available in the second volume.

I am indebted to many individuals in the preparation of the first edition. My initial exposure to multivariate analysis came in courses taught by Rolf Bargmann at

the University of Georgia and D. R. Jensen at Virginia Tech. Additional impetus to

probe the subtleties of this field came from research conducted with Bruce Brown

at BYU. I wish to thank Bruce Brown, Deane Branstetter, Del Scott, Robert Smidt,

and Ingram Olkin for reading various versions of the manuscript and making valuable suggestions. I am grateful to the following students at BYU who helped with

computations and typing: Mitchell Tolland, Tawnia Newton, Marianne Matis Mohr,

Gregg Littlefield, Suzanne Kimball, Wendy Nielsen, Tiffany Nordgren, David Whiting, Karla Wasden, and Rachel Jones.

SECOND EDITION

For the second edition, I have added Chapters 14 and 15, covering cluster analysis,

multidimensional scaling, correspondence analysis, and biplots. I also made numerous corrections and revisions (almost every page) in the first 13 chapters, in an effort

to improve composition, readability, and clarity. Many of the first 13 chapters now

have additional problems.

I have listed the data sets and SAS files on the Wiley ftp site rather than on a

diskette, as in the first edition. I have made improvements in labeling of these files.

I am grateful to the many readers who have pointed out errors or made suggestions

for improvements. The book is better for their caring and their efforts.

xviii

PREFACE

I thank Lonette Stoddard and Candace B. McNaughton for typing and J. D.

Williams for computer support. As with my other books, I dedicate this volume to

my wife, LaRue, who has supplied much needed support and encouragement.

A LVIN C. R ENCHER

Acknowledgments

I thank the authors, editors, and owners of copyrights for permission to reproduce

the following materials:

•

Figure 3.8 and Table 3.2, Kleiner and Hartigan (1981), Reprinted by permission

of Journal of the American Statistical Association

•

Table 3.3, Kramer and Jensen (1969a), Reprinted by permission of Journal of

Quality Technology

•

Table 3.4, Reaven and Miller (1979), Reprinted by permission of Diabetologia

•

Table 3.5, Timm (1975), Reprinted by permission of Elsevier North-Holland

Publishing Company

•

Table 3.6, Elston and Grizzle (1962), Reprinted by permission of Biometrics

•

Table 3.7, Frets (1921), Reprinted by permission of Genetica

•

Table 3.8, O’Sullivan and Mahan (1966), Reprinted by permission of American

Journal of Clinical Nutrition

•

Table 4.3, Royston (1983), Reprinted by permission of Applied Statistics

•

Table 5.1, Beall (1945), Reprinted by permission of Psychometrika

•

Table 5.2, Hummel and Sligo (1971), Reprinted by permission of Psychological

Bulletin

•

Table 5.3, Kramer and Jensen (1969b), Reprinted by permission of Journal of

Quality Technology

•

Table 5.5, Lubischew (1962), Reprinted by permission of Biometrics

•

Table 5.6, Travers (1939), Reprinted by permission of Psychometrika

•

Table 5.7, Andrews and Herzberg (1985), Reprinted by permission of SpringerVerlag

•

Table 5.8, Tintner (1946), Reprinted by permission of Journal of the American

Statistical Association

•

Table 5.9, Kramer (1972), Reprinted by permission of the author

•

Table 5.10, Cameron and Pauling (1978), Reprinted by permission of National

Academy of Science

xix

xx

ACKNOWLEDGMENTS

•

Table 6.2, Andrews and Herzberg (1985), Reprinted by permission of SpringerVerlag

•

Table 6.3, Rencher and Scott (1990), Reprinted by permission of Communications in Statistics: Simulation and Computation

•

Table 6.6, Posten (1962), Reprinted by permission of the author

•

Table 6.8, Crowder and Hand (1990, pp. 21–29), Reprinted by permission of

Routledge Chapman and Hall

•

Table 6.12, Cochran and Cox (1957), Timm (1980), Reprinted by permission

of John Wiley and Sons and Elsevier North-Holland Publishing Company

•

Table 6.14, Timm (1980), Reprinted by permission of Elsevier North-Holland

Publishing Company

•

Table 6.16, Potthoff and Roy (1964), Reprinted by permission of Biometrika

Trustees

•

Table 6.17, Baten, Tack, and Baeder (1958), Reprinted by permission of Quality

Progress

•

Table 6.18, Keuls et al. (1984), Reprinted by permission of Scientia Horticulturae

•

Table 6.19, Burdick (1979), Reprinted by permission of the author

•

Table 6.20, Box (1950), Reprinted by permission of Biometrics

•

Table 6.21, Rao (1948), Reprinted by permission of Biometrika Trustees

•

Table 6.22, Cameron and Pauling (1978), Reprinted by permission of National

Academy of Science

•

Table 6.23, Williams and Izenman (1989), Reprinted by permission of Colorado

State University

•

Table 6.24, Beauchamp and Hoel (1974), Reprinted by permission of Journal

of Statistical Computation and Simulation

•

Table 6.25, Box (1950), Reprinted by permission of Biometrics

•

Table 6.26, Grizzle and Allen (1969), Reprinted by permission of Biometrics

•

Table 6.27, Crepeau et al. (1985), Reprinted by permission of Biometrics

•

Table 6.28, Zerbe (1979a), Reprinted by permission of Journal of the American

Statistical Association

•

Table 6.29, Timm (1980), Reprinted by permission of Elsevier North-Holland

Publishing Company

•

Table 7.1, Siotani et al. (1963), Reprinted by permission of the Institute of Statistical Mathematics

ACKNOWLEDGMENTS

xxi

•

Table 7.2, Reprinted by permission of R. J. Freund

•

Table 8.1, Kramer and Jensen (1969a), Reprinted by permission of Journal of

Quality Technology

•

Table 8.3, Reprinted by permission of G. R. Bryce and R. M. Barker

•

Table 10.1, Box and Youle (1955), Reprinted by permission of Biometrics

•

Tables 12.2, 12.3, and 12.4, Jeffers (1967), Reprinted by permission of Applied

Statistics

•

Table 13.1, Brown et al. (1984), Reprinted by permission of the Journal of

Pascal, Ada, and Modula

•

Correlation matrix in Example 13.6, Brown, Strong, and Rencher (1973),

Reprinted by permission of The Journal of the Acoustical Society of America

•

Table 14.1, Hartigan (1975), Reprinted by permission of John Wiley and Sons

•

Table 14.3, Dawkins (1989), Reprinted by permission of The American Statistician

•

Table 14.7, Hand et al. (1994), Reprinted by permission of D. J. Hand

•

Table 14.12, Sokol and Rohlf (1981), Reprinted by permission of W. H. Freeman and Co.

•

Table 14.13, Hand et al. (1994), Reprinted by permission of D. J. Hand

•

Table 15.1, Kruskal and Wish (1978), Reprinted by permission of Sage Publications

•

Tables 15.2 and 15.5, Hand et al. (1994), Reprinted by permission of D. J. Hand

•

Table 15.13, Edwards and Kreiner (1983), Reprinted by permission of Biometrika

•

Table 15.15, Hand et al. (1994), Reprinted by permission of D. J. Hand

•

Table 15.16, Everitt (1987), Reprinted by permission of the author

•

Table 15.17, Andrews and Herzberg (1985), Reprinted by permission of

Springer Verlag

•

Table 15.18, Clausen (1988), Reprinted by permission of Sage Publications

•

Table 15.19, Andrews and Herzberg (1985), Reprinted by permission of

Springer Verlag

•

Table A.1, Mulholland (1977), Reprinted by permission of Biometrika Trustees

•

Table A.2, D’Agostino and Pearson (1973), Reprinted by permission of

Biometrika Trustees

•

Table A.3, D’Agostino and Tietjen (1971), Reprinted by permission of Biometrika

Trustees

xxii

ACKNOWLEDGMENTS

•

Table A.4, D’Agostino (1972), Reprinted by permission of Biometrika Trustees

•

Table A.5, Mardia (1970, 1974), Reprinted by permission of Biometrika

Trustees

•

Table A.6, Barnett and Lewis (1978), Reprinted by permission of John Wiley

and Sons

•

Table A.7, Kramer and Jensen (1969a), Reprinted by permission of Journal of

Quality Technology

•

Table A.8, Bailey (1977), Reprinted by permission of Journal of the American

Statistical Association

•

Table A.9, Wall (1967), Reprinted by permission of the author, Albuquerque,

NM

•

Table A.10, Pearson and Hartley (1972) and Pillai (1964, 1965), Reprinted by

permission of Biometrika Trustees

•

Table A.11, Schuurmann et al. (1975), Reprinted by permission of Journal of

Statistical Computation and Simulation

•

Table A.12, Davis (1970a,b, 1980), Reprinted by permission of Biometrika

Trustees

•

Table A.13, Kleinbaum, Kupper, and Muller (1988), Reprinted by permission

of PWS-KENT Publishing Company

•

Table A.14, Lee et al. (1977), Reprinted by permission of Elsevier NorthHolland Publishing Company

•

Table A.15, Mathai and Katiyar (1979), Reprinted by permission of Biometrika

Trustees

CHAPTER 1

Introduction

1.1 WHY MULTIVARIATE ANALYSIS?

Multivariate analysis consists of a collection of methods that can be used when several measurements are made on each individual or object in one or more samples. We

will refer to the measurements as variables and to the individuals or objects as units

(research units, sampling units, or experimental units) or observations. In practice,

multivariate data sets are common, although they are not always analyzed as such.

But the exclusive use of univariate procedures with such data is no longer excusable,

given the availability of multivariate techniques and inexpensive computing power

to carry them out.

Historically, the bulk of applications of multivariate techniques have been in the

behavioral and biological sciences. However, interest in multivariate methods has

now spread to numerous other fields of investigation. For example, I have collaborated on multivariate problems with researchers in education, chemistry, physics,

geology, engineering, law, business, literature, religion, public broadcasting, nursing, mining, linguistics, biology, psychology, and many other fields. Table 1.1 shows

some examples of multivariate observations.

The reader will notice that in some cases all the variables are measured in the same

scale (see 1 and 2 in Table 1.1). In other cases, measurements are in different scales

(see 3 in Table 1.1). In a few techniques, such as profile analysis (Sections 5.9 and

6.8), the variables must be commensurate, that is, similar in scale of measurement;

however, most multivariate methods do not require this.

Ordinarily the variables are measured simultaneously on each sampling unit. Typically, these variables are correlated. If this were not so, there would be little use for

many of the techniques of multivariate analysis. We need to untangle the overlapping

information provided by correlated variables and peer beneath the surface to see the

underlying structure. Thus the goal of many multivariate approaches is simplification. We seek to express what is going on in terms of a reduced set of dimensions.

Such multivariate techniques are exploratory; they essentially generate hypotheses

rather than test them.

On the other hand, if our goal is a formal hypothesis test, we need a technique that

will (1) allow several variables to be tested and still preserve the significance level

1

2

INTRODUCTION

Table 1.1. Examples of Multivariate Data

Units

1. Students

2. Students

3. People

4. Skulls

5. Companies

6. Manufactured items

7. Applicants for bank loans

8. Segments of literature

9. Human hairs

10. Birds

Variables

Several exam scores in a single course

Grades in mathematics, history, music, art, physics

Height, weight, percentage of body fat, resting heart

rate

Length, width, cranial capacity

Expenditures for advertising, labor, raw materials

Various measurements to check on compliance with

specifications

Income, education level, length of residence, savings

account, current debt load

Sentence length, frequency of usage of certain words

and of style characteristics

Composition of various elements

Lengths of various bones

and (2) do this for any intercorrelation structure of the variables. Many such tests are

available.

As the two preceding paragraphs imply, multivariate analysis is concerned generally with two areas, descriptive and inferential statistics. In the descriptive realm, we

often obtain optimal linear combinations of variables. The optimality criterion varies

from one technique to another, depending on the goal in each case. Although linear

combinations may seem too simple to reveal the underlying structure, we use them

for two obvious reasons: (1) they have mathematical tractability (linear approximations are used throughout all science for the same reason) and (2) they often perform

well in practice. These linear functions may also be useful as a follow-up to inferential procedures. When we have a statistically significant test result that compares

several groups, for example, we can find the linear combination (or combinations)

of variables that led to rejection of the hypothesis. Then the contribution of each

variable to these linear combinations is of interest.

In the inferential area, many multivariate techniques are extensions of univariate

procedures. In such cases, we review the univariate procedure before presenting the

analogous multivariate approach.

Multivariate inference is especially useful in curbing the researcher’s natural tendency to read too much into the data. Total control is provided for experimentwise

error rate; that is, no matter how many variables are tested simultaneously, the value

of α (the significance level) remains at the level set by the researcher.

Some authors warn against applying the common multivariate techniques to data

for which the measurement scale is not interval or ratio. It has been found, however,

that many multivariate techniques give reliable results when applied to ordinal data.

For many years the applications lagged behind the theory because the computations were beyond the power of the available desktop calculators. However, with

modern computers, virtually any analysis one desires, no matter how many variables

## Statistical Methods of Valuation and Risk Assessment: Empirical Analysis of Equity Markets and Hedge Fund Strategies

## Tài liệu Negotiations in project sales and delivery process: An application of negotiation analysis pdf

## Multivariate Analysis of Ecological Data ppt

## mathematical methods of engineering analysis - e. cinlar, r. vanderbei

## MATRIX METHODS OF CIRCULAR ANALYSIS

## handbook of stochastic analysis with applications - d. kannan , v. lakshmikantham

## methods of behavior analysis in neuroscience - jerry j. buccafusco

## paul levine - the midas method of technical analysis

## the best trendline methods of alan andrews & 5 new trendline techniques

## quantitative methods for inestment analysis

Tài liệu liên quan