Asymptotic Statistics(2000).pdf

1、Asymptotic Statistics This book is an introduction to the field of asymptotic statistics.The treatment is both practical and mathematically rigorous.In addition to most of the standard topics of an asymptotics course,including likelihood inference,M-estimation,asymptotic efficiency,U-statistics,and

2、rank procedures,the book also presents recent research topics such as sernipararnetric models,the bootstrap,and empirical processes and their applications.One of the unifying themes is the approximation by limit experiments.This entails mainly the local approximation of the classical i.i.d.set-up wi

3、th smooth parameters by location experiments involving a single,normally distributed observation.Thus,even the standard subjects of asymptotic statistics are presented in a novel way.Suitable as a text for a graduate or Masters level statistics course,this book also gives researchers in statistics,p

4、robability,and their applications an overview of the latest research in asymptotic statistics.A.W.van der Vaart is Professor of Statistics in the Department of Mathematics and Computer Science at the Vrije Universiteit,Amsterdam.CAMBRIDGE SERIES IN STATISTICAL AND PROBABILISTIC MATHEMATICS Editorial

5、 Board:R.Gill,Department of Mathematics,Utrecht University B.D.Ripley,Department of Statistics,University of Oxford S.Ross,Department of Industrial Engineering,University of California,Berkeley M.Stein,Department of Statistics,University of Chicago D.Williams,School of Mathematical Sciences,Universi

6、ty of Bath This series of high-quality upper-division textbooks and expository monographs covers all aspects of stochastic applicable mathematics.The topics range from pure and applied statistics to probability theory,operations research,optimization,and mathematical programming.The books contain cl

7、ear presentations of new developments in the field and also of the state of the art in classical methods.While emphasizing rigorous treatment of theoretical methods,the books also contain applications and discussions of new techniques made possible by advances in computational practice.Already publi

8、shed 1.Bootstrap Methods and Their Application,by A.C.Davison and D.V.Hinkley 2.Markov Chains,by J.Norris Asymptotic Statistics A.W.VANDER VAART CAMBRIDGE UNIVERSITY PRESS PUBLISHED BY THE PRESS SYNDICATE OF THE UNIVERSITY OF CAMBRIDGE The Pitt Building,Trumpington Street,Cambridge,United Kingdom CA

9、MBRIDGE UNIVERSITY PRESS The Edinburgh Building,Cambridge CB2 2RU,UK http:/www.cup.cam.ac.uk 40 West 20th Street,New York,NY 10011-4211,USA http:/www.cup.org 10 Stamford Road,Oakleigh,Melbourne 3166,Australia Ruiz de Alarcon 13,28014 Madrid,Spain Cambridge University Press 1998 This book is in copyr

10、ight.Subject to statutory exception and to the provisions of relevant collective licensing agreements,no reproduction of any part may take place without the written permission of Cambridge University Press.First published 1998 First paperback edition 2000 Printed in the United States of America Type

11、set in Times Roman 10112.5 pt in LKfff(2 TB A catalog record for this book is available from the British Library Library of Congress Cataloging in Publication data Vaart,A.W.van der Asymtotic statistics I A.W.van der Vaart.p.em.-(Cambridge series in statistical and probablistic mathematics)Includes

12、bibliographical references.1.Mathematical statistics-Asymptotic theory.I.Title.II.Series:cambridge series on statistical and probablistic mathematics.CA276.V22 1998 519.5-dc21 98-15176 ISBN 0 521 49603 9 hardback ISBN 0 521 78450 6 paperback To Maryse and Marianne Contents Preface page xiii Notation

13、 page XV 1.Introduction 1 1.1.Approximate Statistical Procedures 1 1.2.Asymptotic Optimality Theory 2 1.3.Limitations 3 1.4.The Index n 4 2.Stochastic Convergence 5 2.1.Basic Theory 5 2.2.Stochastic o and 0 Symbols 12*2.3.Characteristic Functions 13*2.4.Almost-Sure Representations 17*2.5.Convergence

14、 of Moments 17*2.6.Convergence-Determining Classes 18*2.7.Law of the Iterated Logarithm 19*2.8.Linde berg-Feller Theorem 20*2.9.Convergence in Total Variation 22 Problems 24 3.Delta Method 25 3.1.Basic Result 25 3.2.Variance-Stabilizing Transformations 30*3.3.Higher-Order Expansions 31*3.4.Uniform D

15、elta Method 32*3.5.Moments 33 Problems 34 4.Moment Estimators 35 4.1.Method of Moments 35*4.2.Exponential Families 37 Problems 40 5.M-and Z-Estimators 41 5.1.Introduction 41 5.2.Consistency 44 5.3.Asymptotic Normality 51 Vll Vlll Contents*5.4.Estimated Parameters 60 5.5.Maximum Likelihood Estimators

16、 61*5.6.Classical Conditions 67*5.7.One-Step Estimators 71*5.8.Rates of Convergence 75*5.9.Argmax Theorem 79 Problems 83 6.Contiguity 85 6.1.Likelihood Ratios 85 6.2.Contiguity 87 Problems 91 7.Local Asymptotic Normality 92 7.1.Introduction 92 7.2.Expanding the Likelihood 93 7.3.Convergence to a Nor

17、mal Experiment 97 7.4.Maximum Likelihood 100*7.5.Limit Distributions under Alternatives 103*7.6.Local Asymptotic Normality 103 Problems 106 8.Efficiency of Estimators 108 8.1.Asymptotic Concentration 108 8.2.Relative Efficiency 110 8.3.Lower Bound for Experiments 111 8.4.Estimating Normal Means 112

18、8.5.Convolution Theorem 115 8.6.Almost-Everywhere Convolution Theorem 115*8.7.Local Asymptotic Minimax Theorem 117*8.8.Shrinkage Estimators 119*8.9.Achieving the Bound 120*8.10.Large Deviations 122 Problems 123 9.Limits of Experiments 125 9.1.Introduction 125 9.2.Asymptotic Representation Theorem 12

19、6 9.3.Asymptotic Normality 127 9.4.Uniform Distribution 129 9.5.Pareto Distribution 130 9.6.Asymptotic Mixed Normality 131 9.7.Heuristics 136 Problems 137 10.Bayes Procedures 138 10.1.Introduction 138 10.2.Bernstein-von Mises Theorem 140 Contents ix 10.3.Point Estimators 146*10.4.Consistency 149 Pro

20、blems 152 11.Projections 153 11.1.Projections 153 11.2.Conditional Expectation 155 11.3.Projection onto Sums 157*11.4.Hoeffding Decomposition 157 Problems 160 12.U-Statistics 161 12.1.One-Sample U-Statistics 161 12.2.Two-Sample U-statistics 165*12.3.Degenerate U-Statistics 167 Problems 171 13.Rank,S

21、ign,and Permutation Statistics 173 13.1.Rank Statistics 173 13.2.Signed Rank Statistics 181 13.3.Rank Statistics for Independence 184*13.4.Rank Statistics under Alternatives 184 13.5.Permutation Tests 188*13.6.Rank Central Limit Theorem 190 Problems 190 14.Relative Efficiency of Tests 192 14.1.Asymp

22、totic Power Functions 192 14.2.Consistency 199 14.3.Asymptotic Relative Efficiency 201*14.4.Other Relative Efficiencies 202*14.5.Rescaling Rates 211 Problems 213 15.Efficiency of Tests 215 15.1.Asymptotic Representation Theorem 215 15.2.Testing Normal Means 216 15.3.Local Asymptotic Normality 218 15

23、.4.One-Sample Location 220 15.5.Two-Sample Problems 223 Problems 226 16.Likelihood Ratio Tests 227 16.1.Introduction 227*16.2.Taylor Expansion 229 16.3.Using Local Asymptotic Normality 231 16.4.Asymptotic Power Functions 236 X Contents 16.5.Bartlett Correction 238*16.6.Bahadur Efficiency 238 Problem

24、s 241 17.Chi-Square Tests 242 17.1.Quadratic Forms in Normal Vectors 242 17.2.Pearson Statistic 242 17.3.Estimated Parameters 244 17.4.Testing Independence 247*17.5.Goodness-of-Fit Tests 248*17.6.Asymptotic Efficiency 251 Problems 253 18.Stochastic Convergence in Metric Spaces 255 18.1.Metric and No

25、rmed Spaces 255 18.2.Basic Properties 258 18.3.Bounded Stochastic Processes 260 Problems 263 19.Empirical Processes 265 19.1.Empirical Distribution Functions 265 19.2.Empirical Distributions 269 19.3.Goodness-of-Fit Statistics 277 19.4.Random Functions 279 19.5.Changing Classes 282 19.6.Maximal Ineq

26、ualities 284 Problems 289 20.Functional Delta Method 291 20.1.von Mises Calculus 291 20.2.Hadamard-Differentiable Functions 296 20.3.Some Examples 298 Problems 303 21.Quantiles and Order Statistics 304 21.1.Weak Consistency 304 21.2.Asymptotic Normality 305 21.3.Median Absolute Deviation 310 21.4.Ex

27、treme Values 312 Problems 315 22.L-Statistics 316 22.1.Introduction 316 22.2.Hajek Projection 318 22.3.Delta Method 320 22.4.L-Estimators for Location 323 Problems 324 23.Bootstrap 326 Contents Xl 23.1.Introduction 326 23.2.Consistency 329 23.3.Higher-Order Correctness 334 Problems 339 24.Nonparamet

28、ric Density Estimation 341 24.1 Introduction 341 24.2 Kernel Estimators 341 24.3 Rate Optimality 346 24.4 Estimating a Unimodal Density 349 Problems 356 25.Semiparametric Models 358 25.1 Introduction 358 25.2 Banach and Hilbert Spaces 360 25.3 Tangent Spaces and Information 362 25.4 Efficient Score

29、Functions 368 25.5 Score and Information Operators 371 25.6 Testing 384*25.7 Efficiency and the Delta Method 386 25.8 Efficient Score Equations 391 25.9 General Estimating Equations 400 25.10 Maximum Likelihood Estimators 402 25.11 Approximately Least-Favorable Submodels 408 25.12 Likelihood Equatio

30、ns 419 Problems 431 References 433 Index 439 Preface This book grew out of courses that I gave at various places,including a graduate course in the Statistics Department of Texas A&M University,Masters level courses for mathematics students specializing in statistics at the Vrije Universiteit Amster

31、dam,a course in the DEA program(graduate level)ofUniversite de Paris-sud,and courses in the Dutch AIO-netwerk(graduate level).The mathematical level is mixed.Some parts I have used for second year courses for mathematics students(but they find it tough),other parts I would only recommend for a gradu

32、ate program.The text is written both for students who know about the technical details of measure theory and probability,but little about statistics,and vice versa.This requires brief explanations of statistical methodology,for instance of what a rank test or the bootstrap is about,and there are sim

33、ilar excursions to introduce mathematical details.Familiarity with(higher-dimensional)calculus is necessary in all of the manuscript.Metric and normed spaces are briefly introduced in Chapter 18,when these concepts become necessary for Chapters 19,20,21 and 22,but I do not expect that this would be

34、enough as a first introduction.For Chapter 25 basic knowledge of Hilbert spaces is extremely helpful,although the bare essentials are summarized at the beginning.Measure theory is implicitly assumed in the whole manuscript but can at most places be avoided by skipping proofs,by ignoring the word mea

35、surable or with a bit of handwaving.Because we deal mostly with i.i.d.observations,the simplest limit theorems from probability theory suffice.These are derived in Chapter 2,but prior exposure is helpful.Sections,results or proofs that are preceded by asterisks are either of secondary importance or

36、are out of line with the natural order of the chapters.As the chart in Figure 0.1 shows,many of the chapters are independent from one another,and the book can be used for several different courses.A unifying theme is approximation by a limit experiment.The full theory is not developed(another writin

37、g project is on its way),but the material is limited to the weak topology on experiments,which in 90%of the book is exemplified by the case of smooth parameters of the distribution of i.i.d.observations.For this situation the theory can be developed by relatively simple,direct arguments.Limit experi

38、ments are used to explain efficiency properties,but also why certain procedures asymptotically take a certain form.A second major theme is the application of results on abstract empirical processes.These already have benefits for deriving the usual theorems on M-estimators for Euclidean parameters b

39、ut are indispensable if discussing more involved situations,such as M-estimators with nuisance parameters,chi-square statistics with data-dependent cells,or semiparametric models.The general theory is summarized in about 30 pages,and it is the applications xiii xiv Preface 5 16 23 24 Figure 0.1.Depe

40、ndence chart.A solid arrow means that a chapter is a prerequisite for a next chapter.A dotted arrow means a natural continuation.Vertical or horizontal position has no independent meaning.that we focus on.In a sense,it would have been better to place this material(Chapters 18 and 19)earlier in the b

41、ook,but instead we start with material of more direct statistical relevance and of a less abstract character.A drawback is that a few(starred)proofs point ahead to later chapters.Almost every chapter ends with a Notes section.These are meant to give a rough historical sketch,and to provide entries i

42、n the literature for further reading.They certainly do not give sufficient credit to the original contributions by many authors and are not meant to serve as references in this way.Mathematical statistics obtains its relevance from applications.The subjects of this book have been chosen accordingly.

43、On the other hand,this is a mathematicians book in that we have made some effort to present results in a nice way,without the(unnecessary)lists of regularity conditions that are sometimes found in statistics books.Occasionally,this means that the accompanying proof must be more involved.If this mean

44、s that an idea could go lost,then an informal argument precedes the statement of a result.This does not mean that I have strived after the greatest possible generality.A simple,clean presentation was the main aim.Leiden,September 1997 A.W.van der Vaart A*JE*Cb(T),UC(T),C(T)eoo(T)r(Q),Lr(Q)IIJIIQ,r l

45、iz lloo,liz liT lin C,N,Q,RZ EX,E*X,var X,sdX,Cov X 1Pn,Gn Gp N(tJ-,:E),tn,x;2 Za Xn,a tn,a 1,1.96 if the observations are sampled from the normal or exponential distribution.Normal Exponentiala 5 0.122 0.19 10 0.082 0.14 15 0.070 0.1 1 20 0.065 0.10 25 0.062 0.09 50 0.056 0.07 100 0.053 0.06 a The

46、third column gives approximations based on 10,000 simulations.In many ways the t-test is an uninteresting example.There are many other reasonable test statistics for the same problem.Often their null distributions are difficult to calculate.An asymptotic result similar to the one for the t-statistic

47、 would make them practically applicable at least for large sample sizes.Thus,one aim of asymptotic statistics is to derive the asymptotic distribution of many types of statistics.There are similar benefits when obtaining confidence intervals.For instance,the given approximation result asserts that J

48、fi(Xn-J-L)/Sn is approximately standard normally distributed if J-L is the true mean,whatever its value.This means that,with probability approximately 1-2a,Jfi(Xn-J-L)-za 0 P(d(X11,X)8)-+0.p This is denoted by Xn-+X.In this notation convergence in probability is the same as p d(X11,X)-+0.t More form

49、ally it is a Borel measurable map from some probability space in JFtk.Throughout it is implicitly understood that variables X,g(X),and so forth of which we compute expectations or probabilities are measurable maps on some probability space.5 6 Stochastic Convergence As we shall see,convergence in pr

50、obability is stronger than convergence in distribution.An even stronger mode of convergence is almost-sure convergence.The sequence X11 is said to converge almost surely to X if d(X11,X)-?0 with probability one:P(limd(X11,X)=0)=1.This is denoted by X11 X.Note that convergence in probability and conv