Properties Of Regular Densities

Densities that are "regular" by Definition 17.3 have three properties which are used in establishing the properties of maximum likelihood estimators:

THEOREM 17.2 Moments of the Derivatives of the Log-Likelihood

D1. ln f (yi 10), gi = d ln f (yi | 0)/d0, and Hi = d2 ln f (yi | 9)/d9d9

i = l,...,n, are all random samples of random variables. This statement follows from our assumption of random sampling. The notation gi (0 0) and Hi (00) indicates the derivative evaluated at 0

Condition D1 is simply a consequence of the definition of the density.

For the moment, we allow the range of yi to depend on the parameters; A(9 0) < yi < B(90). (Consider, for example, finding the maximum likelihood estimator of 0/break for a continuous uniform distribution with range [0, 00].) (In the following, the single integral / ...dyi, would be used to indicate the multiple integration over all the elements of a multivariate of yi if that were necessary). By definition,

BiOo)

Now, differentiate this expression with respect to 00. Leibnitz's theorem gives dizo f i y IO o) dyi d O o

dOo dOo

If the second and third terms go to zero, then we may interchange the operations of differentiation and integration. The necessary condition is that limyia) f (yi | 00) = limyi ^B(e0) f (yi I 0 0) = 0. (Note that the uniform distribution suggested above violates this condition.) Sufficient conditions are that the range of the observed random variable, yi, does not depend on the parameters, which means that d A(00)/d00 = d B(00)/d00 = 0 or that the density is zero at the terminal points. This condition, then, is regularity condition R2. The latter is usually assumed, and we will assume it in what follows. So, d¡ f i yi IO o) dyi f d f i yi IO o)

f iyi I O o) dyi = Eo d ln f iyi I O o) d O o d 2 ln f i yi I O o K, , 9 ln f i yi I O o) d f i yi I O o) j yyiIOo) +

-He^ = f (yi 10 0)-W0-, and the integral of a sum is the sum of integrals. Therefore,

■ d2 ln f (yi 100) ] w f\ d ln f (yi 100) d ln f (yi | 00)