The Method of Maximum Likelihood


Likelihood function

The likelihood function denoted by L(\theta;x_1,\cdots,x_n) of n random variables X_1,\cdots,X_n if the joint density of the n random variables, say f_{X_1,\cdots,X_n}(x_1,\cdots,x_n;\theta), which is considerd to be a function of \theta. In particular, if X_1,\cdots,X_n is a random sample from the density f_X(x:\theta), then the likelihood function is f_X(x_1;\theta) \times \cdots \times f_X(x_n;\theta).

Maximum-Likelihood Estimator

Let L(\theta:x_1,\cdots,x_n) be the likelihood function for the ramdom variables X_1,X_n. If \hat{\theta} = \theta(X_1,\cdots,X_n) as a function of the observations (X_1,\cdots,X_n) is the value which maximize L(\theta,x_1,\cdots,x_n), then \hat{\theta} = \theta(X_1,\cdots,X_n) is the maximum likelihood estimator of \theta.