The Method of Maximum Likelihood

Definition

Likelihood function

The likelihood function denoted by $L(\theta;x_1,\cdots,x_n)$ of n random variables $X_1,\cdots,X_n$ if the joint density of the n random variables, say $f_{X_1,\cdots,X_n}(x_1,\cdots,x_n;\theta)$ , which is considerd to be a function of $\theta$ . In particular, if $X_1,\cdots,X_n$ is a random sample from the density $f_X(x:\theta)$ , then the likelihood function is $f_X(x_1;\theta) \times \cdots \times f_X(x_n;\theta)$ .

Maximum-Likelihood Estimator

Let $L(\theta:x_1,\cdots,x_n)$ be the likelihood function for the ramdom variables $X_1,X_n$ . If $\hat{\theta} = \theta(X_1,\cdots,X_n)$ as a function of the observations $(X_1,\cdots,X_n)$ is the value which maximize $L(\theta,x_1,\cdots,x_n)$ , then $\hat{\theta} = \theta(X_1,\cdots,X_n)$ is the maximum likelihood estimator of $\theta$ .