# Maximum Likelihood Estimation – finding the best Parametric Model

Consider the situation where a collection of data samples has been obtained. It is assumed that this data is drawn from a parametric model. The parameters of this model are unknown.

For example, the collection of data samples can be the heights of all the students in a class. It is assumed that this data fits a Gaussian distribution. A Gaussian distribution is defined by two parameters – the mean and the standard deviation . The goal is to find the best Gaussian distribution (defined by the parameters) that fits the data.

To find the best parameters that fit the data samples, the probability function of the parameters is computed using the given data. The goal of Maximum Likelihood Estimate is to find the parameter value(s) which maximizes this probability.

## Likelihood

The Likelihood is defined as . This is a function of both the data and the parameter(s) . The likelihood changes as the parameter of interest changes.

## Maximum Likelihood Estimate (MLE)

The maximum likelihood estimate (MLE) for the parameter is the value of that maximizes the likelihood . That is, the MLE is the value of for which the data is most likely.

The notation is used for the MLE. It can be computed by taking the derivative of the likelihood function and setting it to .

## Example 1

A coin is flipped times. Given that there were heads, find the maximum likelihood estimate for the probability of heads on a single toss.

### Data

The data is the result of the experiment. In this case it is 55 heads.

### Parameter(s) of Interest

The value of the unknown parameter

For a given value of , the probability of getting heads in this experiment is the binomial probability.



This is read as “the probability of heads given that the probability of heads on a single toss is .”

Setting the derivative to



Solving for





Thus the MLE is

## Log Likelihood

If is often easier to work with the natural log of the likelihood function. For short this is simply called the log likelihood. Since is an increasing function, the maxima of the likelihood and log likelihood coincide.

## Example 2

Redoing Example 1 using log likelihood





Thus the MLE is

## Example 3

Suppose that a particular gene occurs as one of two alleles ( and ), where allele has frequency in the population. That is, a random copy of the gene is with probability and with probability . Since a diploid genotype consists of two genes, the probability of each genotype is given by

A test of random sample of people found that are , are , and are . Find the MLE of .

### Data

are , are , and are .

### Parameter(s) of Interest

The Likelihood is given by



The log Likelihood is given by



Set the derivative equal to



Solving for ,



which is simply the fraction of alleles among all the genes in the sampled population.