Lecture 2 —Friday, January 13, 2006

What was covered?

• Bernoulli distribution
• binomial distribution
• Poisson distribution—assumptions of

Terminology Defined

• Bernoulli distribution
• binomial distribution
• permutation
• combination
• binomial coefficient
• expectation of a sum
• variance of a sum
• covariance
• Poisson distribution
• homogeneity

Bernoulli Distribution

• The Bernoulli distribution is not the simplest probability distribution imaginable (a one-point distribution, in which all the probability is assigned to a single point, would probably qualify for that), but it is probably the simplest useful distribution.
• A Bernoulli random variable X, meaning X is a random variable that has a Bernoulli distribution, is one that has the following characteristics.
  1. It is dichotomous, i.e., it can take only one of two values. By convention these values are denoted 1 and 0, or "success" and "failure", or in habitat suitability studies, "present" and "absent".
  2. The probability of a success is p, i.e., P(X = 1) = p. Since this is a probability distribution, in which the sum of the probabilities over all possible values is 1, it follows that P(X = 0) = 1 – p.
• In words we would say X has a Bernoulli distribution with parameter p.
• Mean of a Bernoulli random variable.

• Variance of a Bernoulli random variable. I use the alternate formula for the variance mentioned in lecture 1.

To use this I first need to calculate .

Finally I calculate the variance.

• A typical example where a Bernoulli distribution arises in ecology is in habitat suitability studies. Here we might obtain a random sample of quadrats and then in each quadrat determine whether a particular species of interest is present or absent. Thus we would have a random sample of Bernoulli random variables. We then might use other variables measured for each quadrat to develop a model that predicts species presence or absence. The standard parametric modeling tool when dealing with Bernoulli random variables is logistic regression.

Binomial Distribution

• Sometimes rather than observing individual Bernoulli outcomes we only see the results collectively. Or perhaps we don't care about the individual results but just the total number of successes.
• Thus if are independent and identically distributed Bernoulli random variables each with parameter p, then

is said to have a binomial distribution with parameters n and p. We write this as .

• The explicit assumptions that go into the definition of a binomial random variable X are the following.
  1. The experiment consists of a fixed number, n, of Bernoulli trials.
  2. Because the trials are Bernoulli only one of two possible outcomes can occur on each trial. We refer to these as success and failure.
  3. The Bernoulli trials are identically distributed. Thus the probability of a success is the same on each trial. We denote it by p.
  4. The outcomes of separate Bernoulli trials are independent of each other.

  X then counts the number of successes that occur in these n trials.

• We are interested in a formula for . Note: by the constraints of the problem it is clear that in order for this probability to make sense.
• To make the notation simpler let's consider an explicit example. Suppose n = 5 and we observe k = 3 successes. We wish to calculate .
• Let S denote a success and let F denote a failure. Suppose in our example we actually observed the outcomes in the following order: SSFSF.
• Now if we wanted to know the probability of obtaining 3 successes out of 5 trials when the outcomes were observed in the order SSFSF, that we can do. Denote the individual Bernoulli random variables as . Then the probability of observing SSFSF can be written as

or using more formal mathematical notation as

• Recall that two events A and B are said to be independent if and only if . This generalizes to the intersection of any number of independent events. Since our Bernoulli trials are independent the above probability becomes



• But this is only one specific way of obtaining 3 successes and 2 failures. Another possible outcome would be SFFSS. Clearly we would get for this order too. So what we need to determine is how many different orders of this kind there are and then multiply by that number.
• What we're looking for is the number of ways of arranging 5 objects when 3 of them are of one kind (successes) and two of them are of another kind (failures). In other words we want to know the number of permutations of 5 objects when some of them are indistinguishable from each other.
• Well if all 5 objects were distinguishable, then the fundamental law of counting says there are 5! ways of arranging them in a linear order, where .
  • Why is 5! the answer? Think of this as the number of ways of filling 5 slots with the letters A, B, C, D, E where no letter is repeated. When we start at slot 1 we have 5 letters to choose from. Having chosen a letter for slot 1, there remain 4 letters left for slot 2, then 3 letters for slot 3, then 2 letters for slot 4, and finally only 1 letter for slot 5.
  • The reason these numbers are multiplied can be seen from a tree diagram. In the diagram below I consider the simpler problem of arranging the 3 letters ABC in a row. In the first row of the tree diagram we see the 3 possible choices for slot 1. In the second row are recorded for each of these choices the 2 letters that are remaining to fill slot 2. Then in the third row is the one remaining choice for the letter in slot 3 for each branch of the tree. The total number of branches, in this case 6, is the number of ways of arranging the three letters A, B, and C in a row. It's because of the branching nature of the choices that the numbers are multiplied, in this case .

• So there are 5! ways of obtaining 3 successes in 5 trials if the 3 successes are distinguishable from each other and the two failures are distinguishable. So for the successes that means we are treating the following six identical outcomes as if they were different.

Here I'm permuting the successes as if they were different from each other (using subscripts to distinguish them). Because there are three successes the fundamental law of counting says there are 3! = 6 ways of permuting them. So every time we have a truly distinguishable outcome, such as SSFFS, it ends up getting counted six times in our 5! count, once for each way of permuting the 3 successes. So 5! overcounts the number of outcomes of interest by 6 times.

• But it's worse than that. The same thing is happening with the failures. When we observe SSFSF, 5! counts as different from . So each true distinguishable order is being counted twice because we are also treating the 2! permutations of the failures as if they were distinguishable from each other.
• So now the answer is clear. We need to correct 5! by dividing it by 3!, the number of ways of permuting the successes among themselves, and 2!, the number of ways of permuting the failures among themselves. So we have

Number of ways of obtaining 3 successes in 5 trials =

• An ordering of 5 objects is called a permutation. One of the possible ways of dividing 5 objects into two groups of size 3 and 2, where the order of the objects within the groups should be ignored, is called a combination. The following notations are all used for combinations.

The fourth expression (after the third equality) is referred to as a binomial coefficient. It should be read as "5 choose 3" and is the number of distinct ways of choosing 3 objects from 5 (when the order of the three objects doesn't matter).

• So, we can now return to the general problem of finding .
  • We now know that the probability of obtaining k successes and n – k failures in a particular order is .
  • We also know that the number of distinct arrangements of k successes and n – k failures is .
  • Putting these two results together we obtain

• What is the probability of getting no successes in n trials? This corresponds to observing . There is only one way this can occur and its probability is . According to our formula, we get the following.

Clearly for the formula to work we need 0! = 1. This in fact is how zero factorial is defined. So the factorial operator turns nothing, 0, into something, 1. I think this deserves an exclamation, such as "Wow!" (To be read as wow factorial.)

• Anytime a response variable of interest is a proportion and the proportion can be treated as the number of successes out of a fixed number of trials, then the binomial distribution is relevant. As with Bernoulli response variables, the typical modeling tool when dealing with responses that are binomially distributed is logistic regression.
• Mean of a binomial random variable. To use the expectation definition we would have to calculate the following.

This is not as hard as it looks, but there is an easier way.

• Recall that a binomial random variable with parameters n and p can be thought of as a sum of n independent Bernoulli random variables, , each with parameter p. Thus

• One of the nice properties of the expectation operator is that it is linear. In other words the expectation of a sum is the sum of the expectations.

This follows from the fact that an expectation is either a sum or an integral, and sums and integrals behave this way. (Things are a little bit more complicated than this, but we'll skip the details.) Therefore we have

• Variance of a binomial random variable. We'll try the same trick with the variance as we did with the mean.

• Unfortunately, the variance operator is not linear. (Recall that its definition involves a sum of squares.) There is a formula that is comparable to the expectation of a sum formula, but it is more complicated for the variance.

The term in the second sum involves the covariance operator which is defined as follows.

The covariance measures how much two variables covary together. In general the covariance of two random variables is not zero, but for independent random variables it is. Since a binomial random variable with parameters n and p is the sum of n independent Bernoulli random variables, the covariance term drops out and we have the following.

Poisson Distribution

• Like the binomial, the Poisson distribution deals with counts. Unlike the binomial there are no trials that result in success or failure. We just observe "successes" when or where they occur. Here the observations are occurring over time (such as the number of occurrences of earthquakes in California in a year or the number of cases of Lyme disease in Orange Co, NC in a year) or space (the number of individuals of a particular species of bird observed while walking a transect).
• Unlike the binomial distribution, in the Poisson distribution there is no parameter n or, perhaps closer to the truth, n is very big and for all practical purposes unknowable. The Poisson is a one-parameter distribution and the single parameter is usually designated λ.
• A random variable X is said to have a Poisson distribution with parameter λ if the following three assumptions hold.
  1. Events occur at a constant rate λ. This means that in a time interval of length t we expect to encounter events. Put another way, if we define to be the number of events observed in a time interval of length t, then by this assumption . Put yet a third way, the number of events occurring in an interval is proportional to the length of the interval. Since the rate is constant over time (or space), there is no spatial or temporal heterogeneity in the event occurrence rate. Thus this assumption is usually called the homogeneity assumption. The Poisson distribution is said to be time invariant or space invariant.
  2. Events occurring in one interval have no effect on events occurring in a second interval, as long as the two intervals are disjoint. In other words, events in non-overlapping intervals occur independently of each other. Accordingly, this is called the independence assumption.
  3. The third assumption is a bit technical. In lay language we require that the probability of observing two or more events in an interval, when the interval in question is very small, be approximately zero. More precisely we require that the probability of observing two or more events is negligible compared to the probability of observing one event. Put more formally, if h is a very small interval of time (or space), then

where is a function such that

In words this says that as the interval is shrunk down, the probability of observing two or more events shrinks much faster than the interval itself. In particular, it shrinks faster than the probability of observing exactly one event (which is proportional to the length of the interval).

• Next time, using these properties, we'll derive the formula for the probability mass function of a Poisson random variable. We'll show that if is a Poisson random variable with rate λ, then the probability of observing k events in an interval of length t is the following.

Course Home Page