lacunary - Mathnotes

Probability Basics

Sample Space

Definition: Sample Space \@{sample-space}

The set of all possible outcomes of a statistical experiment is called the sample space and is represented by the symbol S. Each outcome in a sample space is called an element or member of the sample space or simply a sample point.

For example, the sample space representing the outcomes of the rolls of a standard six-sided die is

S={1,2,3,4,5,6}.

Definition: Event \@{event}

An event is a subset of a sample space. For example, we can say the event A that the roll of a die is odd is A={1,3,5}, the event B that the roll of a die is even is B={2,4,6}, and that the event that the roll of a die is less than or equal to 4 is C={1,2,3,4}.

Referenced by (1 direct, 2 transitive)

Direct references:

Transitive (depth 1):

A lot of the rest of basic probability is just set theory from there:

  • The event that the roll of a die is not less than or equal to 4 is the complement of C, written as Cc={5,6}.
  • The event that the roll is both even and less than or equal to 4 is the intersection of A and C, written as AC={2,4}.
  • The event that the roll is either even or less than or equal to 4 is the union of A and C, written as AC={1,2,3,4,6}.

We can also have continuous sample spaces. For example, if we were to randomly choose a number between 0 and 1 inclusive, the sample space would be S=[0,1], and the event that the outcome is a number whose decimal digits are only 2s would be {0.2,0.22,0.222,}.

Probability

Definition: Probability \@{probability}

Note: This applies to sample spaces of discrete events and is a bit hand wavy.

We can assign a probability or weight to each sample point in a sample space by giving it a value ranging from 0 to 1. Events that are more likely to occur have a probability closer to 1, and events that are less likely to occur have a probability closer to 0. We give a probability to all sample points in a sample space such that the sum of the probabilities of all sample points in a sample space is 1.

Then, the probability of an event A is the sum of the probabilities of all sample points in A. Therefore,

0P(A)1,P()=0,P(S)=1.

Definition: Odds Notation: odds \@{odds}

The odds of an event A occurring are defined as

odds(A)=P(A)P(Ac)=P(A)1P(A)

that is, the probability A occurs divided by the probability of the complement of A (i.e., the probability that A does not occur.)

Referenced by (3 direct)

Direct references:

Note \@{note-5}

Odds are often written as a @ratio of positive integers. So we'd say that the odds of drawing an ace from a standard deck of 52 cards is

odds(Ace)=4524852=448=112=1:12.

Note \@{note-7}

Using log probability has some advantages:

  • It converts multiplication of probabilities into addition, which are faster to compute.
  • It improves @numerical stability when very small probabilities are involved.
  • It eliminates @exponentiation.
  • Logarithms are @concave which is useful for @optimization.

Probability Rules

Definition: Independence (also: independent) \@{independence}

The events A and B are said to be independent if

P(AB)=P(A)P(B).

Theorem: Addition and Multiplication Rules \@{probability-rules}

The probability of A or B is the probability of A plus the probability of B minus the probability of A and B occurring together:

P(AB)=P(A or B)=P(A)+P(B)P(A and B)

If A and B are mutually exclusive events, P(AB)=P(A)+P(B).

We have to subtract the overlap between A and B to avoid double counting.

Similarly

P(AB)=P(A)+P(B)P(AB).

The probability of A and B is the probability of A times the probability of B given A, or equivalently, the probability of B times the probability of A given B.

P(AB)=P(A and B)=P(B and A)=P(A)P(B|A)=P(B)P(A|B)

If A and B are independent, this reduces to P(A)P(B).

Definition: Conditional Probability \@{conditional-probability}

From this, we can give the formula for conditional probability. The probability of A given B is

P(A|B)=P(A and B)P(B).

That is to say, the probability of A occurring given B has occurred is the portion of times B occurs that A also occurs.

If P(A)=P(A|B), then A and B are independent events, and P(A)P(B)=P(A and B).

Referenced by (1 direct, 1 transitive)

Direct references:

Transitive (depth 1):