Random Variables and Probability Distributions

• Probability Distributions tell a story. Distributions are related to each other in the way they reframe counting samples

Random Variables §

• A random variable is a function that maps an event in the sample space to a numerical value.

• A discrete random variable operates on a countable sample space. That is, it may be quantified using a probability mass function where

  and

• A continuous random variable operates on an uncountable sample space. It can be quantified using a probability density function wherein the probability is given by

  A corresponding function called the cumulative distribution function is defined such that

• Change of variables extends to probability distributions Given is a continuous random variable from , with probability distribution and , assuming is an invertible function, we have that

  Where denotes the Jacobian for multidimensional .

  • If were discrete, then we simply have

Properties of Probability Distributions §

• The Quantile denoted defined as the value such that

• The mean or expectation or expected value is defined as the average value of the distribution.

  • An alternative notation is which is obtained as

  • Linearity of Expectation.

• The variance is defined as the average spread of the distribution

  • An important formulation of the variance is as follows:

  • The standard deviation is expressed in the same units of . It is simply

  • The coefficient of variation denoted is defined as

    The squared coefficient of variation is simply .

    It serves as a measure of variability. The CV denotes how concentrated the probability distribution is around the mean (higher = more spread out)

• The covariance between and measure the degree to which they are linearly related. It is defined as

  We may define a covariance matrix whose entries contain the pairwise covariances between variables. That is

  • The Covariance satisfies the following

• The precision is the inverse of the variance. It is given by

  The precision matrix is given as the inverse of the covariance matrix

• The Pearson Correlation between and defines a normalized measure of the covariance. It is defined as

  We may define a correlation matrix where

  In fact, indicates that . Correlation is a measure to the degree in which two variables are linearly related.

Theorems §

• Law of Total Expectation also called the Law of Iterated Expectations. States that for random variables we have

• The following property applies to expectations. Let be a joint distribution. We can write the expectation of sampled from this distribution as

  Observe we expanded the joint distribution according to the chain rule of probability. hen, we manipulated the internals sums by pushing them out.

  Remarkably, for a Markov Process, the expectation collapses simply as

• Markov’s Inequality states the following:

  If is a nonnegative random variable and , then

  When , we take for to get

Links §

• Murphy Ch. 2