Contributing Scholar - Alberto Leon-Garcia, University of Toronto

3 Semester Credit Hours

Course Description: Communication systems and computer networks are designed to provide high performance consistently and reliably in the presence of noisy communication channels, equipment faults, a wide range of media applications that combine voice, images and video, and high variability in user demand. Probability models provide the mathematical framework for characterizing random variability and they form the basis for tools to design systems that perform predictably in the face of random inputs and random environment.

The notion of a random variable and its characterization using a probability distribution function and associated moments are reviewed. The focus is on characterizing the joint behavior of multiple random variables to understand their interdependence and to enable prediction of likely outcomes. The joint distribution function as well as the correlation and the covariance functions are essential tools in achieving these objectives. The notion of a random process, consisting of a sequence and even a continuum of random variables, is introduced, and the probability tools are extended to capture joint behavior. Random processes are shown to describe signals and dynamic behavior encountered in engineering systems. The utility of probability models is demonstrated through applications in communication systems, reliability, digital signal processing, and communications networks.

Course Objectives:

For random experiments:

For single, dual, and multiple random variables:

For sequences of random variables:

For random processes:

• Understand the definition of a random process as an indexed family of random variables and the relation of a sample path to a random process

• Apply the joint probability distribution, correlation and autocovariance functions to calculate probabilities and other measures involving a random process
• Identify whether random processes have independent increments or satisfy the Markov property and calculate probabilities using these properties
• Understand and calculate the covariance matrix to determine the joint PDF for jointly Gaussian random variables
• Calculate probabilities for simple events and moments involving the following random processes: iid, sum, Binomial, Random Walk, Poisson process, random telegraph signal, Wiener process and Brownian motion
• Determine whether a random process satisfies various types of stationarity and ergodicity
• Determine whether a random process is continuous, differentiable, or integrable in the mean square sense
• Find the Fourier series expansion for mean square periodic random processes, and the Karhunen-Loeve expansion for general random processes
• Calculate the power spectral density for continuous-time and discrete-time random processes
• For continuous-time and discrete-time systems, calculate the autocorrelation and power spectral density of the output of a linear time-invariant system in terms of the autocorrelation and power spectral density of the input signal
• Apply the orthogonality condition to obtain optimal linear systems for filtering, smoothing, prediction, and estimation
• Determine the transition probability matrix and transition rate matrix for discrete-time and continuous-time Markov chains, respectively
• Determine the classification of states and the limiting probabilities of a Markov chain

Prerequisites