Everything we do, everything that happens around us, obeys the laws of probability. We can no more escape them than we can escape gravity... "Probability," a philosopher once said, "is the very guide of life." We are all gamblers who go through life making countless bets on the outcome of countless actions.

Every field of science is concerned with estimating probability. A physicist calculates the probable path of a particle. A geneticist calculates the chances that a couple will have blue-eyed children. Insurance companies, businessmen, stockbrokers, sociologists, politicians, military experts - all have to be skilled in calculating the probability of the events with which they are concerned.

[Gardner, 1986]

Synopsis

Probability theory is the branch of mathematics that tells us how to estimate degrees of probability. If an event is certain to happen, it is given a probability of 1. If it is certain not to happen, it has a probability of 0.

This course introduces the principles of probability and random processes to undergraduate students in electronics and communication. The topics to be covered include random experiments, events, probability, discrete and continuous random variables, probability density function, cumulative distribution function, functions of random variables, expectations, law of large numbers, central limit theorem, introduction to random processes, Gaussian random process, autocorrelation and power spectral density.

Announcements

General Information

Handouts and Course Material

  • Part I: Introduction, Set Theory, Classical Probability theory, and Combinatorics [Posted @ 5PM on Jun 13].
  • Slides: Introduction [Posted @ 3PM on Jun 14].
  • Slides: Set theory, Clasical Probability, and Combinatorics [Posted @ 3PM on Jun 21, Updated @ 2PM on Jun 30].
  • Part II: Event-based probability theory
  • Part III: Discrete random variables
  • Information regarding the midterm
    • 4 Aug 2011; 09:00 - 12:00 ; BKD 3507
    • OneNote: Review [Posted @10AM on July 8. Updated @2PM on July 21]
    • All commented notes combined in one pdf file [Posted @5PM on July 21]
    • Closed book. Closed notes. No cheat/study sheet.
    • Basic calculator allowed.
    • Cover page of the exam
    • Topics
      • Counting and classical Probability: Approximately 10%
      • Event-based probability theory: Approximately 40%
        • Independence and contidional probability: Approximately 25%
      • Random Variables: Approximately 50%
    • Some info from last year.
      • Note that the exam from last year covered joint pmf (for two random variables). This topic hasn't been discussed this year and hence won't be on the exam.
    • OneNote: All handwritten notes [Posted @8PM on Aug 2]
  • Part III (con't)
    • Part III.3 [Updated @5PM on Aug 9]
    • Part III.4 [Posted @11AM on Aug 8]
      • Commented version [Posted @5PM on Aug 9, Updated @6PM on Aug 18]
      • OneNote: Questions raised in class [Posted @5PM on Aug 16]
      • References
        • 9.1 A Pair of Random Variables: [Y&G] Sections 4.1 to 4.3 and Section 4.10
        • 9.2 Extending the De nitions to Multiple RVs: [Y&G] Sections 5.1 to 5.4
    • Part III.5 [Posted @11PM on Aug 14]
      • Commented version [Posted @6PM on Aug 18, Updated @12PM on Aug 26]
      • OneNote: Review
      • References
        • 9.3 Function of Discrete Random Variables: [Y&G] Section 4.6 (Theorem 4.9)
        • 9.4 Expectation of function of discrete random variables: [Y&G] Sections 4.7 and 6.1
        • 9.5 Linear Dependence: [Y&G] Section 4.7
  • Part IV: Continuous Random Variables
    • Part IV.1 [Posted @12PM on Aug 23]
      • Commented version [Posted @10PM on Aug 30, Updated @4PM on Sep 6]
      • OneNote: Extra comments [Posted @10PM on Aug 30, Updated @3PM on Aug 31]
      • Solution for Quiz 3
      • References:
        • 10.1 From Discrete to Continuous Random Variables: [Y&G] Sections 3.0 to 3.1
        • 10.2 PDF and CDF: [Y&G] Sections 3.1 to 3.2
        • 10.3 Expectation and Variance: [Y&G] Section 3.3
        • 10.4 Families of Continuous Random Variables: : [Y&G] Sections 3.4 to 3.5
        • Table 3.1 and Table 3.2 from [Y&G] [Posted @8AM on Sep 8]
    • Part IV.2 [Posted @12PM on Sep 7]
      • Commented version [Updated @6PM on Sep 20]
      • References: Sections 4.4 to 4.7 [Y&G]
        • 10.5 SISO: [Y&G] Section 3.7; [Z&T] Section 5.2.5
        • 10.6 Pairs of Continuous Random Variables: [Y&G] Sections 4.1, 4.5, and 4.11
        • 10.7 MISO: [Y&G] Section 6.2
      • Tutorial: Calculus [Posted @7PM on Sep 20]
      • Solution for Quiz 4
    • OneNote: Review [Posted @5PM on Sep 13]
  • Part V, VI, and VII.1 [Posted @10AM on Sep 19]
    • Commented version [Posted @3:30PM on Sep 26; Updated @2PM on Sep 29]
    • References:
      • 11 Mixed RVs: [Y&G] Sections 3.6 and 3.7
      • 12 Conditional Probability/Expectation: [Y&G] Sections 4.9 and 4.11
      • 13 Transform Methods: [Y&G] Sections 6.3 and 6.4.
      • 14 Limiting Theorem: [Y&G] Theorem 7.8; [Y&G] Sections 6.6 and 6.7
      • 15 Random Vector: [Y&G] Sections 5.6 and 5.7
  • Part VII.2 [Posted @9PM on Sep 26]
  • Information regarding the final exam
    • 09:00 - 12:00; 11 Oct 2011; BKD 2501
    • Closed book. Closed notes.
    • You will need a basic calculator e.g. FX-991MS.
    • Cover all the materials that we discussed in class and practice in the HWs.
    • One A4 sheet allowed.
      • You can use only one side. (Another side should be Table 3.1 and Table 3.2.)
      • Must be hand-written.
      • No small pieces of paper notes glued/attached on top of it.
      • Indicate your name and id on the upper right corner of the sheet.
      • Submit your formula sheet with your final exam. (You can get it back from me next semester.)
    • 2010 Exam
    • About the 2011 exam:
      • 15 pages, 9 problems
      • Cover page
      • Approximately 25% on discrete random variables (Sections 8,9,12)
      • Approximately 45% on continuous random variables (Section 10)
      • Approximately 10% on mixed random variables (Section 11)
      • Approximately 15% on cha. function and limiting theorems (Sections 13 and 14)
      • When used correctly, Section 15 may help you solve some questions faster but you are not required to directly use the material in there.
      • No question on Section 16.
      • Question 1 (25pt) starts with "Random variables X and Y have the following joint pmf..."
      • Question 3 (16pt) starts with "Random variables X and Y have joint pdf..."
      • Question 9 (6pt) starts with "Kakashi and Gai are eternal rivals...."
    • More information may be posted here. [Updated @ 10AM on Oct 6]

Problem Set

  1. HW 1 (Due: July 6)
    • Solution for HW1
    • Scores (The leftmost column contains the last two digits of your student IDs.)
  2. HW 2 (Due: July 14)
  3. HW 3 (Due: July 25)
  4. HW 4 (Not Due)
  5. Self-Evaluation Form (Due: July 11)
  6. HW5 (Due: Aug 25)
  7. HW6 (Due: Sep 8)
  8. HW7 (Due: Sep 22)
  9. HW8 (Due: Sep 29; Fixed at 3 PM on Sep 25)
  10. HW9 (Not Due)
  11. Self-Evaluation Form (Due: Oct 11)

Calendar



Reading Assignment

Misc. Links