Everything we do, everything that happens around us, obeys the laws of probability. We can no more escape them than we can escape gravity... "Probability," a philosopher once said, "is the very guide of life." We are all gamblers who go through life making countless bets on the outcome of countless actions.

Every field of science is concerned with estimating probability. A physicist calculates the probable path of a particle. A geneticist calculates the chances that a couple will have blue-eyed children. Insurance companies, businessmen, stockbrokers, sociologists, politicians, military experts - all have to be skilled in calculating the probability of the events with which they are concerned.

[Gardner, 1986]

#### Synopsis

Probability theory is the branch of mathematics that tells us how to estimate degrees of probability. If an event is certain to happen, it is given a probability of 1. If it is certain not to happen, it has a probability of 0.

This course introduces the principles of probability and random processes to undergraduate students in electronics and communication. The topics to be covered include random experiments, events, probability, discrete and continuous random variables, probability density function, cumulative distribution function, functions of random variables, expectations, law of large numbers, central limit theorem, introduction to random processes, Gaussian random process, autocorrelation and power spectral density.

#### Announcements

- Final Exam Scores
- Post-midterm Announcements
- A basic RSS feed is created to track and inform updates [Posted @ 5PM on Jun 25]
- This site can be access via prapun.com/ecs315 [Posted @ 5PM on Jun 25]
- Welcome to ECS315! Feel free to look around this site. [Posted @ 5PM on Apr 1]

#### General Information

**Instructor**: Dr. Prapun Suksompong (prapun@siit.tu.ac.th)- Office: BKD3601-7
- Office Hour: Monday 14:40-16:00, Friday: 14:00-16:00

**Course Syllabus****Class information**

- Textbook: [Y&G] R. D. Yates and D. J. Goodman, Probability and Stochastic Processes: A Friendly Introduction for Electrical and Computer Engineers, 2nd ed., Wiley, 2004.
- Call No. QA273 Y384 2005. ISBN: 978-0-471-27214-4
- Student Companion Site

- References
- Older version of the textbook: Probability and stochastic processes : a friendly introduction for electrical and computer engineers / Roy D. Yates, David J. Goodman. Call No. QA273 Y384 1999
- Probability and probabilistic reasoning for electrical engineering / Terrence L. Fine. Call No. QA273 F477 2006
- Probability and random processes for electrical engineering / Alberto Leon-Garcia. Call No. TK153 L425 1994
- [Z&T] Rodger E. Ziemer and William H. Tranter, Principles of Communications, 6th International student edition, John Wiley & Sons Ltd, 2010.
- Probability, random variables, and stochastic processes / Athanasios Papoulis, S. Unnikrishna Pillai. Call No. QA273 P2 2002
- Probability, random variables, and stochastic processes / Athanasios Papoulis. Call No. QA273 P2 1991

- A first course in probability / Sheldon Ross. Call No. QA273 R83 2002
- A first course in probability / Sheldon Ross. Call No. QA273 R83 1976

- Probability models, introduction to / Sheldon M. Ross. Call No. QA273 R84 1997
- Random signals for engineers using MATLAB and Mathcad / Richard C. Jaffe. Call No. TK5102.9 J34 2000
- Stochastic processes / Sheldon M. Ross. Call No. QA274 R65 1996
- Stochastic processes / Emanuel Parzen. Call No. QA273 P278 1962
- Probability theory and its applications, an introduction to / William Feller Call No. QA273 F37 1966
- Davenport, W.B., Probability and Random Processes, McGraw-Hill, New York, 1970. (Excellent introductory text.)
- Feller, W., An Introduction to Probability Theory and its Applications, Vols. 1, 2, John Wiley, New York, 1950. (Definitive work on probability—requires mature mathematical knowledge.)
**Free textbooks***Introduction to Probability*by Charles M. Grinstead and J. Laurie Snell

- MATLAB Primer, 8th edition T. A. Davis. CRC Press, 2010.

#### Handouts and Course Material

- Part I: Introduction, Set Theory, Classical Probability theory, and Combinatorics [Posted @ 5PM on Jun 13].
- Commented version [Posted @ 4PM on Jun 14, Updated @ 2PM on Jun 23].

- Slides: Introduction [Posted @ 3PM on Jun 14].
- Slides: Set theory, Clasical Probability, and Combinatorics [Posted @ 3PM on Jun 21, Updated @ 2PM on Jun 30].
- Part II: Event-based probability theory
- Part II.1: Kolmogorov's Axioms and their consequences [Posted @ 3PM on Jun 22].
- Commented version [Posted @ 2PM on Jun 30, Updated @12PM on July 6]

- Part II.2: Conditional probability and independence [Posted @ 10PM on Jul 4].
- Commented version [Posted @5PM on July 7, Updated @1PM on July 14]
- OneNote: Extra Examples [Posted @ 5PM on Jul 12].

- Slides: Event-based probability theory [Posted @ 5PM on Jul 12, Updated @2PM on July 14].

- Part II.1: Kolmogorov's Axioms and their consequences [Posted @ 3PM on Jun 22].
- Part III: Discrete random variables
- Part III.1: pmf and cdf [Posted @ 11AM on Jul 12].
- Commented version [Posted @1PM on July 14, updated @8PM on July 19]
- Solution for Quiz 2

- Part III.2: Expectation and commonly used discrete random variables [Posted @ 10PM on Jul 18].
- Commented version [Posted @8PM on July 19. Updated @2PM on July 21]

- Part III.3: Functions of a discrete random variable and their expectations [Posted @ 2PM on Jul 20].
- Commented version [Posted @2PM on July 21]

- Part III.1: pmf and cdf [Posted @ 11AM on Jul 12].
- Information regarding the midterm
- 4 Aug 2011; 09:00 - 12:00 ; BKD 3507
- OneNote: Review [Posted @10AM on July 8. Updated @2PM on July 21]
- All commented notes combined in one pdf file [Posted @5PM on July 21]
- Here is the uncommented version. (The links inside the document should works. References are also available at the end.) [Posted @9PM on July 21]

- Closed book. Closed notes. No cheat/study sheet.

- Basic calculator allowed.
- Cover page of the exam
- Topics
- Counting and classical Probability: Approximately 10%
- Event-based probability theory: Approximately 40%
- Independence and contidional probability: Approximately 25%

- Random Variables: Approximately 50%

- Some info from last year.
- Note that the exam from last year covered joint pmf (for two random variables). This topic hasn't been discussed this year and hence won't be on the exam.

- OneNote: All handwritten notes [Posted @8PM on Aug 2]

- Part III (con't)
- Part III.3 [Updated @5PM on Aug 9]
- Part III.4 [Posted @11AM on Aug 8]
- Commented version [Posted @5PM on Aug 9, Updated @6PM on Aug 18]
- OneNote: Questions raised in class [Posted @5PM on Aug 16]
- References
- 9.1 A Pair of Random Variables: [Y&G] Sections 4.1 to 4.3 and Section 4.10
- 9.2 Extending the Denitions to Multiple RVs: [Y&G] Sections 5.1 to 5.4

- Part III.5 [Posted @11PM on Aug 14]
- Commented version [Posted @6PM on Aug 18, Updated @12PM on Aug 26]
- OneNote: Review
- References
- 9.3 Function of Discrete Random Variables: [Y&G] Section 4.6 (Theorem 4.9)
- 9.4 Expectation of function of discrete random variables: [Y&G] Sections 4.7 and 6.1
- 9.5 Linear Dependence: [Y&G] Section 4.7

- Video Tutorial: Here are two extra examples with solutions. The explanation of the solutions is provided via this video for the first example and this video for the second example. You can also download the Excel file that I used in the solution. Those who are familiar with MATLAB may prefer this .m file.

- Part IV: Continuous Random Variables
- Part IV.1 [Posted @12PM on Aug 23]
- Commented version [Posted @10PM on Aug 30, Updated @4PM on Sep 6]
- OneNote: Extra comments [Posted @10PM on Aug 30, Updated @3PM on Aug 31]
- Solution for Quiz 3
- References:
- 10.1 From Discrete to Continuous Random Variables: [Y&G] Sections 3.0 to 3.1
- 10.2 PDF and CDF: [Y&G] Sections 3.1 to 3.2
- 10.3 Expectation and Variance: [Y&G] Section 3.3
- 10.4 Families of Continuous Random Variables: : [Y&G] Sections 3.4 to 3.5
- Table 3.1 and Table 3.2 from [Y&G] [Posted @8AM on Sep 8]

- Part IV.2 [Posted @12PM on Sep 7]
- Commented version [Updated @6PM on Sep 20]
- References: Sections 4.4 to 4.7 [Y&G]
- 10.5 SISO: [Y&G] Section 3.7; [Z&T] Section 5.2.5
- 10.6 Pairs of Continuous Random Variables: [Y&G] Sections 4.1, 4.5, and 4.11
- 10.7 MISO: [Y&G] Section 6.2

- Tutorial: Calculus [Posted @7PM on Sep 20]
- Solution for Quiz 4

- OneNote: Review [Posted @5PM on Sep 13]

- Part IV.1 [Posted @12PM on Aug 23]
- Part V, VI, and VII.1 [Posted @10AM on Sep 19]
- Commented version [Posted @3:30PM on Sep 26; Updated @2PM on Sep 29]
- References:
- 11 Mixed RVs: [Y&G] Sections 3.6 and 3.7
- 12 Conditional Probability/Expectation: [Y&G] Sections 4.9 and 4.11
- 13 Transform Methods: [Y&G] Sections 6.3 and 6.4.
- 14 Limiting Theorem: [Y&G] Theorem 7.8; [Y&G] Sections 6.6 and 6.7
- 15 Random Vector: [Y&G] Sections 5.6 and 5.7

- Part VII.2 [Posted @9PM on Sep 26]
- Information regarding the
**final exam**- 09:00 - 12:00; 11 Oct 2011; BKD 2501
- Closed book. Closed notes.
- You will need a basic calculator e.g. FX-991MS.

- Cover all the materials that we discussed in class and practice in the HWs.
- Strong focus on the materials that haven’t been on the midterm.
- I could ask something that I have never defined in class but, in such a case, I will give you the exact definition on the exam itself.
- These notes are provided for your studying pleasure....
- All commented notes combined in one pdf file
- Post-midterm commented notes
- Uncommented version. (The links inside the document should works. References are also available at the end.)

- All post-midterm OneNote Notes.

- All commented notes combined in one pdf file

- One
**A4 sheet**allowed.- You can use only one side. (Another side should be Table 3.1 and Table 3.2.)
- Must be hand-written.
- No small pieces of paper notes glued/attached on top of it.
- Indicate your name and id on the upper right corner of the sheet.
- Submit your formula sheet with your final exam. (You can get it back from me next semester.)

- 2010 Exam
- About the 2011 exam:
- 15 pages, 9 problems
- Cover page
- Approximately 25% on discrete random variables (Sections 8,9,12)
- Approximately 45% on continuous random variables (Section 10)
- Approximately 10% on mixed random variables (Section 11)
- Approximately 15% on cha. function and limiting theorems (Sections 13 and 14)
- When used correctly, Section 15 may help you solve some questions faster but you are not required to directly use the material in there.
- No question on Section 16.
- Question 1 (25pt) starts with "Random variables X and Y have the following joint pmf..."
- Question 3 (16pt) starts with "Random variables X and Y have joint pdf..."
- Question 9 (6pt) starts with "Kakashi and Gai are eternal rivals...."

- More information may be posted here. [Updated @ 10AM on Oct 6]

#### Problem Set

- HW 1 (Due: July 6)
- HW 2 (Due: July 14)
- HW 3 (Due: July 25)
- HW 4 (Not Due)
- Self-Evaluation Form (Due: July 11)
- HW5 (Due: Aug 25)
- OneNote: Tutorial session on Aug 16
- Solution + MATLAB code

- HW6 (Due: Sep 8)
- HW7 (Due: Sep 22)
- HW8 (Due: Sep 29; Fixed at 3 PM on Sep 25)
- Solution
- MATLAB codes for Q4.7.8 and Q4.7.12 from [Y&G]

- HW9 (Not Due)
- Self-Evaluation Form (Due: Oct 11)

#### Calendar

#### Reading Assignment

#### Misc. Links

- Video: Probability 101
- More information about Monty Hall Problem
- Video: The Monty Hall Problem
- Video: Monty Hall Problem: Numb3rs and 21
- Paper: Monty Hall, Monty Fall, Monty Crawl
- Video: It *could* just be coincidence
- MV: Bill Nye the Science Guy - "50 Fifty"
- Video: Chevalier de Mere's Scandal of Arithmetic
- Free educational software: Orstat2000
- Originally developed to promote probability and operations research in the senior forms of Dutch high schools (and early college).
- Contain modules for coin-tossing, central limit theorem, etc.

- Probability review from MATH REVIEW for Practicing to Take the GRE General Test
- Video: Mlodinow’s talk @ Google
- Video: The Binomial Distribution / Binomial Probability Function
- Video: The Poisson Distribution
- If you want to experience probability theory at a more advance level, one standard textbook that you can refer to is "Probability: Theory and Examples" by Prof. Durrett. Currently, the 4th edition of the textbook is available online.
- Paper: Cheung, Y. L. "Why Poker is Played with Five Cards."
*Math. Gaz.*73, 313-315, 1989. - Video: Peter Donnelly shows how stats fool juries (same clip on youtube)
- Video: Lies, damned lies and statistics (about TEDTalks): Sebastian Wernicke on TED.com
- The Median Isn't the Message by Stephen Jay Gould
- Video: Daniel Kahneman: The riddle of experience vs. memory
- Article: about risk intelligence
- Dylan Evans, How to Beat the Odds at Judging Risk, The Wall Street Journal, May, 2012
- Alison George, What Gamblers and Weather Forecasters Can Teach Us About Risk: An interview with the creator of the "risk quotient" intelligence scale., pp 30-31, New Scientist, May 19, 2012

- Quotations about Statistics
- Video: Statistics - Dream Job of the next decade
- Google Calculator (Cheat Sheet)
- Sometimes the easiest way to get information on a counting problem is to compute a few small values of a function, then look for a match at the sequence server; if you find a hit, you can sometimes get citations to the literature.
- Prapun's Notes on Probability Theory (Cornell Version)
- MATLAB
- MIT OpenCourseWare > Electrical Engineering and Computer Science > 6.094 Introduction to MATLAB (January (IAP) 2009)

- Learn the Greek Alphabet in less than 10 minutes
- The Greek Alphabet Song