Everything we do, everything that happens around us, obeys the laws of probability. We can no more escape them than we can escape gravity... "Probability," a philosopher (Bishop Butler) once said, "is the very guide of life." We are all gamblers who go through life making countless bets on the outcome of countless actions.

Every field of science is concerned with estimating probability. A physicist calculates the probable path of a particle. A geneticist calculates the chances that a couple will have blue-eyed children. Insurance companies, businessmen, stockbrokers, sociologists, politicians, military experts - all have to be skilled in calculating the probability of the events with which they are concerned.

[Gardner, 1986]

#### Synopsis

Probability theory is the branch of mathematics that tells us how to estimate degrees of probability. If an event is certain to happen, it is given a probability of 1. If it is certain not to happen, it has a probability of 0.

This course introduces the principles of probability and random processes to undergraduate students in electronics and communication. The topics to be covered include random experiments, events, probability, discrete and continuous random variables, probability density function, cumulative distribution function, functions of random variables, expectations, law of large numbers, central limit theorem, introduction to random processes, Gaussian random process, autocorrelation and power spectral density.

#### Announcements

- No class on Aug 30. Make-up class on Sep 3 (Monday) 10:40-12:00.
- Slides for Announcements [Posted at 3:30PM on July 17]
- No class on July 19 (TU Wai Kru Day)
- Make up classes [Posted at 3:30PM on July 3]
- July 6 (Friday) 9-10:20
- July 9 (Monday) 10:40-12:00

- A basic RSS feed is created to track and inform updates
- This site can be accessed via prapun.com/ecs315
- Welcome to ECS315! Feel free to look around this site.

#### General Information

**Instructor**: Dr. Prapun Suksompong (prapun@siit.tu.ac.th)- Office: BKD3601-7
- Office Hour: Monday 14:40-16:00, Friday: 14:00-16:00

**Course Syllabus**[Posted @ 11PM on June 25]**Class information**

- Textbook: [Y&G] R. D. Yates and D. J. Goodman, Probability and Stochastic Processes: A Friendly Introduction for Electrical and Computer Engineers, 2nd ed., Wiley, 2004.
- Call No. QA273 Y384 2005. ISBN: 978-0-471-27214-4
- Student Companion Site

- References
- Older version of the textbook: Probability and stochastic processes : a friendly introduction for electrical and computer engineers / Roy D. Yates, David J. Goodman. Call No. QA273 Y384 1999
- Probability and probabilistic reasoning for electrical engineering / Terrence L. Fine. Call No. QA273 F477 2006
- Probability and random processes for electrical engineering / Alberto Leon-Garcia. Call No. TK153 L425 1994
**Free textbooks***Introduction to Probability*by Charles M. Grinstead and J. Laurie Snell

- [Z&T] Rodger E. Ziemer and William H. Tranter, Principles of Communications, 6th International student edition, John Wiley & Sons Ltd, 2010.
- Probability, random variables, and stochastic processes / Athanasios Papoulis, S. Unnikrishna Pillai. Call No. QA273 P2 2002
- Probability, random variables, and stochastic processes / Athanasios Papoulis. Call No. QA273 P2 1991

- A first course in probability / Sheldon Ross. Call No. QA273 R83 2002
- A first course in probability / Sheldon Ross. Call No. QA273 R83 1976

- Probability models, introduction to / Sheldon M. Ross. Call No. QA273 R84 1997
- Random signals for engineers using MATLAB and Mathcad / Richard C. Jaffe. Call No. TK5102.9 J34 2000
- Stochastic processes / Sheldon M. Ross. Call No. QA274 R65 1996
- Stochastic processes / Emanuel Parzen. Call No. QA273 P278 1962
- Probability theory and its applications, an introduction to / William Feller Call No. QA273 F37 1966
- Davenport, W.B., Probability and Random Processes, McGraw-Hill, New York, 1970. (Excellent introductory text.)
- Feller, W., An Introduction to Probability Theory and its Applications, Vols. 1, 2, John Wiley, New York, 1950. (Definitive work on probability—requires mature mathematical knowledge.)
- MATLAB Primer, 8th edition T. A. Davis. CRC Press, 2010.

#### Handouts and Course Material

- Slides: Introduction to ECS315 [Posted @ 6:30 PM on June 26; Updated @ 3PM on July 5]
- Part I: Classical Probability
- Part I.1: Introduction, Set Theory, Classical Probability Theory [Posted @ 3PM on June 25]
- Annotated verion [Posted @ 3:30PM on July 3; Updated @ 3PM on July 5; Updated again @ 3:30 PM on July 6]
- References: [Y&G] Sections 1.0-1.1
- Slides: Slides for Section 1 [Posted @ 3PM on July 5]
- Slides: Slides for Section 2 [Posted @ 3PM on July 5]
- Slides: Slides for Section 3 [Posted @ 3PM on July 5]

- Part I.2: Enumeration / Combinatorics / Counting [Posted @ 4PM on July 2]
- Annotated version [Posted @ 3:30PM on July 6; Updated @ 5PM on July 9; Updated again @ 4PM on July 10; Last updated @ 1:30PM onJuly 12]
- References: [Y&G] Section 1.8
- Slides: Slides for Section 4 [Posted @ 3:30PM on July 6; Updated @ 5PM on July 10; Updated again @ 1:30PM on July 12]

- Part I.3: Extra Examples on Combinatorics [Posted @ 4PM on July 2]

- Part I.1: Introduction, Set Theory, Classical Probability Theory [Posted @ 3PM on June 25]
- Part II: Kolmogorov's Formal Probability Theory and Event-Based Probability Theory
- Probability Foundations, Event-based Independence, and Conditional Probability [Posted @11PM on July 8]
- References: [Y&G] Sections 1.3-1.6
- Annotated version [Posted @ 1:30PM on July 12; Updated @ 3:30PM, @ 3:30PM on July 24, @ 1:30PM on July 26, @ 4PM on July 31, @4PM on Aug 7, and @4PM on Aug 9]
- Extra note: A result from calculus
- Video: Conditional probability satisfies three properties similar to what we have seen in the axioms (P1-P3) of probability. This video provide the proof of these properties.

- Slides: Slides for Section 5 [Posted @ 3:30PM on July 17]
- Slides: Slides for Section 6.1 [Posted @ 3:30PM on July 24; Updated @ 3:30PM on Aug 9]
- Slides: Slides for Sections 6.2 and 6.3 [Posted @ 11PM on Aug 6; Updated @ 3:30PM on Aug 9]
- Quiz 1 Solution
- Quiz 2 Solution

- Probability Foundations, Event-based Independence, and Conditional Probability [Posted @11PM on July 8]
- Part III: Discrete Random Variables
- References: [Y&G] Chapter 2
- Random Variables, Discrete Random Variables, Probability Mass Function, and Cumulative Distribution Function [Posted @10PM on July 29; Fixed @10AM on July 30]
- Annotated version [Posted @ 2:30PM on Aug 9]

- Information regarding the
**midterm exam**- 14 Aug 2012 TIME 13:30 - 16:30
- ROOM BKD 2401
- Closed book. Closed notes. No cheat/study sheet.
- Basic calculators, e.g. FX-991MS, are permitted
- 9 Pages + 1 Cover Page
- 9 Problems + 1 Extra-Credit Problem
- Cover all the materials that we discussed in class and practice in the HWs.
- About 60% of the scores will be on questions based on Section 6 (Conditional Probability, Independence, and Bernoulli trials)
- About 10% on Section 7.
- I could ask something that I have never defined in class but, in such a case, I will give you the exact definition on the exam itself.
- These notes are provided for your studying pleasure....
- All commented notes combined in one pdf file
- Uncommented version. (The links inside the document should works. References are also available at the end.)

- All slides combined in one pdf file
- Review [Last updated @ 3:30PM on Aug 9]
- Solutions for HW 1-3

- All commented notes combined in one pdf file

- 2010 Midterm
- Ignore Q3c, Q5-Q9. The topics havn't been discussed this year and hence won't be on the exam.

- Part III (Con't)
- Annotated version of Part III.1 [Posted @ 4PM on Aug 23; Updated @ 4 PM on Aug 28]
- Part III.2 (Sections 8.3-8.4 and 9) [Posted @ 10PM on Aug 22]
- Annotated version of Part III.2 [Posted @ 4PM on Aug 23; Updated @ 4PM on Sep 3, @ 3PM on Sep 4, @ 2PM on Sep 6, @11:30AM on Sep 12, and @7:30PM on Sep 13]
- Quiz 4 Solution
- Slides: Slides for Section 8 [Updated @ 9AM on Sep 4]
- Slides: Slides for Section 9 [Posted @ 2PM on Sep 6]

- Part III.3 (Sections 10.1-10.2) [Posted @ 10PM on Aug 22]
- Annotated version [Posted @7:30PM on Sep 13; Updated @4PM on Sep 18 and @4PM on Sep 25]

- Part III.4 (Sections 10.3 - 10.5) [Posted @ 10PM on Sep 16]
- Annotated version [Posted @4PM on Sep 25; Updated @5PM on Sep 27 and @ 7PM on Oct 2]
- Slides: Slides for Section 10 [Posted @5PM on Sep 27]
- Quiz 6 Solution

- References:
- A Pair of Random Variables: [Y&G] Sections 4.1 to 4.3 and Section 4.10
- Extending the Definitions to Multiple RVs: [Y&G] Sections 5.1 to 5.4
- Function of Discrete Random Variables: [Y&G] Section 4.6 (Theorem 4.9)
- Expectation of function of discrete random variables: [Y&G] Sections 4.7 and 6.1
- Linear Dependence: [Y&G] Section 4.7

- Part IV
- Part IV.1 (Sections 11.1-11.4) [Posted @ 10AM on Sep 25]
- Annotated version [Posted @ 7PM on Oct 2; Updated @ 2PM on Oct 4]

- Part IV.2 (Sections 11.5-11.7) [Posted @ 2:30PM on Oct 3]
- Annotated version [Posted @ 3PM on Oct 9; Updated @ 9PM on Oct 13]

- References
- From Discrete to Continuous Random Variables: [Y&G] Sections 3.0 to 3.1
- PDF and CDF: [Y&G] Sections 3.1 to 3.2
- Expectation and Variance: [Y&G] Section 3.3
- Families of Continuous Random Variables: [Y&G] Sections 3.4 to 3.5
- Table 3.1 and Table 3.2 from [Y&G]
- SISO: [Y&G] Section 3.7; [Z&T] Section 5.2.5
- Pairs of Continuous Random Variables: [Y&G] Sections 4.1, 4.5, and 4.11
- MISO: [Y&G] Section 6.2

- Part IV.1 (Sections 11.1-11.4) [Posted @ 10AM on Sep 25]
- Part V
- Section 12-14 [Posted @ 5PM on Oct 8]
- Annotated version [Posted @ 9PM on Oct 13]

- Section 12-14 [Posted @ 5PM on Oct 8]
- Appendix
- Calculus [Posted @ 4PM on Oct 4]

- Information regarding the
**final exam**- 13:30 - 16:30; 18 Oct 2012; BKD 2501-2
- Closed book. Closed notes.
- Basic calculators, e.g. FX-991MS, are permitted.

- One
**A4 sheet**allowed.- One side should be Table 3.1 and Table 3.2. (The sheet was distributed in class, but you can print your own sheet using the provided pdf file.)
- Must be hand-written.
- No small pieces of paper notes glued/attached on top of it.
- Indicate your name and id on the upper right corner of the sheet.
- Submit your formula sheet with your final exam. (You can get it back from me next semester.)

- 11 Pages + 1 Cover Page
- 9 Problems + 1 Extra-Credit Problem
- Cover all the materials that we discussed in class and practice in the HWs.
- Strong focus on the materials that haven’t been on the midterm.
- I could ask something that I have never defined in class but, in such a case, I will give you the exact definition on the exam itself.
- These notes are provided for your studying pleasure....
- All commented notes combined in one pdf file
- Post-midterm commented notes
- Uncommented version. (The links inside the document should works. References are also available at the end.)

- All post-midterm OneNote Notes.
- Solutions for HW 4-9
- All slides combined in one pdf file

- All commented notes combined in one pdf file

- Approximate score distribution based on sections taught in lecture:
- 8:5%, 9:25%, 10:40%, 11:20%, 12to14:5%

- More information may be posted here. [Updated @ 10PM on Oct 13]

#### Problem Set

- HW 1 (Due: July 20)
- HW 2 (Due: Aug 1)
- Solution [Posted @ 8:30PM on Aug 3]

- HW 3 (Due: Aug 10)
- Self-Evaluation (Due: Aug 24)
- HW4 (Due: Sep 6)
- HW5 (Due: Sep 13)
- Solution
- Q3e was graded.

- HW6 (Due: Sep 25)
- Solution
- Q2b was graded.

- HW7 (Due: Oct 4)
- Solution
- Q4a was graded.

- HW8 (Due: Oct 11)

Solve the following questions from [Yates & Goodman, 2005]

Q3.1.3, Q3.2.1, Q3.2.3 (Change U,u to W,w), Q3.3.4, Q3.3.6, Q3.4.5- For those of you who still don't have the textbook, you may use these scanned questions.
- Solution

- HW9 (Free)
- For those of you who still don't have the textbook, you may use these scanned questions.
- Table 3.1 and Table 3.2 from [Y&G]
- Solution

- Self-Evaluation (Due: Oct 18)

#### Calendar

#### Reading Assignment

#### Misc. Links

- Video: Probability 101
- More information about theMonty Hall Problem
- Video: The Monty Hall Problem
- Video: Monty Hall Problem: Numb3rs and 21
- Paper: Monty Hall, Monty Fall, Monty Crawl
- Articles: How Random is the iPod Shuffle? [HowStuffWorks]; Is iTunes’ Shuffle Mode Truly Random?[About.com]; iTunes: Just how random is random?[CNET.com.au, 2007]; My IPod for a Random Playlist [wired.com, 2005];
- Video: It *could* just be coincidence
- MV: Bill Nye the Science Guy - "50 Fifty"
- Video: Chevalier de Mere's Scandal of Arithmetic
- Free educational software: Orstat2000
- Originally developed to promote probability and operations research in the senior forms of Dutch high schools (and early college).
- Contain modules for coin-tossing, central limit theorem, etc.

- Probability review from MATH REVIEW for Practicing to Take the GRE General Test
- Video: Mlodinow’s talk @ Google
- Video: The Binomial Distribution / Binomial Probability Function
- Video: The Poisson Distribution
- If you want to experience probability theory at a more advance level, one standard textbook that you can refer to is "Probability: Theory and Examples" by Prof. Durrett. Currently, the 4th edition of the textbook is available online.
- Paper: Cheung, Y. L. "Why Poker is Played with Five Cards."
*Math. Gaz.*73, 313-315, 1989. - Video: Peter Donnelly shows how stats fool juries (same clip on youtube)
- Video: Lies, damned lies and statistics (about TEDTalks): Sebastian Wernicke on TED.com
- Article about clinical/medical decision making: Jill G. Klein, "5 pitfalls in diagnosis and prescribing: psychological biases that can lead to poor judgement," 2005.
- Related topics: Pitfall #1 (representatiove heuristic), Pitfall #2(availability heuristic), and Pitfall #5 (illusory correlation).

- Related topics: Pitfall #1 (representatiove heuristic), Pitfall #2(availability heuristic), and Pitfall #5 (illusory correlation).
- The Median Isn't the Message by Stephen Jay Gould
- Video: Daniel Kahneman: The riddle of experience vs. memory
- Articles on risk intelligence
- Dylan Evans, How to Beat the Odds at Judging Risk, The Wall Street Journal, May, 2012
- Alison George, What Gamblers and Weather Forecasters Can Teach Us About Risk: An interview with the creator of the "risk quotient" intelligence scale., pp 30-31, New Scientist, May 19, 2012

- Generation of random numbers
- Article: Park, S.K., and K.W. Miller. "Random Number Generators: Good Ones Are Hard to Find." Communications of the ACM, 31(10):1192–1201. 1998.
- Article: C. Moler, Random thoughts, "10^435 years is a very long time", MATLAB News and Notes, Fall, 1995
- Article: Ziggurat algorithm generates normally distributed random numbersdescribing the ziggurat algorithm introduced in MATLAB version 5.

- Quotations about Statistics
- Video: Statistics - Dream Job of the next decade
- Google Calculator (Cheat Sheet)
- Sometimes the easiest way to get information on a counting problem is to compute a few small values of a function, then look for a match at the sequence server; if you find a hit, you can sometimes get citations to the literature.
- Prapun's Notes on Probability Theory (Cornell Version)
- MATLAB
- MIT OpenCourseWare > Electrical Engineering and Computer Science > 6.094 Introduction to MATLAB (January (IAP) 2009)

- Learn the Greek Alphabet in less than 10 minutes
- The Greek Alphabet Song