Everything we do, everything that happens around us, obeys the laws of probability. We can no more escape them than we can escape gravity... "Probability," a philosopher (Bishop Butler) once said, "is the very guide of life." We are all gamblers who go through life making countless bets on the outcome of countless actions.
Every field of science is concerned with estimating probability. A physicist calculates the probable path of a particle. A geneticist calculates the chances that a couple will have blue-eyed children. Insurance companies, businessmen, stockbrokers, sociologists, politicians, military experts - all have to be skilled in calculating the probability of the events with which they are concerned.
[Gardner, 1986]
Synopsis
Probability theory is the branch of mathematics that tells us how to estimate degrees of probability. If an event is certain to happen, it is given a probability of 1. If it is certain not to happen, it has a probability of 0.
This course introduces the principles of probability and random processes to undergraduate students in electrical engineering. The topics to be covered include random experiments, events, probability, discrete and continuous random variables, probability density function, cumulative distribution function, functions of random variables, expectations, law of large numbers, central limit theorem, introduction to random processes, Gaussian random process, autocorrelation and power spectral density.
Announcements
- Information regarding the final exam [Posted @ 1PM on Nov 26]
- Check this course website regularly for breaking news about the final.
- Date: December 3, 2020
- TIME: 9:00-11:00
- ROOMs: BKD 3507, 3510
- Basic calculators, e.g. FX-991MS, are permitted.
- 9 pages + one cover page
- 17+1 = 18 questions.
- To save time, read the cover pagebefore going into the exam room.
- Closed book. Closed notes.
- One A4 sheet allowed.
- Use the returned study sheet from the midterm exam.
- Add more formulas or information on the remaining side (or in the remaining space(s)).
- Basically, you have one whole page for the final exam.
- Q: Can I make a brand new study sheet?
A: No.
However, if you think the old one is in a bad shape (falling apart), you can start with a photocopy of your own. (Reduction in size is not allowed though) - Q: What can I do if I lose my study sheet?
A: Dr.Prapun may be able to print out a scanned copy for you. (The writing won’t be as crisp as the original one but it’s better than nothing.) Alternatively, see below. - Q: I think the midterm study sheet (and the thing I wrote on it) is cursed (making me get poor score); I don’t want to see or use anything on it anymore. What can I do?
A: You may bring a brand new sheet for the final exam. However, you can only use one side of it.
- Basically, same rules as the midterm:
- Must be hand-written in your own handwriting.
- No small pieces of paper notes glued/attached on top of it.
- Indicate your name and ID on the upper right corner of the sheet (in portrait orientation).
- Do not modify (,e.g., add/underline/highlight) content on the sheet inside the exam room.
- Submit your A4 sheet with your exam.
- Violating the above instructions may cost you upto 10 pt.
- Use the returned study sheet from the midterm exam.
- Cover all the materials that we discussed in class and practice in the exercises and HWs.
- Material Distribution (score-wise): CH6 (14.5) + CH7&8 (22) + CH9 (20.5) + CH10 (9)
- For your studying pleasure....
- All post-midterm annotated notes combined in one pdf file.
- All post-midterm HWs and their solutions
- Checked HWs can be picked up in front of the EE office on the 6th floor.
- All post-midterm exercises and their solutions
- Graded exercises are posted on Google Drive.
- If you have valid reason for missing class on the day that we have exercise, please indicate the date, exercise number, and the reason in the (second) online self-evaluation form . Make sure that you also submit/email supporting document/evidence to Dr.Prapun (if you haven't done so).
- All post-midterm slides
- See some old exams.
- Midterm results [Posted @ 5:30PM on Oct 7]
- Information regarding the midterm exam [Posted @ 9PM on Sep 16; Last updated @ 5PM on Sep 25]
- Slides [Posted @ 5PM on Sep 25]
- The midterm exam:
- 7 pages (including the cover page)
- To save time, read the cover page before going into the exam room.
- A typo is fixed @ 10:30PM on Sep 25:
- Last line of the page: "first", not "fist".
- 12+1 = 13 questions.
- 50+1 = 51 points.
- Cover all the materials that we discussed in class and practice in the HWs.
- Material Distribution (score-wise): 8 (CH1-2) + 21 (CH3-4) + 13 (CH5) + 8 (Sec 6.1)
- Closed book. Closed notes.
- (1 pt) One A4 page allowed.
- Must be hand-written in your own handwriting.
- Recommendation: Avoid using pencil or erasable pen.
- No small pieces of paper notes glued/attached on top of it.
- Indicate your name and ID on the upper right corner of the sheet (in portrait orientation).
- Do not modify (,e.g., add/underline/highlight) content on the sheet inside the exam room.
- Make sure that another side is blank. This will be used for the final exam.
- Submit your A4 sheet with your exam. (You will get it back before the final exam.)
- Q: I don't need any formulas. What should I do?
A: Bring in and submit a blank sheet of paper with your name and ID.
- Q: Can I use iPad or other devices with pen input to produce this?
A: Yes, but, again, all the content must still be hand-written in your own handwriting.
- Violating the above instructions will cost you 10 pt.
- Must be hand-written in your own handwriting.
- Basic SIIT-approved calculators, e.g,. FX-991 series, are permitted, but borrowing is not allowed.
- For your studying pleasure....
- All pre-midterm annotated notes combined in one pdf file.
- All pre-midterm HWs and their solutions
- All pre-midterm exercises and their solutions
- Graded exercises are posted on Google Drive.
- All pre-midterm slides
- Graded exercises are posted on Google Drive.
- Note that we also share the make-up session with EES351. See Google calendar below.
- Welcome to EES315! Feel free to look around this site.
General Information
- Instructor: Asst. Prof. Dr.Prapun Suksompong (prapun@siit.tu.ac.th)
- Office Hours: See Google calendar below.
- Lectures: See Google calendar below.
- Course Syllabus [Posted @ 4PM on Aug 7]
- Textbook: [Y&G] R. D. Yates and D. J. Goodman, Probability and Stochastic Processes: A Friendly Introduction for Electrical and Computer Engineers, 2nd ed., Wiley, 2004.
- Call No. QA273 Y384 2005. ISBN: 978-0-471-27214-4
- Student Companion Site
- The 3rd Edition
- References
- Draft of the Lecture notes [Posted @ 4PM on Aug 7]
- Caution: The lecture notes will still be updated throughout the semester. Therefore, do not rely on this version for use in class. The arrangement/inclusion/exclusion of topics may be different from the ones posted below and the versions available at the copy center.
- Probability and probabilistic reasoning for electrical engineering / Terrence L. Fine. Call No. QA273 F477 2006
- Probability and random processes for electrical engineering / Alberto Leon-Garcia. Call No. TK153 L425 1994
- Free textbook: Introduction to Probability by Charles M. Grinstead and J. Laurie Snell
- Henk Tijms. Understanding Probability: Chance Rules in Everyday Life. Cambridge University Press, 3rd edition, 2012. Call No. QA273 T48 2012
- Probability, random variables, and stochastic processes / Athanasios Papoulis, S. Unnikrishna Pillai. Call No. QA273 P2 2002
- Probability, random variables, and stochastic processes / Athanasios Papoulis. Call No. QA273 P2 1991
- A first course in probability / Sheldon Ross. 6E Call No. QA273 R83 2002
- A first course in probability / Sheldon Ross. Call No. QA273 R83 1976
- A first course in probability / Sheldon Ross. 9E
- Probability models, introduction to / Sheldon M. Ross. 10E Call No. QA273 R84 2010
- Probability models, introduction to / Sheldon M. Ross. 6E Call No. QA273 R84 1997
- Probability models, introduction to / Sheldon M. Ross. 11E
Handouts and Course Material
- Slides: Course Introduction [Posted @ 4PM on Aug 7; Updated @ 5PM on Aug 14]
- Exercise 1 on Google Classroom
- Exercise 1 Solution [Posted @ 10AM on Sep 8]
- Part I: Introduction, Set Theory, Classical Probability theory, and Combinatorics [Posted @ 4PM on Aug 6]
- Chapter 1: Probability and You
- Annotated version [Posted @ 4:30PM on Aug 19; Updated @ 5PM on Aug 21]
- Slides [Posted @ 4:30PM on Aug 19; Updated @ 5PM on Aug 21]
- Exercise 2 Solution [Posted @ 10AM on Sep 8]
- Chapter 2: Set Theory
- Annotated version [Posted @ 5PM on Aug 21; Updated @ 8PM on Aug 26]
- Slides [Posted @ 5PM on Aug 21]
- Exercise 3 Solution [Posted @ 10AM on Sep 8]
- Chapter 3: Classical Probability
- Annotated version [Posted @ 8PM on Aug 26; Updated @ 9PM on Aug 28]
- Slides [Posted @ 9PM on Aug 28]
- Exercise 4 Solution [Posted @ 10AM on Sep 8]
- Chapter 4: Enumeration / Combinatorics / Counting
- Annotated version [Posted @ 9PM on Aug 28; Updated @ 7PM on Sep 2, @ 9:30AM on Sep 10, and @ 9:30PM on Sep 11]
- Slides [Posted @ 9PM on Aug 28; Updated @ 7PM on Sep 2]
- Exercise 5 Solution [Posted @ 10AM on Sep 14]
- Exercise 6 Solution [Posted @ 10AM on Sep 14]
- Exercise 7 Solution [Posted @ 10AM on Sep 14]
- Combination using the calculator Casio fx-991ES
- Permutation using the calculator Casio fx-991ES
- Part II: Kolmogorov's Formal Probability Theory and Event-Based Probability Theory
- Chapter 5: Probability Foundations [Posted @ 10AM on Sep 8]
- Annotated version [Posted @ 9:30PM on Sep 11; Updated @ 8PM on Sep 16]
- Exercise 8 Solution [Posted @ 10AM on Sep 19]
- Slides [Posted @ 9PM on Sep 23]
- Chapter 6: Event-based Independence and Conditional Probability [Posted @ 10AM on Sep 8]
- Section 6.1: Conditional Probability
- Annotated version [Posted @ 8PM on Sep 16; Updated @10PM on Sep 18 and @ 5PM on Sep 25]
- Slides [Posted @ 9PM on Sep 23; Updated @ 5PM on Oct 9]
- Exercise 9 Solution [Posted @ 11AM on Sep 24]
- Exercise 10 Solution [Posted @ 11AM on Sep 24]
- The Theory That Would Not Die, How Bayes' Rule Cracked the Enigma Code, Hunted Down Russian Submarines & Emerged Triumphant from Two Centuries of Controversy" (Yale University Press, 2011)
- A book review (The New York Times)
- An article (Scientific American)
- An interview (BBC Radio Wales' Sunday Supplement)
- Section 6.2: Event-based Independence
- Annotated version [Posted @ 5:30PM on Oct 7; Updated @ 5PM on Oct 9]
- Slides [Posted @ 5PM on Oct 9]
- Exercise 11 Solution [Posted @ 9:30AM on Oct 27]
- Exercise 12 Solution [Posted @ 9:30AM on Oct 27]
- Section 6.3: Bernoulli Trials
- Annotated version [Posted @ 9:30AM on Oct 15; Updated @ 5PM on Oct 16]
- Slides [Posted @ 5PM on Oct 16]
- Exercise 13 Solution [Posted @ 9:30AM on Oct 27]
- Section 6.1: Conditional Probability
- Part III: Discrete Random Variables
- Chapter 7: Random Variables [Posted @ 11AM on Oct 12]
- Annotated version [Posted @ 5PM on Oct 16; Updated @ 10PM on Oct 23 and @ 7:30PM on Oct 28]
- Slides [Posted @ 11AM on Oct 21]
- Exercise 14 Solution [Posted @ 9:30AM on Oct 27]
- Chapter 8: Discrete Random Variables [Posted @ 11AM on Oct 12]
- Sections 8.1-8.2: PMF and CDF
- Annotated version [Posted @ 7:30PM on Oct 28; Updated @ 8:30AM on Oct 31]
- Section 8.3-8.4: Families of Discrete Random Variables
- Annotated version [Posted @ 4:30PM on Nov 5; Updated @ 11PM on Nov 6 and @ 3:30PM on Nov 12]
- Slides [Posted @ 11AM on Oct 21; Updated @ 4:30PM on Nov 5 and @ 11PM on Nov 6]
- Exercise 15 Solution [Posted @ 12:30PM on Nov 19]
- Exercise 16 Solution [Posted @ 3PM on Nov 19]
- Exercise 17 Solution [Posted @ 3PM on Nov 19]
- Exercise 18 Solution [Posted @ 3PM on Nov 19]
- Sections 8.1-8.2: PMF and CDF
- Chapter 9: Expectation and Variance [Posted @ 2PM on Nov 3]
- Annotated version [Posted @ 3:30PM on Nov 12; Updated @ 11AM on Nov 17]
- Slides [Posted @ 3:30PM on Nov 12]
- Exercise 19 Solution [Posted @ 3PM on Nov 19]
- Exercise 20 Solution [Posted @ 3PM on Nov 19]
- References: [Y&G] Chapter 2
- Part IV: Continuous Random Variables
- Chapter 10: Continuous Random Variables [Posted @ 2PM on Nov 3]
- Annotated version [Posted @ 11PM on Nov 18; Updated @ 1PM on Nov 26 and @ 2PM on Nov 27]
- Slides [Posted @ 11PM on Nov 18]
- Exercise 21 Solution [Posted @ 3PM on Nov 19]
- References
- From Discrete to Continuous Random Variables: [Y&G] Sections 3.0 to 3.1
- PDF and CDF: [Y&G] Sections 3.1 to 3.2
- Expectation and Variance: [Y&G] Section 3.3
- Families of Continuous Random Variables: [Y&G] Sections 3.4 to 3.5
- Part V: Advanced Topics [Posted @ 5PM on Nov 23]
- Chapter 11: Multiple Random Variables
- References:
- A Pair of Random Variables: [Y&G] Sections 4.1 to 4.3 and Section 4.10
- Extending the Definitions to Multiple RVs: [Y&G] Sections 5.1 to 5.4
- Function of Discrete Random Variables: [Y&G] Section 4.6 (Theorem 4.9)
- Expectation of function of discrete random variables: [Y&G] Sections 4.7 and 6.1
- Linear Dependence: [Y&G] Section 4.7
- Chapter 12: Limiting Theorems (LLN and CLT)
- Chapter 13: Three Types of Random Variables
- Appendix
Problem Set
- HW 1 (Due:Sep 2)
- Solution [Posted @ 4:30PM on Sep 10]
- HW2 (Due: Sep 9)
- Solution [Posted @ 4:30PM on Sep 10]
- HW3 (Due: Sep 16)
- Solution [Posted @ 4:30PM on Sep 10]
- HW4 (Due: Sep 23)
- Solution [Posted @ 11AM on Sep 24]
- HW5 (Not Due)
- Solution [Posted @ 4:30PM on Sep 22]
- HW6 (Due: Oct 22)
- Solution [Posted @ 3:30PM on Oct 29]
- HW7 (Due: Nov 6)
- Solution [Posted @ 11:30AM on Nov 24]
- HW8 (Due: Nov 13)
- Solution [Posted @ 11:30AM on Nov 24]
- HW9 (Due: Nov 25)
- Solution [Posted @ 1PM on Nov 26]
- HW10 (Not Due)
- Solution [Posted @ 11:30AM on Nov 24]
Calendar
Reading Assignment
- Section 1.2 in the lecture notes
- Section 2.5 in the lecture notes
Old Exams
- 2018 Midterm Exam
- Midterm 2020 does not cover Q8, last part of Q11a&b, Q12, Q13d-e, Q14
- Annotated version from 2019
- Annotated version from 2020
- The solutions of the in-class exercises in 2019 also contain the solutions for many of these problems.
- 2017 Midterm Exam
- Midterm 2020 does not cover last part of Q7a&b, Q8, Q10
- Annotated version from 2020
- The solutions of the in-class exercises in 2018 also contain the solutions for many of these problems.
- 2016 Midterm Exam
- Midterm 2020 does not cover Q7b, Q8, Q10-13.
- Annotated version from 2020
- Annotated version from 2017
- The solutions of the in-class exercises in 2017 also contain the solutions for many of these problems.
- 2013 Midterm Exam
- Midterm 2020 does not cover Q2a.iv, Q2b.iv, Q2c, Q3b, Q5-7.
- Annotated version from 2017
- Annotated version from 2015
- Annotated version from 2014
- 2010 Midterm Exam
- Midterm 2020 does not cover Q1b-e, Q2c-e, Q5-9.
- Final 2020 does not cover Q6, Q8-9
- Annotated version from 2015
- Annotated version from 2014
- Solution
- 2013 Final Exam
- Final 2020 does not cover Q1.b.5-6 (probability calc. of Gaussian), Q3b, Q3c.ii-iii, Q3d, Q4c, Q5-7, Q8b-c
- Annotated version from 2020
- Annotated version from 2015
- Annotated version from 2014
- 2011 Final Exam
- Problems in here cover many topics that are not discussed in Final 2020. That said, it is still possble to solve Q2a-c,e, Q6a, Q9a
- 2010 Final Exam
- Problems in here cover topics that are not discussed in Final 2020. That said, it is still possble to solve Q1a-d,f, Q1e.ii,
More References
- Older version of the textbook: Probability and stochastic processes : a friendly introduction for electrical and computer engineers / Roy D. Yates, David J. Goodman. Call No. QA273 Y384 1999
- 10-page probability cheatsheet compiled from Harvard's Introduction to Probability course
- Allen B. Downey, Think Stats: Exploratory Data Analysis, O'Reilly Media, 2014 (free book)
- Allen B. Downey, Think Bayes: Bayesian Statistics in Python, O'Reilly Media, 2013 (free book)
- Encyclopædia Britannica Online: Probability Theory
- Random signals for engineers using MATLAB and Mathcad / Richard C. Jaffe. Call No. TK5102.9 J34 2000
- Davenport, W.B., Probability and Random Processes, McGraw-Hill, New York, 1970. (Excellent introductory text.)
- Feller, W., An Introduction to Probability Theory and its Applications, Vols. 1, 2, John Wiley, New York, 1950. (Definitive work on probability—requires mature mathematical knowledge.)
- Call No. QA273 F37 1966
- Peter Olofsson, Probabilities The Little Numbers That Rule Our Lives, Wiley, 2006
- Henk Tijms, Probability: A Lively Introduction, Cambridge University Press, 2017, Call No. QA273 P278 1962
- Stochastic processes / Sheldon M. Ross. Call No. QA274 R65 1996
- Stochastic processes / Emanuel Parzen. Call No. QA273 P278 1962
- MATLAB Primer, 8th edition T. A. Davis. CRC Press, 2010.
- Seventh Edition by T. A. Davis and K. Sigmon: Call No. QA297 D38 2005
- Third Edition by K. Sigmon (Free)
- Second Edition by K. Sigmon (Free)
Misc. Links
- Video: Probability 101
- More information about theMonty Hall Problem
- Video: The Monty Hall Problem
- Video: Monty Hall Problem: Numb3rs and 21
- Paper: Monty Hall, Monty Fall, Monty Crawl
- Articles: How Random is the iPod Shuffle? [HowStuffWorks]; Is iTunes’ Shuffle Mode Truly Random?[About.com]; iTunes: Just how random is random?[CNET.com.au, 2007]; My IPod for a Random Playlist [wired.com, 2005];
- Video: It *could* just be coincidence
- MV: Bill Nye the Science Guy - "50 Fifty"
- Video: Chevalier de Mere's Scandal of Arithmetic
- Free educational software: Orstat2000
- Originally developed to promote probability and operations research in the senior forms of Dutch high schools (and early college).
- Contain modules for coin-tossing, central limit theorem, etc.
- Probability review from MATH REVIEW for Practicing to Take the GRE General Test
- Video: Mlodinow’s talk @ Google
- Video: How many ways can you arrange a deck of cards? (There Are More Ways To Arrange a Deck of Cards Than Atoms on Earth)
- Provide nice animation explaining permutation and factorial.
- "Any time you pick up a well shuffled deck, you are almost certainly holding an arrangement of cards that has never before existed and might not exist again."
- Video: The Binomial Distribution / Binomial Probability Function
- Article: Greenlighting Movies: A High-Risk Game
- Video: The Poisson Distribution
- If you want to experience probability theory at a more advance level, one standard textbook that you can refer to is "Probability: Theory and Examples" by Prof. Durrett. Currently, the 4th edition of the textbook is available online.
- Video: Peter Donnelly shows how stats fool juries (same clip on youtube)
- Video: Lies, damned lies and statistics (about TEDTalks): Sebastian Wernicke on TED.com
- Article about clinical/medical decision making: Jill G. Klein, "5 pitfalls in diagnosis and prescribing: psychological biases that can lead to poor judgement," 2005.
- Related topics: Pitfall #1 (representatiove heuristic), Pitfall #2(availability heuristic), and Pitfall #5 (illusory correlation).
- Related topics: Pitfall #1 (representatiove heuristic), Pitfall #2(availability heuristic), and Pitfall #5 (illusory correlation).
- The Median Isn't the Message by Stephen Jay Gould
- Video: Daniel Kahneman: The riddle of experience vs. memory
- Articles on risk intelligence
- Dylan Evans, How to Beat the Odds at Judging Risk, The Wall Street Journal, May, 2012
- Alison George, What Gamblers and Weather Forecasters Can Teach Us About Risk: An interview with the creator of the "risk quotient" intelligence scale., pp 30-31, New Scientist, May 19, 2012
- Generation of random numbers
- Article: Park, S.K., and K.W. Miller. "Random Number Generators: Good Ones Are Hard to Find." Communications of the ACM, 31(10):1192–1201. 1998.
- Article: Tom McNichol, "Totally Random: How two math geeks with a lava lamp and a webcam are about to unleash chaos on the Internet"
- Article: C. Moler, Random thoughts, "10^435 years is a very long time", MATLAB News and Notes, Fall, 1995
- Article: Ziggurat algorithm generates normally distributed random numbersdescribing the ziggurat algorithm introduced in MATLAB version 5.
- Games of chance
- Poker
- Paper: Cheung, Y. L. "Why Poker is Played with Five Cards." Math. Gaz. 73, 313-315, 1989.
- Tim Farajian's Texas Hold'Em Poker Analyzer in MATLAB
- Allow a user to simulate different scenarios in a Texas Hold'Em game.
- Automatically simulate as many hands as you would like, and display winning probabilities or expected returns.
- Blackjack
- Cleve Moler's Blackjack in MATLAB + article
- Michael Iori's Blackjack in MATLAB
- Poker
- Quotations about Statistics
- Video: Statistics - Dream Job of the next decade
- Virtual Laboratories in Probability and Statistics
- Google Calculator (Cheat Sheet)
- Sometimes the easiest way to get information on a counting problem is to compute a few small values of a function, then look for a match at the sequence server; if you find a hit, you can sometimes get citations to the literature.
- Prapun's Notes on Probability Theory (Cornell Version)
- MATLAB
- MIT OpenCourseWare > Electrical Engineering and Computer Science > 6.094 Introduction to MATLAB (January (IAP) 2009)
- Fundamentals: Academic Tutorial
- Video: Controlling Random Number Generation
- Free online book: Cleve Moler, Experiments with MATLAB, 2008
- Free textbook: Cleve Moler,Numerical Computing with MATLAB, 2004
- Learn the Greek Alphabet in less than 10 minutes
- The Greek Alphabet Song