Everything we do, everything that happens around us, obeys the laws of probability. We can no more escape them than we can escape gravity... "Probability," a philosopher (Bishop Butler) once said, "is the very guide of life." We are all gamblers who go through life making countless bets on the outcome of countless actions.
Every field of science is concerned with estimating probability. A physicist calculates the probable path of a particle. A geneticist calculates the chances that a couple will have blue-eyed children. Insurance companies, businessmen, stockbrokers, sociologists, politicians, military experts - all have to be skilled in calculating the probability of the events with which they are concerned.
[Gardner, 1986]
Synopsis
Probability theory is the branch of mathematics that tells us how to estimate degrees of probability. If an event is certain to happen, it is given a probability of 1. If it is certain not to happen, it has a probability of 0.
This course introduces the principles of probability and random processes to undergraduate students in electronics and communication. The topics to be covered include random experiments, events, probability, discrete and continuous random variables, probability density function, cumulative distribution function, functions of random variables, expectations, law of large numbers, central limit theorem, introduction to random processes, Gaussian random process, autocorrelation and power spectral density.
Announcements
- Information regarding the final exam
- Date: 9 Dec 2015
- Time: 09:00 - 12:00
- ROOM: BKD 3507
- Basic calculators, e.g. FX-991MS, are permitted.
- Closed book. Closed notes.
- Two A4 sheets allowed.
- The first sheet is your midterm study sheet.
- May add more formulas on the remaining side or in the remaining space(s).
- The second sheet must contain the Phi/Q tables
- Distributed in class on Nov 25.
- May reprint if necessary but do not reduce the size.
- May add more formulas in the remaining side or space(s).
- The first sheet is your midterm study sheet.
- Must be hand-written in your own handwriting (except for the Phi/Q tables).
- No small pieces of paper notes glued/attached on top of it.
- Indicate your name and ID on the upper right corner of each sheet (in portrait orientation).
- Do not modify (,e.g., add/underline/highlight) content on the sheet inside the exam room.
- Submit your A4 sheets with your exam.
- 1 pt
- 8 questions (24+18+10+20+1+20+6+1)
- 8 pages + 1 cover page
- Cover all the materials that we discussed in class and practice in the HWs.
- Material Distribution (score-wise; rounded to the nearest multiples of 5%): Ch 8 (10%), Ch 9 (20%), Ch 10 (50%), Ch 11 (20%).
- Ch 13 is simply an application of earlier chapters.
- Ch 12 is not on the exam.
- Material Distribution (score-wise; rounded to the nearest multiples of 5%): Ch 8 (10%), Ch 9 (20%), Ch 10 (50%), Ch 11 (20%).
- For your studying pleasure....
- All post-midterm annotated notes combined in one pdf file
- All post-midterm HWs and their solutions
- Correction: For Q2a, the formula for the variance of the binomial RV should have a factor of n. [Credit: Possawee]
- All post-midterm slides
- Quiz 4 Solution
- 2013 Final Exam
- You may skip the following parts because they are not covered this year: 6, 7, 8b
- Annotated version[Posted @ 4PM on Dec 1; Updated @ 5:30PM on Dec 3]
- Annotated version from 2014
- 2010 Midterm Exam
- Annotated version
- Annotated version from 2014 (which contains some more solutions)
- Solution
- 2011 Final Exam
- We covered quite a different range of topics there because the curriculum was different. However, you can find some relevent problems in there as well.
- 2010 Final Exam
- We covered quite a different range of topics there because the curriculum was different. However, you can find some relevent problems in there as well.
- No class on Nov 12 (Thursday)
- Make-up class: 9-10:20AM on Nov 11 (Wednesday)
- The Thursday class on Nov 19 is replaced by ECS332 lecture.
- Make-up class: 9-10:20AM on Nov 18 (Wednesday)
- See the announcement
- Midterm results [Posted @ 6PM on Oct 15]
- Information regarding the midterm exam
- Date: 30 Sep 2015
- Time: 09:00 - 12:00
- ROOM: BKD 2602, 2605
- Basic SIIT-approved calculators, e.g,. FX-991MS, are permitted.
- 9 pages + 1 cover page
- 12 questions (12+14+10+10+12+8+4+13+6+2+6+2) + 1 pt (A4 submission) + 1 pt extra credit
- Closed book. Closed notes.
- One A4 page allowed.
- Must be hand-written in your own handwriting.
- No small pieces of paper notes glued/attached on top of it.
- Indicate your name and ID on the upper right corner of the sheet (in portrait orientation).
- Do not modify (,e.g., add/underline/highlight) content on the sheet inside the exam room.
- Submit your A4 sheet with your exam. (You can get it back from me after the midterm is graded.)
- 1 pt
- Cover all the materials that we discussed in class and practiced in the HWs.
- Material Distribution (score-wise): 12% (Sec 1-2) + 8% (Sec 3) + 6% (Sec 4) + 14.5% (Sec 5) + 22.5% (Sec 6.1) + 11% (Sec 6.2) + 12% (Sec 6.3) + 2% (Sec 7) + 11% (Sec 8)
- For your studying pleasure....
- All pre-midterm annotated notes combined in one pdf file.
- The orginal notes (no annotation) but with workable links (including the table of contents and references)
- All pre-midterm HWs and their solutions
- Graded HWs can be picked up from the green box outside the EC secretary office (room 3514).
- All pre-midterm quiz solutions
- All pre-midterm slides
- Sample Exams:
- 2010 Midterm Exam
- You may skip the following parts because we haven't studied the corresponding topics yet in class: 5h-5l, 6-9
- Annotated version
- Annotated version from 2014 (which contains some more solutions)
- Solution
- 2013 Midterm Exam
- You may skip the following parts because we haven't studied the corresponding topics yet in class: 6c, 7c, 7d
- Annotated version
- Annotated version from 2014 (which contains some more solutions)
- 2010 Midterm Exam
- For those who have already obtained the textbook [Y&G], this table gives the corresponding positions (if any) in the textbook that correspond to the material dfiscussed in class.
- All pre-midterm annotated notes combined in one pdf file.
- Room change:
- Wednesday: BKD 3511 (We swap with ECS213 and ECS216)
- A basic RSS feed is created to track and inform updates.
- This site can be accessed via ecs315.prapun.com.
- Welcome to ECS315! Feel free to look around this site.
General Information
- Instructor: Asst. Prof. Dr.Prapun Suksompong (prapun@siit.tu.ac.th)
- Office: BKD, 4th floor of Sirindhralai building
- Office Hour: M 9:30-10:30, M 14:00-16:00, R 16:00-17:00
- Course Syllabus [Posted @ 1PM on Aug 10]
- Textbook: [Y&G] R. D. Yates and D. J. Goodman, Probability and Stochastic Processes: A Friendly Introduction for Electrical and Computer Engineers, 2nd ed., Wiley, 2004.
- Call No. QA273 Y384 2005. ISBN: 978-0-471-27214-4
- Student Companion Site
- References
- Probability and probabilistic reasoning for electrical engineering / Terrence L. Fine. Call No. QA273 F477 2006
- Probability and random processes for electrical engineering / Alberto Leon-Garcia. Call No. TK153 L425 1994
- Free textbook: Introduction to Probability by Charles M. Grinstead and J. Laurie Snell
- Henk Tijms. Understanding Probability: Chance Rules in Everyday Life. Cambridge University Press, 3rd edition, 2012. Call No. QA273 T48 2012
- Probability, random variables, and stochastic processes / Athanasios Papoulis, S. Unnikrishna Pillai. Call No. QA273 P2 2002
- Probability, random variables, and stochastic processes / Athanasios Papoulis. Call No. QA273 P2 1991
- A first course in probability / Sheldon Ross. Call No. QA273 R83 2002
- A first course in probability / Sheldon Ross. Call No. QA273 R83 1976
- Probability models, introduction to / Sheldon M. Ross. Call No. QA273 R84 1997
Handouts and Course Material
- Slides: Course Introduction [Posted @ 1PM on Aug 10]
- Full version [Posted @ 5PM on Aug 13]
- Part I: Introduction, Set Theory, Classical Probability theory, and Combinatorics
- Section 1: Probability and You [Posted @ 4PM on Aug 9]
- Annotated version [Posted @ 5PM on Aug 13; Updated @ 1:30PM on Aug 19]
- Slides [Posted @ 5PM on Aug 13]
- Section 2: Review of Set Theory [Posted @ 4PM on Aug 9]
- Annotated version [Posted @ 1:30PM on Aug 19; Updated @ 5PM on Aug 20]
- Section 3: Classical Probability [Posted @ 4PM on Aug 9]
- Annotated version [Posted @ 5PM on Aug 20]
- Slides [Posted @ 5PM on Aug 20]
- Slides: Working with randomness in MATLAB [@ 5PM on Aug 20]
- Section 4: Enumeration / Combinatorics / Counting [Posted @ 4PM on Aug 9]
- Annotated version [Posted @ 1:30PM on Aug 19; Updated @ 5:30PM on Aug 20, @ 8PM on Aug 26, @ 12:30PM on Sep 2, and @ 9:30PM on Sep 24]
- Slides [Posted @ 1:30PM on Aug 19; Updated @ 5:30PM on Aug 20 and @ 8PM on Aug 26]
- Extra reading [Posted @ 4PM on Aug 9]
- Section 1: Probability and You [Posted @ 4PM on Aug 9]
- Part II: Kolmogorov's Formal Probability Theory and Event-Based Probability Theory
- Section 5: Probability Foundations [Posted @ 8:30PM on Aug 25; Distributed in class on Aug 26]
- Annotated version [Posted @ 8PM on Aug 26; Updated @ 9:30PM on Aug 27]
- Slides [Posted @ 8PM on Aug 26; Updated @ 9:30PM on Aug 27]
- Section 5: Probability Foundations [Posted @ 8:30PM on Aug 25; Distributed in class on Aug 26]
- Section 6.1: Conditional Probability [Posted @ 9:30PM on Aug 27; Updated @ 9AM on Sep 2; Distributed in class on Aug 26; More will be distributed in class]
- Annotated version [Posted @ 9:30PM on Aug 27; Updated @ 12:30PM on Sep 2, @ 6PM on Sep 3, and @ 2PM on Sep 9]
- Slides [Posted @ 9:30PM on Aug 27; Updated @ 3:30PM on Sep 9]
- Solution for Quiz 1 [Posted @ 9:30AM on Sep 14]
- Solution for Quiz 2 [Posted @ 9:30AM on Sep 14]
- Section 6.2: Event-based Independence [Posted @ 10PM on Sep 8; Distributed in class on Sep 9]
- Annotated version [Posted @ 9PM on Sep 10; Updated @ 1:30PM on Sep 16]
- Slides [Posted @ 9PM on Sep 10]
- Section 6.3: Bernoulli Trials [Posted @ 10PM on Sep 15]
- Annotated version [Posted @ 1:30PM on Sep 16; Updated @ 9PM on Sep 17 and @ 2PM on Sep 23]
- Solution for Quiz 3 [Posted @ 2PM on Sep 19]
- Part III: Discrete Random Variables
- Section 7: Random Variables [Posted @ 9:30PM on Sep 17]
- Annotated version [Posted @ 2PM on Sep 23]
- Section 8.1-8.2: Discrete RV: PMF and CDF [Posted @ 9:30PM on Sep 21]
- Annotated version [Posted @ 4:30PM on Sep 24]
- Slides [Posted @ 8PM on Sep 24; Updated @ 2:30PM on Oct 14]
- Section 8.3-8.4: Families of Discrete Random Variables
[Posted @ 2:30PM on Oct 14; Distibuted in class on Oct 14]
- Annotated version [Posted @ 2:30PM on Oct 14; Updated @ 6PM on Oct 15, @ 3:30PM on Oct 21, @ 4:30PM on Oct 22, and @ 9PM on Oct29]
- Slides [Posted @ 2:30PM on Oct 14; Updated @ 6PM on Oct 15]
- Quiz 4 Solution
- Tutorial [Posted @ 3:30PM on Oct 21]
- Section 9: Expectation and Variance [Posted @ 10:30PM on Oct 20; Distributed in class on Oct 21]
- Annotated version [Posted @ 4:30PM on Oct 22; Updated @ 4PM on Oct 28 and @ 9PM on Oct29]
- Slides [Posted @ 4:30PM on Oct 22]
- Part IV: Continuous Random Variables
- Section 10.1: pdf [Posted @ 4PM on Oct 28; Distributed in class on Oct 28]
- Annotated version [Posted @ 9PM on Oct29; Updated @ 1PM on Nov 4]
- Tutorial [Posted @ 1PM on Nov 4]
- Section 10.2-10.3
- Annotated version [Posted @ 1PM on Nov 4; Updated @ 5PM on Nov 5]
- Section 10.4: Families of Continuous Random Variables
- Annotated version [Posted @ 5PM on Nov 5; Updated @ 3PM on Nov 11]
- Section 10.5: Function of Continuous Random Variables: SISO
- Annotated version [Posted @ 1PM on Nov 18; Updated @ 2PM on Nov 25]
- Slides [Posted @ 1PM on Nov 18]
- Slides: Working with Randomness using MATLAB [Posted @ 5PM on Nov 5]
- Section 10.1: pdf [Posted @ 4PM on Oct 28; Distributed in class on Oct 28]
- Part V: Multiple Random Variables
- Section 11.1-11.2
[Distributed in class on Nov 18]
- Annotated version [Posted @ 1PM on Nov 18; Updated @ 2PM on Nov 25, @ 8:30PM on Nov 26, and @ 1PM on Dec 2]
- Slides [Posted @ 8:30PM on Nov 26]
- Section 11.3-11.5
- Annotated version [Posted @ 1PM on Dec 2; Updated @ 5:30PM on Dec 3]
- Section 11.1-11.2
[Distributed in class on Nov 18]
- Part VI: Limiting Theorems
- Sections 12.1-12.2 (LLN and CLT)
- Annotated version [Posted @ 5:30PM on Dec 3]
- Sections 12.1-12.2 (LLN and CLT)
- Part VII: Additional Topics
- Section 13
- Annotated version [Posted @ 5:30PM on Dec 3]
- Section 13
Problem Set
- HW 1 (Due: Aug 26, 9:19 AM (in tutorial session))
- Correction: For the last part of Q2, the right boundary of the second interval should be +infinity. The HW1 file above is now updated. [Credit: Natchapon]
- Solution [Posted @ 5PM on Aug 30]
- HW2 (Due: Sep 2, 9:19 AM (in tutorial session))
- Solution [Posted @ 4PM on Sep 2]
- HW3 (Due: Sep 9, 9:19 AM (in tutorial session))
- Correction: For part (a) of the hint in Q8, the answer is "not" 1/6. [Credit: Nicha]
- Solution [Link fixed @ 2PM on Sep 19]
- HW4 (Due: Sep 16, 9:19 AM (in tutorial session))
- Solution [Posted @ 2:30PM on Sep 16]
- HW5 (Due: Sep 23, 9:19 AM (in tutorial session))
- Solution [Posted @ 4:30PM on Sep 24]
- HW6 (Not Due)
- Solution [Posted @ 4:30PM on Sep 24]
- Self-evaluation (Due: October 21 (online))
- HW7 (Due: Oct 28, 9:19 AM (in tutorial session))
- Solution [Posted @ 4PM on Oct 28]
- HW8 (Due: Nov 4, 9:19 AM (in tutorial session))
- Solution [Posted @ 9PM on Nov 4; Fixed @ 10PM on Dec 5]
- Correction: For Q2a, the formula for the variance of the binomial RV should have a factor of n. [Credit: Possawee]
- HW9 (Due: Nov 11, 9:19 AM (right before the tutorial session))
- Solution [Posted @ 2:30PM on Nov 11]
- HW10 (Due: Nov 18, 8:59 AM)
- Solution [Posted @ 8PM on Nov 22]
- HW11 (Due: Nov 25, 9:19 AM)
- Solution [Posted @ 3PM on Nov 25]
- HW12 (Due: Dec 2, 9:19 AM)
- Solution [Posted @ 1PM on Dec 2]
- P_XY_marginal.m, P_XY_marginal_2.m
- HW13 (Not Due)
- Solution [Posted @ 1PM on Dec 2]
- Self-evaluation (2) (Due: December (online))
Calendar
Reading Assignment
More References
- Older version of the textbook: Probability and stochastic processes : a friendly introduction for electrical and computer engineers / Roy D. Yates, David J. Goodman. Call No. QA273 Y384 1999
- Encyclopædia Britannica Online: Probability Theory
- Random signals for engineers using MATLAB and Mathcad / Richard C. Jaffe. Call No. TK5102.9 J34 2000
- Davenport, W.B., Probability and Random Processes, McGraw-Hill, New York, 1970. (Excellent introductory text.)
- Feller, W., An Introduction to Probability Theory and its Applications, Vols. 1, 2, John Wiley, New York, 1950. (Definitive work on probability—requires mature mathematical knowledge.)
- Call No. QA273 F37 1966
- Peter Olofsson, Probabilities The Little Numbers That Rule Our Lives, Wiley, 2006
- Stochastic processes / Sheldon M. Ross. Call No. QA274 R65 1996
- Stochastic processes / Emanuel Parzen. Call No. QA273 P278 1962
- MATLAB Primer, 8th edition T. A. Davis. CRC Press, 2010.
- Seventh Edition by T. A. Davis and K. Sigmon: Call No. QA297 D38 2005
- Third Edition by K. Sigmon (Free)
- Second Edition by K. Sigmon (Free)
Misc. Links
- Video: Probability 101
- More information about theMonty Hall Problem
- Video: The Monty Hall Problem
- Video: Monty Hall Problem: Numb3rs and 21
- Paper: Monty Hall, Monty Fall, Monty Crawl
- Articles: How Random is the iPod Shuffle? [HowStuffWorks]; Is iTunes’ Shuffle Mode Truly Random?[About.com]; iTunes: Just how random is random?[CNET.com.au, 2007]; My IPod for a Random Playlist [wired.com, 2005];
- Video: It *could* just be coincidence
- MV: Bill Nye the Science Guy - "50 Fifty"
- Video: Chevalier de Mere's Scandal of Arithmetic
- Free educational software: Orstat2000
- Originally developed to promote probability and operations research in the senior forms of Dutch high schools (and early college).
- Contain modules for coin-tossing, central limit theorem, etc.
- Probability review from MATH REVIEW for Practicing to Take the GRE General Test
- Video: Mlodinow’s talk @ Google
- Video: How many ways can you arrange a deck of cards? (There Are More Ways To Arrange a Deck of Cards Than Atoms on Earth)
- Provide nice animation explaining permutation and factorial.
- "Any time you pick up a well shuffled deck, you are almost certainly holding an arrangement of cards that has never before existed and might not exist again."
- Video: The Binomial Distribution / Binomial Probability Function
- Article: Greenlighting Movies: A High-Risk Game
- Video: The Poisson Distribution
- If you want to experience probability theory at a more advance level, one standard textbook that you can refer to is "Probability: Theory and Examples" by Prof. Durrett. Currently, the 4th edition of the textbook is available online.
- Video: Peter Donnelly shows how stats fool juries (same clip on youtube)
- Video: Lies, damned lies and statistics (about TEDTalks): Sebastian Wernicke on TED.com
- Article about clinical/medical decision making: Jill G. Klein, "5 pitfalls in diagnosis and prescribing: psychological biases that can lead to poor judgement," 2005.
- Related topics: Pitfall #1 (representatiove heuristic), Pitfall #2(availability heuristic), and Pitfall #5 (illusory correlation).
- Related topics: Pitfall #1 (representatiove heuristic), Pitfall #2(availability heuristic), and Pitfall #5 (illusory correlation).
- The Median Isn't the Message by Stephen Jay Gould
- Video: Daniel Kahneman: The riddle of experience vs. memory
- Articles on risk intelligence
- Dylan Evans, How to Beat the Odds at Judging Risk, The Wall Street Journal, May, 2012
- Alison George, What Gamblers and Weather Forecasters Can Teach Us About Risk: An interview with the creator of the "risk quotient" intelligence scale., pp 30-31, New Scientist, May 19, 2012
- Generation of random numbers
- Article: Park, S.K., and K.W. Miller. "Random Number Generators: Good Ones Are Hard to Find." Communications of the ACM, 31(10):1192–1201. 1998.
- Article: Tom McNichol, "Totally Random: How two math geeks with a lava lamp and a webcam are about to unleash chaos on the Internet"
- Article: C. Moler, Random thoughts, "10^435 years is a very long time", MATLAB News and Notes, Fall, 1995
- Article: Ziggurat algorithm generates normally distributed random numbersdescribing the ziggurat algorithm introduced in MATLAB version 5.
- Games of chance
- Poker
- Paper: Cheung, Y. L. "Why Poker is Played with Five Cards." Math. Gaz. 73, 313-315, 1989.
- Tim Farajian's Texas Hold'Em Poker Analyzer in MATLAB
- Allow a user to simulate different scenarios in a Texas Hold'Em game.
- Automatically simulate as many hands as you would like, and display winning probabilities or expected returns.
- Blackjack
- Cleve Moler's Blackjack in MATLAB + article
- Michael Iori's Blackjack in MATLAB
- Poker
- Quotations about Statistics
- Video: Statistics - Dream Job of the next decade
- Virtual Laboratories in Probability and Statistics
- Google Calculator (Cheat Sheet)
- Sometimes the easiest way to get information on a counting problem is to compute a few small values of a function, then look for a match at the sequence server; if you find a hit, you can sometimes get citations to the literature.
- Prapun's Notes on Probability Theory (Cornell Version)
- MATLAB
- MIT OpenCourseWare > Electrical Engineering and Computer Science > 6.094 Introduction to MATLAB (January (IAP) 2009)
- Fundamentals: Academic Tutorial
- Video: Controlling Random Number Generation
- Free online book: Cleve Moler, Experiments with MATLAB, 2008
- Free textbook: Cleve Moler,Numerical Computing with MATLAB, 2004
- Learn the Greek Alphabet in less than 10 minutes
- The Greek Alphabet Song