Everything we do, everything that happens around us, obeys the laws of probability. We can no more escape them than we can escape gravity... "Probability," a philosopher (Bishop Butler) once said, "is the very guide of life." We are all gamblers who go through life making countless bets on the outcome of countless actions.
Every field of science is concerned with estimating probability. A physicist calculates the probable path of a particle. A geneticist calculates the chances that a couple will have blue-eyed children. Insurance companies, businessmen, stockbrokers, sociologists, politicians, military experts - all have to be skilled in calculating the probability of the events with which they are concerned.
[Gardner, 1986]
Synopsis
Probability theory is the branch of mathematics that tells us how to estimate degrees of probability. If an event is certain to happen, it is given a probability of 1. If it is certain not to happen, it has a probability of 0.
This course introduces the principles of probability and random processes to undergraduate students in electronics and communication. The topics to be covered include random experiments, events, probability, discrete and continuous random variables, probability density function, cumulative distribution function, functions of random variables, expectations, law of large numbers, central limit theorem, introduction to random processes, Gaussian random process, autocorrelation and power spectral density.
Announcements
- There are two HW8 submitted with no name: stu1 and stu2
- Please claim your HWs before our final exam
- Information regarding the final exam
[Updated @ 7PM on Dec 15]
- 16 Dec 2014
- TIME 13:30 - 16:30
- ROOM BKD 2501-2, 2506
- Closed book. Closed notes.
- Basic calculators, e.g. FX-991MS, are permitted.
- One A4 sheet allowed.
- It should contain Table 3.1 and Table 3.2.
- Except the tables above, the rest of the content must be hand-written.
- No small pieces of paper notes glued/attached on top of it.
- Indicate your name and id on the upper-right corner of the sheet (in portrait orientation).
- Submit your formula sheet with your final exam. (You can get it back from me next semester.)
- 7 pages + 1 cover page
- 9 questions (24+21+6+4+16+22+5+1+1)
- Cover all the materials that we discussed in class and practice in the HWs.
- Material Distribution (score-wise): 6% (pre-midterm material) + 19% (Sec 9) + 48% (Sec 10) + 23% (Sec 11) + 4% (Sec 12-13)
- For your studying pleasure....
- All post-midterm annotated notes combined in one pdf file
- A typo is corrected on page 130. [Jirath] (For x > b, the cdf of the uniform RV should be 1.)
- All pre-midterm HWs and their solutions
- All post-midterm quiz solutions
- All post-midterm slides
- Sample Exams:
- 2013 Final Exam
- Annotated version [Posted @ 5PM on Oct 16; Updated @ 9PM on Oct 30 and @ 11PM on Dec 5]
- Some problems are crossed out because they are based on materials that we do not discuss this year.
- Annotated version [Posted @ 5PM on Oct 16; Updated @ 9PM on Oct 30 and @ 11PM on Dec 5]
- 2011 Final Exam
- 2010 Final Exam
- 2013 Final Exam
- All post-midterm annotated notes combined in one pdf file
- Extra office hours
- Dec 11: 9-10AM
- Dec 12: 9-11AM
- The announcement slides [Posted @ 5PM on Sep 25]
- My office hours on Wednesday is extended to 5PM.
- Information regarding the midterm exam
- 7 Oct 2014 TIME 13:30 - 16:30
- Closed book. Closed notes. No cheat/study sheet.
- Basic calculators, e.g. FX-991MS, are permitted
- 10 pages + 1 cover page
- 11 questions (16+15+3+8+6+8+10+19+12+2+1) + 1 extra credit question
- Cover all the materials that we discussed in class and practice in the HWs.
- Material Distribution (score-wise): 5.5% (Sec 1-3) + 8.5% (Sec 4) + 9.5% (Sec 5) + 17% (Sec 6.1) + 15.5% (Sec 6.2) + 7% (Sec 6.3) + 7% (Sec 7) + 30% (Sec 8)
- These notes are provided for your studying pleasure....
- All pre-midterm annotated notes combined in one pdf file
- A typo is corrected on page 38. [Phattaraphol]
- Unannotated notes with clickable links and references
- All pre-midterm HWs and their solutions
- A typo is corrected on page 4-2. [Pusit]
- All pre-midterm slides
- All pre-midterm quiz solutions
- All pre-midterm annotated notes combined in one pdf file
- I could ask something that I have never defined in class but, in such a case, I will give you the exact definition on the exam itself.
- Sample Exams:
- 2013 Midterm Exam
- Annotated version [Posted @ 5PM on Sep 25]
- 2010 Midterm Exam
- Annotated version [Posted @ 8PM on Aug 28; Updated @ 5PM on Sep 4, @9PM on Sep 11, and @ 5PM on Sep 18]
- Solution [Posted @ 5PM on Sep 25]
- 2013 Midterm Exam
- A basic RSS feed is created to track and inform updates.
- This site can be accessed via ecs315.prapun.com.
- Welcome to ECS315! Feel free to look around this site.
General Information
- Instructor: Asst. Prof. Dr.Prapun Suksompong (prapun@siit.tu.ac.th)
- Office: BKD3601-7
- Office Hour:
- Monday 14:00 - 16:00
- Wednesday 14:40 - 16:00 (TRIDI LAB) 16:00-17:00
- Course Syllabus [Posted @ 11AM on Aug 5]
- Textbook: [Y&G] R. D. Yates and D. J. Goodman, Probability and Stochastic Processes: A Friendly Introduction for Electrical and Computer Engineers, 2nd ed., Wiley, 2004.
- Call No. QA273 Y384 2005. ISBN: 978-0-471-27214-4
- Student Companion Site
- References
- Probability and probabilistic reasoning for electrical engineering / Terrence L. Fine. Call No. QA273 F477 2006
- Probability and random processes for electrical engineering / Alberto Leon-Garcia. Call No. TK153 L425 1994
- Free textbook: Introduction to Probability by Charles M. Grinstead and J. Laurie Snell
- Henk Tijms. Understanding Probability: Chance Rules in Everyday Life. Cambridge University Press, 3rd edition, 2012. Call No. QA273 T48 2012
- Probability, random variables, and stochastic processes / Athanasios Papoulis, S. Unnikrishna Pillai. Call No. QA273 P2 2002
- Probability, random variables, and stochastic processes / Athanasios Papoulis. Call No. QA273 P2 1991
- A first course in probability / Sheldon Ross. Call No. QA273 R83 2002
- A first course in probability / Sheldon Ross. Call No. QA273 R83 1976
- Probability models, introduction to / Sheldon M. Ross. Call No. QA273 R84 1997
Handouts and Course Material
- Slides: Course Introduction [Posted @ 5PM on Aug 14][Updated @ 4PM on Aug 19]
- Part I: Introduction, Set Theory, Classical Probability theory, and Combinatorics
- Section 1: Probability and You [Posted @ 9:30AM on Aug 13]
- Annotated version [Posted @ 4PM on Aug 19; Updated @ 5PM on Aug 21]
- Slides [Posted @ 4PM on Aug 19]
- Section 2: Review of Set Theory [Posted @ 9:30AM on Aug 13]
- Annotated version [Posted @ 5PM on Aug 21]
- Slides [Posted @ 5PM on Aug 21]
- Section 3: Classical Probability [Posted @ 9:30AM on Aug 13]
- Annotated version [Posted @ 5PM on Aug 21; Updated @ 3PM on Aug 26]
- Slides [Posted @ 5PM on Aug 21]
- Section 4: Enumeration / Combinatorics / Counting [Posted @ 9:30AM on Aug 13]
- Annotated version [Posted @ 5PM on Aug 14; Updated @ 5PM on Aug 21]
- Slides [Posted @ 11AM on Aug 15; Updated @ 5PM on Sep 4]
- Extra Material [Posted @ 9:30AM on Aug 13]
- Section 1: Probability and You [Posted @ 9:30AM on Aug 13]
- Part II: Kolmogorov's Formal Probability Theory and Event-Based Probability Theory
- Section 5: Probability Foundations [Posted @ 9PM on Aug 24]
- Annotated version [Posted @ 3PM on Aug 26; Updated @ 8PM on Aug 28]
- Slides [Posted @ 9:30PM on Aug 28; Updated @ 8PM on Aug 28]
- Section 6: Event-based Independence and Conditional Probability[Posted @ 9PM on Aug 24]
- Annotated version [Posted @ 8PM on Aug 28; Updated @ 8PM on Sep 2, @ 5PM on Sep 4, @ 3PM on Sep 9, and @ 4PM on Sep 16]
- Slides for Section 6.1 [Posted @ 4:30PM on Sep 1; Updated @ 5PM on Sep 4 and @ 3PM on Sep 9]
- Solution for Quiz 1 [Posted @ 8PM on Sep 2; Updated @ 5PM on Sep 4]
- Slides for Sections 6.2 and 6.3[Posted @ 9PM on Sep 11]
- Section 5: Probability Foundations [Posted @ 9PM on Aug 24]
- Part III: Discrete Random Variables
- Section 7: Random Variables [Posted @ 10 PM on Sep 14]
- Annotated version [Posted @ 4PM on Sep 16; Updated @ 5PM on Sep 18]
- Solution for Quiz 2 [Posted @ 4PM on Sep 16]
- Section 8: Discrete Random Variables [Posted @ 10 PM on Sep 14]
- Annotated version [Posted @ 5PM on Sep 18; Updated @ 5PM on Sep 23 and @ 5PM on Sep 25]
- Solution for Quiz 3 [Posted @ 1:30PM on Sep 19]
- Slides [Posted @ 5PM on Sep 23; Updated @ 5PM on Sep 25]
- Section 9: Expectation and Variance [Posted @ 10 PM on Sep 14]
- Annotated version [Posted @ 3PM on Oct 14; Updated @ 4:30 PM on Oct 16 and @ 5:30PM on Oct 21]
- Solution for Quiz 4 [Posted @ 12PM on Nov 17]
- Slides [Posted @ 3PM on Oct 14]
- Part IV: Continuous Random Variables
- Sections 10.1-10.3 [Posted @ 10AM on Oct 14]
- Annotated version [Posted @ 5:30PM on Oct 21; Updated @ 4PM on Oct 28, @ 9PM on Oct 30, and @ 5:30PM on Nov 6, and @ 4PM on Nov 18]
- Section 10.4 [Posted @ 4PM on Oct 27]
- Annotated version [Posted @ 4:30PM on Nov 4; Updated @ 5:30PM on Nov 6]
- Solution for Quiz 5 [Posted @ 5:30PM on Nov 6; Updated @ 11AM on Nov 11]
- Section 10.5 [Posted @ 4PM on Oct 27]
- Annotated version [Posted @ 4PM on Nov 11]
- Solution for Quiz 6 [Posted @ 4:30PM on Nov 18]
- Slides [Posted @ 4:30PM on Nov 4]
- References
- From Discrete to Continuous Random Variables: [Y&G] Sections 3.0 to 3.1
- PDF and CDF: [Y&G] Sections 3.1 to 3.2
- Expectation and Variance: [Y&G] Section 3.3
- Families of Continuous Random Variables: [Y&G] Sections 3.4 to 3.5
- Table 3.1 and Table 3.2 from [Y&G]
- SISO: [Y&G] Section 3.7; [Z&T] Section 5.2.5
- Sections 10.1-10.3 [Posted @ 10AM on Oct 14]
- Part V: Multiple Random Variables
- Sections 11.1-11.2 [Posted @ 4PM on Nov 18]
- Annotated version [Posted @ 5PM on Nov 20; Updated @ 3:30PM on Nov 25 and @ 10PM on Dec 1]
- Section 11.3-11.5 [Posted @ 10PM on Nov 24]
- Annotated version [Posted @ 10PM on Dec 1]
- Section 11.6
- Annotated version [Posted @ 10PM on Dec 2]
- Slides [Posted @ 4PM on Nov 25; Updated @ 10PM on Dec 1]
- References:
- A Pair of Random Variables: [Y&G] Sections 4.1 to 4.3 and Section 4.10
- Extending the Definitions to Multiple RVs: [Y&G] Sections 5.1 to 5.4
- Function of Discrete Random Variables: [Y&G] Section 4.6 (Theorem 4.9)
- Expectation of function of discrete random variables: [Y&G] Sections 4.7 and 6.1
- Linear Dependence: [Y&G] Section 4.7
- Sections 11.1-11.2 [Posted @ 4PM on Nov 18]
- Part VI: Limiting Theorems [Posted @ 10PM on Nov 24]
- Sections 12.1-12.2 (LLN and CLT)
- Annotated version [Posted @ 10PM on Dec 2]
- Sections 12.1-12.2 (LLN and CLT)
- Part VII: More Advanced Topics
- Section 13 [Posted @ 10PM on Nov 24]
- Annotated version [Posted @ 10PM on Dec 2; Updated @ 11PM on Dec 5]
- Section 13 [Posted @ 10PM on Nov 24]
Problem Set
- HW 1 (Due: Aug 28)
- HW 2 (Due: Sep 4)
- HW3 (Due: Sep 11) [Updated @ 9:30 AM on Sep 11 with more extra questions]
- HW4 (Due: Sep 18)
- HW5 (Due: Sep 25)
- HW6 (Not Due)
- Self-Evaluation (Due: Oct 24)
- HW7 (Due: Oct 24)
- HW8 (Due: Nov 6)
- Solution [Link's fixed at 11PM on Nov 6, 2015. Credit: Nonpawit]
- HW9 (Due: Nov 14 (Friday))
- HW10 (Due: Nov 20)
- HW11 (Due: Nov 27)
- HW12 (Due: Dec 4)
- HW13 (Not Due)
- Self-Evaluation (Due: Dec 16)
Calendar
Reading Assignment
More References
- Older version of the textbook: Probability and stochastic processes : a friendly introduction for electrical and computer engineers / Roy D. Yates, David J. Goodman. Call No. QA273 Y384 1999
- Random signals for engineers using MATLAB and Mathcad / Richard C. Jaffe. Call No. TK5102.9 J34 2000
- Davenport, W.B., Probability and Random Processes, McGraw-Hill, New York, 1970. (Excellent introductory text.)
- Feller, W., An Introduction to Probability Theory and its Applications, Vols. 1, 2, John Wiley, New York, 1950. (Definitive work on probability—requires mature mathematical knowledge.)
- Call No. QA273 F37 1966
- Peter Olofsson, Probabilities The Little Numbers That Rule Our Lives, Wiley, 2006
- Stochastic processes / Sheldon M. Ross. Call No. QA274 R65 1996
- Stochastic processes / Emanuel Parzen. Call No. QA273 P278 1962
- MATLAB Primer, 8th edition T. A. Davis. CRC Press, 2010.
- Seventh Edition by T. A. Davis and K. Sigmon: Call No. QA297 D38 2005
- Third Edition by K. Sigmon (Free)
- Second Edition by K. Sigmon (Free)
Misc. Links
- Video: Probability 101
- More information about theMonty Hall Problem
- Video: The Monty Hall Problem
- Video: Monty Hall Problem: Numb3rs and 21
- Paper: Monty Hall, Monty Fall, Monty Crawl
- Articles: How Random is the iPod Shuffle? [HowStuffWorks]; Is iTunes’ Shuffle Mode Truly Random?[About.com]; iTunes: Just how random is random?[CNET.com.au, 2007]; My IPod for a Random Playlist [wired.com, 2005];
- Video: It *could* just be coincidence
- MV: Bill Nye the Science Guy - "50 Fifty"
- Video: Chevalier de Mere's Scandal of Arithmetic
- Free educational software: Orstat2000
- Originally developed to promote probability and operations research in the senior forms of Dutch high schools (and early college).
- Contain modules for coin-tossing, central limit theorem, etc.
- Probability review from MATH REVIEW for Practicing to Take the GRE General Test
- Video: Mlodinow’s talk @ Google
- Video: How many ways can you arrange a deck of cards? (There Are More Ways To Arrange a Deck of Cards Than Atoms on Earth)
- Provide nice animation explaining permutation and factorial.
- "Any time you pick up a well shuffled deck, you are almost certainly holding an arrangement of cards that has never before existed and might not exist again."
- Video: The Binomial Distribution / Binomial Probability Function
- Article: Greenlighting Movies: A High-Risk Game
- Video: The Poisson Distribution
- If you want to experience probability theory at a more advance level, one standard textbook that you can refer to is "Probability: Theory and Examples" by Prof. Durrett. Currently, the 4th edition of the textbook is available online.
- Video: Peter Donnelly shows how stats fool juries (same clip on youtube)
- Video: Lies, damned lies and statistics (about TEDTalks): Sebastian Wernicke on TED.com
- Article about clinical/medical decision making: Jill G. Klein, "5 pitfalls in diagnosis and prescribing: psychological biases that can lead to poor judgement," 2005.
- Related topics: Pitfall #1 (representatiove heuristic), Pitfall #2(availability heuristic), and Pitfall #5 (illusory correlation).
- Related topics: Pitfall #1 (representatiove heuristic), Pitfall #2(availability heuristic), and Pitfall #5 (illusory correlation).
- The Median Isn't the Message by Stephen Jay Gould
- Video: Daniel Kahneman: The riddle of experience vs. memory
- Articles on risk intelligence
- Dylan Evans, How to Beat the Odds at Judging Risk, The Wall Street Journal, May, 2012
- Alison George, What Gamblers and Weather Forecasters Can Teach Us About Risk: An interview with the creator of the "risk quotient" intelligence scale., pp 30-31, New Scientist, May 19, 2012
- Generation of random numbers
- Article: Park, S.K., and K.W. Miller. "Random Number Generators: Good Ones Are Hard to Find." Communications of the ACM, 31(10):1192–1201. 1998.
- Article: Tom McNichol, "Totally Random: How two math geeks with a lava lamp and a webcam are about to unleash chaos on the Internet"
- Article: C. Moler, Random thoughts, "10^435 years is a very long time", MATLAB News and Notes, Fall, 1995
- Article: Ziggurat algorithm generates normally distributed random numbersdescribing the ziggurat algorithm introduced in MATLAB version 5.
- Games of chance
- Poker
- Paper: Cheung, Y. L. "Why Poker is Played with Five Cards." Math. Gaz. 73, 313-315, 1989.
- Tim Farajian's Texas Hold'Em Poker Analyzer in MATLAB
- Allow a user to simulate different scenarios in a Texas Hold'Em game.
- Automatically simulate as many hands as you would like, and display winning probabilities or expected returns.
- Blackjack
- Cleve Moler's Blackjack in MATLAB + article
- Michael Iori's Blackjack in MATLAB
- Poker
- Quotations about Statistics
- Video: Statistics - Dream Job of the next decade
- Virtual Laboratories in Probability and Statistics
- Google Calculator (Cheat Sheet)
- Sometimes the easiest way to get information on a counting problem is to compute a few small values of a function, then look for a match at the sequence server; if you find a hit, you can sometimes get citations to the literature.
- Prapun's Notes on Probability Theory (Cornell Version)
- MATLAB
- MIT OpenCourseWare > Electrical Engineering and Computer Science > 6.094 Introduction to MATLAB (January (IAP) 2009)
- Fundamentals: Academic Tutorial
- Video: Controlling Random Number Generation
- Free online book: Cleve Moler, Experiments with MATLAB, 2008
- Free textbook: Cleve Moler,Numerical Computing with MATLAB, 2004
- Learn the Greek Alphabet in less than 10 minutes
- The Greek Alphabet Song