Everything we do, everything that happens around us, obeys the laws of probability. We can no more escape them than we can escape gravity... "Probability," a philosopher (Bishop Butler) once said, "is the very guide of life." We are all gamblers who go through life making countless bets on the outcome of countless actions.

Every field of science is concerned with estimating probability. A physicist calculates the probable path of a particle. A geneticist calculates the chances that a couple will have blue-eyed children. Insurance companies, businessmen, stockbrokers, sociologists, politicians, military experts - all have to be skilled in calculating the probability of the events with which they are concerned.

[Gardner, 1986]

#### Synopsis

Probability theory is the branch of mathematics that tells us how to estimate degrees of probability. If an event is certain to happen, it is given a probability of 1. If it is certain not to happen, it has a probability of 0.

This course introduces the principles of probability and random processes to undergraduate students in electronics and communication. The topics to be covered include random experiments, events, probability, discrete and continuous random variables, probability density function, cumulative distribution function, functions of random variables, expectations, law of large numbers, central limit theorem, introduction to random processes, Gaussian random process, autocorrelation and power spectral density.

#### Announcements

- Information regarding the
**final exam**- 10 Oct 2013
- 9:00 - 12:00
- Building: IT & MT; Room: BKD 2601, 2605
- 11 Pages: Cover page + 9 pages + 1 blank page
- 9 Questions: 8 + 1 extra credit.
- Closed book. Closed notes.
- Basic calculators, e.g. FX-991MS, are permitted.
- One
**A4 sheet**allowed.- It should contain Table 3.1 and Table 3.2. (The sheet was distributed in class, but you can print your own sheet using the provided pdf file.)
- Except the tables above, the rest of the content must be hand-written.
- No small pieces of paper notes glued/attached on top of it.
- Indicate your name and id on the upper right corner of the sheet.
- Submit your formula sheet with your final exam. (You can get it back next semester.)

- Cover all the materials that we discussed in class and practice in the HWs.
- Strong focus on the materials that haven’t been on the midterm.
- I could ask something that I have never defined in class but, in such a case, I will give you the exact definition on the exam itself.
- Approximate Material Distribution (score-wise):
- Sections 7-9: 25%
- Section 10: 25%
- Section 11.1-11.5: 30%
- Section 11.6-11.7: 10%
- Section 12-14: 10%

- These notes are provided for your studying pleasure....
- All post-midterm annotated notes combined in one pdf file
- Another version which also includes the annotated pre-midterm notes
- Complete notes without the annotation.
- With workable links and references.

- Solutions for HW 7-13
- All post-midterm quiz solutions
- All post-midterm slides
- 2011 Final Exam
- Some problems are crossed out because they are based on materials that we do not discuss this year.

- 2010 Final Exam
- Again, some problems are not covered in this semester. Mostly, these are the one the are related to conditional expectation and variance.

- All post-midterm annotated notes combined in one pdf file
- Tips:
- Check your answers. To do this, think about alternative ways to find answers. For example, the expected value of Y = g(X) can be found directly from the pdf of X via the LOTUS formula. However, we can also find it from the pdf of Y as well.
- Usually, there are many parts in a problem; some of these parts may also have subparts. If some quantity is defined in a part, then that quantity is used throughout the corresponding subparts. For example, if a random variable X is defined in part (c) of problem 3. Then, this definition of X is valid throughout parts c.i, c.ii, c.iii, ... Take alook at the first problem of the 2011 final exam, note how the definition of the joint pdf is used in part (a) to (l).
- The answer(s) from earlier part(s) of a question may be useful for subsequent part(s) or question(s). You may refer to your own answer(s) from earlier part(s).

- More information may be posted here.

- Midterm scores are now available. Visit Dr.Prapun's office to take a look at your graded exam.
- Average = 71.5
- Plot of all the scores.

- Information regarding the
**midterm exam**- 1 Aug 2013 TIME 9:00-12:00
- ROOM BKD 2601 & 2602
- 11 Pages + 1 Cover Page.
- Read the information on the cover page now so that you don't have to waste time reading it again in the exam room.

- 7 Questions + 1 Extra Credits.
- Closed book. Closed notes. No cheat/study sheet.
- Basic calculators, e.g. FX-991MS, are permitted
- Cover all the materials that we discussed in class and practice in the HWs.
- Approximate Material Distribution (score-wise):
- Sections 1&2: 5%
- Sections 3&4: 20%
- Section 5: 5%
- Section 6: 45%
- Sections 7&8: 25%

- I could ask something that I have never defined in class but, in such a case, I will give you the exact definition on the exam itself.
- These notes are provided for your studying pleasure....
- All annotated notes combined in one pdf file
- Solutions for HW 1-6
- All slides
- Summary
- Solutions for Quiz 1-2
- 2010 Midterm Exam [Posted @2PM on July 5; Updated @ 3PM on July 26]

- Approximate Material Distribution (score-wise):

- A basic RSS feed is created to track and inform updates.
- This site can be accessed via ecs315.prapun.com.
- Welcome to ECS315! Feel free to look around this site.

#### General Information

**Instructor**: Asst. Prof. Dr.Prapun Suksompong (prapun@siit.tu.ac.th)- Office: BKD3601-7
- Office Hour:
- Rangsit Library: Tuesday 16:20-17:20
- BKD3601-7: Thursday 14:40-16:00

**Course Syllabus**[Posted @ 10 AM on June 3]- Textbook: [Y&G] R. D. Yates and D. J. Goodman, Probability and Stochastic Processes: A Friendly Introduction for Electrical and Computer Engineers, 2nd ed., Wiley, 2004.
- Call No. QA273 Y384 2005. ISBN: 978-0-471-27214-4
- Student Companion Site

- References
- Older version of the textbook: Probability and stochastic processes : a friendly introduction for electrical and computer engineers / Roy D. Yates, David J. Goodman. Call No. QA273 Y384 1999
- Probability and probabilistic reasoning for electrical engineering / Terrence L. Fine. Call No. QA273 F477 2006
- Probability and random processes for electrical engineering / Alberto Leon-Garcia. Call No. TK153 L425 1994
**Free textbook:***Introduction to Probability*by Charles M. Grinstead and J. Laurie Snell- Henk Tijms. Understanding Probability: Chance Rules in Everyday Life. Cambridge University Press, 3rd edition, 2012. Call No. QA273 T48 2012
- [Z&T] Rodger E. Ziemer and William H. Tranter, Principles of Communications, 6th International student edition, John Wiley & Sons Ltd, 2010.
- Probability, random variables, and stochastic processes / Athanasios Papoulis, S. Unnikrishna Pillai. Call No. QA273 P2 2002
- Probability, random variables, and stochastic processes / Athanasios Papoulis. Call No. QA273 P2 1991

- A first course in probability / Sheldon Ross. Call No. QA273 R83 2002
- A first course in probability / Sheldon Ross. Call No. QA273 R83 1976

- Probability models, introduction to / Sheldon M. Ross. Call No. QA273 R84 1997
- Random signals for engineers using MATLAB and Mathcad / Richard C. Jaffe. Call No. TK5102.9 J34 2000
- Davenport, W.B., Probability and Random Processes, McGraw-Hill, New York, 1970. (Excellent introductory text.)
- Feller, W., An Introduction to Probability Theory and its Applications, Vols. 1, 2, John Wiley, New York, 1950. (Definitive work on probability—requires mature mathematical knowledge.)
- Call No. QA273 F37 1966

**Free book:**Peter Olofsson, Probabilities The Little Numbers That Rule Our Lives, Wiley, 2006

- MATLAB Primer, 8th edition T. A. Davis. CRC Press, 2010.
- Seventh Edition by T. A. Davis and K. Sigmon: Call No. QA297 D38 2005
- Third Edition by K. Sigmon (Free)
- Second Edition by K. Sigmon (Free)

#### Handouts and Course Material

- Slides: Introduction to ECS315[Updated at 3PM on June 14]
- Part I: Introduction, Set Theory, Classical Probability theory, and Combinatorics
- Probability and You [Posted @ 11AM on June 12]
- Annotated version [Posted @3PM on June 14; Updated @6PM on June 20]
- Slides [Posted @9AM on June 20]

- Review of Set Theory [Posted @ 11AM on June 12]
- Annotated version [Posted @6PM on June 18]
- Slides[Posted @6PM on June 18]

- Classical Probability [Posted @ 11AM on June 12]
- Annotated version[Posted @6PM on June 18; Updated @2PM on June 21]
- Slides[Posted @5PM on June 27]

- Enumeration / Combinatorics / Counting [Posted @ 11AM on June 12]
- Annotated version [Posted @3PM on June 14; Updated @2PM on June 21 and @10PM on June 28]
- Slides[Posted @5PM on June 27; Updated @2PM on July 27]
- More explanation for 4.28 and 4.30 [Posted @10PM on June 28]

- Probability and You [Posted @ 11AM on June 12]
- Part II: Kolmogorov's Formal Probability Theory and Event-Based Probability Theory
- Probability Foundations [Posted @8PM on June 19]
- Annotated version [Posted @2PM on June 21; Updated @5PM on June 27 and @10PM on June 28]
- Slides[Posted @5PM on June 27]
- Quiz 1 Solution

- Event-based Independence and Conditional Probability [Posted @ 8PM on June 19; Updated @ 5PM on July 3]
- Annotated version [Posted @5PM on July 4; Updated @2PM on July 5, @5PM on July 11, @2PM on July 12, @5PM on July 18, and @1PM on July 19]
- Slides for Section 6.1 [Posted @5PM on July 4; Updated @2PM on July 5 and @2PM on July 27]
- Slides for Section 6.2 [Posted @2PM on July 12]

- Probability Foundations [Posted @8PM on June 19]
- Part III: Discrete Random Variables
- Random Variables [Posted @ 9AM on July 11]
- Annotated version [Posted @5PM on July 18; Updated @1PM on July 19]

- Discrete Random Variables [Posted @ 9AM on July 11]
- Annotated version [Posted @2:30PM on July 19; Updated @ 4PM on July 25, @ 3PM on July 26, and @ 5:30PM on August 15]
- Quiz 2 Solution
- Slides [Updated @ 2:30PM on Oct 4]

- Questions for the Tutorial on August 16. [Posted @ 1:30PM on Aug 16]
- Expectation and Variance [Posted @ 5:30PM on August 15]
- Annotated version [Posted @ 2PM on August 16; Updated @ 5PM on Aug 22 and @10PM on Aug 23]
- Quiz 3 Solution[Posted @ 9PM on Sep 15]
- Slides

- Part IV: Continuous Random Variables
- Continuous Random Variables (Part A) [Posted @ 2PM on August 21]
- Annotated version [Posted @ 10PM on Aug 23; Updated @ 5PM on Aug 29, @ 1PM on Aug 30, and @ 2PM on Sep 6]

- Continuous Random Variables (Part B) [Posted @ 4PM on August 28]
- Annotated version [Posted @ 1PM on Aug 30; Updated @ 2PM on Sep 6 and @5PM on Sep 12]
- Table 3.1 and Table 3.2 from [Y&G] [Posted @ 4PM on August 28]

- Slides
- References
- From Discrete to Continuous Random Variables: [Y&G] Sections 3.0 to 3.1
- PDF and CDF: [Y&G] Sections 3.1 to 3.2
- Expectation and Variance: [Y&G] Section 3.3
- Families of Continuous Random Variables: [Y&G] Sections 3.4 to 3.5
- Table 3.1 and Table 3.2 from [Y&G]
- SISO: [Y&G] Section 3.7; [Z&T] Section 5.2.5

- Continuous Random Variables (Part A) [Posted @ 2PM on August 21]
- Part V: Multiple Random Variables
- Multiple Disctere Random Variables [Posted @ 5:30PM on Sep 4]
- Annotated version [Posted @ 10PM on Sep 13; Updated @ 5PM on Sep 19 and @ 8:30PM on Sep 20]
- Some notes from the tutorial on Sep 20.

- Function of Discrete Random Variables [Posted @ 9:30PM on Sep 11]
- Annotated version [Posted @ 8:30PM on Sep 20; Updated @ 4:30PM on Sep 26]
- Quiz 4 Solution

- Multiple Continuous Random Variables [Posted @ 10AM on Sep 18]
- Annotated version [Posted @ 4:30PM on Sep 26; Updated @ 4:30PM on Oct 3]
- Some notes from the tutorial on Sep 27

- Slides
- References:
- A Pair of Random Variables: [Y&G] Sections 4.1 to 4.3 and Section 4.10
- Extending the Definitions to Multiple RVs: [Y&G] Sections 5.1 to 5.4
- Function of Discrete Random Variables: [Y&G] Section 4.6 (Theorem 4.9)
- Expectation of function of discrete random variables: [Y&G] Sections 4.7 and 6.1
- Linear Dependence: [Y&G] Section 4.7
- Pairs of Continuous Random Variables: [Y&G] Sections 4.1, 4.5, and 4.11
- MISO: [Y&G] Section 6.2

- Multiple Disctere Random Variables [Posted @ 5:30PM on Sep 4]
- Part VI
- Three Types of Random Variables [Posted @ 1PM on Sep 25]
- Annotated version [Posted @ 4:30PM on Oct 3]

- Characteristic Functions [Posted @ 1PM on Sep 25]
- Annotated version [Posted @ 4:30PM on Oct 3; Updated @ 1:30PM on Oct 4]

- Three Types of Random Variables [Posted @ 1PM on Sep 25]
- Part VII
- Limit Theorems [Posted @ 5PM on Oct 3]
- Annotated version [Posted @ 1:30PM on Oct 4]

- Limit Theorems [Posted @ 5PM on Oct 3]
- Part VIII
- Random Vectors and Random Processes [Posted @ 5PM on Oct 3]

- Appendix
- Calculus [Posted @ 10AM on Sep 18; Distributed in class on Sep 13]

#### Problem Set

- HW 1 (Due: June 28)
- Solution [Posted @ 4PM on June 30]

- HW2 (Due: July 5)
- Solution [Posted @ 9AM on July 11]

- HW3 (Due: July 12)
- Solution [Posted @1PM on July 19]

- HW4 (Due: July 19)
- Solution [Posted @ 4PM on July 25]

- HW5 (Due: July 26)
- HW6 (Not Due)
- Seff-Evaluation (Due: Aug 23)
- HW7 (Due: Aug 30)
- HW8 (Due: Sep 6)
- HW9 (Due: Sep 13; Updated @8:30 PM on Sep 11)
- With some remarks from the tutorial sessions on Sep 6 and Sep 13.
- Solution [Posted @ 9AM on Sep 25]

- HW10 (Due: Sep 20)
- HW11 (Due: Sep 27)
- Some remarks from the tutorial session on Sep 27.
- Solution
- MATLAB codes

- HW12 (Due: Oct 4)
- HW13 (Not Due)
- Self-Evaluation (Due: Oct 10)

#### Calendar

#### Reading Assignment

#### Misc. Links

- Video: Probability 101
- More information about theMonty Hall Problem
- Video: The Monty Hall Problem
- Video: Monty Hall Problem: Numb3rs and 21
- Paper: Monty Hall, Monty Fall, Monty Crawl
- Articles: How Random is the iPod Shuffle? [HowStuffWorks]; Is iTunes’ Shuffle Mode Truly Random?[About.com]; iTunes: Just how random is random?[CNET.com.au, 2007]; My IPod for a Random Playlist [wired.com, 2005];
- Video: It *could* just be coincidence
- MV: Bill Nye the Science Guy - "50 Fifty"
- Video: Chevalier de Mere's Scandal of Arithmetic
- Free educational software: Orstat2000
- Originally developed to promote probability and operations research in the senior forms of Dutch high schools (and early college).
- Contain modules for coin-tossing, central limit theorem, etc.

- Probability review from MATH REVIEW for Practicing to Take the GRE General Test
- Video: Mlodinow’s talk @ Google
- Video: How many ways can you arrange a deck of cards? (There Are More Ways To Arrange a Deck of Cards Than Atoms on Earth)
- Provide nice animation explaining permutation and factorial.
*"Any time you pick up a well shuffled deck, you are almost certainly holding an arrangement of cards that has never before existed and might not exist again."*

- Video: The Binomial Distribution / Binomial Probability Function
- Video: The Poisson Distribution
- If you want to experience probability theory at a more advance level, one standard textbook that you can refer to is "Probability: Theory and Examples" by Prof. Durrett. Currently, the 4th edition of the textbook is available online.
- Video: Peter Donnelly shows how stats fool juries (same clip on youtube)
- Video: Lies, damned lies and statistics (about TEDTalks): Sebastian Wernicke on TED.com
- Article about clinical/medical decision making: Jill G. Klein, "5 pitfalls in diagnosis and prescribing: psychological biases that can lead to poor judgement," 2005.
- Related topics: Pitfall #1 (representatiove heuristic), Pitfall #2(availability heuristic), and Pitfall #5 (illusory correlation).

- Related topics: Pitfall #1 (representatiove heuristic), Pitfall #2(availability heuristic), and Pitfall #5 (illusory correlation).
- The Median Isn't the Message by Stephen Jay Gould
- Video: Daniel Kahneman: The riddle of experience vs. memory
- Articles on risk intelligence
- Dylan Evans, How to Beat the Odds at Judging Risk, The Wall Street Journal, May, 2012
- Alison George, What Gamblers and Weather Forecasters Can Teach Us About Risk: An interview with the creator of the "risk quotient" intelligence scale., pp 30-31, New Scientist, May 19, 2012

- Generation of random numbers
- Article: Park, S.K., and K.W. Miller. "Random Number Generators: Good Ones Are Hard to Find." Communications of the ACM, 31(10):1192–1201. 1998.
- Article: Tom McNichol, "Totally Random: How two math geeks with a lava lamp and a webcam are about to unleash chaos on the Internet"
- Article: C. Moler, Random thoughts, "10^435 years is a very long time", MATLAB News and Notes, Fall, 1995
- Article: Ziggurat algorithm generates normally distributed random numbersdescribing the ziggurat algorithm introduced in MATLAB version 5.

- Games of chance
- Poker
- Paper: Cheung, Y. L. "Why Poker is Played with Five Cards."
*Math. Gaz.*73, 313-315, 1989. - Tim Farajian's Texas Hold'Em Poker Analyzer in MATLAB
- Allow a user to simulate different scenarios in a Texas Hold'Em game.
- Automatically simulate as many hands as you would like, and display winning probabilities or expected returns.

- Paper: Cheung, Y. L. "Why Poker is Played with Five Cards."
- Blackjack
- Cleve Moler's Blackjack in MATLAB + article
- Michael Iori's Blackjack in MATLAB

- Poker
- Stochastic processes / Sheldon M. Ross. Call No. QA274 R65 1996
- Stochastic processes / Emanuel Parzen. Call No. QA273 P278 1962
- Quotations about Statistics
- Video: Statistics - Dream Job of the next decade
- Virtual Laboratories in Probability and Statistics
- Google Calculator (Cheat Sheet)
- Sometimes the easiest way to get information on a counting problem is to compute a few small values of a function, then look for a match at the sequence server; if you find a hit, you can sometimes get citations to the literature.
- Prapun's Notes on Probability Theory (Cornell Version)
- MATLAB
- MIT OpenCourseWare > Electrical Engineering and Computer Science > 6.094 Introduction to MATLAB (January (IAP) 2009)

- Learn the Greek Alphabet in less than 10 minutes
- The Greek Alphabet Song