8/28/2012
1
Lecture 01: Introduction
Instructor:
Dr. Gleb V. Tcheslavski
Contact: gleb@lamar.edu
Office Hours:
Room 2030
Class web site:
http://www.ee.lamar.edu/
gleb/stat/Index.htm
ELEN 5301 Stochastic Signals and Systems
Fall 2008
2
Syllabus overview
• Prerequisites: Signals and Systems, algebra, calculus
• Meetings: MW 1:25-2:40; Room 2631C
• Required book: A. Papoulis and S. Unnikrishna Pillai.
Probability, random variables, and stochastic processes.
McGraw-Hill, New York, 2002, 4th Edition; ISBN-13: 9780071199810.
• Tests: The midterm and the final exams will be closed
book/notes; frequent pop-up quizzes.
• Homework/Projects: up to ten homework may be assigned.
Solutions may be posted one week after the assignment.
• Attendance policy: Class attendance is mandatory with exceptions for medical
and family emergencies.
• Honor System: Discussions on lecture subject material, to clarify your
understanding, are highly encouraged. However, it is your personal/own
understanding only that should be reflected in all work that you turn in. You may thus
claim credit only for your own work. All graded work is covered by the Academic
Honor Code; violations will be prosecuted.
ELEN 5301 Stochastic Signals and Systems
Fall 2008
1
8/28/2012
3
Styles, notations, legends…
1. Colors:
Normal text and formulas
Something more important (imho)
Important formulas and results
Very Important Formulas
Miscellaneous
Miscellaneous
2. Equations notations: (2.17.3)
Lecture # Slide # Formula #
Slides: all class material (handouts, assignments, possibly solutions)
will be available, in form of pdf slides, through the course web site:
http://www.ee.lamar.edu/gleb/stat/Index.htm
ELEN 5301 Stochastic Signals and Systems
Fall 2008
4
Introduction
Processes (events)
1. Deterministic:
One possible value for any
time instance. Therefore, we
can predict the exact value of
the signal for a desired time
instance.
Barely exist in real world…
ELEN 5301 Stochastic Signals and Systems
2. Random (stochastic):
Many (infinitely) possible values
for any time instance. Therefore,
we can predict only the expected
value of the signal for a desired
time instance.
Stock market, weather, gambling
outcomes, most of real life
signals such as speech, medical
data, digital images,
communication signals,…
Fall 2008
2
8/28/2012
5
Definitions and preliminary
considerations
The probability theory calls every action or occurrence an event (me standing in
front of you, raining outside, student “John Smyth” getting an “F” in this class).
Every event either may or may not occur.
The purpose of the theory is to predict events in terms of their probability. The
probability of an event A is a number P{A} assigned to this event. We can interpret
this number as follows:
If the experiment is performed n times and the event A occurs nA times, then, with
a high degree of certainty, the relative frequency nA/n of the occurrence of A is
close to P{A}:
(1.5.1)
provided that n is sufficiently large.
Different events have different probabilities: the probability of rain in June in
Beaumont is much higher that the probability of hail, which, in turn, is much higher
than the probability of snow. Probability indicates how frequently the event may
occur.
ELEN 5301 Stochastic Signals and Systems
Fall 2008
6
Introduction
This cone
shows the
most likely
(at the time
of its
generation)
path of the
tropical
storm
Isaac…
From
http://www.nhc
.noaa.gov/
Assessed on
8/27/2012 at
noon
ELEN 5301 Stochastic Signals and Systems
Fall 2008
3
8/28/2012
7
Introduction
In the applications of probability to real problems, three steps must be
distinguished:
1. Observation (physical): we assign the probabilities P{Ai} of specific events Ai
by an exact process (for instance, observation) according to (1.5.1). If a fair die
is rolled 600 times and “5” is observed 100 times, then we assign P{“5”} =
100/600 = 1/6.
2. Deduction (conceptual): we assume that probability satisfies certain axioms,
and by deductive reasoning we determine from the probabilities P{Ai} of specific
events Ai the probabilities P{Bi} of other events Bi. In the game with a fair die,
we deduce that the probability of “even” is 3/6 with the following reasoning: If
P{“2”} = P{“4”} = P{“6”} = 1/6, then P{“even”} = 1/6 + 1/6 + 1/6 = 3/6.
3. Prediction (physical): we make a physical prediction based on the numbers
P{Bi} we obtained according to (1.5.1). If we roll a fair die 500 times, our
prediction is that “even” will show about 250 times.
We must make a clear distinction between the data we have determined
empirically and the results we have deduced logically.
ELEN 5301 Stochastic Signals and Systems
Fall 2008
8
Introduction
Steps 1 and 3 are based on inductive reasoning. Say, for instance, we wish to
determine the probability of “heads” of a given coin. Should we toss it 100 times or
1000 times? If we tossed it 1000 times and the number of “heads” is 493, what can
we predict based on this observation? Can we deduce that at next 1000 tosses the
number of “heads” will be 493? We can only answer such questions inductively.
We will consider mainly step 2: from certain probabilities we derive deductively
other probabilities. Moreover, we will interpret the probability as a characteristic of
an event (similarly to the mass or resistance) and will not be concerned about the
“physical meaning” of the probability in the theory development.
For example, a resistor is commonly viewed as a 2-terminal device satisfying:
(1.7.1)
However, this is only a convenient abstraction: a real resistor is a complex device
with distributed inductance and capacitance… Therefore, (1.7.1) can be claimed
only with certain errors, in certain frequency ranges and with other assumptions.
Nevertheless, these uncertainties are usually ignored and we are not concerned
with the true meaning of R.
ELEN 5301 Stochastic Signals and Systems
Fall 2008
4