Uploaded by beyenebekele2

Chapter 3 Operation Management

advertisement
Uncertainty of
the Future
CHAPTER THREE
FORECASTING
What is Forecasting?
FORECAST:


A statement about the future value of a variable of interest
such as demand, price, production amount, interest rate,
profits, changes in productivity, etc.
Forecasts affect decisions and activities throughout an
organization
 Accounting and finance
 Human resources
 Marketing
 MIS
 Operations
 Product /service design
Uses of Forecasts
Accounting
Cost/profit estimates, cash management
Finance
Cash flow, funding, capital structure
Human Resources
Hiring/recruiting/training/layoff planning
Marketing
Pricing,
promotion,
competition strategies
MIS
IT/IS systems, services
Operations
Schedules, MRP, workloads, inventory
planning, make or buy decisions, outsourcing
Product/service design New products and services
strategy,
global
Forecasting within OM: How it all fits
together
Forecasts impact not only other business functions
but all other operations decisions.
Operations managers make many forecasts, such as
the expected demand for a company’s products.
These forecasts are then used to determine:
a. product designs that are expected to sell
b. the quantity of product to be produced
c. the amount of needed supplies and materials
d. determine future space requirements capacity and
location needs , and
e. the amount of labor needed
Forecasting within OM…….Con’td
Forecasts drive strategic operations decisions, such
as:
a. Choice of competitive priorities
b. Changes in processes, and
c. Large technology purchases .
Forecast decisions serve as the basis for tactical
planning; developing worker schedules.
Virtually all operations management decisions are
based on a forecast of the future.
Features Common in All Forecasts
1.Assumes causal system
Past ==> Future
2. Forecasts are rarely perfect because of randomness
3. Forecasts for groups of items tend to more
accurate than forecast for individuals
items because forecasting errors
items in a group usually have a
cancelling effect
4. Forecast accuracy decreases
as time horizon increases
I see that you will
get an A this semester.
Elements of a Good Forecast
Timely
Reliable
Accurate
Written
Elements of a Good Forecast…Cont’d
The forecast should be timely. Usually, a certain amount
of time is needed to respond to the information contained
in a forecast.
2. The forecast should be accurate and the degree of
accuracy should be stated.
3. The forecast should be reliable; it should work
consistently.
4. The forecast should expressed in meaningful units. For
example financial planner need to know how many
dollars will be needed, scheduler need to know what
machines and skills will be required.
The choice of units depends on user needs.
1.
Elements of a Good Forecast...Cont’d
5. The forecast should be in writing.
6. The forecasting technique should be simple to
understand and use.
Users often lack confidence in forecasts based on
sophisticated techniques; they do not understand
either the circumstances in which the techniques are
appropriate or the limitations of the techniques.
Steps in the Forecasting Process
“The forecast”
Step 6: Monitor the forecast
Step 5: Prepare the forecast
Step 4: Gather and analyze data
Step 3: Select a forecasting technique
Step 2: Establish a time horizon
Step 1: Determine purpose of forecast
Steps in the Forecasting Process. …Cont’d
Determine the purpose of the forecast. What is its purpose and
when will it be needed?
This will provide an indication of the level of detail required in the
forecast, the amount of resources (personnel, computer, time,
dollars) that can be justified, the level of accuracy necessary.
2. Establish a time horizon. The forecast must be indicate a time
limit, keeping in mind that accuracy decreases as the time
horizon increases.
3. Select the forecast technique.
4. Gather and analyze relevant data. Before a forecast can be
prepared, data must be gathered and analyzed. Identify any
assumptions that are made in conjunction with preparing and
using the forecast
1.
Steps in the Forecasting Process. …Cont’d
5. Prepare the forecast. Use an appropriate
technique of forecasting.
6. Monitor the forecast. A forecast has to be
mentioned to determine whether it is performed in a
satisfactory manner. If it is not, re examine the
method, assumptions, validity of data so on; modify
as needed; and prepare a revised forecast.
Types of Forecasting
There are two types of forecasting:
1. Qualitative methods – judgmental methods
Forecasts
generated subjectively by the forecaster
Educated guesses
 It includes: Executive opinion, sales force opinions,
consumer survey, outside opinion
2. Quantitative methods – based on mathematical modeling:
Forecasts generated through mathematical modeling
It include:
1) Time series and
2) Associative model
Types of Forecasting....Con'td
1. Qualitative or Judgmental Forecasts
i) Executive opinions: A small group of upper level
managers (e.g. marketing, operations, and finance) may
and collectively develop a forecast.
ii) Sales force opinions: the sales staff of the customer
service staff is often a good source of information because
of their direct contact with consumers.
iii) Consumer surveys: Because it is the consumers who
ultimately determine demand, it seems natural to solicit
input from them. However, there are usually to many
customers or there is no way to identify all potential
customer.
Qualitative forecasting....Cont’d
iv) Outside opinion/Delphi method
It involves circulating a series of questionnaires among
individuals who posses the knowledge and ability to
contribute meaningfully. Responses are kept anonymous,
which tend to encourage honest responses and reduce the
risk that one person’s opinion will prevail.
For example: When digital camera might be purchased by
50% of Ethiopian population?
When a medicine for HIVs might be developed and ready
for mass distribution?
2. Quantitative Forecasts
i) TIME SERIES FORECASTS
a) Trend - long-term movement in data either upward or
downward. It is a gradual long-term directional movement
in the data (growth or decline).
b) Seasonality - short-term fairly regular variations in data.
Related to factors such as calendar and time of day. For
example, restaurants, supermarkets, and theaters
experience weekly and even daily seasonal variations.
c) Cycle - wavelike variations of more than one year’s
duration. Related to variety of economic, political, and
even agricultural products
Time Series Forecasts…cont’d
d) Irregular variations - caused by unusual circumstances
such as severe weather condition, strike, major changes in
product/service
e) Random variations - are sporadic (unpredictable) effects
due to chance and unusual.
Are residual variations that remain after all other behaviours
have been accounted for.
Forecast Variations
Irregular
variatio
n
Trend
Cycles
90
89
88
Seasonal variations
i. Naive Forecasts
Uh, give me a minute....
We sold 250 wheels last
week.... Now, next week
we should sell....
The forecast for any period equals the
previous period’s actual value.
A forecasting technique which
assumes that demand in the
next period is equal to demand
in the most recent period,
Uses for Naive Forecasts

Stable time series data: last data point becomes the
forecast for the next period.


Seasonal variations: the forecast for this “season” is
equal to the value of the series “last Season”.


F(t) = A(t-1)
F(t) = A(t-n)
Data with trends: the forecast is equal to the last
value of the series plus or minus the difference
between the last two values of the series.

F(t) = A(t-1) + (A(t-1) – A(t-2))
Naive Forecasts

Simple to use
 Virtually no cost

Quick and easy to prepare

Easily understandable

Can be a standard for comparison

Cannot provide high accuracy
Example
If demand last week was 50 units, the naive forecast
for the upcoming week is 50 units, similarly, if
demand in the upcoming week turns out to be 54
units, the forecast for the following week would be
54 units.
ii. Techniques for Averaging
Averaging techniques smooth fluctuations in a time series
because the individual highs and lows in the data offset
each other when they are combined in to an average.
A forecast based on average thus tends to exhibit less
variability than the original data
1.
Moving Average
2.
Weighted moving average
3.
Exponential smoothing
1. Moving Averages

Moving average – A technique that averages a number of
recent actual values, updated as new values become
available.
n
Ft=MAn=
A

i
i=1
n
Where:
i= an index that corresponds to periods
n= number of periods (data points) in moving average
Ai=Actual value in period i
MA= Moving average
Ft= Forecast for time period t
Example
1.The demand for tires in a tire store in the past 5
weeks were as follows. Compute a three-period
moving average forecast for demand in week 6.
83 80 85 90 94
MA3= 94+90+85 =89.67
3
2. Compute a two period moving average forecast
given demand for shopping carts for the last five
periods.
42
40 43 40 41
Moving average & Actual demand
2. Weighted moving average

Weighted moving average – More recent values in a series
are given more weight in computing the forecast.
Ft 1 

w
t
At
All weights must add to 100% or 1.00
e.g. wt .5, wt-1 .3, wt-2 .2 (weights add to 1.0)
Example:


For the previous demand data, compute a weighted average
forecast using a weight of .40 for the most recent period, .30
for the next most recent, .20 for the next and .10 for the next.
If the actual demand for week 6 is 91, forecast demand for
week 7 using the same weights.
Solution to example
F6= (94x0.4)+ (90x0.3)+(85x0.2)+(80x0.1)
=89.60
F7=(91x0.4)+(94x0.30)+(90x0.20)+(85x0.10)
=91.10
The Advantage of a weighted average over simple
moving average is that the weighted average is more
reflective of the most recent occurrences.
However the choice of weights is some what arbitrary
and generally involves the use of trial and error to
find a suitable weighting scheme.
3. Exponential Smoothing
Ft = Ft-1 + (At-1 - Ft-1)
Where:
Ft= Forecast for period t
Ft-1= Forecast for the previous period
 Smoothing constant
At-1= Actual demand for the previous period
• The most recent observations might have the
highest predictive value.

Therefore, we should give more weight to the more
recent time periods when forecasting.
Exponential Smoothing

Weighted averaging method based on previous
forecast plus a percentage of the forecast error
 A-F is the error term,  is the % feedback
Example: Suppose the previous forecast was 42
units, actual demand was 40 units, and  is 0.10.
Then new forecast would be computed as follows:
Ft= 42+ 0.10(40-42)
=42+ -0.20
= 41.80
Example - Exponential Smoothing
Period Actual
1
83
2
80
3
85
4
89
5
92
6
95
7
91
8
90
9
88
10
93
11
92
12
0.1
83
82.70
82.93
83.54
84.38
85.44
86.00
86.40
86.56
87.20
87.68
Error
-3.00
2.30
6.07
8.46
10.62
5.56
4.00
1.60
6.44
4.80
0.4
83
81.80
83.08
85.45
88.07
90.84
90.90
90.54
89.53
90.92
91.35
Error
-3
3.20
5.92
6.55
6.93
0.16
-0.90
-2.54
3.47
1.08
Picking a Smoothing Constant
Exponential Smoothing
Actual
Alpha=0.10
Alpha=0.40
100
Demand
95
90
85
80
75
70
2
3
4
5
6
7
Period
8
9
10
11
Problem 1

National Mixer Inc. sells can openers.
Monthly sales for a seven-month period
were as follows:
 Forecast September sales volume using
each of the following:




A five-month moving average
Exponential smoothing with a smoothing
constant equal to .20, assuming a March
forecast of 19.
The naive approach
A weighted average using .60 for August,
.30 for July, and .10 for June.
Month
Sales
(1000)
Feb
19
Mar
18
Apr
15
May
20
Jun
18
Jul
22
Aug
20
Problem 2

A dry cleaner uses exponential smoothing to
forecast equipment usage at its main plant. August
usage was forecast to be 88% of capacity. Actual
usage was 89.6%. A smoothing constant of 0.1 is
used.

Prepare a forecast for September
 Assuming actual September usage of 92%, prepare
a forecast of October usage
Problem 3
An electrical contractor’s records during the last five
weeks indicate the number of job requests:
Week:
1
2
3
4
5
Requests: 20 22 18
21 22

Predict the number of requests for week 6 using each of
these methods:



Naïve
A four-period moving average
Exponential smoothing with a smoothing constant of .30.
Use 20 for week 2 forecast.
iii. Techniques for Trend
• Develop an equation that will suitably describe
trend, when trend is present.
• The trend component may be linear or nonlinear
• We focus on linear trends because these are
fairly common.
Common Nonlinear Trends
Parabolic
Exponential
Growth
Linear Trend Equation
Ft
Ft = a + bt


Ft = Forecast for period t
0 1 2
t = Specified number of time periods
a = Value of Ft at t = 0
b = Slope of the line

Example: Ft =10+2t. Interpret 10 and 2. Plot F


3 4 5
t
Example

Sales for over the last 5 weeks are shown below:
Week:
Sales:

1
2
150 157
3
162
4
166
5
177
Plot the data and visually check to see if a linear
trend line is appropriate.
 Determine the equation of the trend line
 Predict sales for weeks 6 and 7.
Line chart
Sales
180
175
170
Sales
165
160
Sales
155
150
145
140
135
1
2
3
Week
4
5
Calculating a and b
n  (ty) -  t  y
b =
2
2
n t - (  t)
 y - b t
a =
n
Linear Trend Equation Example
t
Week
1
2
3
4
5
2
t
1
4
9
16
25
 t = 15
t = 55
2
(t) = 225
2
y
Sales
150
157
162
166
177
ty
150
314
486
664
885
 y = 812  ty = 2499
Linear Trend Calculation
b =
5 (2499) - 15(812)
5(55) - 225
=
12495-12180
275 -225
= 6.3
812 - 6.3(15)
a =
= 143.5
5
Ft(y) = 143.5 + 6.3t
Linear Trend plot
Actual data
Linear equation
180
175
170
165
160
155
150
145
140
135
1
2
3
4
5
Recall: Problem 1

National Mixer Inc. sells can openers.
Monthly sales for a seven-month period
were as follows:




Plot the monthly data
Forecast September sales volume using
a line trend equation
Which method of forecast seems least
appropriate?
What does use of the term sales rather
than demand presume?
Month
Sales
(1000)
Feb
19
Mar
18
Apr
15
May
20
Jun
18
Jul
22
Aug
20
Line chart
Sales
20
0
F
M
J
A
Month A
M
S
J
Problem 4

A cosmetics manufacturer’s marketing department has
developed a linear trend equation that can be used to predict
annual sales of its popular Hand & Foot Cream:
Ft  80  15t
where
Ft  Annual sales (1000 bottles)
t  0 corresponds to 1990


Are annual sales increasing or decreasing? By how much?
Predict annual sales for the year 2006 using the equation.
iv. Techniques for Seasonality



Seasonal variations in time series data are regularly
repeating upward or downward movements in series values
that can be tied to recurring events.
Seasonality may refer to regular annual variation. There are
two models
Seasonality in a time series is expressed in terms of the
amount that actual values deviate from the average value of
a series.


Additive: expressed as a quantity (e.g., 20 units), which is
added or subtracted from the series average
Multiplicative: a percentage of the average or seasonal
relative (e.g., 1.10), which is used to multiply the value of a
series to incorporate seasonality.
Additive vs. multiplicative
Techniques for Seasonality…cont’d

Knowledge of seasonal variations is an important factor in
retailing planning and scheduling.
 Moreover, it is important in capacity planning for systems
that must be designed to handle peak loads (e.g. Public
transportation, electric power plants, highways, and bridges)
Incorporating seasonality in forecast useful when demand
has both trend (average) and seasonal components. It is
accomplished in the following ways:
1. Obtained trend estimates for the desired periods using a
trend equation
2. Add seasonality to the trend estimates by multiplying
(assuming multiplicative model is appropriate)
Example

A furniture manufacturer wants to predict quarterly demand for a
certain loveseat for periods 15 and 16, which happen to be the
second and third quarters of a particular year. The series consists of
both trend and seasonality. The trend portion of demand is
projected using the equation

Quarter relatives are
Ft  124  7.5t
Q1  1.20, Q2  1.10, Q3  0.75, Q4  0.95

Use this information to predict demand for periods 15 and 16.
Solution
The trend values at t=15 and t=16 are:
F15= 124+7.5 (15)= 236.50
F16= 124+7.5 (16) = 244.00
Multiplying the trend value by the appropriate quarter
relative yields a forecast that includes both trend and
seasonality. Given that t= 15 is a second quarter and t= 16 is
a third quarter, the forecast are:
F15= 236.50(1.10)= 260.15
F16+ 244.00(0.75)= 183.00
Problem

A manager is using the equation below to forecast quarterly
demand for a product:
Y(t) = 6,000 + 80t
where t = 0 at Q2 of last year

Quarter relatives are Q1 = .6, Q2 = .9, Q3 = 1.3, and Q4 = 1.2.

What forecasts are appropriate for the last quarter of this year and the
first quarter of next year?
Problem

A manager of store that sells and installs hot tubs
wants to prepare a forecast for January, February
and March of 2007. Her forecasts are a
combination of trend and seasonality. She uses the
following equation to estimate the trend component
of monthly demand:
Ft  70  5t
Where t=0 is June of 2005. Seasonal relatives are
1.10 for Jan, 1.02 for Feb, and .95 for March. What
demands should she predict?
Computing seasonal relatives
120
100
80
60
40
20
0
1
2 3
4
5 6
7
8 9 10 11 12 13 14 15 16 17 18 19 20 21
If your data appears to have seasonality, how do you compute the
seasonal relatives?
Computing seasonal relatives

Calculate centered moving average for each
period.
 Obtain the ratio of the actual value of the period
over the centered moving average.
 Number of periods needed in a centered moving
average = Number of seasons involved:

Monthly data: a 12-period moving average
 Quarterly data: a 4-period moving average
Example

The manager of a parking lot has computed the
number of cars per day in the lot for three weeks.
Using a seven-period centered moving average,
calculate the seasonal relatives.

Note that a seven period centered moving average
is used because there are seven days (seasons) per
week.
Problem 5

Obtain estimates of quarter relatives for these data:
Year:
1
2
Quarter: 1 2 3 4 1 2 3 4
Demand: 14 18 35 46 28 36 60 71
3
1 2 3 4
45 54 84 88
4
1
58
Problem

The manager of a restaurant believes that her
restaurant does about 10% of its business on Sunday
through Wednesday, 15% on Thursday night, 25%
on Friday night, and 20% on Saturday night.

What seasonal relatives would describe this
situation?
Quantitative Forecast
ii. Associative forecast/Causal forecasts

Associative techniques rely on identification of related
variables that can be used to predict values of the variable
of interest.
For example: Real estate prices are usually related to
property location, size of the house, design of the house,
etc

The essence of associative techniques is the development
of an equation that summarizes the effects of predictor
variables.

The primary method of analysis is known as regression.
Associative/causal forecast...Cont’d
Predictor variables - used to predict values of
variable interest (predicted variables).
Regression - technique for fitting a line to a set of
points
Least squares line - minimizes sum of squared
deviations around the line
Simple Linear Regression
Simple linear regression analysis
analyzes the linear relationship that exists
between two variables.
y  a  bx
where:
y = Value of the dependent variable
x = Value of the independent variable
a = Population’s y-intercept
b = Slope of the population regression line
Simple Linear Regression
The coefficients of the line are
b
or
n xy   x y
n x  ( x )
2
2
y  b x

a
n
a  y  bx
Example
The general manager of a building materials
production plant feels the demand for plaster board
shipment may be rotated to the number of
construction permit issued in the region during the
previous quarter. The manager has collected the data
shown in the companied table below:
Required
Determine a point estimate for plasterboard
shipment when the number of construction is 30
Example....cont’d
Construction permit
Plaster Board
15
6
9
4
40
16
20
6
25
13
25
9
15
10
35
16
solution
X
15
Y
6
XY
90
X²
225
Y²
36
9
40
4
16
36
640
81
1600
16
256
20
25
6
13
120
325
400
625
36
169
25
15
35
9
10
16
225
150
560
625
225
1225
81
100
256
∑X= 184
∑Y= 80
∑XY= 2146 ∑X²=5006
∑Y²=950
Solution....cont’d
b= n(∑XY)-(∑X) (∑Y) y=a+bx
n(∑x²)-(∑X) ²
y=0.915+0.395x
b= 8(2146)-(184) (80)
y= 0.915+0.395(30)
8(5006) –(184)²
y=12.76 =13 shipments
b=0.395
a= (∑Y)-b(∑X)
n
a= 80-0.395(184) =0.915
8
Correlation
The correlation coefficient is a quantitative measure
of the strength of the linear relationship between two
variables. The correlation ranges from + 1.0 to - 1.0.
A correlation of  1.0 indicates a perfect linear
relationship, whereas a correlation of 0 indicates no
linear relationship.
Note that: A correlation of +1 indicates that change in
one variable are always matched by changes in the
other; a correlation of -1 indicates that increases in
one variable are matched by decreases in the other.
An algebraic formula for correlation
coefficient
r
n xy   x y
[n( x )  ( x) ][ n( y )  ( y) ]
2
2
2
2
Problem 7

The manager of a seafood restaurant was
asked to establish a pricing policy on
lobster dinners. Experimenting with
prices produced the following data:



Create the scatter plot and determine if
a linear relationship is appropriate.
Determine the correlation coefficient
and interpret it
Obtain the regression line and interpret
its coefficients.
Sold (y)
Price (x)
200
6.00
190
6.50
188
6.75
180
7.00
170
7.25
162
7.50
160
8.00
155
8.25
156
8.50
148
8.75
140
9.00
133
9.25
Forecast Accuracy

Source of forecast errors:

Model may be inadequate
 Irregular variations
 Incorrect use of forecasting technique
 Random variation

Key to validity is randomness

Accurate models: random errors
 Invalid models: nonrandom errors

Key question: How to determine if forecasting
errors are random?
Error measures

Error is difference between actual value and predicted
value

Error= Actual value-Forecasted value

Mean Absolute Deviation (MAD)


Mean Squared Error (MSE)


Average absolute error
Average of squared error
Mean Absolute Percent Error (MAPE)

Average absolute percent error
Tracking signal
MAD, MSE, and MAPE
MAD
=
 Actual
 forecast
n
MSE
=
 ( Actual
 forecast)
2
n -1
MAPE 

Actual  Forecast
 100
Actual
n
Tracking Signal
A
tracking signal is a measurement of how well a forecast
is predicting actual values.
As forecasts are updated every week, month, or quarter, the
newly available demand data are compared to the forecast
values.
The tracking signal is computed as the cumulative error
divided by the mean absolute deviation (MAD):
Tracking =
Signal
 ( Actual
 forecast)
MAD
Example
Period
1
2
3
4
5
6
7
8
MAD=
MSE=
MAPE=
Actual
217
213
216
210
213
219
216
212
2.75
10.86
1.28
Forecast
215
216
215
214
211
214
217
216
(A-F)
2
-3
1
-4
2
5
-1
-4
-2
|A-F|
2
3
1
4
2
5
1
4
22
(A-F)^2
4
9
1
16
4
25
1
16
76
(|A-F|/Actual)*100
0.92
1.41
0.46
1.90
0.94
2.28
0.46
1.89
10.26
Controlling the Forecast

Control chart
 A visual tool for monitoring forecast errors
 Used to detect non-randomness in errors

Forecasting errors are in control if
 All errors are within the control limits
 No patterns, such as trends or cycles, are present
Controlling the forecast
Control charts

Control charts are based on the following
assumptions:
 when errors are random, they are Normally
distributed around a mean of zero.
 Standard deviation of error is MSE
 95.5% of data in a normal distribution is within 2
standard deviation of the mean
 99.7% of data in a normal distribution is within 3
standard deviation of the mean
 Upper and lower control limits are often determine
via
0  2 MSE or 0  3 MSE
Example

Compute 2s control limits for
forecast errors of previous
example and determine if the
forecast is accurate.
5.41
s  MSE  3.295
2s  6.59
3.41
1.41
-0.59 0



Errors are all between -6.59
and +6.59
No pattern is observed
Therefore, according to control
chart criterion, forecast is
reliable
-2.59
-4.59
-6.59
10
Problem 8

The manager of a travel agency has
been using a seasonally adjusted
forecast to predict demand for
packaged tours. The actual and
predicted values are

Compute MAD, MSE, and MAPE.

Determine if the forecast is working
using a control chart with 2s limits. Use
data from the first 8 periods to develop
the control chart, then evaluate the
remaining data with the control chart.
Compute tracking signal and interpret it

Period
1
2
3
4
5
6
7
8
9
10
11
12
13
14
Demand Predicted
1 29
1 94
1 56
91
85
1 32
1 26
1 26
95
1 49
98
85
1 37
1 34
1 24
200
1 50
94
80
1 40
1 28
1 24
1 00
1 50
94
80
1 40
1 28
Problem

Given the following demand data, prepare a naïve
forecast for periods 2 through 10. Then determine each
forecast error, and use those values to obtain 2s control
limits. If demand in the next two periods turns out to be
125 and 130, can you conclude that the forecasts are in
control?
Period
1
2
3
4
5
6
7
8
9
10
Demand 118 117 120 119 126 122 117 123 121 124
Choosing a Forecasting Technique

No single technique works in every situation
 Two most important factors

Cost
 Accuracy

Other factors include the availability of:

Historical data
 Computers
 Time needed to gather and analyze the data
 Forecast horizon
Using Forecasting Information
A manager can take a reactive or a proactive
approach to a forecast.

A reactive approach views forecasts as probable
descriptions of the future demand, and manager reacts
to meet that demand (e.g. Adjusts production rates,
inventories, the work force).

Proactive approach seeks to actively influence
demand (e.g. By means of advertising, pricing, or
product/service changes).

The End of Chapter Three !
Good Luck!
Read Further William J. Stevenson
Chapter 10 (page 480)
Download
Random flashcards
Arab people

15 Cards

Ethnology

14 Cards

Create flashcards