Illustration of Approximation Method of Sennott
MDP Model of Inventory System
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
A periodic inventory replenishment system is modeled.
At the end of each period of demand, the inventory is
tallied, and a decision is made either to do nothing or
to increase the inventory to a desired level.
The probability distribution is discrete and Poisson.
The state of the system is the inventory position: this
the stock-on-hand if nonnegative; if negative, it is the
absolute value of the number of unfilled backorders.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
MDP Model of Inventory Replenishment Problem
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Approx #1
Maximum inventory level = 6
Maximum # of backorders = 2
Expected demand = 3 (Poisson distribution assumed)
Holding cost h = 1
Ordering cost A = 10
Shortage penalty = 5 per unit short
Distinguished state (for excess probability) = 1
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Approx #1
¯¯¯¯¯¯¯¯¯
14 APR 2001 13:41:33
Optimal Policy found by Value Iteration
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
State
| Action |
V
¯¯¯¯¯
¯¯¯¯¯¯
¯¯¯
BO= two
| SOH= 6 | 72.3583
BO= one
| SOH= 6 | 57.3583
SOH= zero | SOH= 6 | 52.3583
SOH= one
| SOH= 6 | 53.3583
SOH= two
| SOH= 2 | 52.4908
SOH= three | SOH= 3 | 50.4510
SOH= four | SOH= 4 | 49.2100
SOH= five | SOH= 5 | 48.5763
SOH= six
| SOH= 6 | 48.3583
MDP Model of Inventory System
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
A periodic inventory replenishment system is modeled.
At the end of each period of demand, the inventory is
tallied, and a decision is made either to do nothing or
to increase the inventory to a desired level.
The probability distribution is discrete and Poisson.
The state of the system is the inventory position: this
the stock-on-hand if nonnegative; if negative, it is the
absolute value of the number of unfilled backorders.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
MDP Model of Inventory Replenishment Problem
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Approx #2
Maximum inventory level = 7
Maximum # of backorders = 3
Expected demand = 3 (Poisson distribution assumed)
Holding cost h = 1
Ordering cost A = 10
Shortage penalty = 5 per unit short
Distinguished state (for excess probability) = 1
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Approx #2
¯¯¯¯¯¯¯¯¯
14 APR 2001 13:42:54
Optimal Policy found by Value Iteration
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
State
| Action |
V
¯¯¯¯¯
¯¯¯¯¯¯
¯¯¯
BO= three | SOH= 7 | 98.2503
BO= two
| SOH= 7 | 73.2503
BO= one
| SOH= 7 | 58.2503
SOH= zero | SOH= 7 | 53.2503
SOH= one
| SOH= 7 | 54.2503
SOH= two
| SOH= 7 | 55.2503
SOH= three | SOH= 3 | 53.2667
SOH= four | SOH= 4 | 51.3011
SOH= five | SOH= 5 | 50.4785
SOH= six
| SOH= 6 | 50.2025
SOH= seven |
SOH= 7
| 50.2503
MDP Model of Inventory System
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
A periodic inventory replenishment system is modeled.
At the end of each period of demand, the inventory is
tallied, and a decision is made either to do nothing or
to increase the inventory to a desired level.
The probability distribution is discrete and Poisson.
The state of the system is the inventory position: this
the stock-on-hand if nonnegative; if negative, it is the
absolute value of the number of unfilled backorders.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
MDP Model of Inventory Replenishment Problem
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Approx #3
Maximum inventory level = 8
Maximum # of backorders = 4
Expected demand = 3 (Poisson distribution assumed)
Holding cost h = 1
Ordering cost A = 10
Shortage penalty = 5 per unit short
Distinguished state (for excess probability) = 1
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Approx #3
¯¯¯¯¯¯¯¯¯
14 APR 2001 13:46:20
Optimal Policy found by Value Iteration
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
State
| Action |
V
¯¯¯¯¯
¯¯¯¯¯¯
¯¯¯
BO= four
| SOH= 8 | 130.6728
BO= three | SOH= 8 | 95.6728
BO= two
| SOH= 8 | 70.6728
BO= one
| SOH= 8 | 55.6728
SOH= zero | SOH= 8 | 50.6728
SOH= one
| SOH= 8 | 51.6728
SOH= two
| SOH= 8 | 52.6728
SOH= three | SOH= 3 | 51.8500
SOH= four | SOH= 4 | 49.3778
SOH=
SOH=
SOH=
SOH=
five
six
seven
eight
|
|
|
|
SOH=
SOH=
SOH=
SOH=
5
6
7
8
|
|
|
|
48.4689
48.2269
48.3086
48.6728
------------------MDP Model of Inventory System
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
A periodic inventory replenishment system is modeled.
At the end of each period of demand, the inventory is
tallied, and a decision is made either to do nothing or
to increase the inventory to a desired level.
The probability distribution is discrete and Poisson.
The state of the system is the inventory position: this
the stock-on-hand if nonnegative; if negative, it is the
absolute value of the number of unfilled backorders.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
MDP Model of Inventory Replenishment Problem
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Approx #4
Maximum inventory level = 10
Maximum # of backorders = 5
Expected demand = 3 (Poisson distribution assumed)
Holding cost h = 1
Ordering cost A = 10
Shortage penalty = 5 per unit short
Distinguished state (for excess probability) = 1
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Approx #4
¯¯¯¯¯¯¯¯¯
14 APR 2001 13:48:09
Optimal Policy found by Value Iteration
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
State
| Action |
V
¯¯¯¯¯
¯¯¯¯¯¯
¯¯¯
BO= five
| SOH= 10 | 176.7718
BO= four
| SOH= 10 | 131.7718
BO= three
BO= two
BO= one
SOH= zero
SOH= one
SOH= two
SOH= three
SOH= four
SOH= five
SOH= six
SOH= seven
SOH= eight
SOH= nine
SOH= ten
|
|
|
|
|
|
|
|
|
|
|
|
|
|
SOH=
SOH=
SOH=
SOH=
SOH=
SOH=
SOH=
SOH=
SOH=
SOH=
SOH=
SOH=
SOH=
SOH=
10
10
10
10
10
10
3
4
5
6
7
8
9
10
|
|
|
|
|
|
|
|
|
|
|
|
|
|
96.7718
71.7718
56.7718
51.7718
52.7718
53.7718
53.5004
50.7828
49.8438
49.6259
49.7289
50.1051
50.7841
51.7718
-----------------------------------------MDP Model of Inventory System
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
A periodic inventory replenishment system is modeled.
At the end of each period of demand, the inventory is
tallied, and a decision is made either to do nothing or
to increase the inventory to a desired level.
The probability distribution is discrete and Poisson.
The state of the system is the inventory position: this
the stock-on-hand if nonnegative; if negative, it is the
absolute value of the number of unfilled backorders.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
MDP Model of Inventory Replenishment Problem
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Approx #5
Maximum inventory level = 12
Maximum # of backorders = 6
Expected demand = 3 (Poisson distribution assumed)
Holding cost h = 1
Ordering cost A = 10
Shortage penalty = 5 per unit short
Distinguished state (for excess probability) = 1
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Approx #5
¯¯¯¯¯¯¯¯¯
14 APR 2001 13:49:56
Optimal Policy found by Value Iteration
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
State
| Action |
V
¯¯¯¯¯
¯¯¯¯¯¯
¯¯¯
BO= six
| SOH= 10 | 231.8900
BO= five
| SOH= 10 | 176.8900
BO= four
| SOH= 10 | 131.8900
BO= three
| SOH= 10 | 96.8900
BO= two
| SOH= 10 | 71.8900
BO= one
| SOH= 10 | 56.8900
SOH= zero
| SOH= 10 | 51.8900
SOH= one
| SOH= 10 | 52.8900
SOH= two
| SOH= 10 | 53.8900
SOH= three | SOH= 3 | 53.7796
SOH= four
| SOH= 4 | 50.9538
SOH= five
| SOH= 5 | 49.9933
SOH= six
| SOH= 6 | 49.7723
SOH= seven | SOH= 7 | 49.8706
SOH= eight | SOH= 8 | 50.2390
SOH= nine
| SOH= 9 | 50.9098
SOH= ten
| SOH= 10 | 51.8900
SOH= eleven | SOH= 11 | 53.1630
SOH= twelve | SOH= 12 | 54.7082
-------------MDP Model of Inventory System
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
A periodic inventory replenishment system is modeled.
At the end of each period of demand, the inventory is
tallied, and a decision is made either to do nothing or
to increase the inventory to a desired level.
The probability distribution is discrete and Poisson.
The state of the system is the inventory position: this
the stock-on-hand if nonnegative; if negative, it is the
absolute value of the number of unfilled backorders.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
MDP Model of Inventory Replenishment Problem
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Approx #6
Maximum inventory level = 15
Maximum # of backorders = 7
Expected demand = 3 (Poisson distribution assumed)
Holding cost h = 1
Ordering cost A = 10
Shortage penalty = 5 per unit short
Distinguished state (for excess probability) = 1
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Approx #6
¯¯¯¯¯¯¯¯¯
14 APR 2001 13:53:35
Optimal Policy found by Value Iteration
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
State
| Action |
V
¯¯¯¯¯
¯¯¯¯¯¯
¯¯¯
BO= seven
| SOH= 10 | 296.9292
BO= six
| SOH= 10 | 231.9292
BO= five
| SOH= 10 | 176.9292
BO= four
| SOH= 10 | 131.9292
BO= three
| SOH= 10 | 96.9292
BO= two
| SOH= 10 | 71.9292
BO= one
| SOH= 10 | 56.9292
SOH= zero
| SOH= 10 | 51.9292
SOH= one
| SOH= 10 | 52.9292
SOH= two
| SOH= 10 | 53.9292
SOH= three
| SOH= 3 | 53.8742
SOH= four
| SOH= 4 | 51.0097
SOH= five
| SOH= 5 | 50.0426
SOH= six
| SOH= 6 | 49.8208
SOH= seven
| SOH= 7 | 49.9175
SOH= eight
| SOH= 8 | 50.2834
SOH= nine
| SOH= 9 | 50.9515
SOH= ten
| SOH= 10 | 51.9292
SOH= eleven
| SOH= 11 | 53.1999
SOH= twelve
| SOH= 12 | 54.7429
SOH= thirteen | SOH= 13 | 56.5408
SOH= fourteen | SOH= 14 | 58.5790
SOH= fifteen | SOH= 15 | 60.8442