Using NeuralTools
to generate
a pricing model for wool
Kimbal Curtis and John Stanton
Australian Wool Industry
z
z
70% of world trade in apparel wool is
Australian wool
Unlike other commodities
• Each farm lot is fully measured
• Each farm lot has an individual price
z
z
About 450,000 farm lots sold each year
in Australia
Raw wool value of AUD3 billion annually
Wool prices & market reporting
z
Estimates of auction price on individual
lots needed by sellers (farmers)
z
Forecast auction price on individual lots
required by buyers for contracts
z
Market reporting of price paid for
different wool types
Neural nets & wool prices
z
Neural nets attractive because
• Number of records is large
• Prices are dynamic
• Price/attribute relationships are non-linear and
•
•
interactive
Price/attribute relationships are dynamic over
time
The data set is incomplete and imprecise
All Merino fleece lots
(Fremantle Jan-Mar 2006)
Each grey dot represents a parcel of wool
sold at auction i.e. a ‘case’
Long & short fleece lots
(Fremantle Jan-Mar 2006)
Long and short wool
differentiated on price
Merino pieces lots
(Fremantle Jan-Mar 2006)
Pieces wool
(a subset of the wool clip)
The Challenge !
Market Indicators
(Fremantle Jan-Mar 2006)
Market indicators, like a stock
market index, used to price wool
Model development (1)
z
Assemble 6 month data set
• Independent category and numeric variables
• Dependent numeric variable (price)
• Training, testing and prediction data Excel Demo
z
z
z
Use Best Net Search
Evaluate predictive capability
Refine model
Model development (2)
z
z
Assemble a 6 month data set
Use Best Net Search
• GRN – proved best in most cases
•
z
z
(generalised regression neural net)
MLFN – also tried with up to 5 nodes
(multi layer feed-forward neural net)
Evaluate predictive capability
Refine model
Configuration summary
Net Information
Name
Configurations Included in Search
Best Configuration
Location
Independent Category Variables
Independent Numeric Variables
Dependent Variable
Net Trained on Pieces wool sales, weeks 33 38, 2006 (3)
GRNN, MLFN 2 to 3 nodes
GRNN Numeric Predictor
Palisade Conf Curtis v6 BNS 6hrs.xls
8 (Sale centre, Sale week, Sale outcome,
Style, Med Hard Cotts, Unscourable Colour,
Jowls, Dark Stain)
8 (Staple Length, Staple Strength, Vegetable
Matter, Diameter, CV Diameter, Mid Breaks,
Yield, Hauteur)
Numeric Var. (Clean price)
Model development (3)
z
z
z
z
Assemble a 6 month data set
Use Best Net Search
Evaluate predictive capability
Refine model
Model evaluation (1)
z
NeuralTools outputs
• Error measures
• Actual versus Predicted, Residuals
• Variable Impact Analysis
z
z
z
Live Prediction
Relationships between variables
Compare to published market indicators
Model evaluation (1)
Training and Testing summary
Training
Number of Cases
Training Time (h:min:sec)
Number of Trials
Reason Stopped
% Bad Predictions (5% Tolerance)
Root Mean Square Error
Mean Absolute Error
Std. Deviation of Abs. Error
5910
0:39:43
104
Auto-Stopped
14.7377%
24.72
16.42
18.48
Testing
Number of Cases
% Bad Predictions (5% Tolerance)
Root Mean Square Error
Mean Absolute Error
Std. Deviation of Abs. Error
1507
43.3975%
53.18
36.99
38.21
Model evaluation - Training data
(mean absolute error 16 cents)
Model evaluation - Testing data
(mean absolute error 37 cents)
Model evaluation (1)
Testing data (indicators)
Observed versus predicted for the
published Pieces Market indicators
Most points are on the 1:1 line, but a small group hover above
i.e. they have higher predicted values than reported
Model evaluation (1)
Variable impact analysis
Relative Variable Impacts
0%
Diameter
Vegetable Matter
Staple Length
Jowls
Hauteur
Sale outcome
Med Hard Cotts
Yield
CV Diameter
Staple Strength
Sale centre
Sale week
Dark Stain
Style
Unscourable Colour
Mid Breaks
10%
20%
30%
40%
50%
60%
70%
41.3%
18.7%
11.7%
8.8%
7.7%
1.9%
1.8%
1.6%
1.2%
1.2%
1.1%
This is a sensitivity analysis,
0.9%
0.7%
not the percent of variance
0.6%
accounted for by each variable
0.4%
0.4%
Model evaluation (2)
z
NeuralTools outputs
• Error measures
• Actual versus Predicted, Residuals
• Variable Impact Analysis
z
z
z
Live Prediction
Relationships between variables
Compare to published market indicators
Model evaluation (2)
Live prediction
Simple spreadsheet pricing tool.
Sale centre
Sale week
Style
Change any of the values in the
yellow cells, and ‘Live prediction’
updates the clean price
Med Hard Cotts
Unscourable Colour
Jowls
Dark Stain
Fremantle
W38
Average
C0
H0
J0
S0
Diameter
20.0
Yield
Vegetable Matter
50.0
2.5
Staple Length
Staple Strength
Mid Breaks
Hauteur
Clean price
80
35
55
62
664
Model evaluation (3)
z
NeuralTools outputs
• Error measures
• Actual versus Predicted, Residuals
• Variable Impact Analysis
z
z
z
Live Prediction
Relationships between variables
Compare to published market indicators
Model evaluation (3)
relationships between variables
680
670
660
650
Clean 640
Price
630
620
610
600
590
Sydney
Week 38
21 micron
2% VM
45
85
40
80
Staple
Length
75
35
70
30
65
25
Staple
Strength
Model evaluation (3)
relationships between variables
21 micron
Fremantle
Melbourne
635
680
680
630
670
670
625
660
660
650
620
650
Cl ean
P r i ce
C l e an
615
P r i ce
640
Cl ean
640
P r i ce
630
610
630
605
620
610
600
610
600
595
600
45
85
80
St a pl e
620
590
80
St a pl e
St r e ngt h
35
70
30
45
85
40
75
Le ngt h
St a pl e
80
St a pl e
St r e ngt h
35
70
Le ngt h
65
30
40
75
St a pl e
30
65
25
650
640
640
630
630
620
600
610
620
580
C l e an
P r i ce
560
St a pl e
St r e ngt h
35
70
Le ngt h
65
25
620
Cl ean
P r i ce
45
85
40
75
25
22 micron
Sydney
610
Cl ean
P r i ce
600
600
590
580
540
590
570
580
560
520
570
500
550
560
45
85
80
St a pl e
Le ngt h
540
80
40
75
35
70
30
65
25
45
85
St a pl e
St r e ngt h
St a pl e
Le ngt h
35
70
30
65
25
45
85
80
40
75
St a pl e
St r e ngt h
St a pl e
Le ngt h
40
75
35
70
30
65
25
St a pl e
St r e ngt h
Model evaluation (3)
relationships between variables
19 micron
Fremantle
Melbourne
760
840
820
755
830
810
750
820
800
745
810
740
Cl ean
P r i ce
790
Cl ean
P r i ce
735
730
725
800
Cl ean
P r i ce
790
780
770
780
720
770
760
710
760
750
705
750
715
45
85
80
740
80
St a pl e
St r e ngt h
35
70
30
45
85
40
75
St apl e
Lengt h
65
80
St apl e
St r engt h
35
70
30
45
85
40
75
St a pl e
Le ngt h
40
75
St a pl e
Le ngt h
65
25
20 micron
Sydney
70
30
65
25
675
740
670
730
665
720
St a pl e
St r e ngt h
35
25
720
710
700
Cl ean
P r i ce
660
Cl ean
P r i ce
655
710
Cl ean
P r i ce
700
690
680
650
690
645
680
640
670
45
85
80
St apl e
Lengt h
670
660
80
40
75
35
70
30
65
25
45
85
St a pl e
St r e ngt h
St a pl e
Le ngt h
35
70
30
65
25
45
85
80
40
75
St apl e
St r engt h
St a pl e
Le ngt h
40
75
35
70
30
65
25
St a pl e
St r e ngt h
Price spread variation
Model evaluation (4)
z
NeuralTools outputs
• Error measures
• Actual versus Predicted, Residuals
• Variable Impact Analysis
z
z
z
Live Prediction
Relationships between variables
Compare to published market indicators
Model evaluation (4)
predictive capability
20 micron indicator
22 micron indicator
Melbourne
Week 38
Model evaluation (4)
predictive capability
Melbourne
Week 38
Dark blue lots have SL, SS and VM
“similar” to market indicator definition
Model evaluation (4)
predictive capability
Melbourne
Week 37
Model evaluation (4)
predictive capability
Melbourne
Week 37
Model evaluation (4)
predictive capability
Melbourne
Week 36
Model evaluation (4)
predictive capability
Melbourne
Week 35
Model evaluation (4)
predictive capability
Melbourne
Week 34
Model evaluation (4)
predictive capability
Melbourne
Week 33
Model evaluation (4)
predictive capability
Fremantle
Week 37
Model evaluation (4)
predictive capability
Fremantle
Week 38
Model development (4)
z
z
z
z
Assemble a 6 month data set
Use Best Net Search
Evaluate predictive capability
Refine model
• Reduce variables
• Combine selling centres
• Sale week - category variable
Some Neural Net applications
z
z
z
z
z
Market reporting
Price predictor
Validation check for other estimates
Missing sale problem
Generate price matrices
• Using Live Prediction and @Risk
Summary
z
z
Data rich application with
characteristics that looked ideal for
NeuralTools
Solutions generated which can support
industry analysis and generation of
indicators