Effect-Oriented Programming
Creating Reliable Systems with Scala 3 and ZIO 2
Bill Frasure, Bruce Eckel and James Ward
Effect-Oriented Programming
Creating Reliable Systems with Scala 3 and ZIO 2
Bill Frasure, Bruce Eckel and James Ward
This book is for sale at http://leanpub.com/effect-oriented-programming
This version was published on 2023-11-25
This is a Leanpub book. Leanpub empowers authors and publishers with the Lean
Publishing process. Lean Publishing is the act of publishing an in-progress ebook
using lightweight tools and many iterations to get reader feedback, pivot until you
have the right book and build traction once you do.
© 2021 - 2023 Bill Frasure, Bruce Eckel and James Ward
CONTENTS
Contents
Copyright . . . . . . . . . . . . . . .
Effect Oriented Programming
Source Code . . . . . . . . . . .
Edit This Chapter . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1
1
2
5
Preface . . . . . . . . . . . .
Who is the book is for
Code examples . . . .
Edit This Chapter . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
6
6
6
6
What Are Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Edit This Chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7
8
Superpowers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Building a Resilient Process in stages . . . . . . . . . . . . . . . . . . . . . . .
Edit This Chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
9
10
14
Why Functional . . . . . . . . . . . . . . . . . . . . .
A Different Goal . . . . . . . . . . . . . . . . . .
Reuse . . . . . . . . . . . . . . . . . . . . . . . . .
Pure Functions . . . . . . . . . . . . . . . . . . .
Composability . . . . . . . . . . . . . . . . . . . .
Effects . . . . . . . . . . . . . . . . . . . . . . . .
Avoiding Recursion . . . . . . . . . . . . . . . .
Core Differences Between OO and Functional
Summary: Style vs Substance . . . . . . . . . .
Edit This Chapter . . . . . . . . . . . . . . . . . .
15
15
18
19
20
21
21
22
22
23
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
CONTENTS
Composability . . . . . . . . . . . . . . .
Composability Explanation . . . . .
Alternatives and their downsides .
Universal Composability with ZIO
Edit This Chapter . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
24
24
24
26
27
Dependency Injection . . . . . . . . . . . . . . . . . . . . . . . . .
DI-Wow! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Intersections AKA Products AKA Case Classes AKA Ands
Edit This Chapter . . . . . . . . . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
28
29
32
38
The ZIO Type . . . . . . . . . . . . . . . . . .
R - The Environment . . . . . . . . . . .
E - The Error . . . . . . . . . . . . . . . .
A - The Result . . . . . . . . . . . . . . . .
Conversions from standard Scala types
Edit This Chapter . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
39
39
39
40
40
41
Built-in Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
History - TODO Consider deleting. Not crucial to the reader.
Overriding Builtin Services . . . . . . . . . . . . . . . . . . . . .
Edit This Chapter . . . . . . . . . . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
42
42
43
43
Running Effects . . . . . . . . . . . . . . . . . . . .
ZIOs are not their result. . . . . . . . . . . . .
The ZIO Interpreter . . . . . . . . . . . . . . .
Building applications from scratch . . . . . .
Testing code . . . . . . . . . . . . . . . . . . . .
Interop with existing/legacy code via Unsafe
Web Request Handler . . . . . . . . . . . . . .
Processing streams of data . . . . . . . . . . .
Edit This Chapter . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
44
44
45
45
47
49
50
50
50
Layers . . . . . . . . . . .
Creating . . . . . . .
Composing . . . . . .
Historic Approaches
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
51
51
51
52
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
CONTENTS
Traits . . . . . . . . . . . . . . . . . . . . . .
Edit This Chapter . . . . . . . . . . . . . . .
Automatically attached experiments. . . .
Testing Unpredictable Effects . . . . . . .
Random . . . . . . . . . . . . . . . . . . . .
Time . . . . . . . . . . . . . . . . . . . . . .
assertTrue . . . . . . . . . . . . . . . . . .
Test Aspects . . . . . . . . . . . . . . . . . .
Defining a base test class for your project
Edit This Chapter . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
53
55
55
58
58
59
60
61
66
66
Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
67
Hello Failures . . . . . . . . . . . . . . . . . . . . . . .
Historic approaches to Error-handling . . . . . .
ZIO Error Handling . . . . . . . . . . . . . . . . .
Unions AKA Sum Types AKA Enums AKA Ors
Edit This Chapter . . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
68
68
73
77
79
Concurrency High Level
zipPar, zipWithPar . .
validateWithPar? . . .
withParallelism . . . .
Edit This Chapter . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
80
81
81
81
81
Concurrency Low Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Edit This Chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
82
82
Concurrency Interruption . . . . . . . . . . . . . . . . . . .
Why Interruption Is Necessary Throughout the Stack .
Timeout . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.withFinalizer . . . . . . . . . . . . . . . . . . . . . . . . .
Uninterruptable . . . . . . . . . . . . . . . . . . . . . . . .
Future Cancellation . . . . . . . . . . . . . . . . . . . . .
Edit This Chapter . . . . . . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
83
83
83
83
83
84
84
Concurrency State . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
85
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
CONTENTS
Unreliable Counting
Reliable Counting .
Ref.Synchronized . .
Edit This Chapter . .
.
.
.
.
.
.
.
.
.
.
.
.
86
86
89
89
Repeats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Edit This Chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
90
90
Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Edit This Chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
91
91
Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Edit This Chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Automatically attached experiments. . . . . . . . . . . . . . . . . . . . . . . .
92
92
92
Configuration . . . . . . . .
CLI Params . . . . . . .
Config Files . . . . . . .
Environment Variables
ZIO Config . . . . . . . .
Historic Approach . . .
Building a Better Way .
Official ZIO Approach .
Exercises . . . . . . . . .
Edit This Chapter . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
. 94
. 94
. 94
. 94
. 94
. 94
. 96
. 100
. 101
. 103
Streams . . . . . . . . . . . . . . . . . . . .
UI Interactions . . . . . . . . . . . . .
Trend Recognition . . . . . . . . . . .
Edit This Chapter . . . . . . . . . . . .
Automatically attached experiments.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
104
104
104
104
105
Appendix: Significant Indentation . . . .
Motivations . . . . . . . . . . . . . . . .
Concessions / Acknowledgements . .
Rules / examples . . . . . . . . . . . . .
Edge cases that are difficult to defend
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
129
129
129
129
130
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
CONTENTS
Edit This Chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
Appendix: ZIO Direct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
How does defer, etc relate to flatmaps, for comprehensions, etc . . . . . . . 132
Edit This Chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
Experiments . . . . . . . . . . . . . . . . . . . . .
experiments-src-test-scala-Hubs . . . . . . .
Parallelism . . . . . . . . . . . . . . . . . . . .
experiments-src-test-scala-random . . . . .
Hubs . . . . . . . . . . . . . . . . . . . . . . .
executing_external_programs . . . . . . . .
experiments-src-test-scala-running_effects
hello_failures . . . . . . . . . . . . . . . . . .
scenarios . . . . . . . . . . . . . . . . . . . . .
zio_helpers . . . . . . . . . . . . . . . . . . . .
crypto . . . . . . . . . . . . . . . . . . . . . . .
experiments-src-test-scala-test_aspects . .
experiments-src-test-scala-scenarios . . . .
cancellation . . . . . . . . . . . . . . . . . . .
concurrency . . . . . . . . . . . . . . . . . . .
rezilience . . . . . . . . . . . . . . . . . . . . .
cause . . . . . . . . . . . . . . . . . . . . . . .
environment . . . . . . . . . . . . . . . . . . .
experiments-src-test-scala-zio_test . . . . .
experiments-src-test-scala-layers . . . . . .
experiments-src-test-scala-time . . . . . . .
performance . . . . . . . . . . . . . . . . . . .
resourcemanagement . . . . . . . . . . . . .
experiments-src-test-scala-concurrency . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
134
134
137
141
145
150
151
153
155
158
159
161
164
165
168
170
176
180
184
188
189
191
195
197
Copyright
Effect Oriented Programming
By Bill Frasure, Bruce Eckel and James Ward
{{ These are the new ISBNs }}
Copyright ©2021, MindView LLC.
eBook ISBN 978-0-9818725-6-8
Print Book ISBN 978-0-9818725-7-5
The eBook ISBN covers the Leanpub eBook distribution in all formats, available
through www.EffectOrientedProgramming.com.
Please purchase this book through www.EffectOrientedProgramming.com, to
support its continued maintenance and updates.
All rights reserved. Printed in the United States of America. This publication is
protected by copyright, and permission must be obtained from the publisher prior
to any prohibited reproduction, storage in a retrieval system, or transmission in any
form or by any means, electronic, mechanical, photocopying, recording, or likewise.
For information regarding permissions, see EffectOrientedProgramming.com.
Created in Crested Butte, Colorado, USA.
Text printed in the United States
Ebook: Version 1.0, Month Year
First printing Month Year
Cover design by Daniel Will-Harris, www.Will-Harris.com¹
¹http://www.Will-Harris.com
2
Copyright
Many of the designations used by manufacturers and sellers to distinguish their
products are claimed as trademarks. Where those designations appear in this book,
and the publisher was aware of a trademark claim, the designations are printed with
initial capital letters or in all capitals.
The Scala trademark belongs to {{???}}. Java is a trademark or registered trademark
of Oracle, Inc. in the United States and other countries. Windows is a registered
trademark of Microsoft Corporation in the United States and other countries. All
other product names and company names mentioned herein are the property of their
respective owners.
The authors and publisher have taken care in the preparation of this book, but make
no expressed or implied warranty of any kind and assume no responsibility for errors
or omissions. No liability is assumed for incidental or consequential damages in
connection with or arising out of the use of the information or programs contained
herein.
Visit us at www.EffectOrientedProgramming.com.
Source Code
All the source code for this book is available as copyrighted freeware, distributed
via Github². To ensure you have the most current version, this is the official code
distribution site. You may use this code in classroom and other educational situations
as long as you cite this book as the source.
The primary goal of this copyright is to ensure that the source of the code is properly
cited, and to prevent you from republishing the code without permission. (As long
as this book is cited, using examples from the book in most media is generally not a
problem.)
In each source-code file you find a reference to the following copyright notice:
²https://github.com/EffectOrientedProgramming/EOPCode
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
3
Copyright
// Copyright.txt
This computer source code is Copyright ©2021 MindView LLC.
All Rights Reserved.
Permission to use, copy, modify, and distribute this
computer source code (Source Code) and its documentation
without fee and without a written agreement for the
purposes set forth below is hereby granted, provided that
the above copyright notice, this paragraph and the
following five numbered paragraphs appear in all copies.
1. Permission is granted to compile the Source Code and to
include the compiled code, in executable format only, in
personal and commercial software programs.
2. Permission is granted to use the Source Code without
modification in classroom situations, including in
presentation materials, provided that the book "Effect Oriented
Programming" is cited as the origin.
3. Permission to incorporate the Source Code into printed
media may be obtained by contacting:
MindView LLC, PO Box 969, Crested Butte, CO 81224
MindViewInc@gmail.com
4. The Source Code and documentation are copyrighted by
MindView LLC. The Source code is provided without express
or implied warranty of any kind, including any implied
warranty of merchantability, fitness for a particular
purpose or non-infringement. MindView LLC does not
warrant that the operation of any program that includes the
Source Code will be uninterrupted or error-free. MindView
LLC makes no representation about the suitability of the
Source Code or of any software that includes the Source
Code for any purpose. The entire risk as to the quality
and performance of any program that includes the Source
Code is with the user of the Source Code. The user
understands that the Source Code was developed for research
and instructional purposes and is advised not to rely
exclusively for any reason on the Source Code or any
program that includes the Source Code. Should the Source
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
4
Copyright
Code or any resulting software prove defective, the user
assumes the cost of all necessary servicing, repair, or
correction.
5. IN NO EVENT SHALL MINDVIEW LLC, OR ITS PUBLISHER BE
LIABLE TO ANY PARTY UNDER ANY LEGAL THEORY FOR DIRECT,
INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES,
INCLUDING LOST PROFITS, BUSINESS INTERRUPTION, LOSS OF
BUSINESS INFORMATION, OR ANY OTHER PECUNIARY LOSS, OR FOR
PERSONAL INJURIES, ARISING OUT OF THE USE OF THIS SOURCE
CODE AND ITS DOCUMENTATION, OR ARISING OUT OF THE INABILITY
TO USE ANY RESULTING PROGRAM, EVEN IF MINDVIEW LLC, OR
ITS PUBLISHER HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH
DAMAGE. MINDVIEW LLC SPECIFICALLY DISCLAIMS ANY
WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE SOURCE CODE AND DOCUMENTATION PROVIDED
HEREUNDER IS ON AN "AS IS" BASIS, WITHOUT ANY ACCOMPANYING
SERVICES FROM MINDVIEW LLC, AND MINDVIEW LLC HAS NO
OBLIGATIONS TO PROVIDE MAINTENANCE, SUPPORT, UPDATES,
ENHANCEMENTS, OR MODIFICATIONS.
Please note that MindView LLC maintains a Web site which is
the sole distribution point for electronic copies of the
Source Code, where it is freely available under the terms
stated above:
https://github.com/EffectOrientedProgramming/EOPCode
If you think you've found an error in the Source Code,
please submit a correction at:
https://github.com/EffectOrientedProgramming/EOPCode/issues
You may use the code in your projects and in the classroom (including your
presentation materials) as long as the copyright notice that appears in each source
file is retained.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
5
Copyright
Edit This Chapter
Edit This Chapter³
³https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/00_Copyright.md
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Preface
Who is the book is for
Code examples
Edit This Chapter
Edit This Chapter⁴
⁴https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/01_Preface.md
What Are Effects
Any real program has to interact with things outside the programmer’s control.
All external systems are unpredictable.
One of the biggest challenges in building systems that are more predictable / reliable
/ ??? is to isolate and manage the unpredictable parts.
An approach that programmers may use to handle this is to delineate the parts of the
program which use external systems.
By delineating them, programmers then have tools to handle the unpredictable parts
in more predictable ways.
The interactions with external systems can be defined in terms of “Effects” which
create a delineation between the parts of a program that interact with external
systems and those that don’t.
For example, a program that displays the current date requires something that
actually knows the current date.
The program needs to talk to an external system (maybe just the operating system)
to get this information.
These programs are unpredictable because the programmer has no control over what
that external system will say or do.
Effect Oriented systems allow us to apply strategies to mitigate the unpredictability
of using external systems.
Effects can not be un-done.
Once a program has communicated with an external system, (i.e. executed an Effect),
everything that happens on that external systems is out of the program’s control.
(analogies on human communication) Imagine that a friend recently stayed in your
home. 3 days after they leave, you realize that you are missing some money that
What Are Effects
8
had been stored in the guestroom. Now you have a dilemma - do you ask them if
they took the money? Simply by asking, you could permanently change, or even end,
your relationship with this person. They could immediately admit fault, and ask for
forgiveness. Now you know that they are capable of stealing from you - will you
ever trust them in your home again? They could angrily deny the accusation, and
resent you for making it. Or the conversation could go in a million different other
ways that are impossible to predict. We know one thing for certain - you will never
be able to un-ask that question. Even if you ultimately grow closer with this person
after navigating this situation, you can’t go back to a world where you never asked.
Regardless of any apology and forgiveness, your relationship is now different.
The Effect is not what happens on the external system because there is no way to
know the actual impact of what the program caused by the communication.
Edit This Chapter
Edit This Chapter⁵
⁵https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/02_What_Are_Effects.md
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Superpowers
1. The superpowers are the strategies to deal with unpredictability
2. OpeningHook (1-6)
1. Note: Progressive enhancement through adding capabilities
3. Concurrency
1. Race
2. Hedge (to show the relationship to OpeningHook ie progressive enhancement)
4. Sequential
1. ZIO Direct
2. Note: Combine OpeningHook & Concurrency with ZIO Direct
5. And so much more
1. Note: And there are many capabilities you might want to express. In the
future we will dive into these other capabilities.
•
•
•
•
•
•
•
•
•
•
Racing
Timeout
Resource Safety
Mutability that you can trust
Human-readable
Cross-cutting Concerns / Observability / Regular Aspects
– timed
– metrics
– debug
– logging
Interruption/Cancellation
Fibers
Processor Utilization
– Fairness
– Work-stealing
Resource Control/Management
10
Superpowers
object DatabaseError
object TimeoutError
Building a Resilient Process in stages
Successful Code
// works
runDemo:
saveUser:
"mrsdavis"
// User saved
Error Fallback Value
// fails
runDemo:
saveUser:
"mrsdavis"
.orElseFail:
"ERROR: User could not be saved"
// DatabaseError
// ERROR: User could not be saved
Retry Upon Failure
import zio.Schedule.{recurs, spaced}
val aFewTimes =
// TODO Restore original spacing when done
// editing
// recurs(3) && spaced(1.second)
recurs(3) && spaced(1.millis)
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
11
Superpowers
runDemo:
saveUser:
"morty"
.retry:
aFewTimes
.orElseSucceed:
"ERROR: User could not be saved"
// DatabaseError
// DatabaseError
// User saved
Fallback after multiple failures
// fails every time - with retry
runDemo:
saveUser:
"morty"
.retry:
aFewTimes
.orElseSucceed:
"ERROR: User could not be saved"
// DatabaseError
// DatabaseError
// DatabaseError
// DatabaseError
// ERROR: User could not be saved
Timeouts
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
12
Superpowers
// TODO Restore real value when done editing
val timeLimit = 5.millis
// timeLimit: Duration = PT0.005S
// 5.seconds
// first is slow - with timeout and retry
runDemo:
saveUser:
"morty"
.timeoutFail(TimeoutError)(timeLimit)
.retry:
aFewTimes
.orElseSucceed:
"ERROR: User could not be saved"
// Interrupting slow request
// Database Timeout
// User saved
Fallback Effect
// fails - with retry and fallback
runDemo:
saveUser:
"morty"
.timeoutFail(TimeoutError)(timeLimit)
.retry:
aFewTimes
.orElse:
sendToManualQueue:
"morty"
.orElseSucceed:
"ERROR: User could not be saved, even to the fallback system"
// DatabaseError
// DatabaseError
// DatabaseError
// DatabaseError
// User sent to manual setup queue
Concurrently Execute Effect
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
13
Superpowers
// concurrently save & send analytics
runDemo:
saveUser:
"morty"
// todo: maybe this hidden extension method
// goes too far with functionality that
// doesn't really exist
// TODO Should we fireAndForget before the
// retries/fallbacks?
.fireAndForget:
userSignupInitiated:
"morty"
.timeoutFail(TimeoutError)(timeLimit)
.retry:
aFewTimes
.orElse:
sendToManualQueue:
"morty"
.orElseSucceed:
"ERROR: User could not be saved"
// User saved
Ignore failures in Concurrent Effect
Feeling a bit “meh” about this step.
// concurrently save & send analytics, ignoring analytics failures
runDemo:
// TODO Consider ways to dedup morty
// string
saveUser:
"mrsdavis"
.timeoutFail(TimeoutError)(timeLimit)
.retry:
aFewTimes
.orElse:
sendToManualQueue:
"mrsdavis"
.tapBoth(
error =>
userSignUpFailed("mrsdavis", error),
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
14
Superpowers
success =>
userSignupSucceeded("mrsdavis", success)
)
.orElseSucceed:
"ERROR: User could not be saved"
// Analytics sent for signup completion
// User saved
Edit This Chapter
Edit This Chapter⁶
⁶https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/03_Superpowers.md
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Why Functional
1. Functions & Specialized Data Types Are Great
2. Why Functional
In your journey to this book, you have undoubtedly learned at least one
way to think about programming, and possibly several.
You might have had reasonable success using that approach.
You might also have encountered programming constructs inspired by functional
programming. For example, Java 8 introduced lambdas along with library support
for streams and functional primitives like map. Python has always allowed functions
to be passed into, be created by, and be returned from other functions, and includes
other functional support. C++, especially more recent versions, has added a number
of features that support functional-style programming.
If, however, you are coming from an imperative programming background, these
functional-style devices can seem arbitrarily complicated. Why go through the
trouble of an effect-oriented API when plain vars, for loops, and :Unit exist?
Sometimes it seems like functional programmers write code like this just to be fancy.
To understand what’s really behind this different way of thinking about programming, it helps to start with some history.
A Different Goal
In the early days of programming, most code was written in an assembly language
for a particular machine. Assembly language had primitive function-like constructs
called subroutines, but you had to do the work of setting up a call (storing arguments
in registers or on the stack), then inside the subroutine you had to access those
arguments, and then before returning from the subroutine you had to somehow pass
Why Functional
16
the return value back to the caller (again, typically using either a register or the stack).
You had to do all this by hand. It was worth it if you knew you were going to reuse
that subroutine, but if you thought you were only ever going to use that code once
then it was easier and more efficient to just insert the code inline. Even if you wanted
to create reusable code, it was often easier to just “goto” a piece of code and use global
variables, rather than bothering with passing arguments and returning results.
In those early days, a “high level language” meant a language like C that passed
arguments and returned results for you. This suddenly made writing functions
much easier, safer, and faster. But the habits of assembly-language programmers
didn’t vanish overnight, and people were still prone to writing obtuse monolithic
programs—often a single function for the whole program—and jumping around in
their code using gotos. Many programs were written and maintained by a small
number of programmers, or even a single programmer, for whom that code made
sense. Anyone else reading the code would be baffled.
In addition, the idea of calling code written by other people was fairly foreign.
If you didn’t write the code yourself, how would you know if it does what you
want? (Documentation and testing were also primitive, if they existed at all). Thus
“code reuse” was a big hurdle; there were many attempts within companies to
create cultures of code reuse because many programmers were rewriting the same
functionality from scratch, over and over within the same organization.
Monolithic programs that didn’t reuse code were also a maintenance nightmare. It
was not uncommon for such programs to be thrown away and rewritten just to add
some new features. Not surprisingly, writing everything from scratch also took a lot
longer than reusing common functionality. The new question became: “how do we
make code reuse easier?”
Because languages like C and Pascal made it easy to not only write functions
but to call them, they were a big improvement over assembly language, although
the monolithic habits of assembly-language programmers persisted into those new
languages. Libraries grew bigger and more complex, and using those libraries was not
easy. In C, for example, you’d often have to call malloc to allocate memory before
calling a library function, and later free to release the memory when that function
was done with it. You also had to learn how to pass information from one library
function to another. You had to learn how a each library reported errors, which
typically varied in strategy from one library to the next. Code could be reused, but
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Why Functional
17
it wasn’t easy.
At this point, object-oriented programming seemed like a good idea, because it
combined a common data structure, automatic initialization and cleanup of that
data structure, together with all the functions that act upon that data. If you wanted
to reuse some code, you just created an object with some initialization values and
then sent messages to that object to produce the desired results. This did make code
reuse easier, and helped speed up program creation. It also came with the distraction
of inheritance polymorphism and an entire education and consulting industry
explaining how to cram every design into an inheritance hierarchy (inheritance
polymorphism does sometimes prove useful, but not everywhere, all the time).
C++ added object-oriented features from the Simula language while maintaining
backward compatibility with the C language. C++ had a strong emphasis on static
type checking. Java was created as a counterpoint to C++ and was heavily inspired
by the Smalltalk language.
Smalltalk’s success came from its ability to rapidly create systems by adding
functionality to existing objects. This introduced a conundrum, because Smalltalk
is a dynamic language, and Java, like C++, is statically typed. Smalltalk can be
thought of as supporting an experimental style of programming: you send a message
to an object and discover at runtime whether the object knows what to do with
that message. But C++ and Java ensure everything is valid, at compile time (along
with escape mechanisms that effectively disable that type checking). This conundrum
is exemplified by the Liskov Substitution Principle, which says that you shouldn’t
add new methods to an inherited type—and yet that activity is the foundation of
Smalltalk.
The Agile methodologies that began in the early 2000’s were another attempt to
produce software faster, but through a more bottom-up lens. Agile was primarily
focused on improving communication between stakeholders and developers, and
producing more rapid round trips between needs and experiments. This improves the
chance that the stakeholders will get what they need, faster. Agile helped the process
of software development, but again, the focus is on developing software quickly, not
on developing reliable software.
The most important thing to take away from this language history is that the
fundamental goal of the various techniques was speed of creation. There seems to be
an underlying assumption that these approaches will somehow automatically create
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Why Functional
18
more reliable software. As a result, we have languages that quickly create unreliable
software. And in many cases we’ve been able to get by with that. For one thing,
this approach has greatly advanced testing technology, because it was necessary.
Customers have learned to put up with buggy software. They’ve often been willing
to accept buggy software when the alternative is no software at all.
The world has changed. Back then, the drive was to speed up activities that humans
were doing. Those humans could compensate for bugs. Now, however, more and
more software is doing things that humans can’t do, so failures in software cannot
be propped up by humans. Unreliable software is no longer an inconvenience, but a
serious problem.
Quickly creating unreliable software is no longer acceptable. We must delve into the
reasons that software fails—either it doesn’t do what it’s supposed to, or it just breaks.
Reuse
How do we create software? When you first learned to program, you probably solved
problems by writing code using the basic constructs of your language. But at some
point you began realizing that you could only produce and debug so much code
by yourself. If you could use code that was already written and debugged by other
people, you could produce solutions faster.
You might have gone through a cut-and-paste phase before discovering that formalized libraries were easier and more reliable. Even then, library ease of use depended
on the sophistication of your language. For the reasons mentioned, using a C library
could be tricky and difficult. C++ made this much easier and paved the way for
the acceptance of languages like Java, Python, Scala, and Kotlin. Indeed, any new
language that doesn’t support easy code reuse is not taken seriously.
But code reuse in object-oriented languages was still limited. You could either use
objects in a library directly, or you could add those library classes into new classes
using composition. This was a big step and it helped a lot. In contrast, composing C
libraries wasn’t particularly realistic—it was just too messy and complicated.
The problem is reliability. If you create a new class using composition, you combine
problems with the existing class(es) with any bugs you introduced in your new class.
As you build up bigger systems, the problem of bugs multiplies.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Why Functional
19
To compose systems rapidly and reliably, we return to first principles and figure out
how to:
1. Create basic components that are completely reliable.
2. Combine those components in a way that doesn’t introduce new bugs.
To achieve these goals we must examine the fundamentals of how we think about
software.
Pure Functions
Composition in an object-oriented language doesn’t attempt to manage bugs, so it
ends up amplifying them. If we want to compose pieces of software, we must discover
what creates a fundamentally unbreakable piece, then how to assemble those pieces
without producing a broken result.
First, what constitutes a reliable, unbreakable piece of software? We’ve already seen
that objects are not inherently unbreakable, so we’ll move back to a more basic
software component: the function. What are the characteristics of an unbreakable
function?
What we want is the same kind of function we have in math. This means that the
function does nothing except produce a result from its arguments. And given the
same arguments, it always produces the same result.
This behavior imposes additional constraints: The function cannot affect its environment, and the environment cannot affect the function—otherwise, the function
has a history and behaves differently at one point in time vs. another. Running that
function doesn’t necessarily produce the same results from one call to the next.
If a function affects its environment, we call that a side effect. It’s “on the side”
because it’s something other than just producing a result from the function. Many
programming languages have side effects built in, in the form of statements. A
statement doesn’t return a result, so the only reason to execute a statement is for
its side effect. For example, “print” is typically a statement that returns nothing
but causes the side effect of displaying text on a console. On the other hand, an
expression produces (“expresses”) a result. A functional language avoids statements
and attempts to make everything an expression that produces a result.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Why Functional
20
What about the environment affecting the function? This is a bit more subtle, and
requires that we think in more general terms than just basic imperative programming.
In particular, we must consider concurrency. If multiple tasks are running in our
program, then at any point another task might see variables in our function. A
variable can change, so that means this other task might see different values at
different points in the function’s execution. And if that variable is modified by some
other task, we have no way of predicting the result, and we don’t get the reliable
mathematical function that we want.
We solve this problem through immutability. That is, instead of using variables, we
create values that cannot change. This way, it doesn’t matter if an external task sees
our values, because it will only see that one value and not something that is different
from one moment to the next. And the external task cannot change the value and
cause the function to produce a different result.
Functions that behave mathematically, that always produce the same results from
the same inputs and have no side effects, are called pure functions. When we add the
additional constraint of immutability, we produce functions that compose without
introducing points of breakage. We can reliably reason about such functions.
Composability
TODO Reconsider this formal math style. It doesn’t match our other examples, where
we really stick to Scala code
If functions g and h are pure, we should be able to combine them directly (assuming
the types agree) to produce a new function f:
f(a) = g(h(a))
This assumes that all functions involved are complete, meaning that they produce a
legitimate result for every possible value of a. This is not always true. For example,
dividing a number by zero is undefined, and so cannot produce a reasonable number
as a result. Using a key to look up a value in a map is undefined if that key doesn’t
exist in the map. <– TODO Weird wording
An incomplete function requires more operations when using it, to handle the
problematic inputs. You can think of the solution as stepwise composability. Instead
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Why Functional
21
of calling g(h(a)), we break the process into steps: x = h(a), then check the
success of the operation. If successful, pass the result to g. These extra steps make
composability sound like it could get tedious, and languages like Scala that provide
more thorough support for functional programming provide syntax to make this kind
of programming feasible.
Effects
Now we have created this perfect world of pure functions that behave just like
the functions in theoretical mathematics. They have no side effects and cannot be
affected by other functions, and can be neatly and safely composed.
“But,” you wonder, “if all I can do with the result of one pure function is pass it as an
argument to another pure function, what’s the point of all these pure function calls?
If these functions have no effect on the world, they seem like an intellectual exercise
that merely heats up the CPU.”
This is absolutely true. A program that never affects the world is pointless. For a
program to be useful, it must be affected by the world, and it must have effects upon
the world.
The phrase “side effect” implies an incidental or accidental impact on the world. What
we need to do is formalize this idea and bring it under our control. We can then call
it simply an “effect,” without the “side.” The solution is to manage these effects so
they are under our control.
This bridge between pure functions and practical programs with controlled and
managed effects is the reason for the title of this book.
Avoiding Recursion
We will deliberately avoid recursion in most examples. Many functional programming tutorials begin by introducing recursion. While this resonates with some, it
is an additional hurdle that makes the transition harder. We want to demonstrate
powerful, functional code without user-facing recursion.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Why Functional
22
Core Differences Between OO and Functional
An OO language worries about managing state. It “encapsulates” a data structure in
privacy and surrounds it with custom methods (aka member functions) which are
ideally the only way to access and modify the state of that data structure. This is
important because an OO data structure is typically mutable. This OO ceremony
attempts to create predictability by knowing how the data structure can be mutated.
Functional programming abstracts common behavior into reusable functional components. These components are adapted to specific needs using other functions. This
is why lambdas are so important, because you constantly need to adapt general code
to specific purposes, They enable concise code that would otherwise be awkward and
intrusive to right as a standalone function.
Functions in a functional language don’t need to be tied to a particular data structure.
Thus, they can often be written for more general use and to reduce duplication.
Functional languages come with a general set of well-tested, reusable operations that
can be applied almost mathematically in many situations.
A functional language relies on immutability. An immutable data structure doesn’t
need privacy because it is safe for any task to read, and it cannot be written
(only initialized). Because immutability dramatically simplifies everything, objects
in functional languages are simply naked data structures along with constructors.
When everything is immutable, there is no need for private properties or methods to
maintain the state of an object.
Summary: Style vs Substance
Functional programming abstracts common behavior into reusable functional components. These components are adapted to specific needs using other functions. This
is why lambdas are so important, because you constantly need to adapt general code
to specific purposes, often with a brief amount of code that would otherwise be
awkward and intrusive to write as a standalone function.
The two things we do with functions is compose them to make more complex
functions, and adapt to them to our specific problem.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Why Functional
23
We assume that many readers are attracted to this book because they have some
experience with functional programming constructs in other languages such as Java
(version 8 or newer), Kotlin, Python or some other language that provides a modicum
of support. However, we also assume you have heard—or you have a sense—that there
could be significantly more than:
• a function’s ability to create other functions
• transforming elemements in a collection using map
Those are indeed important benefits, but they just dip into the possibilities. Adopting
some of the styles found in functional programming does not make a language
functional.
In this book we want to get to the heart of what it means to be functional. In particular,
we want to show what it takes to make reliable functional code that can be composed
without propagating or amplifying flaws in its components.
Edit This Chapter
Edit This Chapter⁷
⁷https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/04_Why_Functional.md
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Composability
Composability Explanation
1. But Functions & Specialized Data Types Don’t Compose for Effects
2. Composability 1. Limitations of Functions & SDTs 1. Some intro to Universal
Effect Data Types ie ZIO 1. The ways in which ZIOs compose (contrasted
to limitations) 1. Note: Merge chapters: composability, Unit, The_ZIO_Type 1.
Note: Avoid explicit anonymous sum & product types at this point
Alternatives and their downsides
Other framings/techniques and their pros/cons:
Plain functions that throw Exceptions
• We can’t union these error possibilities and track them in the type system
• Cannot attach behavior to deferred functions
Plain functions that block
• We can’t indicate if they block or not
• Too many concurrent blocking operations can prevent progress of other operations
• Very difficult to manage
• Blocking performance varies wildly between environments
Composability
25
Functions that return Either/Option/Try/etc
• We can manage the errors in the type system, but we can’t interrupt the code
that is producing these values
• All of these types must be manually transformed into the other types
• Execution is not deferred
Functions that return a Future
•
•
•
•
•
•
•
Can be interrupted example1[future_interrupted_1] two[future_interrupted_2]
Cleanup is not guaranteed⁸
Manual management of cancellation
Start executing immediately
Must all fail with Exception
Implicits
Are not automatically managed by the compiler, you must explicitly add each
one to your parent function
• Resolving the origin of a provided implicit can be challenging
Try-with-resources
• These are statically scoped
• Unclear who is responsible for acquisition & cleanup
Each of these approaches gives you benefits, but you can’t assemble them all
together. Instead of the best of all worlds, you get the pain of all worlds. eg
Closeable[Future[Either[Throwable, A]]] The ordering of the nesting is significant, and not easily changed.
The number of combinations is something like: PairsIn(numberOfConcepts)
⁸./15_Concurrency_Interruption.md#Future-Cancellation
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Composability
26
Universal Composability with ZIO
ZIOs compose including errors, async, blocking, resource managed, cancellation,
eitherness, environmental requirements.
The types expand through generic parameters. ie composing a ZIO with an error of
String with a ZIO with an error of Int results in a ZIO with an error of String |
Int.
With functions there is one way to compose. f(g(h)) will sequentially apply the
functions from the inside out.
Another term for this form of composition is called andThen in Scala.
With ZIO you can use zio-direct to compose ZIOs sequentially with:
runDemo:
defer:
val topStory =
findTopNewsStory
.run
textAlert:
topStory
.run
// Texting story: Battery Breakthrough
// ()
There are many other ways you can compose ZIOs. The methods for composability
depend on the desired behavior. For example, to compose a ZIO that can produce an
error with a ZIO that logs the error and then produces a default value, you can use
the catchAll like:
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Composability
def logAndProvideDefault(e: Throwable) =
Console
.printLine:
e.getMessage
.as:
"default value"
runDemo:
ZIO
.attempt:
???
.catchAll:
logAndProvideDefault
// an implementation is missing
// default value
Edit This Chapter
Edit This Chapter⁹
⁹https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/05_Composability.md
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
27
Dependency Injection
1. Application startup uses the same tools that you utilize for the rest of your
application
General/Historic discussion
One reason to modularize an application into “parts” is that the relationship between
the parts can be expressed and also changed depending on the needs for a given
execution path.
Typically, this approach to breaking things into parts and expressing what they need,
is called “Dependency Injection.”
… Why is it called “Dependency Injection” ? Avoid having to explicitly pass things
down the call chain.
There is one way to express dependencies.
Let’s consider an example: We want to write a function that fetches Accounts from a
database The necessary parts might be a DatabaseService which provides database
connections and a UserService which provides the access controls. By separating
these dependencies our from the functionality of fetching accounts, tests can “fake”
or “mock” the dependencies to simulate the actual dependency.
In the world of Java these dependent parts are usually expressed through annotations
(e.g. @Autowired in Spring). But these approaches are “impure” (require mutability),
often rely on runtime magic (e.g. reflection), and require everything that uses the
annotations to be created through a Dependency Injection manager, complicating
construction flow.
An alternative to this approach is to use “Constructor Injection” which avoids some
of the pitfalls associated with “Field Injection” but doesn’t resolve some of the
underlying issues, including the ability for dependencies to be expressed at compile
time.
Dependency Injection
29
If instead functionality expressed its dependencies through the type system, the
compiler could verify that the needed parts are in-fact available given a particular
path of execution (e.g. main app, test suite one, test suite two).
What ZIO can provide us.
With ZIO’s approach to dependencies, you get many desirable characteristics at
compile-time, using standard language features. Your services are defined as classes
with constructor arguments, just as in any vanilla Scala application. No annotations
that kick off impenetrable wiring logic outside your normal code.
For any given service in your application, you define what it needs in order to execute.
Finally, when it is time to build your application, all of these pieces can be provided in
one, flat space. Each component will automatically find its dependencies, and make
itself available to other components that need it.
To aid further in understanding your application architecture, you can visualize the
dependency graph with a single line.
You can also do things that simply are not possible in other approaches, such as
sharing a single instance of a dependency across multiple test classes, or even
multiple applications.
DI-Wow!
TODO Values to convey: - Layer Graph - Cycles are a compile error - Visualization
with Mermaid - test implementations - Layer Resourcefulness - Layers can have setup
& teardown (open & close) - “‘scala // Explain private constructor approach case class
Dough private ()
object Dough: val letRise: ZIO[Dough, Nothing, Unit] = ZIO.debug(“Dough is
rising”)
val fresh: ZLayer[Any, Nothing, Dough] = ZLayer .derive[Dough] .tapWithMessage(“Making Fresh Dough”) “‘
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Dependency Injection
30
Step 1: Effects can express dependencies
Effects can’t be run until their dependencies have been fulfilled
TODO: Decide what to do about the compiler error differences between these
approaches
object LetDoughRiseNoDough extends ZIOAppDefault:
override def run = Dough.letRise
// error:
//
//
// ──── ZIO APP ERROR ─────────────────────────────────────────────────\
──
//
// Your effect requires a service that is not in the environment.
// Please provide a layer for the following type:
//
//
1. repl.MdocSession.MdocApp.Dough
//
// Call your effect's provide method with the layers you need.
// You can read more about layers and providing services here:
//
//
https://zio.dev/reference/contextual/
//
// ────────────────────────────────────────────────────────────────────\
──
//
//
//
defer:
// ^
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Dependency Injection
31
// TODO Consider weirdness of provide with no args
runDemo:
Dough.letRise.provide()
// error:
//
//
// ──── ZLAYER ERROR ──────────────────────────────────────────────────\
──
//
// Please provide a layer for the following type:
//
//
1. repl.MdocSession.MdocApp.Dough
//
// ────────────────────────────────────────────────────────────────────\
──
//
//
//
Bread.make.provide(Dough.fresh, Heat.oven)
//
^
Step 2: Provide Dependencies to Effects
Then the effect can be run.
runDemo:
Dough
.letRise
.provide:
Dough.fresh
// Making Fresh Dough
// Dough is rising
// ()
For code organization, and legibility at call sites, we are defining several layers within
the Heat companion object. They will all be used soon.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Dependency Injection
32
case class Heat private ()
object Heat:
val oven: ZLayer[Any, Nothing, Heat] =
ZLayer
.derive[Heat]
.tapWithMessage:
"Heating Oven"
val toaster: ZLayer[Any, Nothing, Heat] =
ZLayer
.derive[Heat]
.tapWithMessage:
"Heating Toaster"
val broken: ZLayer[Any, String, Nothing] =
ZLayer.fail:
"**Power Out**"
Step 3: Effects can require multiple dependencies
Note: The following is copy&pasted and might just need a slight diversion
to &’d typed parameters
Intersections AKA Products AKA Case Classes
AKA Ands
graph TD;
Any-->User;
Any-->Account;
trait Piece1
trait Piece2
def needyFunction()
: ZIO[Piece1 & Piece1, Nothing, Unit] = ???
For your Answer, it can be desirable to give a clear name that is relevant to your
domain.
The requirements for each ZIO are combined as an anonymous product type denoted
by the & symbol.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Dependency Injection
33
// Restore private constructor after failure scenario is dialed in
case class Bread()
object Bread:
val make: ZIO[Heat & Dough, Nothing, Bread] =
ZIO.succeed:
Bread()
// TODO Explain ZLayer.fromZIO in prose
// immediately before/after this
val homemade
: ZLayer[Heat & Dough, Nothing, Bread] =
ZLayer
.fromZIO:
make
.tapWithMessage:
"Making Homemade Bread"
val storeBought: ZLayer[Any, Nothing, Bread] =
ZLayer
.derive[Bread]
.tapWithMessage:
"Buying Bread"
val eat: ZIO[Bread, Nothing, String] =
ZIO.succeed:
"Eating bread!"
end Bread
runDemo:
Bread.make.provide(Dough.fresh, Heat.oven)
// Bread()
Step 4: Dependencies can “automatically” assemble to
fulfill the needs of an effect
Something around how like typical DI, the “graph” of dependencies gets resolved
“for you” This typically happens in some completely new/custom phase, that does
follow standard code paths. Dependencies on effects propagate to effects which use
effects.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Dependency Injection
34
// TODO Figure out why Bread.eat debug isn't showing up
runDemo:
Bread
.eat
.provide(
// Highlight that homemade needs the other
// dependencies.
Bread.homemade,
Dough.fresh,
Heat.oven
)
// Eating bread!
Step 5: Different effects can require the same
dependency
Eventually, we grow tired of eating plain Bread and decide to start making Toast.
Both of these processes require Heat.
// Is it worth the complexity of making this private?
// It would keep people from creating Toasts without using the make met\
hod
case class Toast private ()
object Toast:
val make: ZIO[Heat & Bread, Nothing, Toast] =
ZIO.succeed:
println:
"Making toast"
Toast()
It is possible to also use the oven to provide Heat to make the Toast.
The dependencies are based on the type, so in this case both Toast.make and
Bread.make require heat, but
Notice - Even though we provide the same dependencies in this example, Heat.oven
is also required by Toast.make
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Dependency Injection
35
runDemo:
Toast
.make
.provide(
Bread.homemade,
Dough.fresh,
Heat.oven
)
// Making toast
// Toast()
However, the oven uses a lot of energy to make Toast. It would be great if we can
instead use our dedicated toaster!
Step 6: Dependencies are based on types and must be
uniquely provided
runDemo:
Toast
.make
.provide(
Dough.fresh,
Bread.homemade,
Heat.oven,
Heat.toaster
)
// error:
//
//
// ──── ZLAYER ERROR ──────────────────────────────────────────────────\
──
//
// Ambiguous layers! I cannot decide which to use.
// You have provided more than one layer for the following type:
//
//
repl.MdocSession.MdocApp.Heat is provided by:
//
1. Heat.oven
//
2. Heat.toaster
//
// ────────────────────────────────────────────────────────────────────\
──
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Dependency Injection
36
//
//
Unfortunately our program is now ambiguous. It cannot decide if we should be
making Toast in the oven, Bread in the toaster, or any other combination.
Step 7: Providing Dependencies at Different Levels
This enables other effects that use them to provide their own dependencies of the
same type
runDemo:
val bread =
ZLayer.fromZIO:
Bread.make.provide(Dough.fresh, Heat.oven)
Toast.make.provide(bread, Heat.toaster)
// Making toast
// Toast()
Step 8: Dependencies can fail
TODO Explain .build before using it to demo layer construction
Bread2.fromFriend: ZLayer[Any, String, Bread]
runDemo:
Bread
.eat
.provide:
Bread2.fromFriend
// **Power out**
// **Power out Rez**
Step 9: Dependency Retries
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Dependency Injection
runDemo:
val bread =
Bread2
.fromFriend
.retry:
Schedule.recurs:
3
Bread
.eat
.provide:
bread
// **Power out**
// **Power out**
// Power is on
// Eating bread!
Step 10: Dependency Fallback
runDemo:
val bread =
Bread2
.fromFriend
.orElse:
Bread.storeBought
Toast.make.provide(bread, Heat.toaster)
// **Power out**
// Making toast
// Toast()
Step 11: Layer Retry + Fallback?
Maybe retry on the ZLayer eg. (BreadDough.rancid, Heat.brokenFor10Seconds)
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
37
Dependency Injection
runDemo:
Bread2
.fromFriend
.retry:
Schedule.recurs:
1
.orElse:
Bread.storeBought
.build // TODO Stop using build, if possible
.debug
// **Power out**
// **Power out**
// ZEnvironment(MdocSession::MdocApp::Bread -> Br
Edit This Chapter
Edit This Chapter¹⁰
¹⁰https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/06_Dependency_Injection.md
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
38
The ZIO Type
We need an Answer about this scenario. The scenario requires things and could
produce an error. trait ZIO[Requirements, Error, Answer]
The ZIO trait is at the center of our Effect-oriented world.
trait ZIO[R, E, A]
A trait with 3 type parameters can be intimidating, but each one serves a distinct,
important purpose.
R - The Environment
This is the piece that distinguishes the ZIO monad. It indicates which pieces of the
world we will be observing or changing.
import zio.Console
def print(
msg: String
): ZIO[Console, Nothing, Unit] = ???
This type signature tells us that print needs a Console in its environment to execute.
E - The Error
This parameter tells us how this operation might fail.
The ZIO Type
def parse(
contents: String
): ZIO[Any, IllegalArgumentException, Unit] = ???
A - The Result
This is what our code will return if it completes successfully.
def defaultGreeting()
: ZIO[Any, Nothing, String] = ???
Conversions from standard Scala types
ZIO provides simple interop with may of the built-in Scala data types, namely
•
•
•
•
•
Option
Either
Try
scala.concurrent.Future
Promise
And even some Java types • java.util.concurrent.Future
• AutoCloseable
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
40
The ZIO Type
import scala.concurrent.Future
runDemo:
ZIO.fromFuture(implicit ec =>
Future.successful("Success!")
)
// Success!
runDemo(
ZIO.fromFuture(implicit ec =>
Future.failed(new Exception("Failure :("))
)
)
// java.lang.Exception: Failure :(
Edit This Chapter
Edit This Chapter¹¹
¹¹https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/07_The_ZIO_Type.md
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
41
Built-in Services
Why are Console, Clock, Random, System Built-in?
ZIO[Any, _, _] - System effects are not included in E
Examples of Using System Effects
Some Services are considered fundamental/primitive by ZIO. They are built-in to the
runtime and available to any program.
History - TODO Consider deleting. Not crucial
to the reader.
1.x
Originally, ZIO merely provided default implementations of these services. There
was nothing else special about them. If you wanted to write to the console in your
code, you needed a ZIO[Console, _, _]. If you wanted to write random,
timestamped numbers, accompanied by some system information to the
console, you needed a ZIO[Random with Clock with System with Console, _,
_]. This is maximally informative, but it is also a lot of boilerplate code.
2.x
For ZIO 2.0, the team decided to bake these deeper into the runtime. Now you
can use any of these services without an impact on your method signatures. This
reduces boilerplate, with a trade-off. You can no longer discern which piece of the
Environment/Runtime is being accessed by reading the signature.
Built-in Services
Overriding Builtin Services
Note: This doesn’t work in ZIO 2
object ConsoleSanitized extends Console:
private val socialSecurity =
"\\d{3}-\\d{2}-\\d{4}"
def printLine(
output: String
): ZIO[Any, Nothing, Unit] =
val sanitized =
output.replaceAll(
socialSecurity,
"***-**-****"
)
ConsoleLive.printLine(sanitized)
val leakSensitiveInfo
: ZIO[Console, java.io.IOException, Unit] =
zio
.Console
.printLine("Customer SSN is 000-00-0000")
runDemo(
leakSensitiveInfo.provide(
ZLayer.succeed[Console](ConsoleSanitized)
)
)
Edit This Chapter
Edit This Chapter¹²
¹²https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/08_Builtin_Services.md
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
43
Running Effects
ZIOs are not their result.
They are something that can be executed, that might produce that result. If you have
a ZIO Effect like:
println("A")
// A
ZIO.debug("B")
// res1: ZIO[Any, Nothing, Unit] = Sync(
//
trace = "repl.MdocSession.MdocApp.res1(09_Running_Effects.md:13)",
//
eval = zio.ZIOCompanionVersionSpecific$$Lambda$17027/0x00000008042\
96440@332c4631
// )
println("C")
// C
We will not see the ZIO.debug output.
It only describes something to be done. It is only data (in the ZIO data type), not
instructions. To actually run a ZIO, your program must take the data types and
interpret / run them, executing the logic .
A common mistake when starting with ZIO is trying to return ZIO instances
themselves rather than their result.
println(Random.nextInt)
// Stateful(repl.MdocSession.MdocApp.res3(09_Running_Effects.md:25),zio\
.FiberRef$unsafe$$anon$2$$Lambda$17104/0x000000080431c040@5a51f58d)
This is a mistake because ZIO’s are not their result, they are descriptions of effects
that produce the result.
ZIOs are not automatically executed. The user must determine when/where that
happens.
Running Effects
45
An Option might have a value inside of it, but you can’t safely assume that it does.
Similarly, a ZIO might produce a value, but you have to run it to find out.
You can think of them as recipes for producing a value. You don’t want to return a
recipe from a function, you can only return a value. If it is your friend’s birthday,
they want a cake, not a list of instructions about mixing ingredients and baking.
The defer/direct syntax makes this more explicit
The ZIO Interpreter
Scala compiles code to JVM bytecodes, Similarly ZIO has an interpreter that steps
through and executes your code, much like the JVM interprets JVM bytecodes.
The Zio interpreter is the hidden piece that allows Zio to understand so much
more about the meaning of your code. This includes the ability to decide what to
run concurrently and how to invisibly tune that concurrency–all at runtime. The
interpreter is responsible for deciding when to context-switch between tasks, and is
able to do this because it understands the ZIO code that it’s executing.
The interpreter is also the mechanism that evaluates the various effects described in
the generic type parameters for each ZIO object.
The reason we have the defer directive(method?) in zio-direct is to indicate that this
code will be evaluated by the interpreter later.
Building applications from scratch
One way to run ZIOs is to use a “main method” program (something you can start
in the JVM). However, setting up the pieces needed for this is a bit cumbersome if
done without helpers.
ZIOAppDefault
ZIO provides an easy way to do this with the ZIOAppDefault trait.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Running Effects
46
To use it create a new object that extends the ZIOAppDefault trait and implements
the run method. That method returns a ZIO so you can now give it the example ZIO.debug data: scala object HelloWorld extends zio.ZIOAppDefault:
def run = ZIO.debug: "hello, world"
This can be run on the JVM in the same way as any other class that has a static
void main method.
The ZIOAppDefault trait sets up the ZIO runtime which interprets ZIOs and
provides some out-of-the-box functionality, and then runs the provided data in that
runtime.
If you are learning ZIO, you should start your exploration with ZIOAppDefault. It
is the standard, simplest way to start executing your recipes.
// NOTE We cannot execute invoke main on this
// because it crashes mdoc in the CI process
object RunningZIOs extends ZIOAppDefault:
def run =
// TODO Console/debug don't work
ZIO.attempt:
println:
"Hello World!"
You can provide arbitrary ZIO instances to the run method, as long as you have
provided every piece of the environment. In other words, it can accept ZIO[Any, _,
_].
There is a more flexible ZIOApp that facilitates sharing layers between applications,
but this is advanced and not necessary for most applications.
runDemo
While the ZIOApp* types are great for building real applications, they are not ideal
for demonstrating code for a book. We created the runDemo function to streamline
this use-case. It is a function that takes a ZIO and executes it in a runtime, returning
the result. It uses most of the same techniques that are used in ZIOAppDefault, but
is more single purpose, always immediately executing the ZIO provided to it.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Running Effects
47
runDemo:
ZIO.debug:
"hello, world"
// hello, world
// ()
Testing code
- `runSpec` ?
ZIOSpecDefault
Similar to ZIOAppDefault, there is a ZIOSpecDefault that should be your starting
point for testing ZIO applications. ZIOSpecDefault provides test-specific implementations built-in services, to make testing easier. When you run the same ZIO in these
2 contexts, the only thing that changes are the built-in services provided by the
runtime.
TODO - Decide which scenario to test
import zio.test._
object TestingZIOs extends ZIOSpecDefault:
def spec =
test("Hello Tests"):
defer:
ZIO.console.run
assertTrue:
Random.nextIntBounded(10).run > 10
runSpec
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Running Effects
48
runSpec:
defer:
assertTrue:
Random.nextIntBounded(10).run < 10
// Test: PASSED*
TODO Justify defer syntax over for-comp for multi-statement assertions I think
this example completes the objective TODO Change this to a Console app, where
the logic & testing is more visceral scala runSpec: defer: assertTrue: Random.nextIntBetween(0, 10).run <= 10 && Random.nextIntBetween(10,
20).run <= 20 && Random.nextIntBetween(20, 30).run <= 30 // Test:
PASSED*
runSpec:
for
res1 <- Random.nextIntBetween(0, 10)
res2 <- Random.nextIntBetween(10, 20)
res3 <- Random.nextIntBetween(20, 30)
yield assertTrue:
res1 <= 10 && res2 <= 20 && res3 <= 30
// Test: PASSED*
Consider a Console application: scala val logic = defer: val username
= Console .readLine: "Enter your name\n" .run Console .printLine:
s"Hello $username" .run .orDie If we try to run this code in the same way
as most of the examples in this book, we encounter a problem. scala runDemo:
logic.timeout(1.second) // Defect: scala.NotImplementedError: an
implemen We cannot execute this code and render the results for the book because
it requires interaction with a user. However, even if you are not trying to write
demo code for a book, it is very limiting to need a user at the keyboard for your
program to execute. Even for the smallest programs, it is slow, error-prone, and
boring.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Running Effects
49
runSpec:
defer:
TestConsole
.feedLines:
"Zeb"
.run
logic.run
val capturedOutput: String =
TestConsole.output.run.mkString
val expectedOutput =
s"""|Enter your name
|Hello Zeb
|""".stripMargin
assertTrue:
capturedOutput == expectedOutput
// Test: PASSED*
Interop with existing/legacy code via Unsafe
In some cases your ZIOs may need to be run outside a main program, for example
when embedded into other programs. In this case you can use ZIO’s Unsafe utility
which is called Unsafe to indicate that the code may perform side effects.
To do the same ZIO.debug with Unsafe do:
Unsafe.unsafe { implicit u: Unsafe =>
Runtime
.default
.unsafe
.run:
ZIO.debug:
"hello, world"
.getOrThrowFiberFailure()
}
// hello, world
If needed you can even interop to Scala Futures through Unsafe, transforming the
output of a ZIO into a Future.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Running Effects
50
Web Request Handler
Different enough to warrant its own section? You only ever execute a top-level ZIO,
even if it branches out to multiple other ZIOs
Processing streams of data
Edit This Chapter
Edit This Chapter¹³
¹³https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/09_Running_Effects.md
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Layers
Creating
Composing
Managing and wiring dependencies has been a perennial challenge in software
development.
ZIO provides the ZLayer class to solve many of the problems in this space. If you
pay the modest, consistent cost of constructing pieces of your application as ZLayers,
you will get benefits that scale with the complexity of your project. Consistent with
ZIO itself, ZLayer has 3 type parameters that represent:
• What it needs from the environment
• How it can fail
• What it produces when successful.
With the same type parameters, and many of the same methods, you might be
wondering why we even need a separate data type - why not just use ZIO itself
for our dependencies? The environment type parameter for ZLayer maps directly
to unique, singleton services in your application. The environment type parameter
for ZIO might have many possible instances. ZLayer provides additional behaviors
that are valuable specifically in this domain. Typically, you only want a single
instance of a dependency to exist across your application. This may be to reduce
memory/resource usage, or even to ensure basic correctness. ZLayer output values
are shared maximally by default. They also build in scope management that will
ensure resource cleanup in asynchronous, fallible situations.
==============
Imagine a ServiceX that is needed by 20 diverse functions across your stack. Usually
ServiceX has exactly one instance/implementation should be used throughout your
application.
52
Layers
case class ServiceX():
val retrieveImportantData
: ZIO[Any, Nothing, String] = ???
{{ TODO: Should we show a class-based approach, or just go straight to functions? }}
“‘scala case class UserManagement(serviceX: ServiceX)
case class StatisticsCalculator( serviceX: ServiceX )
case class SecurityModule(serviceX: ServiceX)
case class LandingPage( statisticsCalculator: StatisticsCalculator ) “‘
Historic Approaches
Manual Wiring
case class Application(
userManagment: UserManagement,
securityModule: SecurityModule,
landingPage: LandingPage
)
def construct(): Application =
val serviceX = ServiceX()
Application(
UserManagement(serviceX),
SecurityModule(serviceX),
LandingPage(StatisticsCalculator(serviceX))
)
Even in this tiny example, the downsides are already starting to show.
• We have to copy/paste serviceX numerous times
• We have to manage multiple levels of dependencies. LandingPage and ServiceImplentation have to be manually connected.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
53
Layers
Annotations
Pros - “Easy” in the sense that they do not require much code at the use-site Smoother refactoring, as the injection system will determine what needs to be passed
around
Cons - Does not follow normal control flow or composition - Typically, relies on
some framework-level processing that is not easily controlled by the user
Traits
Before looking at the official ZIO implementation, we will create a simpler version.
TODO: Decide whether explaining this pattern belongs in a standalone section. It
is important in isolation, but probably hard to appreciate without a use-case, and
Console is likely the simplest example.
The pattern used here is fundamental to designing composable, ergonomic ZIO
Services.
1. Create a trait with the needed functions.
2. Create an implementation of the trait.
3. (Optional) Provide implementation instance in a Layer as a object field - live.
We will go through each of these steps in detail in this chapter, and more concisely
in the rest. Steps 1 and 2 steps will be familiar to many programmers. Steps 3 is less
familiar, and might be harder to appreciate. We endeavor in the following chapters
to make a compelling case for them. If we succeed, the reader will use them when
creating their own Effects.
One: Create the trait
This trait represents effectful code that we need to interact with.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
54
Layers
trait Console:
def printLine(
output: String
): ZIO[Any, Nothing, Unit]
Two: Create the implementation
object ConsoleLive extends Console:
def printLine(
output: String
): ZIO[Any, Nothing, Unit] =
// TODO Get this working without Predef
ZIO.succeed(Predef.println(output))
case class Logic(console: Console):
val invoke: ZIO[Any, Nothing, Unit] =
defer:
console.printLine("Hello").run
console.printLine("World").run
However, providing dependencies to the logic is still tedious.
runDemo:
Logic:
ConsoleLive
.invoke
// Hello
// World
// ()
Three: Create object Effect.live field
Rather than making each caller wrap our instance in a Layer, we can do that a single
time in our companion.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
55
Layers
object Console:
val live: ZLayer[Any, Nothing, Console] =
ZLayer.succeed[Console]:
ConsoleLive
More important than removing repetition - using 1 unique Layer instance per type
allows us to share it across our application.
Now executing our code is as simple as describing it.
runDemo:
ZIO
.serviceWithZIO[Logic]:
_.invoke
.provide(
Console.live,
ZLayer.fromFunction(Logic.apply _)
)
// Hello
// World
// ()
Edit This Chapter
Edit This Chapter¹⁴
Automatically attached experiments.
These are included at the end of this chapter because their package in the experiments
directory matched the name of this chapter. Enjoy working on the code with full
editor capabilities :D
experiments/src/main/scala/layers/Festival.scala
¹⁴https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/10_Layers.md
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
56
Layers
package layers
import zio.ZIO.debug
case class Toilets()
val toilets = activityLayer(entity = Toilets())
case class Stage()
val stage: ZLayer[Any, Nothing, Stage] =
activityLayer(
entity = Stage(),
setupSteps = ("Transporting", 2.seconds),
("Building", 4.seconds)
)
case class Permit()
val permit: ZLayer[Any, Nothing, Permit] =
activityLayer(
entity = Permit(),
setupSteps = ("Legal Request", 5.seconds)
)
def activityLayer[T: Tag](
entity: T,
setupSteps: (String, Duration)*
) =
ZLayer.scoped(
ZIO.acquireRelease(
defer:
ZIO
.debug:
entity.toString + " ACQUIRE"
.run
ZIO
.foreach(setupSteps):
case (name, duration) =>
activity(
entity.toString,
name,
duration
)
.run
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
57
Layers
entity
)(_ => debug(entity.toString + " RELEASE"))
)
def activity(
entity: String,
name: String,
duration: Duration
) =
defer:
debug:
s"$entity: BEGIN $name"
.run
debug:
s"$entity: END $name"
.delay(duration)
.run
case class Venue(stage: Stage, permit: Permit)
val venue = ZLayer.fromFunction(Venue.apply)
case class SoundSystem()
val soundSystem
: ZLayer[Any, Nothing, SoundSystem] =
ZLayer.succeed(SoundSystem())
case class Festival(
toilets: Toilets,
venue: Venue,
soundSystem: SoundSystem,
security: Security
)
val festival =
ZLayer.scoped {
ZIO.acquireRelease {
defer:
debug("FESTIVAL: We are all set!").run
Festival(
ZIO.service[Toilets].run,
ZIO.service[Venue].run,
ZIO.service[SoundSystem].run,
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
58
Layers
ZIO.service[Security].run
)
} { _ =>
debug(
"FESTIVAL: Good job, everyone. Close it down!"
)
}
}
case class Security(toilets: Toilets)
val security
: ZLayer[Toilets, Nothing, Security] =
ZLayer.scoped {
ZIO.acquireRelease {
defer:
debug("SECURITY: Ready").run
Security(ZIO.service[Toilets].run)
} { _ =>
debug("SECURITY: Going home")
}
}
Testing Unpredictable Effects
Effects need access to external systems thus are unpredictable.
Tests are ideally predictable so how do we write tests for effects that are predictable?
With ZIO we can replace the external systems with predictable ones when running
our tests.
With ZIO Test we can use predictable replacements for the standard systems effects
(Clock, Random, Console, etc).
Random
An example of this is Random numbers. Randomness is inherently unpredictable. But
in ZIO Test, without changing our Effects we can change the underlying systems with
something predictable:
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
59
Layers
import zio.test.TestRandom
import zio.test.assertTrue
runSpec:
defer:
TestRandom.feedInts(1, 2).run
val result1 = Random.nextInt.run
val result2 = Random.nextInt.run
assertTrue(result1 == 1, result2 == 2)
// Test: PASSED*
The Random Effect uses an injected something which when running the ZIO uses
the system’s unpredictable random number generator. In ZIO Test the Random
Effect uses a different something which can predictably generate “random” numbers.
TestRandom provides a way to define what those numbers are. This example feeds
in the Ints 1 and 2 so the first time we ask for a random number we get 1 and the
second time we get 2.
Anything an effect needs (from the system or the environment) can be substituted
in tests for something predictable. For example, an effect that fetches users from a
database can be simulated with a predictable set of users instead of having to setup
a test database with predictable users.
Time
Even time can be simulated as using the clock is an effect.
import zio.test.*
runSpec:
val thingThatTakesTime = ZIO.sleep(2.seconds)
defer:
val fork =
thingThatTakesTime
.timeout(1.second)
.fork
.run
TestClock.adjust(2.seconds).run
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
60
Layers
val result = fork.join.run
assertTrue(result.isEmpty)
// Test: PASSED*
By default in ZIO Test, the clock does not change unless instructed to. Calling a time
based effect like timeout would hang indefinitely with a warning like: Warning:
A test is using time, but is not advancing the test clock, which
may result in the test hanging. Use TestClock.adjust to manually
advance the time.
To test time based effects we need to fork those effects so that then we can adjust
the clock. After adjusting the clock, we can then join the effect where in this case
the timeout has then been reached causing the effect to return a None.
Using a simulated Clock means that we no longer rely on real-world time for time.
So this example runs in milliseconds of real-world time instead of taking an actual
1 second to hit the timeout. This way our time-based tests run much more quickly
since they are not based on actual system time. They are also more predictable as the
time adjustments are fully controlled by the tests.
Targeting Error-Prone Time Bands
Using real-world time also can be error prone because effects may have unexpected
results in certain time bands. For instance, if you have code that gets the time and it
happens to be 23:59:59, then after some operations that take a few seconds, you get
some database records for the current day, those records may no longer be the day
associated with previously received records. This scenario can be very hard to test
for when using real-world time. When using a simulated clock in tests, you can write
tests that adjust the clock to reliably reproduce the condition.
Todo: The example could be clarified.
assertTrue
In this example we utilize ZIO Test’s assertTrue which provides a non-DSL approach to writing assertions while preserving the negative condition error messages.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
61
Layers
Typically using assertTrue doesn’t give helpful errors, ie true != false, but ZIO
Test provides helpful details for why the assertion was false.
Todo: Can we display with mdoc the nice assertTrue fail message? Todo:
More compelling assertTrue failure
runSpec:
assertTrue(Some("asdf") == None)
// Test: FAILED
Test Aspects
We have seen how to add capabilities and behaviors ZIO’s enabled by manipulating
them as values. We can add behaviors to ZSpecs that are more specific to testing.
Overriding Builtin Services
When testing ZIOs we can provide user-defined Environment types by using .provide. However, the Built-in Services are not part of the Environment, so we need a
different way to override them. By default, tests will get Test versions of the Built-in
Services.
runSpec:
defer:
val thingThatTakesTime = ZIO.sleep(2.seconds)
val result =
defer:
val fork =
thingThatTakesTime
.fork
.run
TestClock.adjust(10.seconds).run
fork.join.run
.timed
.run
println(result)
assertCompletes
// (PT10S,())
// Test: PASSED*
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
62
Layers
runSpec(
defer:
val thingThatTakesTime = ZIO.sleep(2.seconds)
val result =
defer:
val fork =
thingThatTakesTime
.fork
.run
TestClock.adjust(10.seconds).run
fork.join.run
.timed
.run
println(result)
assertCompletes
,
TestAspect.withLiveClock
)
// (PT2.000875687S,())
// Test: PASSED*
Injecting Behavior before/after/around
runSpec(
defer:
println("During test")
assertCompletes
,
TestAspect.around(
ZIO.debug("ZIO IO, before"),
ZIO.succeed(println("plain IO, after")),
)
)
// During test
// plain IO, after
// Test: PASSED*
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
63
Layers
Flakiness
Commonly, as a project grows, the supporting tests become more and more flaky.
This can be caused by a number of factors:
• The code is using shared, live services Shared resources, such as a database or
a file system, might be altered by other processes. These could be other tests in
the project, or even unrelated processes running on the same machine.
• The code is not thread safe Other processes running simultaneously might alter
the expected state of the system.
• Resource limitations A team of engineers might be able to successfully run the
entire test suite on their personal machines. However, the CI/CD system might
not have enough resources to run the tests triggered by everyone pushing to
the repository. Your tests might be occasionally failing due to timeouts or lack
of memory.
runSpec(
defer:
assertTrue:
Random.nextBoolean.run
,
TestAspect.withLiveRandom,
TestAspect.flaky
)
// Test: PASSED*
Forbidding
• nonflaky
We might have sections of the code that absolutely must be reliable, and we want to
express that in our tests. By using nonFlaky we can ensure that the test will fail if it
is flaky, by hammering it with repeated executions. You can dial up the number of
iterations to match your reliability expectations.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
64
Layers
runSpec(
defer:
assertTrue:
Random.nextInt.run != 42
,
TestAspect.withLiveRandom,
TestAspect.nonFlaky
)
// Test: PASSED*
Tolerating/Flagging
In a perfect world, we would fix the underlying issues immediately. However, under
real world constraints, we may need to tolerate flakiness for a time. ZIO Test provides
a few ways to do this.
• flaky This is your goto, easily-applied solution for accommodating legacy
flakiness in your codebase. For the average, undiagnosed “This test fails
sometimes” circumstance, this is the right starting point.
• eventually When you have a test that is flaky, but you don’t know what a
reasonable retry behavior is, use eventually. It’s tolerant of any number of
failures, and will just keep retrying until interrupted by other mechanisms.
Platform concerns
Configuration / Environment
• TestAspect.ifEnv/ifProp
Time
#### Measuring Time Since there is already a .timed method available directly on
ZIO instances, it might seem redundant to have a timed TestAspect. However, they
are distinct enough to justify their existence. ZIOs .timed methods changes the result
type of your code by adding the duration to a tuple in the result. This is useful, but
requires the calling code to handle this new result type. TestAspect.timed is a
non-invasive way to measure the duration of a test. The timing information will be
managed behind the scenes, and printed in the test output, without changing any
other behavior.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
65
Layers
Restricting Time
Sometimes, it’s not enough to simply track the time that a test takes. If you have
specific Service Level Agreements (SLAs) that you need to meet, you want your
tests to help ensure that you are meeting them. However, even if you don’t have
contracts bearing down on you, there are still good reasons to ensure that your tests
complete in a timely manner. Services like GitHub Actions will automatically cancel
your build if it takes too long, but this only happens at a very coarse level. It simply
kills the job and won’t actually help you find the specific test responsible.
A common technique is to define a base test class for your project that all of your
tests extend. In this class, you can set a default upper limit on test duration. When a
test violates this limit, it will fail with a helpful error message.
This helps you to identify tests that have completely locked up, or are taking an
unreasonable amount of time to complete.
For example, if you are running your tests in a CI/CD pipeline, you want to ensure
that your tests complete quickly, so that you can get feedback as soon as possible. you
can use TestAspect.timeout to ensure that your tests complete within a certain
time frame.
What should run?
It would be great if all our tests could run & pass at every moment in time, but
there are times when it’s not feasible. If you are doing Test-Driven Development, you
don’t want the build to be broken until you are completely finished implementing the
feature. If you are rewriting a significant part of your project, you already know there
are going to be test failures until you are finished. Traditionally, we comment out the
tests in these situations. However, this can lead to a lot of noise in the codebase, and
it’s easy to forget to uncomment the tests when you are done. TestAspects provide
a better way to handle this.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
66
Layers
runSpec(
defer:
assertNever:
"Not implemented. Do not run"
)
// Test: FAILED
runSpec(
defer:
assertNever:
"Not implemented. Do not run"
,
TestAspect.ignore
)
// Test: PASSED*
Defining a base test class for your project
Edit This Chapter
Edit This Chapter¹⁵
¹⁵https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/11_Testing_Effects.md
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Errors
1. Creating & Handling
2. Error composability
3. Retry
In a language that cannot throw, following the execution path is simple, following 2
basic rules:
- At a branch, execute only the first match
- Otherwise, Read everything from left-to-right, top-to-bottom,
Once you add throw, the rules are more complicated
t
At a branch, execute only the first match
Otherwise, Read everything from left-to-right, top-to-bottom,
Unless we `throw`, which means immediately jumping through a differen\
dimension away from the code you're viewing
Hello Failures
If you are not interested in the discouraged ways to handle errors, and just want to
see the ZIO approach, jump down to ZIO Error Handling
Historic approaches to Error-handling
In the past, some programs have thrown exceptions to indicate failures. Imagine a
program that displays the local temperature the user based on GPS position and a
network call. There are distinct levels of problems in any given program. They require
different types of handling by the programmer.
Temperature: 30 degrees
class GpsException()
extends RuntimeException
class NetworkException() extends RuntimeException
enum Scenario:
case Success,
NetworkError,
GPSError
def render(value: String) =
s"Temperature: $value"
def calculateTemp(behavior: Scenario): String =
behavior match
case Scenario.GPSError =>
throw GpsException()
case Scenario.NetworkError =>
throw NetworkException()
case Scenario.Success =>
"35 degrees"
Hello Failures
69
def currentTemperatureUnsafe(
behavior: Scenario
): String =
render:
calculateTemp:
behavior
currentTemperatureUnsafe:
Scenario.Success
// res0: String = "Temperature: 35 degrees"
On the happy path, everything looks as desired. If the network is unavailable, what is
the behavior for the caller? This can take many forms. If we don’t make any attempt
to handle our problem, the whole program blows up and shows the gory details to
the user.
// Note - Can't make this output prettier/simpler because it's *not* us\
ing ZIO
currentTemperatureUnsafe:
Scenario.NetworkError
// repl.MdocSession$MdocApp$NetworkException
//
at repl.MdocSession$MdocApp.calculateTemp(12_Errors.md:28)
//
at repl.MdocSession$MdocApp.currentTemperatureUnsafe(12_Errors.md:4\
0)
//
at repl.MdocSession$MdocApp.$init$$$anonfun$1(12_Errors.md:53)
We could take the bare-minimum approach of catching the Exception and returning
null:
def currentTemperatureNull(
behavior: Scenario
): String =
render:
try
calculateTemp:
behavior
catch
case ex: RuntimeException =>
null
currentTemperatureNull:
Scenario.NetworkError
// res1: String = "Temperature: null"
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Hello Failures
70
This is slightly better, as the user can at least see the outer structure of our UI element,
but it still leaks out code-specific details world.
Maybe we could fallback to a sentinel value, such as 0 or -1 to indicate a failure?
def currentTemperature(
behavior: Scenario
): String =
render:
try
calculateTemp:
behavior
catch
case ex: RuntimeException =>
"-1 degrees"
currentTemperature:
Scenario.NetworkError
// res2: String = "Temperature: -1 degrees"
Clearly, this isn’t acceptable, as both of these common sentinel values are valid
temperatures. We can take a more honest and accurate approach in this situation.
def currentTemperature(
behavior: Scenario
): String =
render:
try
calculateTemp:
behavior
catch
case ex: RuntimeException =>
"Unavailable"
currentTemperature:
Scenario.NetworkError
// res3: String = "Temperature: Unavailable"
We have improved the failure behavior significantly; is it sufficient for all cases?
Imagine our network connection is stable, but we have a problem in our GPS
hardware. In this situation, do we show the same message to the user? Ideally, we
would show the user a distinct message for each scenario. The Network issue is
transient, but the GPS problem is likely permanent.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Hello Failures
71
def currentTemperature(
behavior: Scenario
): String =
try
render:
calculateTemp:
behavior
catch
case ex: NetworkException =>
"Network Unavailable"
case ex: GpsException =>
"GPS problem"
currentTemperature:
Scenario.NetworkError
// res4: String = "Network Unavailable"
currentTemperature:
Scenario.GPSError
// res5: String = "GPS problem"
Wonderful! We have specific messages for all relevant error cases. However, this still
suffers from downsides that become more painful as the codebase grows.
• The signature of currentTemperature does not alert us that it might fail
• If we realize it can fail, we must dig through the implementation to discover
the multiple failure values
• We never have certainty about the failure paths of our full application, or any
subset of it.
{{ TODO Tear apart exceptions more }}
Encountering an error during a function call generally means two things:
1. You can’t continue executing the function in the normal fashion.
2. You can’t return a normal result.
Many languages use exceptions for handling errors. An exception throws out of the
current execution path to locate a user-written handler to deal with the error. There
are two goals for exceptions:
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Hello Failures
72
1. Separate error-handling code from “success-path” code, so the success-path
code is easier to understand and reason about.
2. Reduce redundant error-handling code by handling associated errors in a single
place.
Exceptions have problems:
1. They can be “swallowed.” Just because code throws an exception, there’s no
guarantee that issue will be dealt with.
2. They can lose important information. Once an exception is caught, it is
considered to be “handled,” and the program doesn’t need to retain the failure
information.
3. They aren’t typed. Java’s checked exceptions provide a small amount of type
information, but it’s not that helpful compared to a full type system. Unchecked
exceptions provide no information at all.
4. Because they are handled dynamically, the only way to ensure your program
won’t crash is by testing it through all possible execution paths. A staticallytyped error management solution can ensure—at compile time—that all errors
are handled.
5. They don’t scale. {{Need to think about this more to make the case.}}
6. Hard to reason about. {{Also need to make this case}}
7. Difficult or impossible to retry an operation if it fails. Java {{and Scala?}} use
the “termination” model of exception handling. This assumes the error is so
critical there’s no way to get back to where the exception occurred. If you’re
performing an operation that you’d like to retry if it fails, exceptions don’t help
much.
Exceptions were a valiant attempt to produce a consistent error-reporting interface,
and they are definitely better than what’s in C. But they don’t end up solving the
problem very well, and you just don’t know what you’re going to get when you use
exceptions.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Hello Failures
What’s wrong with Try?
ADTS as another step forward
ZIO Error Handling
Now we will explore how ZIO enables more powerful, uniform error-handling.
TODO {{Update verbiage now that ZIO section is first}}
• ZIO Error Handling
• Wrapping Legacy Code
ZIO-First Error Handling
def getTemperatureZ(behavior: Scenario): ZIO[
Any,
GpsException | NetworkException,
String
] =
behavior match
case Scenario.GPSError =>
ZIO.fail:
GpsException()
case Scenario.NetworkError =>
// TODO Use a non-exceptional error
ZIO.fail:
NetworkException()
case Scenario.Success =>
ZIO.succeed:
"35 degrees"
runDemo:
getTemperatureZ:
Scenario.Success
// 35 degrees
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
73
Hello Failures
74
// TODO make MDoc:fail adhere to line limits?
runDemo:
getTemperatureZ:
Scenario.Success
.catchAll:
case ex: NetworkException =>
ZIO.succeed:
"Network Unavailable"
// error:
// match may not be exhaustive.
//
// It would fail on pattern case: _: GpsException
//
TODO Demonstrate ZIO calculating the error types without an explicit annotation
being provided
runDemo:
getTemperatureZ:
Scenario.GPSError
// repl.MdocSession$MdocApp$GpsException
Wrapping Legacy Code
If we are unable to re-write the fallible function, we can still wrap the call We are
re-using the calculateTemp
{{TODO }}
def calculateTempWrapped(
behavior: Scenario
): ZIO[Any, Throwable, String] =
ZIO.attempt:
calculateTemp:
behavior
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Hello Failures
75
def displayTemperatureZWrapped(
behavior: Scenario
): ZIO[Any, Nothing, String] =
calculateTempWrapped:
behavior
.catchAll:
case ex: NetworkException =>
ZIO.succeed:
"Network Unavailable"
case ex: GpsException =>
ZIO.succeed:
"GPS problem"
runDemo:
displayTemperatureZWrapped:
Scenario.Success
// 35 degrees
runDemo:
displayTemperatureZWrapped:
Scenario.NetworkError
// Network Unavailable
This is decent, but does not provide the maximum possible guarantees. Look at what
happens if we forget to handle one of our errors.
def getTemperatureZGpsGap(
behavior: Scenario
): ZIO[Any, Nothing, String] =
calculateTempWrapped:
behavior
.catchAll:
case ex: NetworkException =>
ZIO.succeed:
"Network Unavailable"
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Hello Failures
76
runDemo:
getTemperatureZGpsGap:
Scenario.GPSError
// Defect: GpsException
The compiler does not catch this bug, and instead fails at runtime. Take extra care
when interacting with legacy code, since we cannot automatically recognize these
situations at compile time. We have 2 options in these situations.
First, we can provide a fallback case that will report anything we missed:
def getTemperatureZWithFallback(
behavior: Scenario
): ZIO[Any, Nothing, String] =
calculateTempWrapped:
behavior
.catchAll:
case ex: NetworkException =>
ZIO.succeed:
"Network Unavailable"
case other =>
ZIO.succeed:
"Error: " + other
runDemo:
getTemperatureZWithFallback:
Scenario.GPSError
// Error: repl.MdocSession$MdocApp$GpsException
This lets us avoid the most egregious gaps in functionality, but it does not take full
advantage of ZIO’s type-safety.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Hello Failures
77
def getTemperatureZAndFlagUnhandled(
behavior: Scenario
): ZIO[Any, GpsException, String] =
calculateTempWrapped:
behavior
.catchSome:
case ex: NetworkException =>
ZIO.succeed:
"Network Unavailable"
// TODO Eh, find a better version of this.
.mapError(_.asInstanceOf[GpsException])
runDemo:
getTemperatureZAndFlagUnhandled:
Scenario.GPSError
// repl.MdocSession$MdocApp$GpsException
{{TODO show catchSome}}
Note: The following is copy&pasted and needs work
Unions AKA Sum Types AKA Enums AKA Ors
Note - Avoid official terminology in most prose. Just say “And”/”Or” where appropriate.
Scala 3 automatically aggregates the error types by synthesizing an anonymous sum
type from the combined errors.
Functions usually transform the Answer from one type to another type. Errors often
aggregate.
trait Error1
trait Error2
def failableFunction()
: ZIO[Any, Error1 | Error2, Unit] = ???
Consider 2 error types
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Hello Failures
78
trait UserNotFound
trait PermissionError
In the type system, the most recent ancestor between them is Any.
Unfortunately, you cannot make any meaningful decisions based on this type.
graph TD;
UserNotFound-->Nothing;
PermissionError-->Nothing;
We need a more specific way to indicate that our code can fail with either of these
types. The | (or) tool provides maximum specificity without the need for inheritance.
TODO Figure out how to use pipe symbol in Mermaid
graph TD;
UserNotFound-->UserNotFound_OR_PermissionError;
PermissionError-->UserNotFound_OR_PermissionError;
UserNotFound-->Nothing;
PermissionError-->Nothing;
Often, you do not care that Nothing is involved at all. The mental model can be
simply:
graph TD;
UserNotFound-->UserNotFound_OR_PermissionError;
PermissionError-->UserNotFound_OR_PermissionError;
case class UserService()
trait User
trait SuperUser
def getUser(
userId: String
): ZIO[UserService, UserNotFound, User] = ???
def getSuperUser(
user: User
): ZIO[UserService, PermissionError, SuperUser] =
???
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Hello Failures
def loginSuperUser(userId: String): ZIO[
UserService,
UserNotFound | PermissionError,
SuperUser
] =
defer:
val basicUser = getUser(userId).run
getSuperUser(basicUser).run
trait Status
trait NetworkService
def statusOf(
user: User
): ZIO[NetworkService, UserNotFound, Status] =
???
def check(userId: String): ZIO[
UserService & NetworkService,
UserNotFound,
Status
] =
defer:
val user =
getUser:
userId
.run
statusOf:
user
.run
Edit This Chapter
Edit This Chapter¹⁶
¹⁶https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/12_Errors.md
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
79
Concurrency High Level
1. forEachPar, collectAllPar
TODO
Prose
scala def sleepThenPrint( d: Duration ): ZIO[Any,
java.io.IOException, Duration] = defer: ZIO.sleep(d).run
ZIO.debug(s"${d.render} elapsed").run d
runDemo:
ZIO.foreach(Seq(2, 1)): i =>
sleepThenPrint(i.seconds)
// List(PT2S, PT1S)
runDemo:
ZIO.foreachPar(Seq(2, 1)): i =>
sleepThenPrint(i.seconds)
// List(PT2S, PT1S)
runDemo:
defer:
val durations =
ZIO
.collectAllPar:
Seq(
sleepThenPrint(2.seconds),
sleepThenPrint(1.seconds)
)
.run
val total =
durations.fold(Duration.Zero)(_ + _).render
Console
.printLine:
total
.run
// ()
Concurrency High Level
def slowFailableRandom(duration: Duration) =
defer:
val randInt =
Random.nextIntBetween(0, 100).run
ZIO.sleep(duration).run
ZIO
.when(randInt < 10)(
ZIO.fail("Number is too low")
)
.run
duration
// Massive example
runDemo:
defer:
val durations =
ZIO
.collectAllSuccessesPar:
Seq
.fill(1_000)(1.seconds)
.map(duration =>
slowFailableRandom(duration)
)
.run
durations.fold(Duration.Zero)(_ + _).render
// 14 m 59 s
zipPar, zipWithPar
validateWithPar?
withParallelism
Edit This Chapter
Edit This Chapter¹⁷
¹⁷https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/13_Concurrency_High_Level.md
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
81
Concurrency Low Level
1. Fork join
2. Throwaway reference to STM
TODO
Prose
scala def sleepThenPrint( d: Duration ): ZIO[Any,
java.io.IOException, Duration] = defer { ZIO.sleep(d).run
println(s"${d.render} elapsed") d }
runDemo(
defer {
val f1 = sleepThenPrint(2.seconds).fork.run
val f2 = sleepThenPrint(1.seconds).fork.run
f1.join.run
f2.join.run
}
)
// 1 s elapsed
// 2 s elapsed
// PT1S
Edit This Chapter
Edit This Chapter¹⁸
¹⁸https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/14_Concurrency_Low_Level.md
Concurrency Interruption
Why Interruption Is Necessary Throughout
the Stack
In order for the Runtime to operate and provide the super powers of ZIO, it needs to
be able to interrupt running workflows without resource leaks.
Timeout
## Race
.withFinalizer
## .zipWithPar ## .aquireRelease effects are uninterruptable ## .fromFutureInterrupt
Uninterruptable
// This is duplicate code
def sleepThenPrint(
d: Duration
): ZIO[Any, java.io.IOException, Duration] =
defer:
ZIO
.sleep:
d
.run
println:
s"${d.render} elapsed"
d
Concurrency Interruption
runDemo:
sleepThenPrint:
2.seconds
.race:
sleepThenPrint:
1.seconds
// 1 s elapsed
// PT1S
Future Cancellation
We show that Future’s are killed with finalizers that never run
import scala.concurrent.Future
runDemo:
ZIO
.fromFuture:
Future:
try
println:
"Starting operation"
Thread.sleep:
500
println:
"Ending operation"
finally
println:
"Cleanup"
.timeout:
25.millis
// Starting operation
// None
Edit This Chapter
Edit This Chapter¹⁹
¹⁹https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/15_Concurrency_Interruption.md
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
84
Concurrency State
1. Ref
2. Thundering Herds
Functional programmers often sing the praises of immutability. The advantages
are real and numerous. However, it is easy to find situations that are intrinsically
mutable.
• How many people are currently inside a building?
• How much fuel is in your car?
• How much money is in your bank account?
Rather than avoiding mutability entirely, we want to avoid unprincipled, unsafe
mutability. If we codify and enumerate everything that we need from Mutability,
then we can wield it safely. Required Operations:
• Update the value
• Read the current value
These are both effectful operations.
trait RefZ[A]:
def get: ZIO[Any, Nothing, A]
def update(a: A => A): ZIO[Any, Nothing, Unit]
In order to confidently use this, we need certain guarantees about the behavior:
• The underlying value cannot be changed during a read
• Multiple writes cannot happen concurrently, which would result in lost updates
Less obviously, we also need to create the Mutable reference itself. We are changing
the world, by creating a space that we can manipulate. This operation can live in the
companion object:
Concurrency State
86
object RefZ:
def make[A](a: A): ZIO[Any, Nothing, RefZ[A]] =
???
Unreliable Counting
val unreliableCounting =
var counter = 0
val increment =
ZIO.succeed:
counter = counter + 1
defer:
ZIO
.foreachParDiscard(Range(0, 100000)): _ =>
increment
.run
// It's not obvious to the reader why
// we need to wrap counter in .succeed
"Final count: " + ZIO.succeed(counter).run
runDemo:
unreliableCounting
// Final count: 98239
Due to the unpredictable nature of shared mutable state, we do not know exactly
what the final count above is. Each time we publish a copy of this book, the code is reexecuted and a different wrong result is generated. However, conflicts are extremely
likely, so some of our writes get clobbered by others, and we end up with less than
the expected 100,000. Ultimately, we lose information with this approach.
TODO Consider making a diagram parallel writes
Performing our side effects inside ZIO’s does not magically make them safe. We need
to fully embrace the ZIO components, utilizing Ref for correct mutation.
Reliable Counting
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Concurrency State
87
lazy val reliableCounting =
def incrementCounter(counter: Ref[Int]) =
counter.update:
_ + 1
defer:
val counter = Ref.make(0).run
ZIO
.foreachParDiscard(Range(0, 100000)): _ =>
incrementCounter:
counter
.run
"Final count: " + counter.get.run
runDemo:
reliableCounting
// Final count: 100000
Now we can say with full confidence that our final count is 100000. Additionally,
these updates happen without blocking. This is achieved through a strategy called
“Compare & Swap”, which we will not cover in detail. TODO Link/reference supplemental reading
Although there are significant advantages; a basic Ref is not the solution for
everything. We can only pass pure functions into update. The API of the plain
Atomic Ref steers you in the right direction by not accepting ZIOs as parameters to
any of its methods. To demonstrate why this restriction exists, we will deliberately
undermine the system by sneaking in a side effect. First, we will create a helper
function that imitates a long-running calculation.
def expensiveCalculation() =
Thread.sleep:
35
Our side effect will be a mock alert that is sent anytime our count is updated: scala
def sendNotification() = println: "Alert: updating count!"
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Concurrency State
88
def update(counter: Ref[Int]) =
counter.update: previousValue =>
expensiveCalculation()
sendNotification()
previousValue + 1
runDemo:
defer:
val counter = Ref.make(0).run
ZIO
.foreachParDiscard(Range(0, 4)): _ =>
update(counter)
.run
"Final count: " + counter.get.run
// Alert: updating count!
// Alert: updating count!
// Alert: updating count!
// Alert: updating count!
// Alert: updating count!
// Alert: updating count!
// Alert: updating count!
// Alert: updating count!
// Alert: updating count!
// Alert: updating count!
// Final count: 4
What is going on?! Previously, we were losing updates because of unsafe mutability.
Now, we have the opposite problem! We are sending far more alerts than intended,
even though we can see that our final count is 4.
TODO This section will need significant attention and polish
Now we must consider the limitations of the “Compare & Swap” system. It achieves
lock-free performance by letting each fiber freely make their updates, and then doing
a last-second check to see if the underlying value changed during its update. If the
value has not changed, the update is made. If it has changed, then the entire function
that was passed into update is re-executed until it completes with a stable value. The
higher the parallelism, or the longer the operation takes, the higher the likelihood of
a compare-and-swap retry.
This retry behavior is safe with pure functions, which can be executed an arbitrary
number of times. However, it is completely inappropriate for effects, which should
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Concurrency State
89
only be executed a single time. For these situations, we need a specialized variation
of Ref
Ref.Synchronized
Ref.Synchronized guarantees only a single execution of the update body and any
of the effects contained inside. The only change required is replacing Ref.make with
Ref.Synchronized.make
val sideEffectingUpdatesSync =
defer:
val counter = Ref.Synchronized.make(0).run
ZIO
.foreachParDiscard(Range(0, 4)): _ =>
counter.update: previousValue =>
expensiveCalculation()
sendNotification()
previousValue + 1
.run
"Final count: " + counter.get.run
runDemo:
sideEffectingUpdatesSync
// Alert: updating count!
// Alert: updating count!
// Alert: updating count!
// Alert: updating count!
// Final count: 4
Now we see exactly the number of alerts that we expected. This correctness comes
with a cost though, as the name of this type implies. Each of your updates will run
sequentially, despite initially launching them all in parallel. This is the only known
way to avoid retries. Try to structure your code to minimize the coupling between
effects and updates, and use this type only when necessary.
Edit This Chapter
Edit This Chapter²⁰
²⁰https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/16_Concurrency_State.md
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Repeats
## Number of occurances ## Spacing/timing of repeats ## Until a condition is met ##
While a condition is met ## Until a failure occurs
Edit This Chapter
Edit This Chapter²¹
²¹https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/17_Repeats.md
Resources
1. Open / Close around an Effect
Resources are finite / large overhead allocatable pools of things: - Connections File handles - Player Slots (Rule based or Fractional Computational Power - The
usefulness of the game degrades at some scale) - STM? Is it based on ZManaged?
Connection with dining philosophers
Externalizes the resource management so that the logic that acts on the resource can
be reused, refactored, composed.
Assembly of resources works the same as a single resource. If a resource is more than
1 resource, the logic acting on any / all resources doesn’t have to know what cleanup.
Similarly, the logic is unconcerned with the ability for all needed resources to be
available. Logic is only ever to be applied when all resources are available.
Edit This Chapter
Edit This Chapter²²
²²https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/18_Resources.md
Logging
Edit This Chapter
Edit This Chapter²³
Automatically attached experiments.
These are included at the end of this chapter because their package in the experiments
directory matched the name of this chapter. Enjoy working on the code with full
editor capabilities :D
experiments/src/main/scala/logging/Logging.scala
package logging
import zio.logging.*
import zio.logging.LogFormat.{
label,
line,
quoted,
text
}
object Logging extends ZIOAppDefault:
lazy val minimal: LogFormat =
label("message", quoted(line)).highlight
lazy val locationLogger: LogFormat =
location(new WackyGps) |-|
label("message", quoted(line)).highlight
²³https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/19_Logging.md
93
Logging
lazy val coloredLogger =
Runtime.removeDefaultLoggers >>>
consoleLogger(
ConsoleLoggerConfig(
//
LogFormat.colored
locationLogger,
LogFilter.logLevel(LogLevel.Info)
)
)
def run = ZIO.log("Hi").provide(coloredLogger)
def location(gps: Gps): LogFormat =
text {
gps.currentLocation().toString
}
end Logging
enum Continent:
case NorthAmerica,
SouthAmerica,
Europe,
Asia,
Antarctica,
Australia,
Africa
trait Gps:
def currentLocation(): Continent
class WackyGps extends Gps:
def currentLocation(): Continent =
Continent
.values
.apply(
scala
.util
.Random
.between(0, Continent.values.length)
)
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Configuration
Changing things based on the running environment.
CLI Params
Config Files
Environment Variables
ZIO Config
Historic Approach
Environment Variables are a common way of providing dynamic and/or sensitive
data to your running application. A basic use-case looks like this:
val apiKey = sys.env.get("API_KEY")
// apiKey: Option[String] = Some(value = "SECRET_API_KEY")
This seems rather innocuous; however, it can be an annoying source of problems as
your project is built and deployed across different environments. Given this API:
case class Hotel(name: String)
case class Error(msg: String)
To augment the built-in environment function, we will create a wrapper.
Configuration
95
def envRequiredUnsafe(
variable: String
): Either[Error, String] =
sys
.env
.get(variable)
.toRight(Error("Unconfigured Environment"))
toRight is an Option method that turns the Option into an Either.
Our business logic now looks like this:
def fancyLodgingUnsafe(
hotelApi: HotelApi
): Either[Error, Hotel] =
for
apiKey <- envRequiredUnsafe("API_KEY")
hotel <- hotelApi.cheapest("90210", apiKey)
yield hotel
When you look up an Environment Variable, you are accessing information that was
not passed into your function as an explicit argument. Now we will simulate running
the function with the same arguments in 3 different environments.
Your Machine:
fancyLodgingUnsafe(HotelApi())
// res0: Either[Error, Hotel] = Right(
//
value = Hotel(name = "Eddy's Roach Motel")
// )
Collaborator’s Machine:
fancyLodgingUnsafe(HotelApi())
// res2: Either[Error, Hotel] = Left(
//
value = Error(msg = "Invalid API Key")
// )
Continuous Integration Server:
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Configuration
96
fancyLodgingUnsafe(HotelApi())
// res4: Either[Error, Hotel] = Left(
//
value = Error(
//
msg = "Unconfigured Environment"
//
)
// )
On your own machine, everything works as expected. However, your collaborator
has a different value stored in this variable, and gets a failure when they execute this
code. Finally, the CI server has not set any value, and fails at runtime.
Building a Better Way
ZIO has a full System implementation of, but we will consider just 1 function for the
moment.
def envZ(
variable: String
): ZIO[Any, Nothing, Option[String]] =
ZIO.succeed(sys.env.get("API_KEY"))
This merely wraps our original function call. This is safe, but it is not the easiest code
to use or read. We want to convert an empty Option into an error state.
object SystemStrict:
val live: ZLayer[Any, Nothing, SystemStrict] =
ZLayer.fromZIO(
defer {
SystemStrict()
}
)
case class SystemStrict():
def envRequired(
variable: String
): ZIO[Any, Error, String] =
defer {
val variableAttempt = envZ(variable).run
ZIO
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Configuration
.fromOption(variableAttempt)
.mapError(_ =>
Error("Missing value for: " + variable)
)
.run
}
Similarly, we wrap our API in one that leverages ZIO.
case class HotelApiZ(
system: SystemStrict,
hotelApi: HotelApi
):
def cheapest(
zipCode: String
): ZIO[Any, Error, Hotel] =
defer {
val apiKey =
system.envRequired("API_KEY").run
ZIO
.fromEither(
hotelApi.cheapest(zipCode, apiKey)
)
.run
}
object HotelApiZ:
val live: ZLayer[
SystemStrict with HotelApi,
Nothing,
HotelApiZ
] =
ZLayer.fromZIO(
defer {
HotelApiZ(
ZIO.service[SystemStrict].run,
ZIO.service[HotelApi].run
)
}
)
This helps us keep a flat Error channel when we write our domain logic.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
97
Configuration
98
This was quite a process; where did it get us? Our fully ZIO-centric, side-effect-free
logic looks like this:
// TODO This produces large, wide output that does not adhere to the wi\
dth of the page.
// TODO This has fallen out of sync with the "identical" code below
// TODO Make this a case class?
def fancyLodging(
hotelApiZ: HotelApiZ
): ZIO[Any, Error, Hotel] =
hotelApiZ.cheapest("90210")
Original, unsafe:
def fancyLodgingUnsafe(
hotelApi: HotelApi
): Either[Error, Hotel] =
hotelApi.cheapest("90210")
// error:
// missing argument for parameter apiKey of method cheapest in class Ho\
telApi: (zipCode: String, apiKey: String):
//
Either[repl.MdocSession.MdocApp.Error, repl.MdocSession.MdocApp.Ho\
tel]
The logic is identical to our original implementation! The only difference is the result
type.
This is what it looks like in action:
Your Machine:
// TODO Do this for CI environment too
// TODO "originalAuthor" Don't know why it's called that?
val originalAuthor = HotelApiZ.live
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Configuration
99
val logic =
defer {
fancyLodging(ZIO.service[HotelApiZ].run)
}
// logic: ZIO[HotelApiZ, Nothing, ZIO[Any, Error, Hotel]] = OnSuccess(
//
trace = "zio.direct.ZioMonad.Success.$anon.map(ZioMonad.scala:18)",
//
first = OnSuccess(
//
trace = "repl.MdocSession.MdocApp.<local MdocApp>.logic(20_Confi\
guration.md:233)",
//
first = Sync(
//
trace = "repl.MdocSession.MdocApp.<local MdocApp>.logic(20_Con\
figuration.md:233)",
//
eval = zio.ZIOCompanionVersionSpecific$$Lambda$17027/0x0000000\
804296440@3f8ce416
//
),
//
successK = zio.ZIO$$$Lambda$17035/0x00000008042a8040@3724614b
//
),
//
successK = zio.ZIO$$Lambda$17083/0x000000080430f040@1e8e292a
// )
runDemo(
logic.provide(
SystemStrict.live,
ZLayer.succeed(HotelApi()),
originalAuthor
)
)
// OnSuccess(zio.direct.ZioMonad.Success.$anon.fl
Collaborator’s Machine:
// TODO Do this for CI environment too
val collaborater = HotelApiZ.live
val colaboraterLayer = collaborater
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Configuration
100
runDemo(
defer {
fancyLodging(ZIO.service[HotelApiZ].run)
}.provide(
SystemStrict.live,
ZLayer.succeed(HotelApi()),
collaborater
)
)
// OnSuccess(zio.direct.ZioMonad.Success.$anon.fl
Continuous Integration Server:
val ci = HotelApiZ.live
runDemo(
defer {
fancyLodging(ZIO.service[HotelApiZ].run)
}.provide(
SystemStrict.live,
ZLayer.succeed(HotelApi()),
ci
)
)
// OnSuccess(zio.direct.ZioMonad.Success.$anon.fl
TODO{{The actual line looks the same, which I highlighted as a problem before. How
should we indicate that the Environment is different?}}
When constructed this way, it becomes very easy to test. We create a second implementation that accepts test values and serves them to the caller. TODO{{Reorder
things so that the official ZIO TestSystem is used.}}
Official ZIO Approach
ZIO provides a more complete System API in the zio.System. This is always
available as a standard service from the ZIO runtime.
TODO
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Configuration
def fancyLodgingBuiltIn(
hotelApiZ: HotelApiZ
): ZIO[Any, SecurityException | Error, Hotel] =
defer {
val apiKey = zio.System.env("API_KEY").run
hotelApiZ
.cheapest(
apiKey.get // unsafe! TODO Use either
)
.run
}
Exercises
import zio.test.TestSystem
import zio.test.TestSystem.Data
// TODO Use real tests once Scala3 & ZIO2 are
// updated
Exercise 1: Create a function will report missing Environment Variables as
NoSuchElementException failures, instead of an Option success case.
trait Exercise1:
def envOrFail(variable: String): ZIO[
zio.System,
SecurityException | NoSuchElementException,
String
]
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
101
Configuration
object Exercise1Solution extends Exercise1:
def envOrFail(variable: String): ZIO[
zio.System,
SecurityException | NoSuchElementException,
String
] =
// TODO Direct instead of flatmap
zio
.System
.env(variable)
.flatMap(
_.fold(
ZIO.fail(new NoSuchElementException())
)(ZIO.succeed(_))
)
import zio.test.*
runSpec(
defer {
val res = Exercise1Solution
.envOrFail("key")
.provide(
TestSystem.live(
Data(envs = Map("key" -> "value"))
)
)
.run
assertTrue(res == "value")
}
)
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
102
Configuration
import zio.test.*
runSpec(
defer {
val res =
Exercise1Solution
.envOrFail("key")
.catchSome {
case _: NoSuchElementException =>
ZIO.succeed("Expected Error")
}
.provide(
TestSystem.live(Data(envs = Map()))
)
.run
assertTrue(res == "Expected Error")
}
)
Exercise 2: Create a function will attempt to parse a value as an Integer
and report errors as a NumberFormatException.
trait Exercise2:
def envInt(variable: String): ZIO[
Any,
NoSuchElementException |
NumberFormatException,
Int
] = ???
Edit This Chapter
Edit This Chapter²⁴
²⁴https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/20_Configuration.md
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
103
Streams
If you want to get all items in a defined range, eg July 1st - July 3rd, then you
might not need a Stream. However, if you want to get all items in a range that is not
bounded, eg From July 1st onward, then you need a Stream.
UI Interactions
UI events are a great use-case for streams. When you present a UI to a user, it is
impossible to know how they will interact with it. They might click a few buttons,
and finish in a few minutes. They might never interact with it at all. They might
leave the page open for days, only occasionally interacting with it. It is impossible
to run a function at any point in time that returns a List[Event] that represents all
interactions, because it is an unbounded, ongoing concept.
In addition, it’s unlikely that you want to hold all of UI events at one time. It is
enough to process them individually, or in small batches, as they occur.
Trend Recognition
Possible Scenarios: - Fraud Prevention - Self-harm prevention - Anti-terrorism Surge pricing / smoothing - Disease spread tracking
Edit This Chapter
Edit This Chapter²⁵
²⁵https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/21_Streams.md
105
Streams
Automatically attached experiments.
These are included at the end of this chapter because their package in the experiments
directory matched the name of this chapter. Enjoy working on the code with full
editor capabilities :D
experiments/src/main/scala/streams/Alphabet.scala
package streams
import zio.stream.*
object Alphabet1 extends ZIOAppDefault:
override def run =
ZStream
.fromIterable('a' to 'z')
.debug
.runDrain
object Alphabet2 extends ZIOAppDefault:
override def run =
ZStream
.fromIterable('a' to 'z')
.forever
.debug
.runDrain
object Alphabet3 extends ZIOAppDefault:
override def run =
ZStream
.fromIterable('a' to 'z')
.mapZIO { c =>
defer {
val d = Random.nextIntBounded(5).run
ZIO.sleep(d.seconds).run
ZIO.debug(c).run
}.fork
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
106
Streams
}
.runDrain // exits before all forks are completed
object Alphabet4 extends ZIOAppDefault:
override def run =
ZStream
.fromIterable('a' to 'z')
.schedule(
Schedule.fixed(1.second).jittered
)
.aggregateAsyncWithin(
ZSink.collectAll,
Schedule.fixed(3.seconds)
)
.debug("Elements in past 3 seconds")
.map(_.length)
.debug("Rate per 3 seconds")
.runDrain
// doesn't chunk into time-oriented groups as we'd expect
object Alphabet5 extends ZIOAppDefault:
override def run =
ZStream
.fromIterable('a' to 'z')
.schedule(Schedule.spaced(10.millis))
.throttleShape(1, 1.second) { chunk =>
println(chunk)
1
}
.debug
.runDrain
// grouping as many items as can fit in one second, with a cap of 1000
// Note: Int.MaxValue causes OOM
object Alphabet6 extends ZIOAppDefault:
def run =
ZStream
.fromIterable('a' to 'z')
.schedule(Schedule.spaced(100.millis))
.groupedWithin(1000, 1.second)
.debug
.runDrain
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
107
Streams
experiments/src/main/scala/streams/CommitStream.scala
package streams
import zio.stream.*
trait CommitStream:
def commits: Stream[Nothing, Commit]
case class Commit(
project: Project,
author: Author,
message: String,
added: Int,
removed: Int
)
object CommitStream:
object Live extends CommitStream:
def commits: Stream[Nothing, Commit] =
ZStream.repeatZIO(randomCommit)
private val randomCommit =
defer {
val author = Author.random.run
val project = Project.random.run
val message = Message.random.run
val linesAdded =
Random.nextIntBounded(500).run
val linesRemoved =
Random.nextIntBounded(500).run
Commit(
project,
author,
message,
linesAdded,
-linesRemoved
)
}
end CommitStream
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
108
Streams
object Message:
private val generic =
List(
"Refactor code",
"Add documentation",
"Update dependencies",
"Format code",
"Fix bug",
"Add feature",
"Add tests",
"Remove unused code"
)
def random: ZIO[Any, Nothing, String] =
randomElementFrom(generic)
case class Project(
name: String,
language: Language
)
object Project:
private val entries =
List(
Project("ZIO", Language.Scala),
Project("Tapir", Language.Scala),
Project("Kafka", Language.Java),
Project("Flask", Language.Python),
Project("Linux", Language.C)
)
val random: ZIO[Any, Nothing, Project] =
randomElementFrom(entries)
enum Language:
case Scala,
Java,
C,
CPlusPlus,
Go,
Rust,
Python,
Unison,
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
109
Streams
Ruby
enum Author:
case Kit,
Adam,
Bruce,
James,
Bill
object Author:
val random: ZIO[Any, Nothing, Author] =
randomElementFrom(Author.values.toList)
experiments/src/main/scala/streams/Counter.scala
package streams
import zio.{Ref, ZIO}
case class Counter(count: Ref[Int]):
val get: ZIO[Any, Nothing, Int] =
count.getAndUpdate(_ + 1)
object Counter:
val make = Ref.make(0).map(Counter(_))
experiments/src/main/scala/streams/DataFountain.scala
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
110
Streams
package streams
case class DataFountain(
tweets: TweetStream,
commitStream: CommitStream,
httpRequestStream: HttpRequestStream,
rate: Schedule[Any, Nothing, Long] =
Schedule.spaced(1.second)
):
def withRate(newValue: Int) =
copy(rate =
Schedule
.spaced(1.second.dividedBy(newValue))
)
object DataFountain:
def userFriendlyConstructor(rate: Int) =
DataFountain(
TweetStream.Live,
CommitStream.Live,
HttpRequestStream.Live,
Schedule.spaced(1.second.dividedBy(rate))
)
val live =
DataFountain(
TweetStream.Live,
CommitStream.Live,
HttpRequestStream.Live
)
//
// TODO More throttle investigation
tweets.throttleEnforce(1, 1.second, 1)(_.length)
experiments/src/main/scala/streams/DeliveryCenter.scala
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
111
Streams
package streams
import zio.stream.*
case class Order()
/** Possible stages to demo:
*
1. Ship individual orders as they come 2.
*
Queue up multiple items and then send 3.
*
Ship partially-filled truck if it has
*
been waiting too long
*/
object DeliveryCenter extends ZIOAppDefault:
sealed trait Truck
case class TruckInUse(
queued: List[Order],
fuse: Promise[Nothing, Unit],
capacity: Int = 3
) extends Truck:
val isFull: Boolean =
queued.length == capacity
val waitingTooLong =
fuse.isDone.map(done => !done)
def handle(
order: Order,
staged: Ref[Option[TruckInUse]]
) =
def shipIt(reason: String) =
defer:
ZIO
.debug(reason + " Ship the orders!")
.run
staged
.get
.flatMap(_.get.fuse.succeed(()))
.run
staged.set(None).run
val loadTruck =
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
112
Streams
defer {
val latch =
Promise.make[Nothing, Unit].run
val truck =
staged
.updateAndGet(truck =>
truck match
case Some(t) =>
Some(
t.copy(queued =
t.queued :+ order
)
)
case None =>
Some(
TruckInUse(
List(order),
latch
)
)
)
.map(_.get)
.run
ZIO
.debug(
"Loading order: " +
truck.queued.length + "/" +
truck.capacity
)
.run
truck
}
def shipIfWaitingTooLong(truck: TruckInUse) =
ZIO
.whenZIO(truck.waitingTooLong)(
shipIt(reason =
"Truck has bit sitting half-full too long."
)
)
.delay(4.seconds)
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
113
Streams
defer {
val truck = loadTruck.run
if (truck.isFull)
shipIt(reason = "Truck is full.").run
else
ZIO
.when(truck.queued.length == 1)(
ZIO.debug("Adding timeout daemon") *>
shipIfWaitingTooLong(truck)
)
.forkDaemon
.run
}
end handle
def run =
defer {
val stagedItems =
Ref.make[Option[TruckInUse]](None).run
val orderStream =
ZStream.repeatWithSchedule(
Order(),
Schedule
.exponential(1.second, factor = 1.8)
)
orderStream
.foreach(handle(_, stagedItems))
.timeout(12.seconds)
.run
}
end DeliveryCenter
experiments/src/main/scala/streams/DemoDataFountain.scala
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
114
Streams
package streams
import zio.*
import zio.stream.*
def specifyStream[A](
f: DataFountain => Stream[Nothing, A]
) =
f(DataFountain.live)
.take(10)
.foreach(ZIO.debug(_))
object DemoDataFountainTweets
extends ZIOAppDefault:
def run =
specifyStream:
_.tweets.tweets
object DemoDataFountainHttpRequests
extends ZIOAppDefault:
def run =
specifyStream:
_.httpRequestStream.requests
object DemoDataFountainCommits
extends ZIOAppDefault:
def run =
specifyStream:
_.commitStream.commits
object RecognizeBurstOfBadRequests
extends ZIOAppDefault:
def run =
DataFountain
.live
.httpRequestStream
.requests
.groupedWithin(10, 1.second)
.debug
.foreach(requests =>
ZIO.when(
requests
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
115
Streams
.filter: r =>
r.response == Code.Forbidden
.length > 2
)(ZIO.debug("Too many bad requests"))
)
.timeout:
5.seconds
end RecognizeBurstOfBadRequests
experiments/src/main/scala/streams/HelloStreams.scala
package streams
import zio.stream.*
object HelloStreams extends ZIOAppDefault:
def run =
for
_ <- ZIO.debug("Stream stuff!")
greetingStream =
ZStream.repeatWithSchedule(
"Hi",
Schedule.spaced(1.seconds)
)
insultStream =
ZStream.repeatWithSchedule(
"Dummy",
Schedule.spaced(2.seconds)
)
combinedStream =
ZStream.mergeAllUnbounded()(
greetingStream,
insultStream
)
aFewElements = combinedStream.take(6)
res <- aFewElements.runCollect
_
<- ZIO.debug("Res: " + res)
yield ()
end HelloStreams
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
116
Streams
experiments/src/main/scala/streams/HttpRequestStream.scala
package streams
import zio.stream.*
case class Request(response: Code, path: Path)
trait HttpRequestStream:
def requests: Stream[Nothing, Request]
object HttpRequestStream:
object Live extends HttpRequestStream:
override def requests
: Stream[Nothing, Request] =
ZStream
.repeatZIO(randomRequest)
.schedule(Schedule.spaced(100.millis))
private val randomRequest =
defer {
val code = Code.random.run
val path = Path.random.run
Request(code, path)
}
enum Code:
case Ok,
BadRequest,
Forbidden
object Code:
val random =
randomElementFrom(Code.values.toList)
case class Path(segments: Seq[String]):
override def toString: String =
segments.mkString("/")
object Path:
val random: ZIO[Any, Nothing, Path] =
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
117
Streams
defer {
val generator =
randomElementFrom(Random.generators).run
generator.run
}
def apply(first: String, rest: String*): Path =
Path(Seq(first) ++ rest)
private object Random:
private val generic
: ZIO[Any, Nothing, Path] =
val genericPaths =
List(
"login",
"preferences",
"settings",
"home",
"latest",
"logout"
)
defer {
val section =
randomElementFrom(genericPaths).run
Path(s"/$section")
}
private val user: ZIO[Any, Nothing, Path] =
val userSections =
List(
"activity",
"status",
"collaborators"
)
defer {
val userId =
zio.Random.nextIntBounded(1000).run
val section =
randomElementFrom(userSections).run
Path(s"/user/$userId/$section")
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
118
Streams
}
val generators
: List[ZIO[Any, Nothing, Path]] =
List(generic, user)
end Random
end Path
private[streams] def randomElementFrom[T](
collection: List[T]
): ZIO[Any, Nothing, T] =
for idx <Random.nextIntBounded(collection.length)
yield collection(idx)
experiments/src/main/scala/streams/MultipleConcurrentStrea
package streams
import zio.stream.*
import java.io.File
object MultipleConcurrentStreams
extends ZIOAppDefault:
val userActions =
ZStream(
"login",
"post:I feel happy",
"post: I want to buy something!",
"updateAccount",
"logout",
"post:I want to buy something expensive"
).mapZIO(action =>
ZIO.succeed(action).delay(1.seconds)
)
//
.throttleShape(1, 1.seconds, 2)(_.length)
// Note: I tried to bake this into the mapZIO
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
119
Streams
// call above, but that resulted in additional
// printing
// for every consumer of the stream.
// Surprising, but I'm sure there's good
// reasoning behind it.
val userActionAnnouncements =
userActions.mapZIO(action =>
ZIO.debug("Incoming event: " + action)
)
val actionBytes: ZStream[Any, Nothing, Byte] =
userActions.flatMap(action =>
ZStream
.fromIterable((action + "\n").getBytes)
)
val filePipeline
: ZPipeline[Any, Throwable, Byte, Long] =
ZPipeline.fromSink(
ZSink.fromFile(new File("target/output"))
)
val writeActionsToFile =
actionBytes >>> filePipeline
val marketingData =
userActions
.filter(action => action.contains("buy"))
val marketingActions =
marketingData.mapZIO(marketingDataPoint =>
ZIO.debug(
" $$ info: " + marketingDataPoint
)
)
val accountAuthentication =
userActions.filter(action =>
action == "login" || action == "logout"
)
val auditingReport =
accountAuthentication.mapZIO(event =>
ZIO.debug(" Security info: " + event)
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
120
Streams
)
def run =
ZStream
.mergeAllUnbounded()(
userActionAnnouncements,
marketingActions,
auditingReport,
writeActionsToFile
)
.runDrain
end MultipleConcurrentStreams
experiments/src/main/scala/streams/Scanning.scala
package streams
import zio.*
import zio.stream.*
object Scanning extends ZIOAppDefault:
enum GdpDirection:
case GROWING,
SHRINKING
enum EconomicStatus:
case GOOD_TIMES,
RECESSION
import EconomicStatus.*
import GdpDirection.*
case class EconomicHistory(
quarters: Seq[GdpDirection],
economicStatus: EconomicStatus
)
object EconomicHistory:
def apply(
quarters: Seq[GdpDirection]
): EconomicHistory =
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
121
Streams
EconomicHistory(
quarters,
if (
quarters
.sliding(2)
.toList
.lastOption
.contains(List(SHRINKING, SHRINKING))
)
RECESSION
else
GOOD_TIMES
)
val gdps =
ZStream(
GROWING,
SHRINKING,
GROWING,
SHRINKING,
SHRINKING
)
val economicSnapshots =
gdps.scan(EconomicHistory(List.empty))(
(history, gdp) =>
EconomicHistory(history.quarters :+ gdp)
)
def run =
economicSnapshots.runForeach(snapShot =>
ZIO.debug(snapShot)
)
end Scanning
experiments/src/main/scala/streams/TweetFactory.scala
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
122
Streams
package streams
case class TweetFactory(counter: Counter):
val randomTweet
: ZIO[Any, Nothing, SimpleTweet] =
defer {
val subject =
TweetFactory.randomSubject.run
val adjective =
TweetFactory.randomAdjective.run
val id = counter.get.run
SimpleTweet(
id,
s"$subject is the $adjective thing ever!"
)
}
private object TweetFactory:
val make: ZIO[Any, Nothing, TweetFactory] =
Counter.make.map(TweetFactory(_))
val superlatives =
List("best", "greatest", "most awesome")
val derogatory =
List("worst", "most terrible", "most awful")
val allAdjectives = superlatives ++ derogatory
val allSubjects =
List(
"Ice cream",
"The sunrise",
"Rain",
"ZIO",
"PHP",
"Skiing",
"Music"
)
val randomAdjective =
defer {
val index =
Random
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
123
Streams
.nextIntBounded(allAdjectives.size)
.run
allAdjectives(index)
}
val randomSubject =
defer {
val index =
Random
.nextIntBounded(allSubjects.size)
.run
allSubjects(index)
}
end TweetFactory
experiments/src/main/scala/streams/TweetStream.scala
package streams
import zio.stream.*
case class SimpleTweet(id: Int, text: String)
trait TweetStream:
def tweets: Stream[Nothing, SimpleTweet]
val slowTweetStream: Stream[
Nothing,
SimpleTweet
]
object TweetStream:
object Live extends TweetStream:
private val tweetService =
ZLayer.fromZIO(TweetFactory.make)
private val tweetsPerSecond = 6000
private val tweetRate =
Schedule.spaced(
1.second.dividedBy(tweetsPerSecond)
)
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
124
Streams
val tweets
: ZStream[Any, Nothing, SimpleTweet] =
ZStream
.repeatZIO(
ZIO.serviceWithZIO[TweetFactory](
_.randomTweet
)
)
.schedule(tweetRate)
.provideLayer(tweetService)
val slowTweetStream =
tweets.throttleShape(1, 1.second)(_.length)
end Live
end TweetStream
experiments/src/main/scala/streams/TwitterCustomerSupport
package streams
import zio.stream.*
import java.nio.file.{Files, Paths}
// This currently runs against the dataset available here:
// https://www.kaggle.com/datasets/thoughtvector/customer-support-on-tw\
itter?resource=download
object TwitterCustomerSupport
extends ZIOAppDefault:
val fileName =
//
"../datasets/sample.csv"
//
"../datasets/twcs/twcs.csv"
//
"small"
"medium"
//
"twcs_tiny.csv"
def isHappy(tweet: Tweet): Boolean =
List(
"fantastic",
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
125
Streams
"awesome",
"great",
"wonderful"
).exists(tweet.text.toLowerCase.contains(_))
def isAngry(tweet: Tweet): Boolean =
List("stupid", "dumb", "idiot", "shit")
.exists(tweet.text.toLowerCase.contains(_))
def trackActiveCompanies(
tweets: ZStream[Any, Throwable, Tweet]
) =
defer {
val activeCompanies =
Ref.make[Map[String, Int]](Map.empty).run
val mostActiveCompanyAtEachMoment =
tweets.mapZIO(tweet =>
defer {
val companies =
activeCompanies
.updateAndGet(
incrementCompanyActivity(
_,
tweet
)
)
.run
companies
.map(x => x)
.toList
.sortBy(x => -x._2)
}
)
val res =
mostActiveCompanyAtEachMoment.runLast.run
res.get
}
end trackActiveCompanies
override def run =
defer {
val dataset =
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
126
Streams
ZIOAppArgs
.getArgs
.map(_.headOption.getOrElse(fileName))
.run
val tweets =
ZStream
.fromJavaStream(
Files.lines(
Paths.get(
//
"..",
"datasets",
"twcs",
dataset + ".csv"
)
)
)
.map(l => Tweet(l))
.filter(_.isRight)
.map(_.getOrElse(???))
val happyTweetFilter: ZPipeline[
Any,
Nothing,
Tweet,
Tweet
] = ZPipeline.filter(isHappy)
val angryTweetFilter: ZPipeline[
Any,
Nothing,
Tweet,
Tweet
] = ZPipeline.filter(isAngry)
(tweets >>> happyTweetFilter)
.runCount
.debug("Number of happy tweets")
.run
(tweets >>> angryTweetFilter)
.runCount
.debug("Number of angry tweets")
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
127
Streams
.run
//
gatherHappyTweets
//
.timed
//
.map(_._1)
//
.debug("Happy duration") <&>
//
gatherAngryTweets <&>
trackActiveCompanies(tweets)
.map(_.take(3).mkString(" : "))
.debug("ActiveCompanies")
.timed
.map(_._1)
.debug("Active Company duration")
.run
}
end run
//
.timeout(60.seconds)
private def incrementCompanyActivity(
value1: Map[String, Int],
tweet: Tweet
): Map[String, Int] =
value1.updatedWith(tweet.author_id) {
case Some(value) =>
Some(value + 1)
case None =>
Some(1)
}
case class ParsingError(msg: String)
case class Tweet(
tweet_id: String,
author_id: String,
inbound: Boolean,
created_at: String,
text: String,
response_tweet_id: Option[String],
in_response_to_tweet_id: Option[String]
)
object Tweet:
def apply(
csvLine: String
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
128
Streams
): Either[ParsingError, Tweet] =
val pieces = csvLine.split(",")
Either.cond(
pieces.length == 7,
pieces match
case Array(
tweet_id,
author_id,
inbound,
created_at,
text,
response_tweet_id,
in_response_to_tweet_id
) =>
Tweet(
tweet_id,
author_id,
inbound == "True",
created_at,
text,
Some(response_tweet_id),
Some(in_response_to_tweet_id)
)
case _ =>
???
,
ParsingError("Bad value: " + pieces)
)
end apply
end Tweet
end TwitterCustomerSupport
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Appendix: Significant
Indentation
We have taken a bit of a risk with the style used in this book. We are embracing
significant indentation nearly to the max. These syntastic features were some of the
most contentious changes from Scala 3
Motivations
• Reduce the space taken up by small demos
– Often, closing parens/braces would double the vertical space
• Avoid getting lost scanning a long line
Concessions / Acknowledgements
• This style will be new/unfamiliar to many programmers - even seasoned Scala
2 users!
• The syntax is not yet fully supported by all editors // TODO Confirm this
• Sometimes this style actually increases the vertical space
Rules / examples
• Generally, when providing a block as an argument, use a colon and place the
argument on the following line
Appendix: Significant Indentation
130
def sendMessage(msg: String) =
println("Sent: " + msg)
sendMessage:
val name
= "Alice"
val greeting = "Hello"
s"$greeting, $name"
// Sent: Hello, Alice
However, when the function accepts:
• Multiple parameters
• Multiple lists of parameters
We use parentheses to group/collect/etc the arguments
multiply(7, 6)
// res1: Int = 42
This is necessary, because the alternative would just be evaluated as one block. The
first parameter is flagged as an unused value, and we only provide the 2nd parameter
to the function.
multiply:
7
6
// error:
// A pure expression does nothing in statement position; you may be omi\
tting necessary parentheses
//
"Hello"
//
^
Edge cases that are difficult to defend
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Appendix: Significant Indentation
131
runDemo:
defer:
ZIO.debug("Hello").run
ZIO.debug("World").run
// Hello
// World
// ()
VS
runDemo:
defer:
ZIO
.debug:
"Hello"
.run
ZIO
.debug:
"World"
.run
// Hello
// World
// ()
Edit This Chapter
Edit This Chapter²⁶
md
²⁶https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/22_Appendix_Significant_Indentation.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Appendix: ZIO Direct
How does defer, etc relate to flatmaps, for
comprehensions, etc
We don’t need to talk about monads, etc as zio-direct will just be “the way” so
we don’t need to explain why zio-direct or what it is. It just is. — We have been
experimenting with the zio-direct style of writing ZIO applications. Our theory is
that it is easier to teach this style of code to beginners. “Program as values” is a core
concept when using ZIO. ZIOs are just unexecuted values until they are run.
If not using zio-direct, you must explain many language details in order to write ZIO
code.
- If you want to sequence ZIO operations, you need to `flatmap` them
- To avoid indentation mountain, you should use a `for comprehension`
- How a `for` comprehension turns into `flatMap`s followed by a final `\
map`
- If you want to create a `val` in the middle of this, you need to use \
a `=` instead of `<-`
After you have accomplished all of that, you have trained your student to write
concise, clean code… that only makes sense to those versed in this style.
With zio-direct, you can write ZIO code that looks like imperative code.
Here are the concepts you need to understand for zio-direct
Appendix: ZIO Direct
133
- If you want to sequence ZIO operations, you need to write them inside\
of a `defer` block
- Code in `defer` will be captured in a ZIO
- Inside defer you can use `.run` to indicate when effects should be ex\
ecuted
Note- `.run` calls are *only* allowed inside `defer` blocks.
After you have accomplished that, you have trained your student to write slightly
less concise code… that most programmers will be comfortable with.
Gotchas -Something about mutable collection operations. TODO More info from
James - Cannot end a defer block with a ZIO[_,_,Nothing] It currently fails with
a very cryptic missing argument message
Edit This Chapter
Edit This Chapter²⁷
²⁷https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/23_Appendix_ZIO_Direct.md
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Experiments
These experiments are not currently attached to a chapter, but are included for
previewing. Before publication, we should not have any lingering experiments here.
experiments-src-test-scala-Hubs
experiments/src/test/scala/Hubs/QuizGameSpec.scala
package Hubs
import Hubs.QuizGame.cahootGame
import zio.test.*
object QuizGameSpec extends ZIOSpecDefault:
val
val
val
val
frop
zeb
shtep
cheep
=
=
=
=
Player("Frop")
Player("Zeb")
Player("Shtep")
Player("Cheep")
val players: List[Player] =
List(frop, zeb, shtep, cheep)
def spec =
suite("QuizGameSpec")(
test("roundWithMultipleCorrectAnswers") {
val roundWithMultipleCorrectAnswers =
RoundDescription(
Question(
"What is the southern-most European country?",
"Spain"
),
Seq(
Answer(zeb, "Germany", 2.seconds),
Answer(frop, "Spain", 1.seconds),
135
Experiments
Answer(cheep, "Spain", 3.seconds),
Answer(shtep, "Spain", 4.seconds)
)
)
val rounds =
Seq(roundWithMultipleCorrectAnswers)
defer {
val results =
QuizGame
.cahootGame(rounds, players)
.run
assertTrue(
results ==
List(
RoundResults(List(frop, cheep))
)
)
}
},
test("roundWithOnly1CorrectAnswer") {
val roundWithOnly1CorrectAnswer =
RoundDescription(
Question(
"What is the lightest element?",
"Hydrogen"
),
Seq(
Answer(frop, "Lead", 2.seconds),
Answer(zeb, "Hydrogen", 1.seconds),
Answer(cheep, "Gold", 3.seconds),
Answer(
shtep,
"Hydrogen",
10.seconds
)
)
)
val rounds =
Seq(roundWithOnly1CorrectAnswer)
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
136
Experiments
defer {
val results =
QuizGame
.cahootGame(rounds, players)
.run
assertTrue(
results ==
List(RoundResults(List(zeb)))
)
}
},
test("roundWhereEverybodyIsWrong") {
val roundWhereEverybodyIsWrong =
RoundDescription(
Question(
"What is the average airspeed of an unladen swallow?",
"INSUFFICIENT DATA FOR MEANINGFUL ANSWER"
),
Seq(
Answer(frop, "3.0 m/s", 1.seconds),
Answer(zeb, "Too fast", 1.seconds),
Answer(
cheep,
"Not fast enough",
1.seconds
),
Answer(shtep, "Scary", 1.seconds)
)
)
val rounds =
Seq(roundWhereEverybodyIsWrong)
defer {
val results =
QuizGame
.cahootGame(rounds, players)
.run
assertTrue(
results == List(RoundResults(List()))
)
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
137
Experiments
}
}
) @@ TestAspect.withLiveClock
end QuizGameSpec
Parallelism
experiments/src/main/scala/Parallelism/Finalizers.scala
package Parallelism
import zio.Console.printLine
import java.io.IOException
import scala.io.Source.*
object Finalizers extends zio.ZIOAppDefault:
//
//
//
//
//
//
In this example, we create a ZIO that uses
file IO. It opens a file to read it, but
gets failed half way through.
We use a finalizer to ensure that even if
the ZIO fails unexpectedly, the file will
still be closed.
def finalizer(
source: scala.io.Source
) = // Define the finalizer behavior here
ZIO.succeed {
println("Finalizing: Closing file reader")
source.close // Close the input source
}
val readFileContents
: ZIO[Scope, Throwable, Vector[String]] =
ZIO
.acquireRelease(
ZIO.succeed(
scala
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
138
Experiments
.io
.Source
.fromFile(
"src/main/scala/Parallelism/csvFile.csv"
)
)
)(finalizer)
.map {
bufferedSource => // Use the bracket method with the finalizer \
defined above to define behavior on fail.
val lines = bufferedSource.getLines
if (
true
) // Simulating an enexpected error/exception
throw new IOException("Boom!")
Vector() ++ lines
}
def run = // Use App's run function
println("In main")
val ioExample: ZIO[
Scope,
Throwable,
Unit
] = // Define the ZIO contexts
defer {
val fileLines = readFileContents.run
printLine(fileLines.mkString("\n"))
.run // Combine the strings of the output vector into a singl\
e string, separated by \n
}
ioExample
.catchAllDefect(exception =>
printLine(
"Ultimate error message: " +
exception.getMessage
)
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
139
Experiments
)
.exitCode // Call the Zio with exitCode.
end run
end Finalizers
experiments/src/main/scala/Parallelism/ParallelSleepers.scala
package Parallelism
val sleepNow =
defer:
ZIO
.debug:
"Yawn, going to sleep"
.run
ZIO
.sleep:
1.seconds
.run
ZIO
.debug:
"Okay, I am awake!"
.run
import zio_helpers.timedSecondsDebug
@main
def quick =
runDemo:
defer:
for _ <- 1 to 3 do
sleepNow.run
.timedSecondsDebug("Serial Sleepers")
object SerialSleepers extends ZIOAppDefault:
override def run =
defer:
for _ <- 1 to 3 do
sleepNow.run
.timedSecondsDebug("Serial Sleepers")
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
140
Experiments
object ParallelSleepers extends ZIOAppDefault:
override def run =
defer(Use.withParallelEval):
for _ <- 1 to 3 do
sleepNow.run
.timedSecondsDebug("AllSleepers")
val sleepers =
Seq(
1.seconds,
2.seconds,
3.seconds,
4.seconds,
5.seconds
)
object ParallelSleepers2 extends ZIOAppDefault:
override def run =
ZIO
.foreach(sleepers)(ZIO.sleep(_))
.timed
.debug
object ParallelSleepers3 extends ZIOAppDefault:
override def run =
ZIO
.foreachPar(sleepers)(ZIO.sleep(_))
.timed
.debug
object ParallelSleepers4 extends ZIOAppDefault:
override def run =
val racers = sleepers.map(ZIO.sleep(_))
ZIO
.raceAll(racers.head, racers.tail)
.timed
.debug
object ParallelSleepers5 extends ZIOAppDefault:
override def run =
ZIO.withParallelism(2) {
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
141
Experiments
ZIO
.foreachPar(sleepers)(ZIO.sleep(_))
.timed
.debug
}
experiments/src/main/scala/Parallelism/PrimeSeeker.scala
package Parallelism
object PrimeSeeker extends ZIOAppDefault:
override def run =
ZIO
.foreachPar(1 to 16): _ =>
ZIO
.succeed:
crypto.nextPrimeAfter:
100_000_000
.debug:
"Found prime:"
.timed
.debug:
"Found a bunch of primes"
experiments-src-test-scala-random
experiments/src/test/scala/random/OfficialZioRandomSpec.sca
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
142
Experiments
package random
import zio.test.*
object OfficialZioRandomSpec
extends ZIOSpecDefault:
def spec =
suite("random")(
test("hello random")(
defer {
// Note: Once the test exhausts these
// values, it goes
// back to true random values.
TestRandom
.feedInts(
1, 2, 3, 4, 5, 6, 7, 8, 9, 10
)
.run
val result =
Random.nextIntBounded(1000).run
assertTrue(result == 1)
val result2 =
Random.nextIntBounded(1000).run
assertTrue(result2 == 2)
}
),
test(
"rosencrants and guildenstern are dead"
)(
defer {
val coinToss =
defer {
if (Random.nextBoolean.run)
ZIO
.debug("ROSENCRANTZ: Heads.")
.run
else
ZIO
.fail(
"Tails encountered. Ending performance."
)
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
143
Experiments
.run
}
TestRandom
.feedBooleans(Seq.fill(100)(true)*)
.run
coinToss.repeatN(4).run
ZIO
.debug(
"GUILDENSTERN: There is an art to the building up of susp\
ense."
)
.run
coinToss.run
ZIO
.debug(
"GUILDENSTERN: Though it can be done by luck alone."
)
.run
coinToss.run
assertCompletes
}
),
test("asdf")(
defer:
TestRandom.feedInts(1, 2).run
val result1 = Random.nextInt.run
val result2 = Random.nextInt.run
// val result3 = Random.nextInt.run //
// this falls back to system Random
assertTrue(
result1 == 1,
result2 == 2
// result3 == 5
)
),
test("timeout"):
val thingThatTakesTime =
ZIO.sleep(2.seconds)
defer:
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
144
Experiments
val fork =
thingThatTakesTime
.timeout(1.second)
.fork
.run
TestClock.adjust(2.seconds).run
val result = fork.join.run
assertTrue(result.isEmpty)
// test("failure"):
// assertTrue(Some("asdf") == None)
)
end OfficialZioRandomSpec
experiments/src/test/scala/random/RunEffectfulGuessingGame
package random
import zio.test.*
import zio.internal.stacktracer.SourceLocation
object RunEffectfulGuessingGameSpec
extends ZIOSpecDefault:
def spec =
suite("GuessingGame")(
suite("Effectful")(
test("Untestable randomness")(
defer {
TestConsole
.feedLines(Seq.fill(100)("3")*)
.run
val res =
sideEffectingGuessingGame.run
assertTrue(res == "You got it!")
}
) @@
TestAspect
.flaky, // Highlight that we shouldn't need this TestAspect.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
145
Experiments
test("Testable")(
defer {
TestConsole.feedLines("3").run
TestRandom.feedInts(3).run
val res = effectfulGuessingGame.run
assertTrue(res == "You got it!")
}
) @@ TestAspect.nonFlaky(10)
)
)
end RunEffectfulGuessingGameSpec
Hubs
experiments/src/main/scala/Hubs/BasicHub.scala
package Hubs
// The purpose of this example to to create a
// very basic hub that displays small
// capabilities.
object BasicHub extends zio.ZIOAppDefault:
// This example makes a hub, and publishes a
// String. Then, two entities take the
// published string and print it.
val logic1 =
Hub
.bounded[String](2)
.flatMap { Hub =>
ZIO.scoped {
Hub
.subscribe
.zip(Hub.subscribe)
.flatMap { case (left, right) =>
defer {
Hub.publish("Hub message").run
val leftItem = left.take.run
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
146
Experiments
Console
.printLine(
"Left item: " + leftItem
)
.run
val rightItem = right.take.run
Console
.printLine(
"Right item: " + rightItem
)
.run
}
}
}
}
def run = logic1.exitCode
end BasicHub
experiments/src/main/scala/Hubs/QuizGame.scala
package Hubs
import zio.Console.printLine
import java.io.IOException
case class Player(name: String)
case class Question(
text: String,
correctResponse: String
)
case class Answer(
player: Player,
text: String,
delay: Duration
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
147
Experiments
)
case class RoundDescription(
question: Question,
answers: Seq[Answer]
)
case class RoundResults(
correctRespondents: List[Player]
)
object QuizGame:
// TODO Return result that can be tested
def cahootGame(
rounds: Seq[RoundDescription],
players: List[Player]
) =
defer {
val questionHub =
Hub.bounded[Question](1).run
val answerHub: Hub[Answer] =
Hub.bounded[Answer](players.size).run
val (
questions: Dequeue[Question],
answers: Dequeue[Answer]
) =
questionHub
.subscribe
.zip(answerHub.subscribe)
.run
ZIO
.foreach(rounds)(roundDescription =>
questionHub.publish(
roundDescription.question
) *>
playARound(
roundDescription,
questions,
answerHub,
answers
)
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
148
Experiments
)
.run
}
private[Hubs] def playARound(
roundDescription: RoundDescription,
questions: Dequeue[Question],
answerHub: Hub[Answer],
answers: Dequeue[Answer]
): ZIO[Any, IOException, RoundResults] =
defer {
val correctRespondents =
Ref.make[List[Player]](List.empty).run
printLine(
"Question for round: " +
roundDescription.question.text
).run
// TODO This should happen *before*
// playARound is invoked
questions.take.run
ZIO
.collectAllPar(
Seq(
submitAnswersAfterDelay(
answerHub,
roundDescription.answers
),
recordCorrectAnswers(
roundDescription
.question
.correctResponse,
answers,
correctRespondents
).repeat(
untilWinnersAreFound(
correctRespondents
)
)
)
)
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
149
Experiments
.timeout(4.second)
.run
RoundResults(correctRespondents.get.run)
}
private def untilWinnersAreFound(
correctRespondents: Ref[List[Player]]
) =
Schedule.recurUntilZIO(_ =>
correctRespondents.get.map(_.size == 2)
)
private def submitAnswersAfterDelay(
answerHub: Hub[Answer],
answers: Seq[Answer]
) =
ZIO.foreachParDiscard(answers) { answer =>
defer {
ZIO.sleep(answer.delay).run
answerHub.publish(answer).run
}
}
private def recordCorrectAnswers(
correctAnswer: String,
answers: Dequeue[Answer],
correctRespondents: Ref[List[Player]]
) =
defer:
val answer = answers.take.run
val output =
if (answer.text == correctAnswer)
correctRespondents
.update(_ :+ answer.player)
.run
"Correct response from: " +
answer.player
else
"Incorrect response from: " +
answer.player
printLine(output).run
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
150
Experiments
end QuizGame
executing_external_programs
experiments/src/main/scala/executing_external_programs/Gource.scala
package executing_external_programs
import zio.process.Command
/* Possibilities:
* - Show a certain time period
* - More recent activity
* - Cycle between different repositories */
object GourceDemo extends ZIOAppDefault:
def gource(repoDir: String) =
Command(
"gource",
//
"--follow-user", "bfrasure", // Highlights user, but still show\
s others
"--user-show-filter",
"bfrasure|Bill Frasure", // Only shows user
repoDir
)
val projects =
List(
"/Users/bfrasure/Repositories/book",
"/Users/bfrasure/Repositories/TestFrameworkComparison"
)
def showActivityForAWhile(repoDir: String) =
defer {
val run1 = gource(repoDir).run.run
ZIO.sleep(5.seconds).run
run1.killForcibly.run
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
151
Experiments
}
def randomProjectActivity =
defer {
val idx =
Random
.nextIntBounded(projects.length)
.run
showActivityForAWhile(projects(idx)).run
}
def run = randomProjectActivity.repeatN(2)
end GourceDemo
experiments/src/main/scala/executing_external_programs/Say.scala
package executing_external_programs
import zio.process.Command
def say(message: String) =
Command("say", message)
object SayDemo extends ZIOAppDefault:
def run = say("Hello, world!").run
experiments-src-test-scala-running_effects
experiments/src/test/scala/running_effects/ExampleConsoleSpec.scala
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
152
Experiments
package running_effects
import zio.test.*
object FeedLinesDemo extends ZIOSpecDefault:
def spec =
test("feedSomeLines"):
defer:
TestConsole.feedLines("Zep").run
assertCompletes
object ExampleConsoleSpec extends ZIOSpecDefault:
val promptForUsername = ZIO.succeed("Zeb")
def notificationFor(username: String) =
ZIO.succeed("Meeting @ 9")
def spec =
test("console IO"):
defer:
TestConsole.feedLines("Zeb").run
val username =
Console
.readLine("Enter your name\n")
.run
Console.printLine(s"Hello $username").run
val notification =
notificationFor(username).run
Console.printLine(notification).run
val capturedOutput: Vector[String] =
TestConsole.output.run
val expectedOutput =
s"""|Enter your name
Hello Zeb
Meeting @ 9
""".stripMargin
assertTrue(
capturedOutput.mkString("") ==
expectedOutput
)
end ExampleConsoleSpec
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
153
Experiments
hello_failures
experiments/src/main/scala/hello_failures/KeepSuccesses.scala
package hello_failures
import zio.Console.printLine
object KeepSuccesses extends zio.ZIOAppDefault:
val allCalls =
List("a", "b", "large payload", "doomed")
case class GoodResponse(payload: String)
case class BadResponse(payload: String)
val initialRequests =
allCalls.map:
fastUnreliableNetworkCall
val moreStructuredLogic =
defer:
val results =
ZIO
.partition(allCalls)(
fastUnreliableNetworkCall
)
.run
results match
case (failures, successes) =>
defer:
printFailures(failures).run
val recoveries =
attemptFallbackFor(failures).run
successes ++ recoveries
.debug:
"All successes"
.run
def run = moreStructuredLogic
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
154
Experiments
private def printFailures(
failures: Iterable[BadResponse]
) =
ZIO.foreach(failures): e =>
printLine:
"Error: " + e +
". Should retry on other server."
private def attemptFallbackFor(
failures: Iterable[BadResponse]
) =
ZIO.collectAllSuccesses:
failures.map: failure =>
slowMoreReliableNetworkCall:
failure.payload
.tapError: e =>
printLine:
"Giving up on: " + e
def fastUnreliableNetworkCall(input: String) =
if (input.length < 5)
ZIO.succeed:
GoodResponse(input)
else
ZIO.fail:
BadResponse(input)
def slowMoreReliableNetworkCall(
input: String
) =
if (input.contains("a"))
ZIO.succeed:
GoodResponse(input)
else
ZIO.fail:
BadResponse(input)
end KeepSuccesses
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
155
Experiments
scenarios
experiments/src/main/scala/scenarios/SecuritySystem.scala
package scenarios
import izumi.reflect.Tag
import time.scheduledValues
import scala.concurrent.TimeoutException
case class SecuritySystemX(
motionDetector: MotionDetector,
acousticDetectorX: AcousticDetectorX
):
val securityLoop: ZIO[
Any,
scala.concurrent.TimeoutException,
Unit
] =
defer:
val noise =
acousticDetectorX.monitorNoise.run
val motion =
motionDetector.amountOfMotion.run
ZIO
.debug:
s"Motion: $motion Noise: $noise"
.run
val securityResponse =
determineResponse(motion, noise)
securityResponse match
case Relax =>
ZIO
.debug:
"No need to panic"
.run
case LoudSiren =>
ZIO
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
156
Experiments
.debug:
"WOOOO EEEE WOOOOO EEEE"
.run
@annotation.nowarn
def shouldAlertServices()
: ZIO[Any, TimeoutException, String] =
defer:
securityLoop
.repeat:
Schedule.recurs(5) &&
Schedule.spaced(1.seconds)
.run
"Fin"
def determineResponse(
amountOfMotion: Pixels,
noise: Decibels
): SecurityResponse =
val numberOfAlerts =
determineBreaches(amountOfMotion, noise)
.size
if (numberOfAlerts == 0)
Relax
else
LoudSiren
def determineBreaches(
amountOfMotion: Pixels,
noise: Decibels
): Set[SecurityBreach] =
List(
Option.when(amountOfMotion.value > 50)(
SignificantMotion
),
Option.when(noise.value > 15)(LoudNoise)
).flatten.toSet
end SecuritySystemX
object SecuritySystemX:
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
157
Experiments
val live =
ZLayer.fromFunction(SecuritySystemX.apply _)
trait SecurityBreach
object LoudNoise
extends SecurityBreach
object SignificantMotion extends SecurityBreach
trait SecurityResponse
object Relax
extends SecurityResponse
object LoudSiren extends SecurityResponse
case class Decibels(value: Int)
case class Pixels(value: Int)
trait MotionDetector:
val amountOfMotion: ZIO[Any, Nothing, Pixels]
object MotionDetector:
object LiveMotionDetector
extends MotionDetector:
override val amountOfMotion
: ZIO[Any, Nothing, Pixels] =
ZIO.succeed(Pixels(30))
val amountOfMotion
: ZIO[MotionDetector, Nothing, Pixels] =
ZIO
.service[MotionDetector]
.flatMap(_.amountOfMotion)
val live
: ZLayer[Any, Nothing, MotionDetector] =
ZLayer.succeed(LiveMotionDetector)
case class AcousticDetectorX(
valueProducer: ZIO[
Any,
TimeoutException,
Decibels
]
):
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
158
Experiments
val monitorNoise
: ZIO[Any, TimeoutException, Decibels] =
valueProducer
object AcousticDetectorX:
def apply(
value: (Duration, Decibels),
values: (Duration, Decibels)*
): ZLayer[Any, Nothing, AcousticDetectorX] =
ZLayer.fromZIO:
defer:
val valueProducer: ZIO[
Any,
TimeoutException,
Decibels
] = scheduledValues(value, values*).run
// that same service we wrote above
AcousticDetectorX(valueProducer)
zio_helpers
experiments/src/main/scala/zio_helpers/Helpers.scala
package zio_helpers
extension (z: ZIO.type)
def repeatNPar[R, E, A](
numTimes: Int
)(op: ZIO[R, E, A]): ZIO[R, E, Seq[A]] =
z.foreachPar(0 until numTimes)((_: Int) =>
op
)
extension [R, E, A](z: ZIO[R, E, A])
def timedSecondsDebug(
message: String
): ZIO[R, E, A] =
z.timed
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
159
Experiments
.tap: (duration, res) =>
res match
// Don't bother printing Unit results
case () =>
ZIO.debug:
message + " [took " +
duration.getSeconds + "s]"
case _ =>
ZIO.debug:
message + ": " + res + " [took " +
duration.getSeconds + "s]"
.map(_._2)
crypto
experiments/src/main/scala/crypto/Mining.scala
package crypto
import zio.Random.nextIntBetween
import zio.ZIO.debug
import scala.annotation.tailrec
object Mining extends ZIOAppDefault:
def run =
defer:
val chain =
Ref.make[BlockChain](BlockChain()).run
raceForNextBlock(chain).repeatN(5).run
chain.get.debug("Final").run
private val miners =
Seq("Zeb", "Frop", "Shtep")
.flatMap(minerName =>
Range(1, 50)
.map(i => new Miner(minerName + i))
)
def raceForNextBlock(
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
160
Experiments
chain: Ref[BlockChain]
): ZIO[Any, Nothing, Unit] =
defer:
val raceResult = findNextBlock(miners).run
val (winner, winningPrime) = raceResult
chain
.update(chainCurrent =>
chainCurrent.copy(blocks =
chainCurrent.blocks :+ winningPrime
)
)
.run
debug(
s"$winner mined block: $winningPrime"
).run
case class BlockChain(
blocks: List[Int] = List.empty
)
class Miner(val name: String):
def mine(
num: Int
): ZIO[Any, Nothing, (String, Int)] =
defer {
val duration = nextIntBetween(1, 4).run
ZIO.sleep(duration.second).run
(name, nextPrimeAfter(num))
}
def findNextBlock(
miners: Seq[Miner]
): ZIO[Any, Nothing, (String, Int)] =
defer:
val startNum =
nextIntBetween(80000000, 160000000).run
ZIO
.raceAll(
miners.head.mine(startNum),
miners.tail.map(_.mine(startNum))
)
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
161
Experiments
.run
end Mining
// TODO Consider putting math functions somewhere else to avoid clutter\
ing example
def isPrime(num: Int): Boolean = (2 until num)
.forall(divisor => num % divisor != 0)
@tailrec
def nextPrimeAfter(num: Int): Int =
if (isPrime(num))
num
else
nextPrimeAfter(num + 1)
experiments-src-test-scala-test_aspects
experiments/src/test/scala/test_aspects/DemoBaseSpec.scala
package test_aspects
import zio.*
import zio.test.*
trait DemoBaseSpec extends ZIOSpecDefault {
val trackStats =
aroundAllWith(ZIO.debug("Starting"))( (_: Unit) => ZIO.debug("Finis\
hing"))
override def aspects: Chunk[TestAspectAtLeastR[TestEnvironment]] =
if (TestPlatform.isJVM)
Chunk(TestAspect.timeout(10.seconds), TestAspect.timed, trackStat\
s)
else
Chunk(TestAspect.timeout(10.seconds), TestAspect.sequential, Test\
Aspect.timed, trackStats)
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
162
Experiments
def aroundWith[R0, E0, A0](
before: ZIO[R0, E0, A0]
)(after: A0 => ZIO[R0, Nothing, Any]): Test\
Aspect[Nothing, R0, E0, Any] =
new TestAspect.PerTest[Nothing, R0, E0, Any] {
def perTest[R <: R0, E >: E0](test: ZIO[R, TestFailure[E], TestSu\
ccess])(implicit
\
trace: Trace
): ZIO[R, TestFailure[E], TestSuccess] =
ZIO.acquireReleaseWith(before.catchAllCause(c => ZIO.fail(TestF\
ailure.Runtime(c))))(after)(_ => test)
}
def walk[R, E](spec: Spec[R, E], labels: Chunk[String] = Chunk.empty)\
: Unit =
println("Walking")
spec.caseValue match
case Spec.ExecCase(exec, spec) => ()
case Spec.LabeledCase(label, spec) =>
println("Walk Label: " + label)
walk(spec, labels.appended(label))
case Spec.ScopedCase(scoped) => ()
case Spec.MultipleCase(specs) =>
println("Multi case")
specs.foreach(s => walk(s, labels))
case Spec.TestCase(test, annotations) =>
println("test labels: " + labels.mkString(" - "))
()
def aroundAllWith[R0, E0, A0](
before: ZIO[R0, E0, A0]
)(after: A0 => ZIO[R0, Nothing, Any]): T\
estAspect[Nothing, R0, E0, Any] =
new TestAspect[Nothing, R0, E0, Any] {
def some[R <: R0, E >: E0](spec: Spec[R, E])(implicit trace: Trac\
e): Spec[R, E] =
walk(spec)
Spec.scoped[R](
ZIO.acquireRelease(before)(after).mapError(TestFailure.fail).\
as(spec)
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
163
Experiments
)
}
}
object Demo1Spec extends DemoBaseSpec:
def spec = suite("Demo1Spec")(
test("test1") {
assertCompletes
},
test("test2") {
assertCompletes
}
)
experiments/src/test/scala/test_aspects/WithLiveSpec.scala
package test_aspects
import zio.test.*
import zio.test.TestAspect.*
object WithLiveSpec extends ZIOSpecDefault:
def halfFlaky[A](a: A): ZIO[Any, String, A] =
defer {
val b = zio.Random.nextBoolean.debug.run
ZIO
.cond(b, a, "failed")
.tapError(ZIO.logError(_))
.run
}
val song =
defer {
halfFlaky("works").debug.run
assertCompletes
}
val song1: Spec[Any, String] =
test("Song 1")(song)
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
164
Experiments
val songFlaky
: Spec[Live & Annotations, String] =
test("Song Flaky")(song) @@ flaky(10) @@
withLiveRandom
val spec =
suite("Play some music")(
song1,
songFlaky,
test("Song 2")(assertCompletes)
)
end WithLiveSpec
experiments-src-test-scala-scenarios
experiments/src/test/scala/scenarios/SecuritySystemSpec.scala
package scenarios
import zio.test.*
import zio.Console.printLine
import scala.concurrent.TimeoutException
object SecuritySystemSpec extends ZIOSpecDefault:
def spec =
suite("SecuritySystemSpec")(
suite("Module pattern version")(
test("runs out of data")(
defer {
val system =
ZIO.service[SecuritySystemX].run
val res =
system
.shouldAlertServices()
.catchSome {
case _: TimeoutException =>
printLine(
"Invalid Scenario. Ran out of sensor data."
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
165
Experiments
)
}
.run
ZIO.debug("Final result: " + res).run
assertCompletes
}
).provide(
SecuritySystemX.live,
MotionDetector.live ++
AcousticDetectorX(
(2.seconds, Decibels(11)),
(1.seconds, Decibels(20))
)
) @@ TestAspect.withLiveClock @@
TestAspect.tag("important", "slow") @@
TestAspect.flaky @@
TestAspect.silent @@ TestAspect.timed
)
)
end SecuritySystemSpec
cancellation
experiments/src/main/scala/cancellation/FutureCancellation.s
package cancellation
import scala.concurrent.Future
// We show that Future's are killed with finalizers that never run
object FutureNaiveCancellation
extends ZIOAppDefault:
def run =
ZIO
.fromFuture:
Future:
try Thread.sleep(500)
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
166
Experiments
finally println("Cleanup")
"Success!"
.timeout(25.millis)
.debug
experiments/src/main/scala/cancellation/ZIOCancellation.scala
package cancellation
val longRunning =
createProcess(
"LongRunning",
ZIO.sleep(5.seconds)
)
object HelloCancellation extends ZIOAppDefault:
def run = longRunning.timeout(2.seconds)
def createProcess(
label: String,
innerProcess: ZIO[Any, Nothing, Unit]
) =
defer:
ZIO.debug(s"Started $label").run
innerProcess.run
ZIO.debug(s"Finished $label").run
// TODO Consider rewriting to avoid
// dot-chaining on block
.onInterrupt(ZIO.debug(s"Interrupted $label"))
object HelloCancellation2 extends ZIOAppDefault:
val complex =
createProcess("Complex", longRunning)
def run = complex.timeout(2.seconds)
object CancellationWeb extends ZIOAppDefault:
def spawnLevel(
level: Int,
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
167
Experiments
limit: Int,
parent: String
): ZIO[Any, Nothing, Unit] =
ZIO
.foreachPar(List("L", "R"))(label =>
createProcess(
" " *
(level + 1 * 2) + parent +
s"-$label",
ZIO
.when(level < limit)(
spawnLevel(
level + 1,
limit,
" " *
(level + 1 * 2) + parent +
s"-$label"
)
)
.unit
)
)
.delay(level.seconds)
.unit
def run =
spawnLevel(0, 3, "Root").timeout(3.seconds)
end CancellationWeb
object FailureDuringFork extends ZIOAppDefault:
def run =
defer:
val fiber1 =
createProcess(
"Fiber 1",
ZIO.sleep(5.seconds)
).fork.run
val fiber2 =
createProcess(
"Fiber 2",
ZIO.sleep(5.seconds)
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
168
Experiments
).fork.run
// Once we fail here, the fibers will be
// interrupted.
ZIO.fail("Youch!").run
fiber1.join.run
fiber2.join.run
end FailureDuringFork
import org.apache.commons.lang3.RandomStringUtils
import org.apache.commons.text.similarity.LevenshteinDistance
val input = RandomStringUtils.random(70_000)
val target = RandomStringUtils.random(70_000)
val leven =
LevenshteinDistance.getDefaultInstance
object PlainLeven extends App:
leven(input, target)
object CancellingATightLoop
extends ZIOAppDefault:
val scenario =
ZIO.attempt(leven(input, target))
def run =
// For timeouts, you need fibers and
// cancellation
scenario
// TODO This is running for 16 seconds
// nomatter what.
.timed.debug("Time:").timeout(2.seconds)
concurrency
experiments/src/main/scala/concurrency/ServiceThatCanHand
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
169
Experiments
package concurrency
import zio.cache.{Cache, Lookup}
import java.nio.file.Path
import zio.Console.printLine
// TODO Move this all to concurrency_state prose when we can bring test\
s over in a decent way
case class FileContents(contents: List[String])
trait FileService:
def retrieveContents(
name: Path
): ZIO[Any, Nothing, FileContents]
// These are just for demos
val hits: ZIO[Any, Nothing, Int]
val misses: ZIO[Any, Nothing, Int]
// TODO Figure if these functions belong in the object instead.
trait FileSystem:
def readFileExpensive(
name: Path
): ZIO[Any, Nothing, FileContents] =
defer:
printLine("Reading from FileSystem")
.orDie
.run
ZIO.sleep(2.seconds).run
FileSystem.hardcodedFileContents
object FileSystem:
val hardcodedFileContents =
FileContents(
List("viralImage1", "viralImage2")
)
val live = ZLayer.succeed(new FileSystem {})
case class ServiceThatCanHandleThunderingHerds(
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
170
Experiments
cache: Cache[Path, Nothing, FileContents]
) extends FileService:
override def retrieveContents(
name: Path
): ZIO[Any, Nothing, FileContents] =
cache.get(name)
override val hits: ZIO[Any, Nothing, Int] =
defer {
cache.cacheStats.run.hits.toInt
}
override val misses: ZIO[Any, Nothing, Int] =
defer {
cache.cacheStats.run.misses.toInt
}
object ServiceThatCanHandleThunderingHerds:
val make =
defer:
val retrievalFunction =
ZIO
.service[FileSystem]
.map(_.readFileExpensive)
.run
val cache
: Cache[Path, Nothing, FileContents] =
Cache
.make(
capacity = 100,
timeToLive = Duration.Infinity,
lookup = Lookup(retrievalFunction)
)
.run
ServiceThatCanHandleThunderingHerds(cache)
end ServiceThatCanHandleThunderingHerds
rezilience
experiments/src/main/scala/rezilience/Bulkhead.scala
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
171
Experiments
package rezilience
import nl.vroste.rezilience.*
import nl.vroste.rezilience.Bulkhead.BulkheadError
/** In this demo, we can visualize all the
* requests that are currently in flight
*/
// TODO - Demonstrate when maxQueueing is reached
val makeBulkhead: ZIO[Scope, Nothing, Bulkhead] =
Bulkhead
.make(maxInFlightCalls = 3, maxQueueing = 32)
object BulkheadDemo extends ZIOAppDefault:
def run =
defer:
val currentRequests =
Ref.make[List[Int]](List.empty).run
val bulkhead = makeBulkhead.run
val statefulResource =
StatefulResource(currentRequests)
ZIO
.foreachPar(1 to 10): _ =>
bulkhead(statefulResource.request)
.debug("All requests done: ")
.run
case class StatefulResource(
currentRequests: Ref[List[Int]]
):
def request: ZIO[Any, Throwable, Int] =
defer:
val res = Random.nextIntBounded(1000).run
// Add the request to the list of current
// requests
currentRequests
.updateAndGet(res :: _)
.debug("Current requests: ")
.run
// Simulate a long-running request
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
172
Experiments
ZIO.sleep(1.second).run
removeRequest(res).run
res
private def removeRequest(i: Int) =
currentRequests.update(_ diff List(i))
end StatefulResource
experiments/src/main/scala/rezilience/CircuitBreakerDemo.sca
package rezilience
import nl.vroste.rezilience.*
import nl.vroste.rezilience.CircuitBreaker.*
object Scenario:
enum Step:
case Success,
Failure
import rezilience.Scenario.Step
object CircuitBreakerDemo extends ZIOAppDefault:
case class ExternalSystem(
requests: Ref[Int],
steps: List[Step]
):
// TODO: Better error type than Throwable
def call(): ZIO[Any, Throwable, Int] =
defer:
val requestCount =
requests.getAndUpdate(_ + 1).run
steps.apply(requestCount) match
case Scenario.Step.Success =>
ZIO
.succeed:
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
173
Experiments
requestCount
.run
case Scenario.Step.Failure =>
ZIO
.fail:
Exception:
"Something went wrong"
.run
.tapError: e =>
ZIO.debug(s"External failed: $e")
end ExternalSystem
val makeCircuitBreaker
: ZIO[Scope, Nothing, CircuitBreaker[
Any
]] =
CircuitBreaker.make(
trippingStrategy =
TrippingStrategy
.failureCount(maxFailures = 2),
resetPolicy =
Retry
.Schedules
.exponentialBackoff(
min = 1.second,
max = 1.minute
)
)
def callProtectedSystem(
cb: CircuitBreaker[Any],
system: ExternalSystem
) =
defer {
ZIO.sleep(500.millis).run
cb(system.call())
.catchSome:
case CircuitBreakerOpen =>
ZIO.debug:
"Circuit breaker blocked the call to our external system"
case WrappedError(e) =>
ZIO.debug:
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
174
Experiments
s"External system threw an exception: $e"
.tap: result =>
ZIO.debug:
s"External system returned $result"
.run
}
def run =
defer:
val cb
= makeCircuitBreaker.run
val requests = Ref.make[Int](0).run
import Scenario.Step.*
val steps =
List(Success, Failure, Failure, Success)
val system =
ExternalSystem(requests, steps)
callProtectedSystem(cb, system)
.repeatN(5)
.run
end CircuitBreakerDemo
experiments/src/main/scala/rezilience/RateLimiter.scala
package rezilience
import nl.vroste.rezilience.*
/** This is useful for scenarios such as:
*
- Making sure you don't suddenly spike your
*
AWS bill
*
- Not accidentally DDOSing a service
*/
val makeRateLimiter
: ZIO[Scope, Nothing, RateLimiter] =
RateLimiter.make(max = 1, interval = 1.second)
// We use Throwable as error type in this example
def rsaKeyGenerator: ZIO[Any, Throwable, Int] =
Random.nextInt
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
175
Experiments
object RateLimiterDemo extends ZIOAppDefault:
def run =
defer:
val rateLimiter = makeRateLimiter.run
rateLimiter(rsaKeyGenerator)
// Repeats as fast as the limiter allows
.repeatN(5).debug("Result").run
object RateLimiterDemoWithLogging
extends ZIOAppDefault:
// TODO Put in book-side ZIO helpers?
extension [R, E, A](z: ZIO[R, E, A])
def timedSecondsDebug(
message: String
): ZIO[R, E, A] =
z.timed
.tap: (duration, res) =>
ZIO.debug:
message + ": " + res + " [took " +
duration.getSeconds + "s]"
.map(_._2)
def run =
defer:
val rateLimiter = makeRateLimiter.run
rateLimiter(rsaKeyGenerator)
// Print the time to generate each key:
.timedSecondsDebug("Generated key")
// Repeat as fast as the limiter allows:
.repeatN(5)
// Print the last result
.timedSecondsDebug("Result").run
end RateLimiterDemoWithLogging
object RateLimiterDemoGlobal
extends ZIOAppDefault:
// TODO Put in book-side ZIO helpers?
extension (z: ZIO.type)
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
176
Experiments
def repeatNPar[R, E, A](numTimes: Int)(
op: Int => ZIO[R, E, A]
): ZIO[R, E, Seq[A]] =
z.foreachPar(0 until numTimes)(op)
def run =
defer:
val rateLimiter = makeRateLimiter.run
ZIO
.repeatNPar(4): i =>
rateLimiter(
rsaKeyGenerator.debug(i.toString)
)
// Repeats as fast as allowed
.repeatN(5).debug(s"Result $i")
.run
end RateLimiterDemoGlobal
cause
experiments/src/main/scala/cause/MalcomInTheMiddle.scala
package cause
object MalcomInTheMiddle extends ZIOAppDefault:
@annotation.nowarn
def run =
def turnOnLights() = throw new BurntBulb()
class BurntBulb() extends Exception
def getNewBulb() = throw new WobblyShelf()
class WobblyShelf() extends Exception
def grabScrewDriver() =
throw new SqueakyDrawer()
class SqueakyDrawer() extends Exception
def sprayWD40() = throw new EmptyCan()
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
177
Experiments
class EmptyCan() extends Exception
def driveToStore() = throw new DeadCar()
class DeadCar() extends Exception
def repairCar() = throw new Nagging()
class Nagging() extends Exception
//
//
//
//
try
turnOnLights()
catch
case burntBulb: BurntBulb =>
try
getNewBulb()
catch
case wobblyShelf: WobblyShelf =>
try
grabScrewDriver()
catch
case squeakyDrawer: SqueakyDrawer =>
try
sprayWD40()
catch
case emptyCan: EmptyCan =>
try
driveToStore()
catch
case deadCar: DeadCar =>
try repairCar()
finally
ZIO
.debug(
"What does it look like I'm doing?!"
)
.exitCode
finally
println
end try
finally
ZIO
.debug(
"What does it look like I'm doing?!"
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
178
Experiments
//
//
)
.exitCode
end run
/** try { turnOnLights } catch { case
* burntLightBulb => try {
*/
end MalcomInTheMiddle
experiments/src/main/scala/cause/MalcomInTheMiddleZ.scala
package cause
object MalcomInTheMiddleZ extends ZIOAppDefault:
def run =
def turnOnLights() = ZIO.fail(BurntBulb())
class BurntBulb()
extends Exception("Burnt Bulb!")
def getNewBulb() =
ZIO.attempt(
throw new Exception("Wobbly Shelf!")
)
def grabScrewDriver() =
ZIO.fail(Exception("SqueakyDrawer"))
defer {
turnOnLights()
.catchAllCause(originalError =>
getNewBulb().catchAllCause(bulbError =>
grabScrewDriver()
.mapErrorCause(screwDriverError =>
(originalError ++ bulbError) ++
screwDriverError
)
)
)
.run
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
179
Experiments
ZIO.debug("Preserve failures!").run
}.catchAllCause(bigError =>
ZIO.debug(
"Final error: " +
simpleStructureAlternative(bigError)
)
)
end run
end MalcomInTheMiddleZ
def simpleStructure(
cause: Cause[Throwable]
): String =
cause match
case Cause.Empty =>
???
case Cause.Fail(value, trace) =>
value.getMessage
case Cause.Die(value, trace) =>
???
case Cause.Interrupt(fiberId, trace) =>
???
case Cause.Stackless(cause, stackless) =>
???
case Cause.Then(left, right) =>
"Then(" + simpleStructure(left) + ", " +
simpleStructure(right) + ")"
case Cause.Both(left, right) =>
???
def simpleStructureAlternative(
cause: Cause[Throwable]
): String =
cause match
case Cause.Fail(value, trace) =>
value.getMessage
case Cause.Then(left, right) =>
simpleStructureAlternative(left) + " => " +
simpleStructureAlternative(right)
case Cause.Both(left, right) =>
???
case _ =>
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
180
Experiments
???
environment
experiments/src/main/scala/environment/DatabaseConnection
package environment
opaque type UserId = String
object UserId:
def apply(str: String): UserId = str
case class DbConnection(
actionLog: Ref[Seq[String]]
):
def execute(label: String) =
actionLog.update(_.appended(label))
object DbConnection:
val make: ZLayer[Any, Nothing, DbConnection] =
ZLayer.scoped(
ZIO.acquireRelease(
defer {
val actionLog: Ref[Seq[String]] =
Ref.make(Seq.empty[String]).run
val connection =
DbConnection(actionLog)
connection.execute("OPEN").run
connection
}
)(connection =>
defer {
connection.execute("CLOSE").run
pprint.apply(connection)
}.debug
)
)
end DbConnection
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
181
Experiments
object DatabaseConnectionSimple
extends ZIOAppDefault:
def executeUserQueries(
userId: UserId
): ZIO[DbConnection, Nothing, Unit] =
// TODO Consider alternative version of this
// where the defer happens inside of a
// serviceWithZIO call
defer {
val connection =
ZIO.service[DbConnection].run
connection
.execute(
s"QUERY for $userId preferences"
.stripMargin
)
.run
ZIO.sleep(1.second).run
connection
.execute(
s"QUERY for $userId accountHistory"
.stripMargin
)
.run
}
def run =
defer {
executeUserQueries(UserId("Alice")).run
executeUserQueries(UserId("Bob")).run
}.provide(DbConnection.make)
end DatabaseConnectionSimple
object DatabaseConnectionInterleavedQueries
extends ZIOAppDefault:
def executeUserQueries(
userId: UserId
): ZIO[DbConnection, Nothing, Unit] =
// TODO Consider alternative version of this
// where the defer happens inside of a
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
182
Experiments
// serviceWithZIO call
defer {
val connection =
ZIO.service[DbConnection].run
connection
.execute(
s"QUERY for $userId preferences"
.stripMargin
)
.run
ZIO.sleep(1.second).run
connection
.execute(
s"QUERY for $userId accountHistory"
.stripMargin
)
.run
}
def run =
ZIO
.foreachPar(
List(UserId("Alice"), UserId("Bob"))
)(executeUserQueries)
.provide(DbConnection.make)
end DatabaseConnectionInterleavedQueries
experiments/src/main/scala/environment/ToyEnvironment.sca
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
183
Experiments
package environment
import scala.reflect.{ClassTag, classTag}
case class DBService(url: String)
// Yada yada yada lets talk about the environment
trait ToyEnvironmentT[+R]:
def add[A: ClassTag](
a: A
): ToyEnvironmentT[R & A]
def get[A >: R: ClassTag]: A
class ToyEnvironment[+R](
typeMap: Map[ClassTag[_], Any]
) extends ToyEnvironmentT[R]:
def add[A: ClassTag](
a: A
): ToyEnvironment[R & A] =
ToyEnvironment(typeMap + (classTag[A] -> a))
def get[A >: R: ClassTag]: A =
typeMap(classTag[A]).asInstanceOf[A]
@annotation.nowarn
@main
def demoToyEnvironment =
val env: ToyEnvironment[_] =
ToyEnvironment(Map.empty)
val env1: ToyEnvironment[String] =
env.add("hi")
val env2: ToyEnvironment[String & DBService] =
env1.add(DBService("blah"))
val env3: ToyEnvironment[
String & DBService & List[String]
] = env2.add(List("a", "b"))
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
184
Experiments
println(env3.get[String])
println(env3.get[DBService])
println(env3.get[List[String]])
// We get some amount of compile time safety
// here, but not much
// println(env.get(classOf[List[DBService]]))
// Downside of the current approach is that it
// doesn't prevent duplicate types
env3.add("hi") // is accepted
end demoToyEnvironment
// Consider this runtime de-duping
class ToyEnvironmentRuntimeDeduplication[+R](
typeMap: Map[ClassTag[_], Any]
):
def add[A: ClassTag](
a: A
): ToyEnvironment[R & A] =
if (typeMap.contains(classTag[A]))
throw new IllegalArgumentException(
s"Cannot add ${classTag[A]} to environment, it already exists"
)
else
ToyEnvironment(
typeMap + (classTag[A] -> a)
)
experiments-src-test-scala-zio_test
experiments/src/test/scala/zio_test/Shared.scala
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
185
Experiments
package zio_test
object Shared:
val layer: ZLayer[Any, Nothing, Ref[Int]] =
ZLayer.scoped {
ZIO.acquireRelease(
Ref.make(0) <* ZIO.debug("Initializing!")
)(
_.get
.debug(
"Number of tests that used shared layer"
)
)
}
case class Scoreboard(value: Ref[Int]):
def display(): ZIO[Any, Nothing, String] =
defer {
val current = value.get.run
s"**$current**"
}
val scoreBoard: ZLayer[
Scope with Ref[Int],
Nothing,
Scoreboard
] =
ZLayer.fromZIO {
defer {
val value = ZIO.service[Ref[Int]].run
ZIO
.acquireRelease(
ZIO.succeed(Scoreboard(value)) <*
ZIO.debug(
"Initializing scoreboard!"
)
)(_ =>
ZIO.debug("Shutting down scoreboard")
)
.run
}
}
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
186
Experiments
end Shared
experiments/src/test/scala/zio_test/UseComplexLayer.scala
package zio_test
import zio.test.*
import zio_test.Shared.Scoreboard
object UseComplexLayer
extends ZIOSpec[Scoreboard]:
def bootstrap
: ZLayer[Any, Nothing, Scoreboard] =
ZLayer.make[Scoreboard](
Shared.layer,
Shared.scoreBoard,
Scope.default
)
def spec =
test("use scoreboard") {
defer {
ZIO
.serviceWithZIO[Scoreboard](
_.display()
)
.debug
.run
assertCompletes
}
}
end UseComplexLayer
experiments/src/test/scala/zio_test/UseSharedLayerA.scala
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
187
Experiments
package zio_test
import zio.test.{ZIOSpec, assertCompletes}
object UseSharedLayerA extends ZIOSpec[Ref[Int]]:
def bootstrap = Shared.layer
def spec =
test("Test A") {
for _ <ZIO.serviceWithZIO[Ref[Int]](
_.update(_ + 1)
)
yield assertCompletes
}
experiments/src/test/scala/zio_test/UseSharedLayerB.scala
package zio_test
import zio.test.{ZIOSpec, assertCompletes}
import zio.{Ref, ZIO}
object UseSharedLayerB extends ZIOSpec[Ref[Int]]:
def bootstrap = Shared.layer
def spec =
test("Test B") {
for _ <ZIO.serviceWithZIO[Ref[Int]](count =>
count.update(_ + 1)
)
yield assertCompletes
}
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
188
Experiments
experiments-src-test-scala-layers
experiments/src/test/scala/layers/FestivalShortedOutSoundSys
package layers
import zio.test.*
// TODO Replace with some toilet issue?
object FestivalShortedOutSoundSystemSpec
extends ZIOSpecDefault:
val brokenFestival
: ZLayer[Any, String, Festival] =
ZLayer.make[Festival](
festival,
stage,
soundSystem,
toilets,
security,
venue,
permit
)
val spec =
suite("Play some music")(
test("Good festival")(
(
for _ <- ZIO.service[Festival]
yield assertCompletes
).provide(brokenFestival)
.withClock(Clock.ClockLive)
.catchAll(e =>
ZIO.debug("Expected error: " + e) *>
ZIO.succeed(assertCompletes)
)
)
)
end FestivalShortedOutSoundSystemSpec
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
189
Experiments
experiments/src/test/scala/layers/FestivalSpec.scala
package layers
import zio.test.*
object FestivalSpec extends ZIOSpecDefault:
val spec =
suite("Play some music")(
test("Good festival")(
ZIO.service[Festival].as(assertCompletes)
)
).provide(
festival,
stage,
soundSystem,
toilets,
security,
venue,
permit
//
ZLayer.Debug.mermaid
) @@ TestAspect.withLiveClock
experiments-src-test-scala-time
experiments/src/test/scala/time/ScheduledValuesSpec.scala
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
190
Experiments
package time
import zio.test.*
import scala.concurrent.TimeoutException
object ScheduledValuesSpec
extends ZIOSpecDefault:
def spec =
suite("ScheduledValues")(
suite("scheduledValues")(
test(
"querying after no time has passed returns the first value, i\
f duration was > 0"
)(
defer {
val valueAccessor =
scheduledValues(
(1.seconds, "First Section")
).run
assertTrue(
valueAccessor.run ==
"First Section"
)
}
),
test(
"querying when no time has passed fails when the duration == \
0"
)(
defer {
val valueAccessor =
scheduledValues(
(0.seconds, "First Section")
).run
valueAccessor.flip.run
assertCompletes
}
),
test(
"next value is returned after enough time has elapsed"
)(
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
191
Experiments
defer {
val valueAccessor =
scheduledValues(
(1.seconds, "First Section"),
(2.seconds, "Second Section")
).run
TestClock.adjust(1.seconds).run
val secondValue = valueAccessor.run
assertTrue(
secondValue == "Second Section"
)
}
),
test("time range end is not inclusive")(
defer {
val valueAccessor =
scheduledValues(
(1.seconds, "First Section")
).run
TestClock.adjust(1.seconds).run
valueAccessor.flip.run
assertCompletes
}
)
)
)
end ScheduledValuesSpec
performance
experiments/src/main/scala/performance/Hedging.scala
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
192
Experiments
package performance
import zio_helpers.repeatNPar
object Hedging extends ZIOAppDefault:
extension [R, E, A](z: ZIO[R, E, A])
def hedge(
wait: zio.Duration,
depth: Int = 1
): ZIO[R, E, A] =
depth match
case 0 =>
z
case other =>
z.delay:
wait
.race:
hedge(wait, depth - 1)
def hedgedRequestNarrow =
handleRequest
.race(handleRequest.delay(25.millis))
.race(handleRequest.delay(25.millis))
.race(handleRequest.delay(25.millis))
def hedgedRequestGeneral =
handleRequest.hedge(25.millis, 3)
def run =
defer:
val timeBuckets =
Ref
.make[Map[Percentile, RequestStats]]:
Map()
.run
ZIO
.repeatNPar(50_000):
demoRequest:
timeBuckets
.run
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
193
Experiments
pprint.pprintln(
timeBuckets.get.run,
width = 47
)
def demoRequest(
timeBuckets: Ref[
Map[Percentile, RequestStats]
]
) =
hedgedRequestGeneral.tap(requestTime =>
timeBuckets.update(results =>
results.updatedWith(
Percentile
.fromDuration(requestTime.duration)
):
case Some(value) =>
Some(
value.copy(
count = value.count + 1,
totalTime =
value
.totalTime
.plus(requestTime.duration)
)
)
case None =>
Some(
RequestStats(
count = 1,
totalTime = requestTime.duration
)
)
)
)
end Hedging
val handleRequest =
defer {
Percentile.random.run match
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
194
Experiments
case Percentile._50 =>
ResponseTimeCutoffs.Fast
case Percentile._95 =>
ResponseTimeCutoffs.Acceptable
case Percentile._999 =>
ResponseTimeCutoffs.BreachOfContract
}.tap(dimension =>
ZIO.sleep(dimension.duration)
)
// TODO Fix name
enum ResponseTimeCutoffs(val duration: Duration):
case Fast
extends ResponseTimeCutoffs(21.millis)
case Acceptable
extends ResponseTimeCutoffs(70.millis)
case BreachOfContract
extends ResponseTimeCutoffs(1800.millis)
enum Percentile:
case _50,
_95,
_999
object Percentile:
def random =
defer:
val int = Random.nextIntBounded(1_000).run
if (int < 950)
Percentile._50
else if (int < 999)
Percentile._95
else
Percentile._999
def fromDuration(d: Duration) =
d match
case ResponseTimeCutoffs.Fast.duration =>
Percentile._50
case ResponseTimeCutoffs
.Acceptable
.duration =>
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
195
Experiments
Percentile._95
case ResponseTimeCutoffs
.BreachOfContract
.duration =>
Percentile._999
case _ =>
???
end Percentile
case class RequestStats(
count: Int,
totalTime: Duration
)
resourcemanagement
experiments/src/main/scala/resourcemanagement/ChatSlots.s
package resourcemanagement
import zio.Console.printLine
case class Slot(id: String)
case class Player(name: String, slot: Slot)
object ChatSlots extends zio.ZIOAppDefault:
enum SlotState:
case Closed,
Open
def run =
@annotation.nowarn
def acquire(ref: Ref[SlotState]) =
defer:
printLine:
"Took a speaker slot"
.run
ref
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
196
Experiments
.set:
SlotState.Open
.run
"Use Me"
def release(ref: Ref[SlotState]) =
defer:
printLine:
"Freed up a speaker slot"
.orDie
.run
ref
.set:
SlotState.Closed
.run
defer {
val ref =
Ref
.make[SlotState]:
SlotState.Closed
.run
val managed =
ZIO.acquireRelease(acquire(ref))(_ =>
release:
ref
)
val reusable =
managed.map:
printLine(_)
reusable.run
reusable.run
ZIO
.scoped:
// TODO Get rid of flatmap if
// possible...
managed.flatMap: s =>
defer:
printLine:
s
.run
printLine:
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
197
Experiments
"Blowing up"
.run
if (true)
throw Exception:
"Arggggg"
.run
}
end run
end ChatSlots
experiments-src-test-scala-concurrency
experiments/src/test/scala/concurrency/ThunderingHerdsSpec
package concurrency
import zio.Console.printLine
import zio.test.*
import java.nio.file.Path
object ThunderingHerdsSpec
extends ZIOSpecDefault:
val testInnards =
defer {
val users = List("Bill", "Bruce", "James")
val herdBehavior =
defer {
val fileService =
ZIO.service[FileService].run
val fileResults =
ZIO
.foreachPar(users)(user =>
fileService.retrieveContents(
Path.of("awesomeMemes")
)
)
.run
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
198
Experiments
ZIO.debug("=========").run
fileService
.retrieveContents(
Path.of("awesomeMemes")
)
.run
fileResults
}
printLine("Capture?").run
val logicFork = herdBehavior.fork.run
TestClock.adjust(2.seconds).run
val res = logicFork.join.run
val misses =
ZIO
.serviceWithZIO[FileService](_.misses)
.run
ZIO.debug("Eh?").run
assertTrue(
misses == 1,
res.forall(singleResult =>
singleResult ==
FileSystem.hardcodedFileContents
)
)
}
end testInnards
override def spec =
suite("ThunderingHerdsSpec")(
test(
"classic happy path using zio-cache library"
) {
testInnards
}.provide(
FileSystem.live,
ZLayer.fromZIO(
ServiceThatCanHandleThunderingHerds
.make
)
)
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
199
Experiments
)
end ThunderingHerdsSpec
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward