Effect-Oriented Programming
Creating Reliable Systems with Scala 3 and ZIO 2
Bill Frasure, Bruce Eckel and James Ward
Effect-Oriented Programming
Creating Reliable Systems with Scala 3 and ZIO 2
Bill Frasure, Bruce Eckel and James Ward
This book is for sale at http://leanpub.com/effect-oriented-programming
This version was published on 2024-01-25
This is a Leanpub book. Leanpub empowers authors and publishers with the Lean
Publishing process. Lean Publishing is the act of publishing an in-progress ebook
using lightweight tools and many iterations to get reader feedback, pivot until you
have the right book and build traction once you do.
© 2021 - 2024 Bill Frasure, Bruce Eckel and James Ward
CONTENTS
Contents
Copyright . . . . . . . . . . . . . . .
Effect Oriented Programming
Source Code . . . . . . . . . . .
Edit This Chapter . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1
1
2
5
Preface . . . . . . . . . . . .
Who is the book is for
Code examples . . . .
Acknowledgements .
Edit This Chapter . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
6
6
6
7
7
What Are Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Introduction . . . . . . . . . . . . .
Untangling the Chaos . . . . .
The State of Software . . . . .
The Software Crisis . . . . . .
Reliability . . . . . . . . . . . .
Dealing with unpredictability
Effects can not be un-done . .
What is an Effect? . . . . . . .
The Interpreter . . . . . . . . .
Edit This Chapter . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
9
9
10
11
12
13
14
14
16
17
Superpowers with Effects . . . . . . . . .
Building a Resilient Process in stages
Step 1: Happy Path . . . . . . . . . . .
Step 2: Provide Error Fallback Value
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
18
18
18
18
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
CONTENTS
Step 3: Retry Upon Failure . . . . . . . . . .
Step 4: Fallback after multiple failures . . .
Step 5: Timeouts . . . . . . . . . . . . . . . .
Step 6: Fallback Effect . . . . . . . . . . . . .
Step 7: Concurrently Execute Effect . . . . .
Step 8: Ignore failures in Concurrent Effect
Step 9: Rate Limit TODO . . . . . . . . . . .
Edit This Chapter . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
19
19
20
20
21
22
22
24
Why Functional . . . . . . . . . . . . . . . . . . . . .
A Different Goal . . . . . . . . . . . . . . . . . .
Reuse . . . . . . . . . . . . . . . . . . . . . . . . .
Pure Functions . . . . . . . . . . . . . . . . . . .
Composability . . . . . . . . . . . . . . . . . . . .
Effects . . . . . . . . . . . . . . . . . . . . . . . .
Core Differences Between OO and Functional
Summary: Style vs Substance . . . . . . . . . .
Edit This Chapter . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
25
25
28
29
30
31
32
32
33
Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
General/Historic discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . .
What ZIO Provides Us. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
DI-Wow! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
TODO Rewrite this without . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Step 1: Effects can express dependencies . . . . . . . . . . . . . . . . . . . . .
Step 2: Provide Dependency Layers to Effects . . . . . . . . . . . . . . . . . .
Step 3: Effects can require multiple dependencies . . . . . . . . . . . . . . .
Step 4: Dependencies can “automatically” assemble to fulfill the needs of
an effect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Step 5: Different effects can require the same dependency . . . . . . . . . .
Step 6: Dependencies must be fulfilled by unique types . . . . . . . . . . . .
Step 7: Providing Dependencies at Different Levels . . . . . . . . . . . . . .
Step 8: Dependencies can fail . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Step 9: Dependency Retries . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Step 10: Fallback Dependencies . . . . . . . . . . . . . . . . . . . . . . . . . .
Step 11: Layer Retry + Fallback? . . . . . . . . . . . . . . . . . . . . . . . . . .
34
34
35
35
35
36
36
37
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
39
39
40
41
42
42
43
43
CONTENTS
Testing Effects . . .
Random . . . . . .
Time . . . . . . . .
Edit This Chapter .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
44
44
46
48
Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Historic approaches to Error-handling . . . . . .
ZIO Error Handling . . . . . . . . . . . . . . . . .
Unions AKA Sum Types AKA Enums AKA Ors
Edit This Chapter . . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
49
49
55
59
61
Contract-Based Composability . . . . .
Composability Explanation . . . . . .
Alternatives and their downsides . .
Universal Composability with ZIO .
All The Thing Example . . . . . . . .
Hedging . . . . . . . . . . . . . . . . .
Repeats . . . . . . . . . . . . . . . . . .
Edit This Chapter . . . . . . . . . . . .
Automatically attached experiments.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
62
64
64
66
67
68
68
68
69
Concurrency High Level
zipPar, zipWithPar . .
validateWithPar? . . .
withParallelism . . . .
Edit This Chapter . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
75
76
76
76
76
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
path
. . . .
. . . .
77
77
77
77
77
77
79
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Concurrency Interruption . . . . . . . . . . . . . . . . . . . . . .
Why Interruption Is Necessary Throughout the Stack . . . .
Timeout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Race . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.withFinalizer . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Fork Interruption . . . . . . . . . . . . . . . . . . . . . . . . . .
Uninterruptable . . . . . . . . . . . . . . . . . . . . . . . . . . .
Future Cancellation (Contra-example. Not necessary for
explanation) . . . . . . . . . . . . . . . . . . . . . . . .
Edit This Chapter . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
happy
. . . . .
. . . . .
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
79
80
CONTENTS
State . . . . . . . . . . . .
Unreliable Counting
Reliable Counting .
Ref.Synchronized . .
Edit This Chapter . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
81
81
82
85
85
Reliability . . . . . . . . . . . . . . . . .
Caching . . . . . . . . . . . . . . .
Staying under rate limits . . . . .
Constraining concurrent requests
Circuit Breaking . . . . . . . . . .
Hedging . . . . . . . . . . . . . . .
Edit This Chapter . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
86
86
87
87
89
89
89
ZIO and ZLayer . . . .
ZIO . . . . . . . . .
ZLayer . . . . . . .
Composing . . . . .
ZIO <-> ZLayer . .
Edit This Chapter .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
90
90
91
91
92
92
Appendix Running Effects . . . . . . . . . . . . .
Building applications from scratch . . . . . .
Testing code . . . . . . . . . . . . . . . . . . . .
Interop with existing/legacy code via Unsafe
Edit This Chapter . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
93
93
94
97
97
Experiments . .
rezilience . .
performance
zio_helpers .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
. 98
. 98
. 106
. 108
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Copyright
Effect Oriented Programming
By Bill Frasure, Bruce Eckel and James Ward
{{ These are the new ISBNs }}
Copyright ©2021, MindView LLC.
eBook ISBN 978-0-9818725-6-8
Print Book ISBN 978-0-9818725-7-5
The eBook ISBN covers the Leanpub eBook distribution in all formats, available
through www.EffectOrientedProgramming.com.
Please purchase this book through www.EffectOrientedProgramming.com, to
support its continued maintenance and updates.
All rights reserved. Printed in the United States of America. This publication is
protected by copyright, and permission must be obtained from the publisher prior
to any prohibited reproduction, storage in a retrieval system, or transmission in any
form or by any means, electronic, mechanical, photocopying, recording, or likewise.
For information regarding permissions, see EffectOrientedProgramming.com.
Created in Crested Butte, Colorado, USA.
Text printed in the United States
Ebook: Version 1.0, Month Year
First printing Month Year
Cover design by Daniel Will-Harris, www.Will-Harris.com¹
¹http://www.Will-Harris.com
2
Copyright
Many of the designations used by manufacturers and sellers to distinguish their
products are claimed as trademarks. Where those designations appear in this book,
and the publisher was aware of a trademark claim, the designations are printed with
initial capital letters or in all capitals.
The Scala trademark belongs to {{???}}. Java is a trademark or registered trademark
of Oracle, Inc. in the United States and other countries. Windows is a registered
trademark of Microsoft Corporation in the United States and other countries. All
other product names and company names mentioned herein are the property of their
respective owners.
The authors and publisher have taken care in the preparation of this book, but make
no expressed or implied warranty of any kind and assume no responsibility for errors
or omissions. No liability is assumed for incidental or consequential damages in
connection with or arising out of the use of the information or programs contained
herein.
Visit us at www.EffectOrientedProgramming.com.
Source Code
All the source code for this book is available as copyrighted freeware, distributed
via Github². To ensure you have the most current version, this is the official code
distribution site. You may use this code in classroom and other educational situations
as long as you cite this book as the source.
The primary goal of this copyright is to ensure that the source of the code is properly
cited, and to prevent you from republishing the code without permission. (As long
as this book is cited, using examples from the book in most media is generally not a
problem.)
In each source-code file you find a reference to the following copyright notice:
²https://github.com/EffectOrientedProgramming/EOPCode
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
3
Copyright
// Copyright.txt
This computer source code is Copyright ©2021 MindView LLC.
All Rights Reserved.
Permission to use, copy, modify, and distribute this
computer source code (Source Code) and its documentation
without fee and without a written agreement for the
purposes set forth below is hereby granted, provided that
the above copyright notice, this paragraph and the
following five numbered paragraphs appear in all copies.
1. Permission is granted to compile the Source Code and to
include the compiled code, in executable format only, in
personal and commercial software programs.
2. Permission is granted to use the Source Code without
modification in classroom situations, including in
presentation materials, provided that the book "Effect Oriented
Programming" is cited as the origin.
3. Permission to incorporate the Source Code into printed
media may be obtained by contacting:
MindView LLC, PO Box 969, Crested Butte, CO 81224
MindViewInc@gmail.com
4. The Source Code and documentation are copyrighted by
MindView LLC. The Source code is provided without express
or implied warranty of any kind, including any implied
warranty of merchantability, fitness for a particular
purpose or non-infringement. MindView LLC does not
warrant that the operation of any program that includes the
Source Code will be uninterrupted or error-free. MindView
LLC makes no representation about the suitability of the
Source Code or of any software that includes the Source
Code for any purpose. The entire risk as to the quality
and performance of any program that includes the Source
Code is with the user of the Source Code. The user
understands that the Source Code was developed for research
and instructional purposes and is advised not to rely
exclusively for any reason on the Source Code or any
program that includes the Source Code. Should the Source
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
4
Copyright
Code or any resulting software prove defective, the user
assumes the cost of all necessary servicing, repair, or
correction.
5. IN NO EVENT SHALL MINDVIEW LLC, OR ITS PUBLISHER BE
LIABLE TO ANY PARTY UNDER ANY LEGAL THEORY FOR DIRECT,
INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES,
INCLUDING LOST PROFITS, BUSINESS INTERRUPTION, LOSS OF
BUSINESS INFORMATION, OR ANY OTHER PECUNIARY LOSS, OR FOR
PERSONAL INJURIES, ARISING OUT OF THE USE OF THIS SOURCE
CODE AND ITS DOCUMENTATION, OR ARISING OUT OF THE INABILITY
TO USE ANY RESULTING PROGRAM, EVEN IF MINDVIEW LLC, OR
ITS PUBLISHER HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH
DAMAGE. MINDVIEW LLC SPECIFICALLY DISCLAIMS ANY
WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE SOURCE CODE AND DOCUMENTATION PROVIDED
HEREUNDER IS ON AN "AS IS" BASIS, WITHOUT ANY ACCOMPANYING
SERVICES FROM MINDVIEW LLC, AND MINDVIEW LLC HAS NO
OBLIGATIONS TO PROVIDE MAINTENANCE, SUPPORT, UPDATES,
ENHANCEMENTS, OR MODIFICATIONS.
Please note that MindView LLC maintains a Web site which is
the sole distribution point for electronic copies of the
Source Code, where it is freely available under the terms
stated above:
https://github.com/EffectOrientedProgramming/EOPCode
If you think you've found an error in the Source Code,
please submit a correction at:
https://github.com/EffectOrientedProgramming/EOPCode/issues
You may use the code in your projects and in the classroom (including your
presentation materials) as long as the copyright notice that appears in each source
file is retained.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
5
Copyright
Edit This Chapter
Edit This Chapter³
³https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/00_Copyright.md
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Preface
Who is the book is for
This book uses Scala 3 but the focus is not on the language. The code examples should
be relatable to programmers who are familiar with: - Chaining operations on objects
("asdf ".trim.length) - Strong static typing - Functions that take parameters, can
be named, and referenced - TODO
This book also uses Scala ZIO, an effect-oriented library for Scala, but it does not go
into depth on ZIO. To learn ZIO, see: TODO
Code examples
The code examples are available at: TODO
The code in this book uses language syntax that might be unfamiliar.
• significant indentation
• colon syntax
Teaching With This Book
You may use the examples, exercises and solutions in classroom and other educational situations as long as you cite this book as the source. See the Copyright⁴ page
for further details.
The primary goal of the copyright is to ensure that the source of the code is properly
cited, and to prevent you from republishing the code without permission. As long
as this book is cited, using examples from the book in most media is generally not a
problem.
⁴\protect\char”007B\relax\protect\char”007B\relax???\protect\char”007D\relax\protect\char”007D\relax
7
Preface
Acknowledgements
Kit Langton, for being a kindred spirit in software interests and inspiring contributor
to the open source world.
Wyett Considine, for being an enthusiastic intern and initial audience.
Hali Frasure, for dozens of dinners and facilitating our book nights generally.
Edit This Chapter
Edit This Chapter⁵
⁵https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/01_Preface.md
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
What Are Effects
Introduction
TODO: Combine with “Reliability” ?
08:07 AM January 13, 2018
Televisions, Radios, and Cell Phones across Hawaii suddenly flash an alert:
“BALLISTIC MISSILE INBOUND THREAT TO HAWAII. SEEK IMMEDIATE
SHELTER. THIS IS NOT A DRILL”
Local communities sound alarms.
Calls to 911 jam the phone lines.
Panicked internet searches overwhelm data networks.
Students sprint from classrooms to fallout shelters.
Parents say goodbye to their children.
Untangling the Chaos
Thankfully, no missiles were launched that day.
During what should have been a quiet system test, an employee at the Hawaii
Emergency Management Agency accidentally pushed the wrong button. From the
Washington Post⁶:
“He clicked the button to send out an actual notification on Hawaii’s emergency
alert interface during what was intended to be a test of the state’s ballistic missile
preparations computer program.” The employee was prompted to choose between
the options “test missile alert” and “missile alert”, had selected the latter, initiating
the alert sent out across the state.
⁶https://www.washingtonpost.com/news/post-nation/wp/2018/01/14/hawaii-missile-alert-how-one-employeepushed-the-wrong-button-and-caused-a-wave-of-panic/
10
Introduction
Here is the system’s control screen:
This cluster of inconsistently named links increased the likelihood of mistakes. Basic
changes would drastically simplify proper use of the alerts. Imagine the earlier
mishaps that moved “False Alarm” to the top of the list.
We believe the system was doomed long before the interface was created. The fatal
flaw was that both “live” and “test” alerts were available in the running application.
A safe system makes these behaviors mutually exclusive.
The effects of this system were:
• Sending messages to Cell Phones
• Playing warnings on Radio frequencies
• Displaying banners on Television stations
The State of Software
There are many other examples of carefully-built software systems failing disastrously:
• The Ariane 5 rocket self-destructed on 4 June 1996 because of a malfunction in
the control software (the program tried to stuff a 64-bit number into a 16-bit
space).
• The American Northeast Power Blackout, August 14 2003.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
11
Introduction
• The NASA Mars Climate Orbiter, September 23, 1999. The orbiter was programmed for metric but ground control software used non-metric English.
The list goes on; just search for something like “Famous Software Failures” to see
more. And consider security- all the applications you use that are constantly being
updated with security patches. What about those that aren’t? Are they that good, or
is security being ignored?
How did things get so bad?
The Software Crisis
In the 70’s and 80’s, the idea of the Software Crisis emerged. This can be summarized
as: “We can’t create software fast enough.” One of the most popular attempts to solve
this problem was Structured Analysis & Design, which was a way to understand a
problem and design a solution using existing imperative languages.
The real problem that Structured Analysis & Design set out to solve was big
monolithic pieces of code. When one programmer was able to solve the entire
problem, the structure of the program didn’t matter as much. But as software needs
grew, this approach didn’t scale. In particular, you couldn’t finish a project more
quickly by adding more programmers, because there wasn’t a way to hand off
portions of a program to multiple programmers. To do that, teams needed some way
to break down the complexity of the program into individual functions—functions
that might someday be reused. This was seen as the reason for the Software Crisis.
Structured Analysis was an attempt to discover the individual functions in a program.
But it was a top-down approach, and it assumed these functions could be determined
before any code is written. Structured Analysis & Design continued the approach of
“big up-front design.” The analyst produced the structure, and then the programmers
implemented it.
Experienced programmers know that a design that cannot evolve during development is doomed to failure: both programmers and stakeholders learn things during
development. You discover much of your structure as you’re building the program,
and not on a whiteboard. Building a program reveals things you didn’t know were
important when you designed the solution.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
12
Introduction
From this book’s perspective, the most fundamental problem with Structured Analysis & Design was that it only paid lip service to the idea of reliability. There was
nothing about reliability truly integrated into Structured Analysis & Design.
Structured Analysis & Design was motivated by a business problem: “how do we
create software faster?” Virtually every language that came out in its aftermath
focused on development speed. Not reliability. So we produced a lot of languages
to quickly create unreliable software.
Reliability
A reliable system does not break. // TODO Discuss
If you’ve been programming for a while, this sounds far-fetched or even impossible.
Most existing languages are built for rapid development. You create a system as
quickly as possible, then begin isolating areas of failure, finding and fixing bugs until
the system is tolerable and can be delivered. Throughout the lifetime of the system,
bugs are regularly discovered and fixed. There is no realistic expectation that you will
ever achieve a completely bug-free system, just one that seems to work well enough
to meet the requirements. This is the reality programmers have come to accept.
If each piece of a traditional system is unreliable, when you combine these pieces
you get a multiplicative effect – the resulting parts are significantly less reliable than
their component pieces.
What if we could change our thinking around the problem of building software
systems? Imagine building small pieces that can each be reasoned about and made
rock-solid. Now suppose there is a way to combine these reliable pieces to make
bigger parts that are just as reliable. Each time you combine smaller parts to create a
bigger part, the result inherits the reliability of its components. Instead of multiplying
unreliability, you maintain reliability. The resulting system is as reliable as any of
its components.
This is what functional programming together with effects management can achieve.
This is what we want to teach you in this book.
The biggest impact on you as a programmer is the requirement for patience. With
most languages, the first thing you want to do is figure out how to write “Hello,
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
13
Introduction
World!”, then start accumulating the other language features as standalone concepts.
In functional programming we start by examining the impact of each concept on
reliability. We then combine the smaller concepts, ensuring reliability at each step.
A reliable system isolates parts that are always the same (pure functions) from the
parts that can change (effects). This mathematical rigor produces a reliable system.
Some aspects of writing code in this style might seem onerous. Most of us are used
to the more immediate feedback and satisfaction of getting something working, so
this can be challenging. But would you rather create an unreliable system quickly?
We assume you are reading this book because you do not.
The Effect is not what happens on the external system because there is no way to
know the actual impact of what the program caused by the communication.
Dealing with unpredictability
Any real program has to interact with things outside the programmer’s control.
All external systems are unpredictable.
One of the biggest challenges in building systems that are more predictable / reliable
/ ??? is to isolate and manage the unpredictable parts.
An approach that programmers may use to handle this is to delineate the parts of the
program which use external systems.
By delineating them, programmers then have tools to handle the unpredictable parts
in more predictable ways.
The interactions with external systems can be defined in terms of “Effects” which
create a delineation between the parts of a program that interact with external
systems and those that don’t.
For example, a program that displays the current date requires something that
actually knows the current date.
The program needs to talk to an external system (maybe just the operating system)
to get this information.
These programs are unpredictable because the programmer has no control over what
that external system will say or do.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
14
Introduction
Effect Oriented systems allow us to apply strategies to mitigate the unpredictability
of using external systems.
Effects can not be un-done
Once a program has communicated with an external system, (i.e. executed an Effect),
everything that happens on that external systems is out of the program’s control.
(analogies on human communication) Imagine that a friend recently stayed in your
home. 3 days after they leave, you realize that you are missing some money that
had been stored in the guestroom. Now you have a dilemma - do you ask them if
they took the money? Simply by asking, you could permanently change, or even end,
your relationship with this person. They could immediately admit fault, and ask for
forgiveness. Now you know that they are capable of stealing from you - will you
ever trust them in your home again? They could angrily deny the accusation, and
resent you for making it. Or the conversation could go in a million different other
ways that are impossible to predict. We know one thing for certain - you will never
be able to un-ask that question. Even if you ultimately grow closer with this person
after navigating this situation, you can’t go back to a world where you never asked.
Regardless of any apology and forgiveness, your relationship is now different.
What is an Effect?
An effect is an interaction with the world outside your CPU. An application might
generate any number of effects, which fall into two categories:
• Observing the World
• Changing the World
Effects cannot be undone. If you 3D-print a figurine, you cannot reclaim that material.
Once you send a Tweet, you can delete it but people might have already read it. Even
if you provide database DELETE statements paired with INSERT statements, it must
still be considered effectful: Another program might read your data before you delete
it, or a database trigger might activate during an INSERT.
TODO {{Explain: Optionality, Asynchronicity, Blocking – In a later chapter. }}
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
15
Introduction
Observing the World
Observation can be very basic:
• Accepting user input from the console
• Getting the current time from the system clock
• Taking the output of a random number generator
Observations can also be complex and domain-specific:
• Sensing slippage in an anti-lock braking system
• Getting the current price of a stock
• Detecting the current from a pacemaker
• Checking the temperature of a nuclear reactor
We explore similar scenarios throughout the book.
Changing the World
Just as with observations, changes can be basic:
• Displaying on the console
• Writing to a file
• Mutating a variable
• Saving to a database
They can be advanced:
• 3D printing a model
• Triggering an alarm
• Stabilizing an airplane
• Detonating explosives
• ZIOs are not their result.
Effects Defined as Data
TODO
One approach to defining effects is…
The effects have not been executed when defined.
A common mistake when starting with ZIO is trying to return ZIO instances
themselves rather than their result.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
16
Introduction
println(Random.nextInt)
// Stateful(repl.MdocSession.MdocApp.res0(02_What_Are_Effects.md:8),zio\
.FiberRef$unsafe$$anon$2$$Lambda$14980/0x0000000803cea840@3701ae45)
We will not see a random number printed out; we see some inscrutable type
information.
This is a mistake because ZIO’s are not their result, they are descriptions of effects
that produce the result. The ZIO instance only describes something to be done.
ZIOs are not automatically executed. To actually run a ZIO, your program must take
the data types and interpret / run them, executing the logic . The user must determine
when/where that happens.
Consider the Option type in the standard library. An Option might have a value
inside of it, but you can’t safely assume that it does. Similarly, a ZIO might produce
a value, but you have to run it to find out.
You can think of them as recipes for producing a value. You don’t want to return a
recipe from a function, you can only return a value. If it is your friend’s birthday,
they want a cake, not a list of instructions about mixing ingredients and baking.
The Interpreter
Scala compiles code to JVM bytecodes, An interpreter steps through and executes
your code, much like the JVM interprets JVM bytecodes. The interpreter is the
hidden piece understands so much more about the meaning of your code. This
includes the ability to decide what to run concurrently and how to invisibly tune
that concurrency–all at runtime. The interpreter is responsible for deciding when to
context-switch between tasks, and is able to do this because it understands the ZIO
code that it’s executing.
The interpreter is also the mechanism that evaluates the various effects described in
the generic type parameters for each ZIO object.
The reason we have the defer directive(method?) in zio-direct is to indicate that this
code will be evaluated by the interpreter later.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
17
Introduction
Edit This Chapter
Edit This Chapter⁷
⁷https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/02_What_Are_Effects.md
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Superpowers with Effects
TODO: A couple sentences about the superpowers
Building a Resilient Process in stages
We want to evolve our process from a simple happy path to a more resilient process.
Progressive enhancement through adding capabilities
TODO: Is “Step” necessary? Some other word?
Step 1: Happy Path
runDemo:
saveUser:
"mrsdavis"
// User saved
Step 2: Provide Error Fallback Value
object DatabaseError
object TimeoutError
Superpowers with Effects
runDemo:
saveUser:
"Robert'); DROP TABLE USERS"
.orElseFail:
"ERROR: User could not be saved"
// DatabaseError
// ERROR: User could not be saved
Step 3: Retry Upon Failure
import zio.Schedule.{recurs, spaced}
val aFewTimes =
// TODO Restore original spacing when done
// editing
// recurs(3) && spaced(1.second)
recurs(3) && spaced(1.millis)
runDemo:
saveUser:
"morty"
.retry:
aFewTimes
.orElseSucceed:
"ERROR: User could not be saved"
// DatabaseError
// DatabaseError
// User saved
Step 4: Fallback after multiple failures
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
19
Superpowers with Effects
runDemo:
saveUser:
"morty"
.retry:
aFewTimes
.orElseSucceed:
"ERROR: User could not be saved"
// DatabaseError
// DatabaseError
// DatabaseError
// DatabaseError
// ERROR: User could not be saved
Step 5: Timeouts
// TODO Restore real value when done editing
val timeLimit = 5.millis
// timeLimit: Duration = PT0.005S
// 5.seconds
// first is slow - with timeout and retry
runDemo:
saveUser:
"morty"
.timeoutFail(TimeoutError)(timeLimit)
.retry:
aFewTimes
.orElseSucceed:
"ERROR: User could not be saved"
// Interrupting slow request
// Database Timeout
// User saved
Step 6: Fallback Effect
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
20
Superpowers with Effects
// fails - with retry and fallback
runDemo:
saveUser:
"morty"
.timeoutFail(TimeoutError)(timeLimit)
.retry:
aFewTimes
.orElse:
sendToManualQueue:
"morty"
.orElseSucceed: // TODO Delete?
"ERROR: User could not be saved, even to the fallback system"
// DatabaseError
// DatabaseError
// DatabaseError
// DatabaseError
// User sent to manual setup queue
Step 7: Concurrently Execute Effect
TODO Consider deleting. Uses an extension
// concurrently save & send analytics
runDemo:
saveUser:
"morty"
// todo: maybe this hidden extension method
// goes too far with functionality that
// doesn't really exist
// TODO Should we fireAndForget before the
// retries/fallbacks?
.fireAndForget:
userSignupInitiated:
"morty"
.timeoutFail(TimeoutError)(timeLimit)
.retry:
aFewTimes
.orElse:
sendToManualQueue:
"morty"
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
21
Superpowers with Effects
.orElseSucceed:
"ERROR: User could not be saved"
// User saved
Step 8: Ignore failures in Concurrent Effect
Feeling a bit “meh” about this step.
// concurrently save & send analytics, ignoring analytics failures
runDemo:
// TODO Consider how to dedup strings
saveUser:
"mrsdavis"
.timeoutFail(TimeoutError)(timeLimit)
.retry:
aFewTimes
.orElse:
sendToManualQueue:
"mrsdavis"
.tapBoth(
error =>
userSignUpFailed("mrsdavis", error),
success =>
userSignupSucceeded("mrsdavis", success)
)
.orElseSucceed:
"ERROR: User could not be saved"
// Analytics sent for signup completion
// User saved
Step 9: Rate Limit TODO
TODO Slot these in:
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
22
Superpowers with Effects
23
Time
#### Measuring Time Since there is already a .timed method available directly on
ZIO instances, it might seem redundant to have a timed TestAspect. However, they
are distinct enough to justify their existence. ZIOs .timed methods changes the result
type of your code by adding the duration to a tuple in the result. This is useful, but
requires the calling code to handle this new result type. TestAspect.timed is a
non-invasive way to measure the duration of a test. The timing information will be
managed behind the scenes, and printed in the test output, without changing any
other behavior.
Restricting Time
Sometimes, it’s not enough to simply track the time that a test takes. If you have
specific Service Level Agreements (SLAs) that you need to meet, you want your
tests to help ensure that you are meeting them. However, even if you don’t have
contracts bearing down on you, there are still good reasons to ensure that your tests
complete in a timely manner. Services like GitHub Actions will automatically cancel
your build if it takes too long, but this only happens at a very coarse level. It simply
kills the job and won’t actually help you find the specific test responsible.
A common technique is to define a base test class for your project that all of your
tests extend. In this class, you can set a default upper limit on test duration. When a
test violates this limit, it will fail with a helpful error message.
This helps you to identify tests that have completely locked up, or are taking an
unreasonable amount of time to complete.
For example, if you are running your tests in a CI/CD pipeline, you want to ensure
that your tests complete quickly, so that you can get feedback as soon as possible. you
can use TestAspect.timeout to ensure that your tests complete within a certain
time frame.
Flakiness
Commonly, as a project grows, the supporting tests become more and more flaky.
This can be caused by a number of factors:
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Superpowers with Effects
24
• The code is using shared, live services Shared resources, such as a database or
a file system, might be altered by other processes. These could be other tests in
the project, or even unrelated processes running on the same machine.
• The code is not thread safe Other processes running simultaneously might alter
the expected state of the system.
• Resource limitations A team of engineers might be able to successfully run the
entire test suite on their personal machines. However, the CI/CD system might
not have enough resources to run the tests triggered by everyone pushing to
the repository. Your tests might be occasionally failing due to timeouts or lack
of memory.
Edit This Chapter
Edit This Chapter⁸
⁸https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/03_Superpowers.md
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Why Functional
TODO: Combine with “What are Effects” ?
In your journey to this book, you might have already learned one way to
think about programming, and possibly several.
You might have succeeded creating a project using that approach.
You might also have encountered programming constructs inspired by functional
programming.
For example, many languages have introduced lambdas along with support for
streams and functional operations like map.
Some languages allow functions to be - be created anonymously - stored in variables
- passed as parameters to other functions - returned from other functions
C++, especially more recent versions, has added a number of features that support
functional-style programming.
If, however, you are coming from an imperative programming background, these
functional-style devices can seem arbitrarily complicated. Why go through the
trouble of an effect-oriented API when plain vars, for loops, and :Unit exist?
Sometimes it seems like functional programmers write code like this just to be fancy.
To understand what’s really behind this different way of thinking about programming, we will start with some history.
A Different Goal
In the early days of programming, most code was written in an assembly language
for a particular machine.
Why Functional
26
Assembly language had primitive function-like constructs called subroutines, but
they were so much work to set up and safely use that programmers often just wrote
the code inline.
Even if you wanted to create reusable code, it was often easier to just “goto” a piece
of code and use global variables, rather than bothering with passing arguments and
returning results.
In those early days, a “high level language” meant a language like C that passed
arguments and returned results for you. This suddenly made writing functions much
easier, safer, and faster.
But the habits of assembly-language programmers didn’t vanish overnight, and
people were still prone to writing obtuse monolithic programs —often a single
function for the whole program— and jumping around in their code using gotos.
Many programs were written and maintained by a small number of programmers,
or even a single programmer, for whom that code made sense. Anyone else reading
the code would be baffled.
In addition, the idea of calling code written by other people was fairly foreign. If
you didn’t write the code yourself, how would you know if it does what you want?
Documentation and testing were also primitive, if they existed at all. Thus “code
reuse” was a big goal. There were many attempts within companies to create cultures
of code reuse, because many programmers were rewriting the same functionality
from scratch, over and over within the same organization.
Monolithic programs that didn’t reuse code were also a maintenance nightmare. It
was not uncommon for such programs to be thrown away and rewritten just to add
some new features. Not surprisingly, writing everything from scratch also took a lot
longer than reusing common functionality. The new question became: “how do we
make code reuse easier?”
Languages like C and Pascal made it easy to both write and call functions. They
were a big improvement over assembly language, However, the monolithic habits
of assembly-language programmers persisted into those new languages. Libraries
grew bigger and more complex, and using those libraries was not easy. In C, for
example, you would often have to allocate memory before calling a function, and
then remember to release the memory after that function returned.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Why Functional
27
You also had to learn how to pass information from one library function to another.
You had to learn how each library reported errors, which typically varied in strategy
from one library to the next. Code could be reused, but it wasn’t easy.
At this point, object-oriented programming brought several good ideas to the world.
It allowed you to package data structures with automatic initialization and cleanup,
with all the functions that act upon that data.
If you wanted to reuse some code, you created an object with some initialization
values, then sent messages to that object to produce the desired results. This did
make code reuse easier, and helped speed up program creation. It also came with the
distraction of inheritance polymorphism. This birthed an entire education and consulting industry explaining how to cram every design into an inheritance hierarchy
Inheritance polymorphism does sometimes prove useful, but not everywhere, all the
time.
C++ added object-oriented features from the Simula language while maintaining
backward compatibility with the C language. C++ had a strong emphasis on static
type checking. Java was created as a counterpoint to C++ and was heavily inspired
by the Smalltalk language.
Smalltalk’s success came from its ability to rapidly create systems by adding
functionality to existing objects. This introduced a conundrum, because Smalltalk
is a dynamic language, and Java, like C++, is statically typed. Smalltalk can be
thought of as supporting an experimental style of programming: you send a message
to an object and discover at runtime whether the object knows what to do with
that message. But C++ and Java ensure everything is valid, at compile time (along
with escape mechanisms that effectively disable that type checking). This conundrum
is exemplified by the Liskov Substitution Principle, which says that you shouldn’t
add new methods to an inherited type — and yet that activity is the foundation of
Smalltalk.
The Agile methodologies that began in the early 2000’s were another attempt to
produce software faster, but through a more bottom-up lens. Agile was primarily
focused on improving communication between stakeholders and developers, and
producing more rapid round trips between needs and experiments. This improves the
chance that the stakeholders will get what they need, faster. Agile helped the process
of software development, but again, the focus is on developing software quickly, not
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Why Functional
28
on developing reliable software.
The most important thing to take away from this language history is that the
fundamental goal of the various techniques was speed of creation. There seems to be
an underlying assumption that these approaches will somehow automatically create
more reliable software. As a result, we have languages that quickly create unreliable
software. And in many cases we’ve been able to get by with that. For one thing,
this approach has greatly advanced testing technology, because it was necessary.
Customers have learned to put up with buggy software. They’ve often been willing
to accept buggy software when the alternative is no software at all.
The world has changed. Back then, the drive was to speed up activities that humans
were doing. Those humans could compensate for bugs. Now, however, more and
more software is doing things that humans can’t do, so failures in software cannot
be propped up by humans. Unreliable software is no longer an inconvenience, but a
serious problem.
Quickly creating unreliable software is no longer acceptable. We must delve into
the reasons that software fails —either it doesn’t do what it’s supposed to, or it just
breaks.
Reuse
How do we create software? When you first learned to program, you probably solved
problems by writing code using the basic constructs of your language. But at some
point you began realizing that you could only produce and debug so much code
by yourself. If you could use code that was already written and debugged by other
people, you could produce solutions faster.
You might have gone through a cut-and-paste phase before discovering that formalized libraries were easier and more reliable. Even then, library ease of use depended
on the sophistication of your language. For the reasons mentioned, using a C library
could be tricky and difficult. C++ made this much easier and paved the way for
the acceptance of languages like Java, Python, Scala, and Kotlin. Indeed, any new
language that doesn’t support easy code reuse is not taken seriously.
But code reuse in object-oriented languages was still limited. You could either use
objects in a library directly, or you could add those library classes into new classes
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Why Functional
29
using composition. This was a big step, and it helped a lot. In contrast, composing C
libraries wasn’t particularly realistic —it was just too messy and complicated.
The problem is reliability. If you create a new class using composition, you combine
problems with the existing class(es) with any bugs you introduced in your new class.
As you build up bigger systems, the problem of bugs multiplies.
To compose systems rapidly and reliably, we return to first principles and figure out
how to:
1. Create basic components that are completely reliable.
2. Combine those components in a way that does not introduce new bugs.
To achieve these goals we must examine the fundamentals of how we think about
software.
Pure Functions
Composition in an object-oriented language doesn’t attempt to manage bugs, so it
ends up amplifying them. If we want to compose pieces of software, we must discover
what creates a fundamentally unbreakable piece, then how to assemble those pieces
without producing a broken result.
First, what constitutes a reliable,unbreakable piece of software? We’ve already seen
that objects are not inherently unbreakable, so we’ll move back to a more basic
software component: the function. What are the characteristics of an unbreakable
function?
What we want is the same kind of function we have in math. This means that the
function does nothing except produce a result from its arguments. And given the
same arguments, it always produces the same result.
This behavior imposes additional constraints: The function cannot affect its environment, and the environment cannot affect the function —otherwise, the function
has a history and behaves differently at one point in time vs. another. Running that
function doesn’t necessarily produce the same results from one call to the next.
If a function affects its environment, we call that a side effect. It’s “on the side”
because it’s something other than just producing a result from the function. Many
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Why Functional
30
programming languages have side effects built in, in the form of statements. A
statement doesn’t return a result, so the only reason to execute a statement is for
its side effect. For example, “print” is typically a statement that returns nothing
but causes the side effect of displaying text on a console. On the other hand, an
expression produces (“expresses”) a result. A functional language avoids statements
and attempts to make everything an expression that produces a result.
What about the environment affecting the function? This is a bit more subtle, and
requires that we think in more general terms than just basic imperative programming.
In particular, we must consider concurrency. If multiple tasks are running in our
program, then at any point another task might see variables in our function. A
variable can change, so that means this other task might see different values at
different points in the function’s execution. And if that variable is modified by some
other task, we have no way of predicting the result, and we don’t get the reliable
mathematical function that we want.
We solve this problem through immutability. That is, instead of using variables, we
create values that cannot change. This way, it doesn’t matter if an external task sees
our values, because it will only see that one value and not something that is different
from one moment to the next. And the external task cannot change the value and
cause the function to produce a different result.
Functions that behave mathematically, that always produce the same results from
the same inputs and have no side effects, are called pure functions. When we add the
additional constraint of immutability, we produce functions that compose without
introducing points of breakage. We can reliably reason about such functions.
Composability
TODO Reconsider this formal math style. It doesn’t match our other examples, where
we really stick to Scala code
If functions g and h are pure, we should be able to combine them directly (assuming
the types agree) to produce a new function f:
f(a) = g(h(a))
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Why Functional
31
This assumes that all functions involved are complete, meaning that they produce a
legitimate result for every possible value of a. This is not always true. For example,
dividing a number by zero is undefined, and so cannot produce a reasonable number
as a result. Using a key to look up a value in a map is undefined if that key doesn’t
exist in the map. <– TODO Weird wording
An incomplete function requires more operations when using it, to handle the
problematic inputs. You can think of the solution as stepwise composability. Instead
of calling g(h(a)), we break the process into steps: x = h(a), then check the
success of the operation. If successful, pass the result to g. These extra steps make
composability sound like it could get tedious, and languages like Scala that provide
more thorough support for functional programming provide syntax to make this kind
of programming feasible.
Effects
Now we have created this perfect world of pure functions that behave just like
the functions in theoretical mathematics. They have no side effects and cannot be
affected by other functions, and can be neatly and safely composed.
“But,” you wonder, “if all I can do with the result of one pure function is pass it as an
argument to another pure function, what’s the point of all these pure function calls?
If these functions have no effect on the world, they seem like an intellectual exercise
that merely heats up the CPU.”
This is absolutely true. A program that never affects the world is pointless. For a
program to be useful, it must be affected by the world, and it must have effects upon
the world.
The phrase “side effect” implies an incidental or accidental impact on the world. What
we need to do is formalize this idea and bring it under our control. We can then call
it simply an “effect,” without the “side.” The solution is to manage these effects, so
they are under our control.
This bridge between pure functions and practical programs with controlled and
managed effects is the reason for the title of this book.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Why Functional
32
Core Differences Between OO and Functional
An OO language worries about managing state. It “encapsulates” a data structure in
privacy and surrounds it with custom methods (aka member functions) which are
ideally the only way to access and modify the state of that data structure. This is
important because an OO data structure is typically mutable. This OO ceremony
attempts to create predictability by knowing how the data structure can be mutated.
Functional programming abstracts common behavior into reusable functional components. These components are adapted to specific needs using other functions. This
is why lambdas are so important, because you constantly need to adapt general code
to specific purposes, They enable concise code that would otherwise be awkward and
intrusive to right as a standalone function.
Functions in a functional language don’t need to be tied to a particular data structure.
Thus, they can often be written for more general use and to reduce duplication.
Functional languages come with a general set of well-tested, reusable operations that
can be applied in many situations.
A functional language relies on immutability. An immutable data structure doesn’t
need privacy because it is safe for any task to read, and it cannot be written (only
initialized). Ojects in functional languages are simply naked data structures along
with constructors. When everything is immutable, there is no need for private
properties or methods to maintain the state of an object.
Summary: Style vs Substance
Functional programming abstracts common behavior into reusable functional components. These components are adapted to specific needs using other functions. This
is why lambdas are so important, because you constantly need to adapt general code
to specific purposes, often with a brief amount of code that would otherwise be
awkward and intrusive to write as a standalone function.
The two things we do with functions is compose them to make more complex
functions, and adapt to them to our specific problem.
However, there could be significantly more than:
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Why Functional
33
• a function’s ability to create other functions
• transforming elements in a collection using map
Those are indeed important benefits, but they just dip into the possibilities. Adopting
some of the styles found in functional programming does not make a language
functional.
In this book we want to get to the heart of what it means to be functional. In particular,
we want to show what it takes to make reliable functional code that can be composed
without propagating or amplifying flaws in its components.
Edit This Chapter
Edit This Chapter⁹
⁹https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/04_Why_Functional.md
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Configuration
1. Application startup uses the same tools that you utilize for the rest of your
application
General/Historic discussion
One reason to modularize an application into “parts” is that the relationship between
the parts can be expressed and also changed depending on the needs for a given
execution path.
Typically, this approach to breaking things into parts and expressing what they need,
is called “Dependency Injection.”
… Why is it called “Dependency Injection” ? Avoid having to explicitly pass things
down the call chain.
There is one way to express dependencies.
Let’s consider an example: We want to write a function that fetches Accounts from a
database The necessary parts might be a DatabaseService which provides database
connections and a UserService which provides the access controls. By separating
these dependencies our from the functionality of fetching accounts, tests can “fake”
or “mock” the dependencies to simulate the actual dependency.
In the world of Java these dependent parts are usually expressed through annotations
(e.g. @Autowired in Spring). But these approaches are “impure” (require mutability),
often rely on runtime magic (e.g. reflection), and require everything that uses the
annotations to be created through a Dependency Injection manager, complicating
construction flow.
An alternative to this approach is to use “Constructor Injection” which avoids some
of the pitfalls associated with “Field Injection” but doesn’t resolve some of the
underlying issues, including the ability for dependencies to be expressed at compile
time.
Configuration
35
If instead functionality expressed its dependencies through the type system, the
compiler could verify that the needed parts are in-fact available given a particular
path of execution (e.g. main app, test suite one, test suite two).
What ZIO Provides Us.
With ZIO’s approach to dependencies, you get many desirable characteristics at
compile-time, using standard language features. Your services are defined as classes
with constructor arguments, just as in any vanilla Scala application. No annotations
that kick off impenetrable wiring logic outside your normal code.
For any given service in your application, you define what it needs in order to execute.
Finally, when it is time to build your application, all of these pieces can be provided in
one, flat space. Each component will automatically find its dependencies, and make
itself available to other components that need it.
To aid further in understanding your application architecture, you can visualize the
dependency graph with a single line.
You can also do things that simply are not possible in other approaches, such as
sharing a single instance of a dependency across multiple test classes, or even
multiple applications.
DI-Wow!
TODO Values to convey: - Layer Graph - Cycles are a compile error - Visualization
with Mermaid - test implementations - Layer Resourcefulness - Layers can have setup
& teardown (open & close)
TODO Rewrite this without
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Configuration
36
// Explain private constructor approach
case class Dough():
val letRise =
ZIO.debug:
"Dough is rising"
object Dough:
val fresh =
ZLayer
.derive[Dough]
.tapWithMessage("Making Fresh Dough")
Step 1: Effects can express dependencies
Effects can’t be run until their dependencies have been fulfilled
TODO: Decide what to do about the compiler error differences between these
approaches
TODO: Can we avoid the .provide() and still get a good compile
error
in
mdoc
scala runDemo: ZIO.serviceWithZIO[Dough](_-
.letRise) .provide() // error: // // // ──── ZLAYER ERROR
──────────────────────────────────────────────────── //
// Please provide a layer for the following type: // // 1.
repl.MdocSession.MdocApp.Dough // // ───────────────────────────────────────
// //
Step 2: Provide Dependency Layers to Effects
Then the effect can be run.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Configuration
37
runDemo:
ZIO.serviceWithZIO[Dough]:
_.letRise
.provide:
Dough.fresh
// Making Fresh Dough
// Dough is rising
// ()
For code organization, and legibility at call sites, we are defining several layers within
the Heat companion object. They will all be used soon.
case class Heat()
object Heat:
val oven =
ZLayer
.derive[Heat]
.tapWithMessage:
"Heating Oven"
val toaster =
ZLayer
.derive[Heat]
.tapWithMessage:
"Heating Toaster"
val broken =
ZLayer.fail:
"**Power Out**"
Step 3: Effects can require multiple
dependencies
Note: The following is copy&pasted and might just need a slight diversion
to &’d typed parameters
Intersections AKA Products AKA Case Classes AKA Ands
The requirements for each ZIO operation are combined as an anonymous product
type denoted by the & symbol.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Configuration
38
// TODO Restore private constructor after failure scenario is dialed in
// TODO Can we make Bread a trait,
//
Then we would have BreadHomemade & BreadStoreBought
trait Bread:
val eat =
ZIO.debug:
"Eating bread!"
case class BreadHomeMade(heat: Heat, dough: Dough) extends Bread
// TODO Move StoreBought further down?
case class BreadStoreBought() extends Bread
object Bread:
// TODO Explain ZLayer.fromZIO in prose
// immediately before/after this
val homemade =
ZLayer
.derive[BreadHomeMade]
.tapWithMessage:
"Making Homemade Bread"
val storeBought =
ZLayer
.derive[BreadStoreBought]
.tapWithMessage:
"Buying Bread"
end Bread
runDemo:
Bread
.homemade
.build
.provide(
Dough.fresh,
Heat.oven,
Scope.default
)
// ZEnvironment(MdocSession::MdocApp::BreadHomeMa
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Configuration
39
Step 4: Dependencies can “automatically”
assemble to fulfill the needs of an effect
Something around how like typical DI, the “graph” of dependencies gets resolved
“for you” This typically happens in some completely new/custom phase, that does
follow standard code paths. Dependencies on effects propagate to effects which use
effects.
// TODO Figure out why Bread.eat debug isn't showing up
runDemo:
ZIO.serviceWithZIO[Bread]:
_.eat
.provide(
// Highlight that homemade needs the other
// dependencies.
Bread.homemade,
Dough.fresh,
Heat.oven
)
// ()
Step 5: Different effects can require the same
dependency
Eventually, we grow tired of eating plain Bread and decide to start making Toast.
Both of these processes require Heat.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Configuration
40
// Is it worth the complexity of making this private?
// It would keep people from creating Toasts without using the make met\
hod
case class Toast (heat: Heat, bread: Bread)
object Toast:
val make =
ZLayer.derive[Toast]
.tapWithMessage("Making Toast")
It is possible to also use the oven to provide Heat to make the Toast.
TODO Update refs here The dependencies are based on the type, so in this case both
Toast.make and Bread.make require heat, but
Notice - Even though we provide the same dependencies in this example, Heat.oven
is also required by Toast.make
runDemo:
ZLayer.make[Toast](
Toast.make,
Bread.homemade,
Dough.fresh,
Heat.oven
).build
// ZEnvironment(MdocSession::MdocApp::Toast -> To
However, the oven uses a lot of energy to make Toast. It would be great if we can
instead use our dedicated toaster!
Step 6: Dependencies must be fulfilled by
unique types
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Configuration
41
runDemo:
ZLayer.make[Toast](
Toast
.make,
Dough.fresh,
Bread.homemade,
Heat.oven,
Heat.toaster
).build
// error:
//
//
// ──── ZLAYER ERROR ──────────────────────────────────────────────────\
──
//
// Ambiguous layers! I cannot decide which to use.
// You have provided more than one layer for the following type:
//
//
repl.MdocSession.MdocApp.Heat is provided by:
//
1. Heat.oven
//
2. Heat.toaster
//
// ────────────────────────────────────────────────────────────────────\
──
//
//
Unfortunately our program is now ambiguous. It cannot decide if we should be
making Toast in the oven, Bread in the toaster, or any other combination.
Step 7: Providing Dependencies at Different
Levels
This enables other effects that use them to provide their own dependencies of the
same type
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Configuration
runDemo:
val bread =
ZLayer.make[Bread](
Bread.homemade,
Dough.fresh,
Heat.oven
)
ZLayer.make[Toast](
Toast.make,
bread,
Heat.toaster
).build
// ZEnvironment(MdocSession::MdocApp::Toast -> To
Step 8: Dependencies can fail
TODO Explain .build before using it to demo layer construction
Bread2.fromFriend: ZLayer[Any, String, Bread]
runDemo:
ZIO.serviceWithZIO[Bread]:
_.eat
.provide:
Bread2.fromFriend
// **Power out**
// **Power out Rez**
Step 9: Dependency Retries
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
42
Configuration
runDemo:
val bread =
Bread2
.fromFriend
.retry:
Schedule.recurs:
3
ZIO.serviceWithZIO[Bread]:
_.eat
.provide:
bread
// **Power out**
// **Power out**
// Power is on
// ()
Step 10: Fallback Dependencies
runDemo:
val bread =
Bread2
.fromFriend
.orElse:
Bread.storeBought
ZLayer.make[Toast](
Toast.make,
bread,
Heat.toaster
).build
// **Power out**
// ZEnvironment(MdocSession::MdocApp::Toast -> To
Step 11: Layer Retry + Fallback?
Maybe retry on the ZLayer eg. (BreadDough.rancid, Heat.brokenFor10Seconds)
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
43
Configuration
44
runDemo:
Bread2
.fromFriend
.retry:
Schedule.recurs:
1
.orElse:
Bread.storeBought
.build // TODO Stop using build, if possible
.debug
// **Power out**
// **Power out**
// ZEnvironment(MdocSession::MdocApp::BreadStoreB
Changing things based on the running environment.
• CLI Params
• Config Files
• Environment Variables
Testing Effects
Effects need access to external systems thus are unpredictable.
Tests are ideally predictable so how do we write tests for effects that are predictable?
With ZIO we can replace the external systems with predictable ones when running
our tests.
With ZIO Test we can use predictable replacements for the standard systems effects
(Clock, Random, Console, etc).
Random
An example of this is Random numbers. Randomness is inherently unpredictable. But
in ZIO Test, without changing our Effects we can change the underlying systems with
something predictable:
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Configuration
import zio.test.{TestRandom, TestAspect, assertCompletes}
val coinToss =
defer:
if (Random.nextBoolean.run)
ZIO
.debug:
"R: Heads."
.run
else
ZIO
.fail:
"Tails encountered. Ending performance."
.run
val rosencrantzAndGuildensternAreDead =
defer:
TestRandom
.feedBooleans(Seq.fill(8)(true) *)
.run
ZIO
.debug:
"*Performance Begins*"
.run
coinToss.repeatN(4).run
ZIO
.debug:
"G: There is an art to building suspense."
.run
coinToss.run
ZIO
.debug:
"G: Though it can be done by luck alone."
.run
coinToss.run
ZIO
.debug:
"G: ...probability"
.run
coinToss.run
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
45
Configuration
46
runSpec:
defer:
rosencrantzAndGuildensternAreDead.run
assertCompletes
// Test: PASSED*
val spec =
defer:
rosencrantzAndGuildensternAreDead.run
assertCompletes
// todo: maybe we can change runSpec so that this spec structure matche\
s the previous
runSpec(spec, TestAspect.withLiveRandom, TestAspect.eventually)
// Test: PASSED*
The Random Effect uses an injected something which when running the ZIO uses
the system’s unpredictable random number generator. In ZIO Test the Random
Effect uses a different something which can predictably generate “random” numbers.
TestRandom provides a way to define what those numbers are. This example feeds
in the Ints 1 and 2 so the first time we ask for a random number we get 1 and the
second time we get 2.
Anything an effect needs (from the system or the environment) can be substituted
in tests for something predictable. For example, an effect that fetches users from a
database can be simulated with a predictable set of users instead of having to setup
a test database with predictable users.
When your program treats randomness as an effect, testing unusual scenarios
becomes straightforward. You can preload “Random” data that will result in deterministic behavior. ZIO gives you built-in methods to support this.
Time
Even time can be simulated as using the clock is an effect.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Configuration
47
import zio.test.*
runSpec:
val slowOperation =
ZIO.sleep:
2.seconds
defer:
val fork =
slowOperation
.timeout:
1.second
.fork
.run
TestClock
.adjust:
2.seconds
.run
val result = fork.join.run
assertTrue:
result.isEmpty
// Test: PASSED*
By default in ZIO Test, the clock does not change unless instructed to. Calling a time
based effect like timeout would hang indefinitely with a warning like: Warning:
A test is using time, but is not advancing the test clock, which
may result in the test hanging. Use TestClock.adjust to manually
advance the time.
To test time based effects we need to fork those effects so that then we can adjust
the clock. After adjusting the clock, we can then join the effect where in this case
the timeout has then been reached causing the effect to return a None.
Using a simulated Clock means that we no longer rely on real-world time for time.
So this example runs in milliseconds of real-world time instead of taking an actual
1 second to hit the timeout. This way our time-based tests run much more quickly
since they are not based on actual system time. They are also more predictable as the
time adjustments are fully controlled by the tests.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Configuration
48
Targeting Error-Prone Time Bands
Using real-world time also can be error prone because effects may have unexpected
results in certain time bands. For instance, if you have code that gets the time and it
happens to be 23:59:59, then after some operations that take a few seconds, you get
some database records for the current day, those records may no longer be the day
associated with previously received records. This scenario can be very hard to test
for when using real-world time. When using a simulated clock in tests, you can write
tests that adjust the clock to reliably reproduce the condition.
Edit This Chapter
Edit This Chapter¹⁰
¹⁰https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/05_Configuration.md
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Errors
1. Creating & Handling
2. Error composability
3. Retry
In a language that cannot throw, following the execution path is simple, following 2
basic rules:
- At a branch, execute only the first match
- Otherwise, Read everything from left-to-right, top-to-bottom,
Once you add throw, the rules are more complicated
t
At a branch, execute only the first match
Otherwise, Read everything from left-to-right, top-to-bottom,
Unless we `throw`, which means immediately jumping through a differen\
dimension away from the code you're viewing
Historic approaches to Error-handling
If you are not interested in the discouraged ways to handle errors, and just want to
see the ZIO approach, jump down to ZIO Error Handling
In the past, some programs have thrown exceptions to indicate failures. Imagine a
program that displays the local temperature the user based on GPS position and a
network call. There are distinct levels of problems in any given program. They require
different types of handling by the programmer.
Temperature: 30 degrees
50
Errors
class GpsException()
extends RuntimeException
class NetworkException() extends RuntimeException
enum Scenario:
case Success,
NetworkError,
GPSError
def render(value: String) =
s"Temperature: $value"
def calculateTemp(behavior: Scenario): String =
behavior match
case Scenario.GPSError =>
throw GpsException()
case Scenario.NetworkError =>
throw NetworkException()
case Scenario.Success =>
"35 degrees"
def currentTemperatureUnsafe(
behavior: Scenario
): String =
render:
calculateTemp:
behavior
currentTemperatureUnsafe:
Scenario.Success
// res0: String = "Temperature: 35 degrees"
On the happy path, everything looks as desired. If the network is unavailable, what is
the behavior for the caller? This can take many forms. If we don’t make any attempt
to handle our problem, the whole program blows up and shows the gory details to
the user.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
51
Errors
// Note - Can't make this output prettier/simpler because it's *not* us\
ing ZIO
currentTemperatureUnsafe:
Scenario.NetworkError
// repl.MdocSession$MdocApp$NetworkException
//
at repl.MdocSession$MdocApp.calculateTemp(06_Errors.md:28)
//
at repl.MdocSession$MdocApp.currentTemperatureUnsafe(06_Errors.md:4\
0)
//
at repl.MdocSession$MdocApp.$init$$$anonfun$1(06_Errors.md:53)
We could take the bare-minimum approach of catching the Exception and returning
null:
def currentTemperatureNull(
behavior: Scenario
): String =
render:
try
calculateTemp:
behavior
catch
case ex: RuntimeException =>
null
currentTemperatureNull:
Scenario.NetworkError
// res1: String = "Temperature: null"
This is slightly better, as the user can at least see the outer structure of our UI element,
but it still leaks out code-specific details world.
Maybe we could fallback to a sentinel value, such as 0 or -1 to indicate a failure?
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
52
Errors
def currentTemperature(
behavior: Scenario
): String =
render:
try
calculateTemp:
behavior
catch
case ex: RuntimeException =>
"-1 degrees"
currentTemperature:
Scenario.NetworkError
// res2: String = "Temperature: -1 degrees"
Clearly, this isn’t acceptable, as both of these common sentinel values are valid
temperatures. We can take a more honest and accurate approach in this situation.
def currentTemperature(
behavior: Scenario
): String =
render:
try
calculateTemp:
behavior
catch
case ex: RuntimeException =>
"Unavailable"
currentTemperature:
Scenario.NetworkError
// res3: String = "Temperature: Unavailable"
We have improved the failure behavior significantly; is it sufficient for all cases?
Imagine our network connection is stable, but we have a problem in our GPS
hardware. In this situation, do we show the same message to the user? Ideally, we
would show the user a distinct message for each scenario. The Network issue is
transient, but the GPS problem is likely permanent.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
53
Errors
def currentTemperature(
behavior: Scenario
): String =
try
render:
calculateTemp:
behavior
catch
case ex: NetworkException =>
"Network Unavailable"
case ex: GpsException =>
"GPS problem"
currentTemperature:
Scenario.NetworkError
// res4: String = "Network Unavailable"
currentTemperature:
Scenario.GPSError
// res5: String = "GPS problem"
Wonderful! We have specific messages for all relevant error cases. However, this still
suffers from downsides that become more painful as the codebase grows.
• The signature of currentTemperature does not alert us that it might fail
• If we realize it can fail, we must dig through the implementation to discover
the multiple failure values
• We never have certainty about the failure paths of our full application, or any
subset of it.
{{ TODO Tear apart exceptions more }}
Encountering an error during a function call generally means two things:
1. You can’t continue executing the function in the normal fashion.
2. You can’t return a normal result.
Many languages use exceptions for handling errors. An exception throws out of the
current execution path to locate a user-written handler to deal with the error. There
are two goals for exceptions:
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
54
Errors
1. Separate error-handling code from “success-path” code, so the success-path
code is easier to understand and reason about.
2. Reduce redundant error-handling code by handling associated errors in a single
place.
Exceptions have problems:
1. They can be “swallowed.” Just because code throws an exception, there’s no
guarantee that issue will be dealt with.
2. They can lose important information. Once an exception is caught, it is
considered to be “handled,” and the program doesn’t need to retain the failure
information.
3. They aren’t typed. Java’s checked exceptions provide a small amount of type
information, but it’s not that helpful compared to a full type system. Unchecked
exceptions provide no information at all.
4. Because they are handled dynamically, the only way to ensure your program
won’t crash is by testing it through all possible execution paths. A staticallytyped error management solution can ensure—at compile time—that all errors
are handled.
5. They don’t scale. {{Need to think about this more to make the case.}}
6. Hard to reason about. {{Also need to make this case}}
7. Difficult or impossible to retry an operation if it fails. Java {{and Scala?}} use
the “termination” model of exception handling. This assumes the error is so
critical there’s no way to get back to where the exception occurred. If you’re
performing an operation that you’d like to retry if it fails, exceptions don’t help
much.
Exceptions were a valiant attempt to produce a consistent error-reporting interface,
and they are definitely better than what’s in C. But they don’t end up solving the
problem very well, and you just don’t know what you’re going to get when you use
exceptions.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
55
Errors
What’s wrong with Try?
ADTS as another step forward
ZIO Error Handling
Now we will explore how ZIO enables more powerful, uniform error-handling.
TODO {{Update verbiage now that ZIO section is first}}
• ZIO Error Handling
• Wrapping Legacy Code
ZIO-First Error Handling
def getTemperatureZ(behavior: Scenario) =
behavior match
case Scenario.GPSError =>
ZIO.fail:
GpsException()
case Scenario.NetworkError =>
// TODO Use a non-exceptional error
ZIO.fail:
NetworkException()
case Scenario.Success =>
ZIO.succeed:
"35 degrees"
runDemo:
getTemperatureZ:
Scenario.Success
// 35 degrees
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
56
Errors
// TODO make MDoc:fail adhere to line limits?
runDemo:
getTemperatureZ:
Scenario.Success
.catchAll:
case ex: NetworkException =>
ZIO.succeed:
"Network Unavailable"
// error:
// match may not be exhaustive.
//
// It would fail on pattern case: _: GpsException
//
TODO Demonstrate ZIO calculating the error types without an explicit annotation
being provided
runDemo:
getTemperatureZ:
Scenario.GPSError
// repl.MdocSession$MdocApp$GpsException
Wrapping Legacy Code
If we are unable to re-write the fallible function, we can still wrap the call We are
re-using the calculateTemp
{{TODO }}
def calculateTempWrapped(
behavior: Scenario
) =
ZIO.attempt:
calculateTemp:
behavior
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
57
Errors
def displayTemperatureZWrapped(
behavior: Scenario
) =
calculateTempWrapped:
behavior
.catchAll:
case ex: NetworkException =>
ZIO.succeed:
"Network Unavailable"
case ex: GpsException =>
ZIO.succeed:
"GPS problem"
runDemo:
displayTemperatureZWrapped:
Scenario.Success
// 35 degrees
runDemo:
displayTemperatureZWrapped:
Scenario.NetworkError
// Network Unavailable
This is decent, but does not provide the maximum possible guarantees. Look at what
happens if we forget to handle one of our errors.
def getTemperatureZGpsGap(
behavior: Scenario
) =
calculateTempWrapped:
behavior
.catchAll:
case ex: NetworkException =>
ZIO.succeed:
"Network Unavailable"
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
58
Errors
runDemo:
getTemperatureZGpsGap:
Scenario.GPSError
// Defect: GpsException
The compiler does not catch this bug, and instead fails at runtime. Take extra care
when interacting with legacy code, since we cannot automatically recognize these
situations at compile time. We have 2 options in these situations.
First, we can provide a fallback case that will report anything we missed:
def getTemperatureZWithFallback(
behavior: Scenario
) =
calculateTempWrapped:
behavior
.catchAll:
case ex: NetworkException =>
ZIO.succeed:
"Network Unavailable"
case other =>
ZIO.succeed:
"Error: " + other
runDemo:
getTemperatureZWithFallback:
Scenario.GPSError
// Error: repl.MdocSession$MdocApp$GpsException
This lets us avoid the most egregious gaps in functionality, but it does not take full
advantage of ZIO’s type-safety.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
59
Errors
def getTemperatureZAndFlagUnhandled(
behavior: Scenario
) =
calculateTempWrapped:
behavior
.catchSome:
case ex: NetworkException =>
ZIO.succeed:
"Network Unavailable"
// TODO Eh, find a better version of this.
.mapError(_.asInstanceOf[GpsException])
runDemo:
getTemperatureZAndFlagUnhandled:
Scenario.GPSError
// repl.MdocSession$MdocApp$GpsException
{{TODO show catchSome}}
Note: The following is copy&pasted and needs work
Unions AKA Sum Types AKA Enums AKA Ors
Note - Avoid official terminology in most prose. Just say “And”/”Or” where appropriate.
Scala 3 automatically aggregates the error types by synthesizing an anonymous sum
type from the combined errors.
Functions usually transform the Answer from one type to another type. Errors often
aggregate.
Consider 2 error types
case class UserNotFound()
case class PermissionError()
In the type system, the most recent ancestor between them is Any.
Unfortunately, you cannot make any meaningful decisions based on this type.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
60
Errors
graph TD;
UserNotFound-->Nothing;
PermissionError-->Nothing;
We need a more specific way to indicate that our code can fail with either of these
types. The | (or) tool provides maximum specificity without the need for inheritance.
TODO Figure out how to use pipe symbol in Mermaid
graph TD;
UserNotFound-->UserNotFound_OR_PermissionError;
PermissionError-->UserNotFound_OR_PermissionError;
UserNotFound-->Nothing;
PermissionError-->Nothing;
Often, you do not care that Nothing is involved at all. The mental model can be
simply:
graph TD;
UserNotFound-->UserNotFound_OR_PermissionError;
PermissionError-->UserNotFound_OR_PermissionError;
case class UserService()
case class User(name: String)
case class SuperUser(name: String)
def getUser(
userId: String
) =
if (userId == "morty" || userId = "rick")
ZIO.succeed:
User(userId)
else
ZIO.fail:
UserNotFound()
def getSuperUser(
user: User
) =
if (user.name = "rick")
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
61
Errors
ZIO.succeed:
SuperUser(user.name)
else
ZIO.fail:
PermissionError()
def loginSuperUser(userId: String) =
defer:
val basicUser = getUser(userId).run
getSuperUser(basicUser).run
Edit This Chapter
Edit This Chapter¹¹
¹¹https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/06_Errors.md
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Contract-Based Composability
Good contracts make good composability.
contracts are what makes composability work at scale our effects put in place
contracts on how things can compose exceptions do not put in place a contract
maybe something about how exceptions do not convey a contract in either direction.
Anything can be wrapped with a try. Things that produce exceptions don’t need to
be wrapped with trys.
possible example of Scope for Environment contracts
possible contract on provide for things not needed
ZIO.succeed("asdf").someOrFail("error")
// error:
//
// This operator requires that the output type be a subtype of Option[A\
ny]
// But the actual type was String..
// I found:
//
//
IsSubtypeOfOutput.impl[A, B](/* missing */summon[A <:< B])
//
// But no implicit values were found that match type A <:< B.
// def logAndProvideDefault(e: Throwable) =
//
this works as the contract is that the
Contract-Based Composability
63
ZIO.succeed(maybeThing()).someOrFail("error")
// res1: ZIO[Any, String, Unit] = OnSuccess(
//
trace = "repl.MdocSession.MdocApp.res1(07_Composability.md:20)",
//
first = Sync(
//
trace = "repl.MdocSession.MdocApp.res1(07_Composability.md:20)",
//
eval = zio.ZIOCompanionVersionSpecific$$Lambda$14961/0x000000080\
3cd6840@5d634918
//
),
//
successK = zio.ZIO$$Lambda$17731/0x0000000801e5b040@2e13e6d2
// )
ZIO
.succeed(println("Always gonna work"))
.retryN(100)
// error:
// This error handling operation assumes your effect can fail. However,\
your effect has Nothing for the error type, which means it cannot fail\
, so there is no need to handle the failure. To find out which method y\
ou can use instead of this operation, please see the reference chart at\
: https://zio.dev/can_fail.
// I found:
//
//
CanFail.canFail[E](/* missing */summon[util.NotGiven[E =:= Nothi\
ng]])
//
// But no implicit values were found that match type util.NotGiven[E =:\
= Nothing].
// def logAndProvideDefault(e: Throwable) =
//
ZIO
.attempt(println("This might work"))
.retryN(100)
// res3: ZIO[Any, Throwable, Unit] = OnSuccess(
//
trace = "repl.MdocSession.MdocApp.res3(07_Composability.md:35)",
//
first = Sync(
//
trace = "repl.MdocSession.MdocApp.res3(07_Composability.md:35)",
//
eval = zio.ZIOCompanionVersionSpecific$$Lambda$14961/0x000000080\
3cd6840@43f99fc3
//
),
//
successK = zio.ZIO$$$Lambda$14963/0x0000000803cd5040@2da21ecd
// )
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Contract-Based Composability
64
is this about surfacing the hidden information through a “bookkeeper” that conveys
the constraints to the caller
An essential part of creating programs is the ability to combine small pieces into
larger pieces.
Different languages / paradigms provide different ways to accomplish these combinations.
Objects can be combined by creating objects that contain other objects.
Functions can be combined by creating new functions that call other functions.
These are types of “composition” but these traditional approaches do not address all
of the aspects of a program.
For example, functions that use resources which need to be opened and closed, do
not compose.
ZIOs compose including errors, async, blocking, resource managed, cancellation,
eitherness, environmental requirements.
Composability Explanation
1. But Functions & Specialized Data Types Don’t Compose for Effects
2. Composability 1. Limitations of Functions & SDTs 1. Some intro to Universal
Effect Data Types ie ZIO 1. The ways in which ZIOs compose (contrasted
to limitations) 1. Note: Merge chapters: composability, Unit, The_ZIO_Type 1.
Note: Avoid explicit anonymous sum & product types at this point
Alternatives and their downsides
Other framings/techniques and their pros/cons:
Plain functions that return Unit
Unit can be viewed as the bare minimum of effect tracking.
Consider a function
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Contract-Based Composability
65
def saveInformation(info: String): Unit = ???
If we look only at the types, this function is an Any=>Unit. Unit is the single, blunt
tool to indicate effectful functions in plain Scala. When we see it, we know that some
type of side-effect is being performed.
When a function returns Unit, we know that the only reason we are calling the
function is to perform an effect. Alternatively, if there are no arguments to the
function, then the input is Unit, indicating that an effect is used to produce the result.
Unfortunately, we can’t do things like timeout/race/etc these functions. We can either
execute them, or not, and that’s about it, without resorting to additional tools for
manipulating their execution.
Plain functions that throw Exceptions
• We cannot union these error possibilities and track them in the type system
• Cannot attach behavior to deferred functions
Plain functions that block
• We can’t indicate if they block or not
• Too many concurrent blocking operations can prevent progress of other operations
• Very difficult to manage
• Blocking performance varies wildly between environments
Functions that return Either/Option/Try/etc
• We can manage the errors in the type system, but we can’t interrupt the code
that is producing these values
• All of these types must be manually transformed into the other types
• Execution is not deferred
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Contract-Based Composability
66
Functions that return a Future
•
•
•
•
•
•
•
Can be interrupted example1[future_interrupted_1] two[future_interrupted_2]
Cleanup is not guaranteed¹²
Manual management of cancellation
Start executing immediately
Must all fail with Exception
Implicits
Are not automatically managed by the compiler, you must explicitly add each
one to your parent function
• Resolving the origin of a provided implicit can be challenging
Try-with-resources
• These are statically scoped
• Unclear who is responsible for acquisition & cleanup
Each of these approaches gives you benefits, but you can’t assemble them all
together. Instead of the best of all worlds, you get the pain of all worlds. eg
Closeable[Future[Either[Throwable, A]]] The ordering of the nesting is significant, and not easily changed.
The number of combinations is something like: PairsIn(numberOfConcepts)
Universal Composability with ZIO
ZIOs compose including errors, async, blocking, resource managed, cancellation,
eitherness, environmental requirements.
The types expand through generic parameters. ie composing a ZIO with an error of
String with a ZIO with an error of Int results in a ZIO with an error of String |
Int.
¹²./15_Concurrency_Interruption.md#Future-Cancellation
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Contract-Based Composability
67
With functions there is one way to compose. f(g(h)) will sequentially apply the
functions from the inside out.
Another term for this form of composition is called andThen in Scala.
With ZIO you can use zio-direct to compose ZIOs sequentially with:
runDemo:
defer:
val topStory = findTopNewsStory.run
textAlert:
topStory
.run
// Texting story: Battery Breakthrough
// ()
There are many other ways you can compose ZIOs. The methods for composability
depend on the desired behavior. For example, to compose a ZIO that can produce an
error with a ZIO that logs the error and then produces a default value, you can use
the catchAll like:
def logAndProvideDefault(e: Throwable) =
Console
.printLine:
e.getMessage
.as:
"default value"
runDemo:
ZIO
.attempt:
???
.catchAll:
logAndProvideDefault
// an implementation is missing
// default value
All The Thing Example
….
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Contract-Based Composability
68
Hedging
TODO Determine final location for Hedging ### Why? This technique snips off the
most extreme end of the latency tail.
Determine the average response time for the top 50% of your requests. If you make
a call that does not get a response within this average delay, make an additional,
identical request. However, you do not give up on the 1st request, since it might
respond immediately after sending the 2nd. Instead, you want to race them and use
the first response you get
To be clear - this technique will not reduce the latency of the fastest requests at all.
It only alleviates the pain of the slowest responses.
You have 1/n chance of getting the worst case response time. This approach turns
that into a 1/n^2 chance. The cost of this is only ∼3% more total requests made.
Citations needed
Further, if this is not enough to completely eliminate your extreme tail, you can
employ the exact same technique once more. Then, you end up with 1/n^3 chance
of getting that worst performance.
Repeats
Repeating is a form of composability, because you are composing a program with
itself
Injecting Behavior before/after/around
Edit This Chapter
Edit This Chapter¹³
¹³https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/07_Composability.md
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Contract-Based Composability
69
Automatically attached experiments.
These are included at the end of this chapter because their package in the experiments
directory matched the name of this chapter. Enjoy working on the code with full
editor capabilities :D
experiments/src/main/scala/composability/AllTheThings.scala
package composability
import zio.*
import scala.concurrent.Future
import zio.direct.*
import scala.Option
import scala.util.Try
trait NewsService:
def getHeadline(): Future[String]
trait ContentAnalyzer:
def findTopicOfInterest(
content: String
): Option[String]
trait HistoricalRecord:
case class DetailedHistory(content: String)
case class NoRecordsAvailable(reason: String)
def summaryFor(
topic: String
): Either[NoRecordsAvailable, DetailedHistory]
trait CloseableFile extends AutoCloseable:
def existsInFile(searchTerm: String): Boolean
def write(entry: String): Try[Unit]
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Contract-Based Composability
case class Scenario(
newsService: NewsService,
contentAnalyzer: ContentAnalyzer,
historicalRecord: HistoricalRecord,
closeableFile: CloseableFile
):
def asyncThing(i: Int) = ZIO.sleep(i.seconds)
val resourcefulThing =
val open =
defer:
ZIO.debug("open").run
"asdf"
val close = (_: Any) => ZIO.debug("close")
ZIO.acquireRelease(open)(close)
val logic =
defer:
// todo: useful order, maybe async first or
// near first?
// maybe something parallel in here too?
// Convert from AutoCloseable
// maybe add Future or make asyncThing a
// Future `
val headline: String =
ZIO
.from:
newsService.getHeadline()
.run
val topic =
ZIO
.from:
contentAnalyzer.findTopicOfInterest:
headline
.run
val summaryFileZ =
ZIO
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
70
Contract-Based Composability
.fromAutoCloseable:
ZIO.succeed:
closeableFile
.run
val topicIsFresh =
summaryFileZ.existsInFile:
topic
if (topicIsFresh)
val newInfo =
ZIO
.from:
historicalRecord.summaryFor:
topic
.run
ZIO
.from:
summaryFileZ.write:
newInfo.content
.run
ZIO
.debug:
"topicIsFresh: " + topicIsFresh
.run
asyncThing(1).run
// todo: some error handling to show that
// the errors weren't lost along the way
.catchAll:
case t: Throwable =>
???
case _: Any =>
???
end Scenario
object AllTheThings extends ZIOAppDefault:
type Nail = ZIO.type
/* If ZIO is your hammer, it's not that you
* _see_ everything as nails.
* You can actually _convert_ everything into
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
71
Contract-Based Composability
72
* nails. */
/*
*
*
*
*
*
*
*
*
*
*
*
*
Possible scenario:
Get headline - Future Analyze for
topic/persons of interest - Option Check if
we have made an entry for them in today's
summary file - Resource If not:
Dig up supporting information on the topic
from a DB - Try Make new entry in today's
summary file - Resource
Is Either different enough to demo here?
It basically splits the difference between
Option/Try I think if we show both of them,
we can skip Either. */
override def run =
Scenario(
Implementations.newsService,
Implementations.contentAnalyzer,
Implementations.historicalRecord,
Implementations.closeableFile
).logic
end AllTheThings
experiments/src/main/scala/composability/Invisible.scala
package composability
import scala.concurrent.Future
import scala.util.Try
object Implementations:
val newsService =
new NewsService:
def getHeadline(): Future[String] =
Future.successful:
"The stock market is crashing!"
val contentAnalyzer =
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Contract-Based Composability
new ContentAnalyzer:
override def findTopicOfInterest(
content: String
): Option[String] =
Option.when(
content.contains("stock market")
):
"stock market"
val historicalRecord =
new HistoricalRecord:
override def summaryFor(
topic: String
): Either[
NoRecordsAvailable,
DetailedHistory
] =
topic match
case "stock market" =>
Right(
DetailedHistory("detailed history")
)
case "TODO_OTHER_MORE_OBSCURE_TOPIC" =>
Left(NoRecordsAvailable("blah blah"))
val closeableFile =
new CloseableFile:
override def close =
println("Closing file now!")
override def existsInFile(
searchTerm: String
): Boolean = searchTerm == "stock market"
override def write(
entry: String
): Try[Unit] =
println("Writing to file: " + entry)
if (entry == "stock market")
Try(
throw new Exception(
"Stock market already exists!"
)
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
73
Contract-Based Composability
)
else
Try(())
end Implementations
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
74
Concurrency High Level
1. forEachPar, collectAllPar
TODO Prose
def sleepThenPrint(
d: Duration
) =
defer:
ZIO.sleep(d).run
ZIO.debug(s"${d.render} elapsed").run
d
runDemo:
ZIO.foreach(Seq(2, 1)): i =>
sleepThenPrint(i.seconds)
// List(PT2S, PT1S)
runDemo:
ZIO.foreachPar(Seq(2, 1)): i =>
sleepThenPrint(i.seconds)
// List(PT2S, PT1S)
runDemo:
defer:
val durations =
ZIO
.collectAllPar:
Seq(
sleepThenPrint(2.seconds),
sleepThenPrint(1.seconds)
)
.run
val total =
durations.fold(Duration.Zero)(_ + _).render
Console
.printLine:
total
.run
// ()
Concurrency High Level
def slowFailableRandom(duration: Duration) =
defer:
val randInt =
Random.nextIntBetween(0, 100).run
ZIO.sleep(duration).run
ZIO
.when(randInt < 10)(
ZIO.fail("Number is too low")
)
.run
duration
// Massive example
runDemo:
defer:
val durations =
ZIO
.collectAllSuccessesPar:
Seq
.fill(1_000)(1.seconds)
.map(duration =>
slowFailableRandom(duration)
)
.run
durations.fold(Duration.Zero)(_ + _).render
// 15 m 9 s
zipPar, zipWithPar
validateWithPar?
withParallelism
Edit This Chapter
Edit This Chapter¹⁴
¹⁴https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/08_Concurrency.md
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
76
Concurrency Interruption
Why Interruption Is Necessary Throughout
the Stack
In order for the Runtime to operate and provide the super powers of ZIO, it needs to
be able to interrupt running workflows without resource leaks.
Timeout
Since we can reliably interrupt arbitrary ZIOs, we can attach an upper bound for the
runtime of the ZIO. This does not change the type of the workflow, it only changes
the runtime behavior.
Race
If we have 2 ZIO’s that each produce the same types, we can race them, acquiring the
first result and cancel the ongoing calculation. We have taken 2 completely separate
workflows and fused them into one.
.withFinalizer
## .onInterrupt
Fork Interruption
Interruption is explicit in the previous situations, but there is an implicit interruption
point that you should be aware of. If an operation is forked, and we exit the scope
that created it without joining, then it will be interrupted.
Concurrency Interruption
78
runDemo:
defer:
ZIO
.debug:
"About to sleep forever"
.run
ZIO
.sleep:
Duration.Infinity
.onInterrupt:
ZIO.succeed:
// More mdoc console weirdness :(
println:
"Interrupted the eternal sleep"
.fork
.run
// About to sleep forever
// Interrupted the eternal sleep
// zio.internal.FiberRuntime@3ae9036c
If we encounter an error between forking and joining, the fibers will also be
interrupted.
runDemo:
defer:
val fiber1 =
createProcess(
"Fiber 1",
ZIO.sleep(5.seconds)
).fork.run
val fiber2 =
createProcess(
"Fiber 2",
ZIO.sleep(5.seconds)
).fork.run
// Once we fail here, the fibers will be
// interrupted.
ZIO.fail("Youch!").run
fiber1.join.run
fiber2.join.run
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Concurrency Interruption
79
// Interrupt Fiber 2
// Interrupt Fiber 1
// Youch!
Uninterruptable
### .acquireRelease effects are uninterruptible There are certain cases where you
want to ensure code is not interrupted. For example, when you have a finalizer that
needs to free up resources, you need to ensure it completes.
Tight loops
Tight loops that aren’t performing ZIO operations cannot be interrupted by the ZIO
runtime.
def longOperation() = "**TODO**"
runDemo:
defer:
ZIO
.succeed:
longOperation()
.timeout(1.seconds)
.timed
.debug("Time:")
.run
// (PT0.000481772S,Some(**TODO**))
We can see 2 significant behaviors here:
• timeout did cause the long operation to return None
• It did not interrupt the operation early.
Future Cancellation (Contra-example. Not
necessary for happy path explanation)
We show that Future’s are killed with finalizers that never run
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Concurrency Interruption
import scala.concurrent.Future
runDemo:
ZIO
.fromFuture:
Future:
try
println:
"Starting operation"
Thread.sleep:
500
println:
"Ending operation"
finally
println:
"Cleanup"
.timeout:
25.millis
// Starting operation
// None
.fromFutureInterrupt
Edit This Chapter
Edit This Chapter¹⁵
¹⁵https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/09_Interruption.md
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
80
State
1. Ref
2. Thundering Herds
Functional programmers often sing the praises of immutability. The advantages
are real and numerous. However, it is easy to find situations that are intrinsically
mutable.
• How many people are currently inside a building?
• How much fuel is in your car?
• How much money is in your bank account?
Rather than avoiding mutability entirely, we want to avoid unprincipled, unsafe
mutability. If we codify and enumerate everything that we need from Mutability,
then we can wield it safely. Required Operations:
• Update the value
• Read the current value
These are both effectful operations. In order to confidently use them, we need certain
guarantees about the behavior:
• The underlying value cannot be changed during a read
• Multiple writes cannot happen concurrently, which would result in lost updates
Less obviously, we also need to create the Mutable reference itself. We are changing
the world, by creating a space that we can manipulate.
Unreliable Counting
82
State
val unreliableCounting =
var counter = 0
val increment =
ZIO.succeed:
counter = counter + 1
defer:
ZIO
.foreachParDiscard(Range(0, 100000)): _ =>
increment
.run
// It's not obvious to the reader why
// we need to wrap counter in .succeed
"Final count: " + ZIO.succeed(counter).run
runDemo:
unreliableCounting
// Final count: 99799
Due to the unpredictable nature of shared mutable state, we do not know exactly
what the final count above is. Each time we publish a copy of this book, the code is reexecuted and a different wrong result is generated. However, conflicts are extremely
likely, so some of our writes get clobbered by others, and we end up with less than
the expected 100,000. Ultimately, we lose information with this approach.
TODO Consider making a diagram parallel writes
Performing our side effects inside ZIO’s does not magically make them safe. We need
to fully embrace the ZIO components, utilizing Ref for correct mutation.
Reliable Counting
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
83
State
lazy val reliableCounting =
def incrementCounter(counter: Ref[Int]) =
counter.update:
_ + 1
defer:
val counter = Ref.make(0).run
ZIO
.foreachParDiscard(Range(0, 100000)): _ =>
incrementCounter:
counter
.run
"Final count: " + counter.get.run
runDemo:
reliableCounting
// Final count: 100000
Now we can say with full confidence that our final count is 100000. Additionally,
these updates happen without blocking. This is achieved through a strategy called
“Compare & Swap”, which we will not cover in detail. TODO Link/reference supplemental reading
Although there are significant advantages; a basic Ref is not the solution for
everything. We can only pass pure functions into update. The API of the plain
Atomic Ref steers you in the right direction by not accepting ZIOs as parameters to
any of its methods. To demonstrate why this restriction exists, we will deliberately
undermine the system by sneaking in a side effect. First, we will create a helper
function that imitates a long-running calculation.
def expensiveCalculation() =
Thread.sleep:
35
Our side effect will be a mock alert that is sent anytime our count is updated: scala
def sendNotification() = println: "Alert: updating count!"
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
84
State
def update(counter: Ref[Int]) =
counter.update: previousValue =>
expensiveCalculation()
sendNotification()
previousValue + 1
runDemo:
defer:
val counter = Ref.make(0).run
ZIO
.foreachParDiscard(Range(0, 4)): _ =>
update(counter)
.run
"Final count: " + counter.get.run
// Alert: updating count!
// Alert: updating count!
// Alert: updating count!
// Alert: updating count!
// Alert: updating count!
// Alert: updating count!
// Alert: updating count!
// Alert: updating count!
// Alert: updating count!
// Alert: updating count!
// Final count: 4
What is going on?! Previously, we were losing updates because of unsafe mutability.
Now, we have the opposite problem! We are sending far more alerts than intended,
even though we can see that our final count is 4.
TODO This section will need significant attention and polish
Now we must consider the limitations of the “Compare & Swap” system. It achieves
lock-free performance by letting each fiber freely make their updates, and then doing
a last-second check to see if the underlying value changed during its update. If the
value has not changed, the update is made. If it has changed, then the entire function
that was passed into update is re-executed until it completes with a stable value. The
higher the parallelism, or the longer the operation takes, the higher the likelihood of
a compare-and-swap retry.
This retry behavior is safe with pure functions, which can be executed an arbitrary
number of times. However, it is completely inappropriate for effects, which should
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
85
State
only be executed a single time. For these situations, we need a specialized variation
of Ref
Ref.Synchronized
Ref.Synchronized guarantees only a single execution of the update body and any
of the effects contained inside. The only change required is replacing Ref.make with
Ref.Synchronized.make
val sideEffectingUpdatesSync =
defer:
val counter = Ref.Synchronized.make(0).run
ZIO
.foreachParDiscard(Range(0, 4)): _ =>
counter.update: previousValue =>
expensiveCalculation()
sendNotification()
previousValue + 1
.run
"Final count: " + counter.get.run
runDemo:
sideEffectingUpdatesSync
// Alert: updating count!
// Alert: updating count!
// Alert: updating count!
// Alert: updating count!
// Final count: 4
Now we see exactly the number of alerts that we expected. This correctness comes
with a cost though, as the name of this type implies. Each of your updates will run
sequentially, despite initially launching them all in parallel. This is the only known
way to avoid retries. Try to structure your code to minimize the coupling between
effects and updates, and use this type only when necessary.
Edit This Chapter
Edit This Chapter¹⁶
¹⁶https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/10_State.md
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Reliability
Caching
One way to achieve better reliability is with caching.
val thunderingHerdsScenario =
defer:
val popularService =
ZIO.service[PopularService].run
ZIO
.foreachPar(List.fill(100)(())): _ =>
popularService.retrieve:
Path.of("awesomeMemes")
.run
val cloudStorage =
ZIO.service[CloudStorage].run
cloudStorage.invoice.debug.run
object PopularService:
private def lookup(key: Path) =
defer:
val cloudStorage =
ZIO.service[CloudStorage].run
cloudStorage.retrieve(key).run
val make =
defer:
val cloudStorage =
ZIO.service[CloudStorage].run
PopularService(cloudStorage.retrieve)
// james don't like
87
Reliability
val makeCached =
defer:
val cache =
Cache
.make(
capacity = 100,
timeToLive = Duration.Infinity,
lookup = Lookup(lookup)
)
.run
PopularService(cache.get)
end PopularService
runDemo:
thunderingHerdsScenario.provide(
CloudStorage.live,
ZLayer.fromZIO(PopularService.make)
)
// Amount owed: $100
runDemo:
thunderingHerdsScenario.provide(
CloudStorage.live,
ZLayer.fromZIO(PopularService.makeCached)
)
// Amount owed: $1
Staying under rate limits
Constraining concurrent requests
If we want to ensure we don’t accidentally DDOS a service, we can restrict the
number of concurrent requests to it.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
88
Reliability
runDemo:
defer:
ZIO
.foreachPar(1 to 10): _ =>
//
bulkhead:
ZIO.serviceWithZIO[DelicateResource](
_.request
)
.as("All Requests Succeeded")
.catchAll(err => ZIO.succeed(err))
.debug
.run
.provideSome[Scope]:
DelicateResource.live
// Delicate Resource constructed.
// Do not make more than 3 concurrent requests!
// Killed the server!!
import nl.vroste.rezilience.Bulkhead
runDemo:
defer:
val bulkhead: Bulkhead =
Bulkhead.make(maxInFlightCalls = 3).run
ZIO
.foreachPar(1 to 10): _ =>
bulkhead:
ZIO.serviceWithZIO[DelicateResource](
_.request
)
.as("All Requests Succeeded")
.catchAll(err => ZIO.succeed(err))
.debug
.run
.provideSome[Scope]:
DelicateResource.live
// All Requests Succeeded
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
89
Reliability
Circuit Breaking
Hedging
Edit This Chapter
Edit This Chapter¹⁷
¹⁷https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/11_Reliability.md
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
ZIO and ZLayer
Connect the DI & Errors chapter to how they are represented in the ZIO data type.
The way we get good compile errors is by having data types which “know” the …
ZIO
We need an Answer about this scenario. The scenario requires things and could
produce an error. trait ZIO[Requirements, Error, Answer]
The ZIO trait is at the center of our Effect-oriented world.
trait ZIO[R, E, A]
A trait with 3 type parameters can be intimidating, but each one serves a distinct,
important purpose.
R - Requirements
TODO
E - Errors
This parameter tells us how this operation might fail.
def parse(
contents: String
): ZIO[Any, IllegalArgumentException, Unit] = ???
A - Answer
This is what our code will return if it completes successfully.
ZIO and ZLayer
91
def defaultGreeting()
: ZIO[Any, Nothing, String] = ???
ZLayer
R - Requirements
### E - Errors ### A - Answer
Composing
Managing and wiring dependencies has been a perennial challenge in software
development.
ZIO provides the ZLayer class to solve many of the problems in this space. If you
pay the modest, consistent cost of constructing pieces of your application as ZLayers,
you will get benefits that scale with the complexity of your project. Consistent with
ZIO itself, ZLayer has 3 type parameters that represent:
• What it needs from the environment
• How it can fail
• What it produces when successful.
With the same type parameters, and many of the same methods, you might be
wondering why we even need a separate data type - why not just use ZIO itself
for our dependencies? The environment type parameter for ZLayer maps directly
to unique, singleton services in your application. The environment type parameter
for ZIO might have many possible instances. ZLayer provides additional behaviors
that are valuable specifically in this domain. Typically, you only want a single
instance of a dependency to exist across your application. This may be to reduce
memory/resource usage, or even to ensure basic correctness. ZLayer output values
are shared maximally by default. They also build in scope management that will
ensure resource cleanup in asynchronous, fallible situations.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
ZIO and ZLayer
ZIO <-> ZLayer
ZLayer.fromZIO
Edit This Chapter
Edit This Chapter¹⁸
¹⁸https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/12_ZIO_and_ZLayer.md
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
92
Appendix Running Effects
TODO: Make an appendix?
Building applications from scratch
One way to run ZIOs is to use a “main method” program (something you can start
in the JVM). However, setting up the pieces needed for this is a bit cumbersome if
done without helpers.
ZIOAppDefault
ZIO provides an easy way to do this with the ZIOAppDefault trait.
To use it create a new object that extends the ZIOAppDefault trait and implements
the run method. That method returns a ZIO so you can now give it the example
ZIO.debug data:
object HelloWorld extends zio.ZIOAppDefault:
def run =
ZIO.debug:
"hello, world"
This can be run on the JVM in the same way as any other class that has a static
void main method.
The ZIOAppDefault trait sets up the ZIO runtime which interprets ZIOs and
provides some out-of-the-box functionality, and then runs the provided data in that
runtime.
If you are learning ZIO, you should start your exploration with ZIOAppDefault. It
is the standard, simplest way to start executing your recipes.
Appendix Running Effects
94
// NOTE We cannot execute invoke main on this
// because it crashes mdoc in the CI process
object RunningZIOs extends ZIOAppDefault:
def run =
// TODO Console/debug don't work
ZIO.attempt:
println:
"Hello World!"
You can provide arbitrary ZIO instances to the run method, as long as you have
provided every piece of the environment. In other words, it can accept ZIO[Any, _,
_].
There is a more flexible ZIOApp that facilitates sharing layers between applications,
but this is advanced and not necessary for most applications.
runDemo
While the ZIOApp* types are great for building real applications, they are not ideal
for demonstrating code for a book. We created the runDemo function to streamline
this use-case. It is a function that takes a ZIO and executes it in a runtime, returning
the result. It uses most of the same techniques that are used in ZIOAppDefault, but
is more single purpose, always immediately executing the ZIO provided to it.
runDemo:
ZIO.debug:
"hello, world"
// hello, world
// ()
Testing code
ZIOSpecDefault
Similar to ZIOAppDefault, there is a ZIOSpecDefault that should be your starting
point for testing ZIO applications. ZIOSpecDefault provides test-specific implementations built-in services, to make testing easier. When you run the same ZIO in these
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Appendix Running Effects
95
2 contexts, the only thing that changes are the built-in services provided by the
runtime.
TODO - Decide which scenario to test
import zio.test._
object TestingZIOs extends ZIOSpecDefault:
def spec =
test("Hello Tests"):
defer:
ZIO.console.run
assertTrue:
Random.nextIntBounded(10).run > 10
runSpec
runSpec:
defer:
assertTrue:
Random.nextIntBounded(10).run < 10
// Test: PASSED*
TODO Justify defer syntax over for-comp for multi-statement assertions I think this
example completes the objective TODO Change this to a Console app, where the
logic & testing is more visceral
runSpec:
defer:
assertTrue:
Random.nextIntBetween(0, 10).run <= 10 &&
Random.nextIntBetween(10, 20).run <=
20 &&
Random.nextIntBetween(20, 30).run <= 30
// Test: PASSED*
Consider a Console application:
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Appendix Running Effects
96
val logic =
defer:
val username =
Console
.readLine:
"Enter your name\n"
.run
Console
.printLine:
s"Hello $username"
.run
.orDie
If we try to run this code in the same way as most of the examples in this book, we
encounter a problem. scala runDemo: logic.timeout(1.second) // Defect:
scala.NotImplementedError: an implemen We cannot execute this code and
render the results for the book because it requires interaction with a user. However,
even if you are not trying to write demo code for a book, it is very limiting to need
a user at the keyboard for your program to execute. Even for the smallest programs,
it is slow, error-prone, and boring.
runSpec:
defer:
TestConsole
.feedLines:
"Zeb"
.run
logic.run
val capturedOutput: String =
TestConsole.output.run.mkString
val expectedOutput =
s"""|Enter your name
|Hello Zeb
|""".stripMargin
assertTrue:
capturedOutput == expectedOutput
// Test: PASSED*
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Appendix Running Effects
97
Interop with existing/legacy code via Unsafe
In some cases your ZIOs may need to be run outside a main program, for example
when embedded into other programs. In this case you can use ZIO’s Unsafe utility
which is called Unsafe to indicate that the code may perform side effects.
To do the same ZIO.debug with Unsafe do:
Unsafe.unsafe { implicit u: Unsafe =>
Runtime
.default
.unsafe
.run:
ZIO.debug:
"hello, world"
.getOrThrowFiberFailure()
}
// hello, world
If needed you can even interop to Scala Futures through Unsafe, transforming the
output of a ZIO into a Future.
Edit This Chapter
Edit This Chapter¹⁹
¹⁹https://github.com/EffectOrientedProgramming/book/edit/main/Chapters/13_Appendix_RunningEffects.md
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
Experiments
These experiments are not currently attached to a chapter, but are included for
previewing. Before publication, we should not have any lingering experiments here.
rezilience
experiments/src/main/scala/rezilience/CircuitBreakerDemo.sca
package rezilience
import nl.vroste.rezilience.*
import nl.vroste.rezilience.CircuitBreaker.*
import zio.{Schedule, ZLayer}
case class Cost(value: Int)
case class Analysis(content: String)
trait ExpensiveSystem:
def call: ZIO[Any, String, Analysis]
val billToDate: ZIO[Any, String, Cost]
object Scenario:
enum Step:
case Success,
Failure
object CircuitBreakerDemo extends ZIOAppDefault:
val makeCircuitBreaker
: ZIO[Scope, Nothing, CircuitBreaker[
Any
]] =
CircuitBreaker.make(
99
Experiments
trippingStrategy =
TrippingStrategy
.failureCount(maxFailures = 2),
resetPolicy = Retry.Schedules.common(),
.exponentialBackoff(
min = 1.second,
max = 4.second,
),
//
//
//
//
onStateChange =
state =>
ZIO.debug(s"State change: $state")
)
def run =
defer:
ZIO
.serviceWithZIO[ExpensiveSystem](_.call)
.ignore
.repeat(
Schedule.recurs(8) &&
Schedule.spaced(200.millis)
)
.run
ZIO
.serviceWithZIO[ExpensiveSystem](
_.billToDate
)
.debug
.run
.provide:
//
ExternalSystem // TOGGLE
ExternalSystemProtected // TOGGLE
.live
end CircuitBreakerDemo
case class ExternalSystemProtected(
externalSystem: ExpensiveSystem,
circuitBreaker: CircuitBreaker[String]
) extends ExpensiveSystem:
val billToDate: ZIO[Any, String, Cost] =
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
100
Experiments
externalSystem.billToDate
def call: ZIO[Any, String, Analysis] =
circuitBreaker:
externalSystem.call
.tap(r => ZIO.debug(s"Result: $r"))
.mapError:
case CircuitBreakerOpen =>
"Circuit breaker blocked the call to our external system"
case WrappedError(e) =>
println("ignored boom?")
s"External system threw an exception: $e"
.tapError(e => ZIO.debug(e))
object ExternalSystemProtected:
val live
: ZLayer[Any, Nothing, ExpensiveSystem] =
ZLayer.fromZIO:
defer:
ExternalSystemProtected(
ZIO.service[ExpensiveSystem].run,
CircuitBreakerDemo
.makeCircuitBreaker
.run
)
.provide(
ExternalSystem.live,
Scope.default
)
experiments/src/main/scala/rezilience/CircuitBreakerInvisible.
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
101
Experiments
package rezilience
import java.time.Instant
import scala.concurrent.TimeoutException
// Invisible mdoc fencess
import Scenario.Step.*
object ExternalSystem:
val live
: ZLayer[Any, Nothing, ExpensiveSystem] =
ZLayer.fromZIO:
defer:
val valueProducer =
scheduledValues(
(300.millis, Success),
(200.millis, Failure),
// TODO Restore when I can get CB to
// reconnect :(
(400.millis, Failure),
(5.seconds, Success)
).run
ExternalSystem(
Ref.make(0).run,
valueProducer
)
case class ExternalSystem(
requests: Ref[Int],
responseAction: ZIO[
Any, // access time
TimeoutException,
Scenario.Step
]
) extends ExpensiveSystem:
// TODO: Better error type than Throwable
val billToDate: ZIO[Any, String, Cost] =
requests
.get
.map:
Cost(_)
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
102
Experiments
def call: ZIO[Any, String, Analysis] =
defer:
ZIO
.debug(
"Called underlying ExternalSystem"
)
.run
val requestCount =
requests.updateAndGet(_ + 1).run
responseAction.orDie.run match
case Success =>
ZIO
.succeed:
Analysis:
s"Expensive report #$requestCount"
.run
case Failure =>
ZIO
.debug:
"boom"
.run
ZIO
.fail:
"Something went wrong"
.run
end ExternalSystem
// TODO Consider deleting
object InstantOps:
extension (i: Instant)
def plusZ(duration: zio.Duration): Instant =
i.plus(duration.asJava)
import InstantOps._
/*
*
*
*
*
Goal: If I accessed this from:
0-1 seconds, I would get "First Value" 1-4
seconds, I would get "Second Value" 4-14
seconds, I would get "Third Value" 14+
seconds, it would fail */
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
103
Experiments
// TODO Consider TimeSequence as a name
def scheduledValues[A](
value: (Duration, A),
values: (Duration, A)*
): ZIO[
Any, // construction time
Nothing,
ZIO[
Any, // access time
TimeoutException,
A
]
] =
defer {
val startTime = Clock.instant.run
val timeTable =
createTimeTableX(
startTime,
value,
values* // Yay Scala3 :)
)
accessX(timeTable)
}
// TODO Some comments, tests, examples, etc to
// make this function more obvious
private def createTimeTableX[A](
startTime: Instant,
value: (Duration, A),
values: (Duration, A)*
): Seq[ExpiringValue[A]] =
values.scanLeft(
ExpiringValue(
startTime.plusZ(value._1),
value._2
)
) {
case (
ExpiringValue(elapsed, _),
(duration, value)
) =>
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
104
Experiments
ExpiringValue(
elapsed.plusZ(duration),
value
)
}
/** Input: (1 minute, "value1") (2 minute,
* "value2")
*
* Runtime: Zero value: (8:00 + 1 minute,
* "value1")
*
* case ((8:01, _) , (2.minutes, "value2")) =>
* (8:01 + 2.minutes, "value2")
*
* Output: ( ("8:01", "value1"), ("8:03",
* "value2") )
*/
private def accessX[A](
timeTable: Seq[ExpiringValue[A]]
): ZIO[Any, TimeoutException, A] =
defer {
val now = Clock.instant.run
ZIO
.getOrFailWith(
new TimeoutException("TOO LATE")
) {
timeTable
.find(_.expirationTime.isAfter(now))
.map(_.value)
}
.run
}
private case class ExpiringValue[A](
expirationTime: Instant,
value: A
)
experiments/src/main/scala/rezilience/RateLimiter.scala
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
105
Experiments
package rezilience
import nl.vroste.rezilience.*
/** This is useful for scenarios such as:
*
- Making sure you don't suddenly spike your
*
AWS bill
*
- Not accidentally DDOSing a service
*/
val makeRateLimiter
: ZIO[Scope, Nothing, RateLimiter] =
RateLimiter.make(max = 1, interval = 1.second)
// We use Throwable as error type in this example
def rsaKeyGenerator: ZIO[Any, Throwable, Int] =
Random.nextIntBounded(1000)
import zio_helpers.timedSecondsDebug
object RateLimiterDemoWithLogging
extends ZIOAppDefault:
def run =
defer:
val rateLimiter = makeRateLimiter.run
rateLimiter(rsaKeyGenerator)
// Print the time to generate each key:
.timedSecondsDebug("Generated key")
// Repeat as fast as the limiter allows:
.repeatN(3)
// Print the last result
.timedSecondsDebug("Result").run
object RateLimiterDemoGlobal
extends ZIOAppDefault:
import zio_helpers.repeatNPar
def run =
defer:
val rateLimiter = makeRateLimiter.run
ZIO
.repeatNPar(3): i =>
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
106
Experiments
rateLimiter:
rsaKeyGenerator
.timedSecondsDebug(
s"${i.toString} generated a key"
)
// Repeats as fast as allowed
.repeatN(2).debug(s"Result $i")
.unit
.timedSecondsDebug("Total time")
.run
end RateLimiterDemoGlobal
performance
experiments/src/main/scala/performance/Hedging.scala
package performance
object Hedging extends ZIOAppDefault:
def run =
defer:
val fastResponses
= Ref.make(0).run
val contractBreaches = Ref.make(0).run
ZIO
.foreachPar(List.fill(50_000)(())):
_ => // james still hates this
defer:
val randomResponse =
Response.random
val hedged =
randomResponse.race:
randomResponse.delay:
25.millis
// todo: extract to invisible
// function
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
107
Experiments
hedged.run match
case Response.Fast =>
fastResponses.update(_ + 1).run
case Response.BreachOfContract =>
contractBreaches
.update(_ + 1)
.run
.run
fastResponses
.get
.debug("Fast responses")
.run
contractBreaches
.get
.debug("Slow responses")
.run
end Hedging
// invisible below
enum Response:
case Fast,
BreachOfContract
object Response:
def random: ZIO[Any, Nothing, Response] =
defer:
val random =
Random.nextIntBounded(1_000).run
random match
case 0 =>
ZIO
.sleep:
3.seconds
.run
ZIO
.succeed:
Response.BreachOfContract
.run
case _ =>
Response.Fast
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward
108
Experiments
zio_helpers
experiments/src/main/scala/zio_helpers/Helpers.scala
package zio_helpers
extension (z: ZIO.type)
def repeatNPar[R, E, A](
numTimes: Int
)(op: Int => ZIO[R, E, A]): ZIO[R, E, Seq[A]] =
z.foreachPar(0 until numTimes)(op)
def repeatNPar[R, E, A](
numTimes: Int
)(op: ZIO[R, E, A]): ZIO[R, E, Seq[A]] =
z.foreachPar(0 until numTimes)(_ => op)
extension [R, E, A](z: ZIO[R, E, A])
def timedSecondsDebug(
message: String
): ZIO[R, E, A] =
z.timed
.tap: (duration, res) =>
res match
// Don't bother printing Unit results
case () =>
ZIO.debug:
message + " [took " +
duration.getSeconds + "s]"
case _ =>
ZIO.debug:
message + ": " + res + " [took " +
duration.getSeconds + "s]"
.map(_._2)
Effect-Oriented Programming ©2021 Bill Frasure, Bruce Eckel & James Ward