Scala
- where are are
- where we plan to go
Martin Odersky
Welcome to ScalaDays!
• 155 attendees, coming from all corners of the planet.
• We are packed. Sadly, many more people could not be
admitted for lack of space.
• This shows the drive and energy of this community!
• The Scala team is looking forward to talk with you and
learn from you during the next two days.
• We are particularly glad to have you here at EPFL, the
place where Scala was born.
History (incomplete)
2001
2003
2004
2005
2006
2007
2008
2009
2010
Scala design begins, as a more practical
successor to Funnel.
First experimental release, first course
taught with it.
Scalable Component Abstractions paper.
Scala 2.0, written in Scala.
Industrial adoption starts.
First Lift release.
First Scala LiftOff, Twitter adopts Scala.
Big increase in adoption, IDEs mature.
First ScalaDays.
The main idea behind Scala
Scala is a Unifier
Agile, with lightweight syntax
Object-Oriented
Scala
Functional
Safe and performant, with strong static typing
The Philosophy behind Scala
Put productivity and creativity back in the hands of
developers.
“Joy is underrated as a metric for a language’s potential
success in a development organization.” a3lx@twitter
- address professional developers
- trust them & value their expertise
- (don’t tell them how they should do their job)
Some users in industry
Open Source Ecosystem
Many great projects, including:
lift
(David P. & friends)
akka
(Jonas B.)
kestrel, cachet, queroulous
(Twitter)
migrations
(Sony)
sbt
(Mark H.)
ScalaTest
(Bill V. & friends)
specs, ScalaCheck, ScalaModules, ScalaQL, ....
( the list goes on, many more tools & projects presented at
this conference )
Scala 2.8
named and default
parameters
copy operation
Now: 2.8
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays
and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
First release candidate Scala 2.8 RC1 is out!
This is primarily a consolidating release
• Some new language features
• More complete libraries
• Redesigns of a few features that did not
work out well.
• Big improvements in tooling.
• Big improvements in stability.
• Laying the groundwork for the future.
Scala 2.8
Named and Default
Parameters
named and default
parameters
copy operation
nested annotations
package objects
def newTable(size: Int = 100, load: Float = 0.5f) =
safer package nesting
new HashMap[String, String]{
@specialized
new collection API
override val initialSize = size
vectors
override val loadFactor = (load * 1000).toInt
hash tries
new treatment of arrays
and strings
manifests
better implicits
}
}
newTable(50, 0.75f)
better type inference
newTable()
continuations
newTable(load = 0.5f)
packrat parsing
newTable(size = 20, load = 0.5f)
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
Defaults can be arbitrary expressions, not just
constants.
Default evaluation is dynamic.
Scala 2.8
named and default
parameters
copy operation
Copy Operation
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays
and strings
manifests
better implicits
better type inference
Problem: Case classes are great for pattern matching,
but how do you transform data?
Often need to update (functionally) some specific field(s)
of a date structure defined by case classes.
Solution: Selective copy method.
case class Tree[+T](elem: T,
left: Tree[T],
right: Tree[T])
def incElem(t: Tree[Int]) = t copy (elem = t.elem + 1)
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
Q: How to define “copy”?
Scala 2.8
Defining “copy”
named and default
parameters
copy operation
nested annotations
package objects
safer package nesting
@specialized
Problem: A class with n parameters allows 2n possible
combinations of parameters to copy.
Default parameters to the rescue.
new collection API
vectors
hash tries
new treatment of arrays
and strings
case class Tree[+T](elem: T,
left: Tree[T],
right: Tree[T]) {
// generated automatically:
manifests
def copy[U](elem: U = this.elem,
better implicits
better type inference
left: Tree[U] = this.left,
continuations
right: Tree[U] = this.right) =
packrat parsing
Branch(elem, left, right)
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
}
Scala 2.8
named and default
parameters
Nested Annotations
copy operation
nested annotations
package objects
•
safer package nesting
@specialized
new collection API
•
vectors
hash tries
new treatment of arrays
and strings
manifests
•
•
Scala 2.7 largely followed Java annotation syntax, but
left out some of the more convoluted constructions.
 Nested annotations were not possible.
Scala 2.8 throws out (deprecates) all special annotation
syntax.
Instead uses class constructors, which can have named
parameters.
This greatly simplifies the syntax, and allows all
annotations to be expressed, including nested ones.
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
@JoinTable(
joinColumns = Array(new JoinColumn(name = "a name“)
) var test: String = _
Scala 2.8
named and default
parameters
@specialized
copy operation
nested annotations
@specialized
new collection API
vectors
hash tries
Problem: Generics cause a performance hit because they
require a uniform representation: everything needs to be
boxed.
Example:
trait Function1[-T, +U] { def apply(x: T): U }
new treatment of arrays
and strings
def app(x: Int, fn: Int => Int) = fn(x)
package objects
safer package nesting
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
Translated
by scalac
2.7 to:
app(7, x => x * x)
trait Function1 {
def apply(x: Object): Object }
class anonfun extends Function1 {
def apply(x: Object): Object =
Int.box(apply(Int.unbox(x)))
def apply(x: Int): Int = x * x
}
new scaladoc
def app(x: Int, fn: Function1) =
Int.unbox(fn(Int.box(x)))
better REPL
app(7, new anonfun)
interactive compiler API
Scala 2.8
named and default
parameters
copy operation
nested annotations
@specialized
new collection API
vectors
hash tries
new treatment of arrays
and strings
package objects
safer package nesting
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
If Function1 is @specialized ...
trait Function1[@specialized -T, @specialized +U] {
def apply(x: T): U }
def app(x: Int, fn: Int => Int) = fn(x)
app(7, x => x * x)
... scalac 2.8 generates instead:
trait Function1 {
def apply(x: Object): Object
def apply$$II(x: Int): Int =
Int.unbox(apply(Int.box(x)))
// specializations for other primitive types
}
class anonfun extends (Int => Int) {
def apply(x: Object): Object =
Int.box(applyIntInt(Int.unbox(x)))
override def apply$$II(x: Int): Int = x * x
}
interactive compiler API
def app(x: Int, fn: Function1) = fn.apply$II(x)
new scaladoc
app(7, anonfun)
better REPL
Scala 2.8
named and default
parameters
copy operation
Scala Collections
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays
and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
Collection Properties:
object-oriented
generic: List[T], Map[K, V]
optionally persistent, e.g.
collection.immutable.Seq
higher-order, with methods
such as foreach, map,
filter.
Uniform return type principle:
Operations return
collections of the same
type (constructor) as their
left operand.
scala> val ys = List(1, 2, 3)
ys: List[Int] = List(1, 2, 3)
scala> val xs: Seq[Int] = ys
xs: Seq[Int] = List(1, 2, 3)
scala> xs map (_ + 1)
res0: Seq[Int] = List(2, 3, 4)
scala> ys map (_ + 1)
res1: List[Int] = List(2, 3, 4)
Scala 2.8
named and default
parameters
copy operation
Old Collection Structure
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays
and strings
manifests
better implicits
better type inference
continuations
packrat parsing
trait Iterable[A] {
def filter(p: A => Boolean): Iterable[A] = ...
def partition(p: A => Boolean) =
(filter(p(_)), filter(!p(_)))
def map[B](f: A => B): Iterable[B] = ...
}
trait Seq[A] extends Iterable[A] {
def filter(p: A => Boolean): Seq[A] = ...
override def partition(p: A => Boolean) =
(filter(p(_)), filter(!p(_)))
def map[B](f: A => B): Seq[B] = ...
}
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
Types force duplication!
Scala 2.8
named and default
parameters
copy operation
New Collection API
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays
and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
• Avoids type & code duplication through
– uniform framework of traversers and builders.
– supported by: implicits, higher-kinded types
• Uniform package organization reflects mutability:
– scala.collection
– scala.collection.mutable
– scala.collection.immutable
• Richer and more regular set of collection
operations.
• New collection implementations: vectors and hash
tries.
• More info: “Fighting bit rot with types”, FSTTCS09
Scala 2.8
named and default
parameters
Vectors and Hash Tries
copy operation
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays
and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
•
•
•
•
•
Trees with branch factor of 32.
Persistent data structures with very efficient sequential
and random access.
Invented by Phil Bagwell, then adopted in Clojure.
New: Persistent prepend/append/update in constant
amortized time.
Next: Fast splits and joins for parallel transformations.
Scala 2.8
named and default
parameters
copy operation
Package Objects
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays
and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
Motivation:
• The new collection design reshuffled a lot of
classes.
• For instance:
scala.List  scala.collection.immutable.List
• How can we keep the old aliases around?
• How can we migrate old to new?
• Easy: put
type List = scala.collection.immutable.List
val List = scala.collection.immutable.List
into the scala package.
• But how to we put a type alias and a val definition
into a package?
Scala 2.8
named and default
parameters
Package Objects
copy operation
nested annotations
package objects
safer package nesting
@specialized
new collection API
Problem: Everything has to live inside an object or class.
=> No top-level value definitions, type aliases,
implicits.
vectors
hash tries
new treatment of arrays
and strings
Package objects make packages more like objects.
They also let you define your own ``Predefs’’.
manifests
better implicits
better type inference
package object scala {
type List[+T] = scala.collection.immutable.List
continuations
packrat parsing
val List = scala.collection.immutable.List
faster actors
...
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
}
Scala 2.8
named and default
parameters
Safer Package Nesting
copy operation
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays
and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
•
•
•
•
Java knows only absolute packages
But Scala’s packages nest
This can lead to an impedance mismatch.
For instance:
package net.liftweb
java.lang.System.print(“hello”)
• This used to fail if net contains a subpackage java.
• New rules: Intermediate packages only visible in
multiple package clauses.
• To access net.java as java, you’d need to write
package net
package liftweb
...
Scala 2.8
named and default
parameters
copy operation
New Treatment of Arrays
nested annotations
package objects
• Arrays were “magical” up to 2.8
safer package nesting
– sometimes primitive
– sometimes wrapped
– automatic boxing/unboxing conversions based
on static type.
@specialized
new collection API
vectors
hash tries
new treatment of arrays
and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
•
•
•
•
This could be confusing and slow.
2.8 removes the magic.
Arrays are always Java arrays
Two implicit conversions make sure that
– arrays are compatible with sequences.
– arrays support all sequence operations
• Strings are treated similarly to arrays.
Scala 2.8
named and default
parameters
copy operation
Manifests
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays
and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
• Question: How to create an Array[T] where T
is a type parameter?
• No magic allowed!
• Solution: supply the run-time type as a type
descriptor called a Manifest or ClassManifest:
def evens[T: ClassManifest](xs: Vector[T]): Array[T] = {
val arr = new Array[T]((xs.length + 1) / 2)
for (i <- 0 until xs.length by 2)
arr(i / 2) = xs(i)
arr
}
The first line is equivalent to:
def evens[T](xs: Vector[T])
(implicit ev: ClassManifest[T]): Array[T]
Scala 2.8
named and default
parameters
copy operation
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays
and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
Better Implicits
Better Type Inference
• Type inference for implicits has been
refined.
• Previously: Infer type first, search implicit
for inferred type afterwards.
• Now: Implicit search can drive type
inference.
• This makes implicits more powerful.
• Other advance: Type constructors of
higher-kinded types can now be inferred.
Scala 2.8
named and default
parameters
copy operation
Delimited Continuations
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays
and strings
manifests
better implicits
better type inference
delimited continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
• Continuations capture the return stack as
a data structure.
• Delimited continuations capture only up to
some enclosing scope.
• Lost of advanced control structures can be
built on them, e.g:
–
–
–
–
coroutines
event-based I/O
reactive programming
lightweight actors
• Scala 2.8 supports delimited continuations
in a compiler plugin.
Scala 2.8
named and default
parameters
copy operation
Better IDEs
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays
and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
• The Scala plugins for Eclipse, IntelliJ,
Netbeans all have matured a lot.
• The Eclipse and Netbeans plugins are
now based on completely redesigned
compiler interfaces for command
completion, semantic highlighting and
incremental building.
• If you have discounted IDEs before for
Scala, maybe now is the time to try again!
Scala 2.8
named and default
parameters
copy operation
Docs for Advanced Types
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays
and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
Problem: Advanced statically typed APIs pose challenges for
presentation and documentation.
Scala 2.8
named and default
parameters
copy operation
Simpler Type Views
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays
and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
• How to explain
def map[B, To](f: A => B)
(implicit cbf: CanBuildFrom[Coll, B, To]): To
to a beginner?
• Key observation: We can approximate the type of map.
• For everyone but the most expert user
def map[B](f: A => B): Traversable[B] // in class Traversable
def map[B](f: A => B): Seq[B]
// in class Seq, etc
is detailed enough.
• These types are correct, they are just not as general as
the type that’s actually implemented.
• These are documented in the new ScalaDoc as use
cases.
Scala 2.8
named and default
parameters
copy operation
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays
and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
New ScalaDoc
Why 2.8, not 3.0?
•
Scala 2.8 is numerically a point release, but feels like a major release.
•
So why did we not name it Scala 3.0?
•
2.8 has been 18 months in the making, and acquired more changes on the
way.
•
Originally, we just wanted to rewrite collections.
•
That took longer than expected.
•
Meanwhile, a lot more additions and changes got included.
•
But 2.8 was already announced, and referred to in books and other
documentation.
•
In light of this, we judged it too confusing to change the version number in
mid-flight.
And now...
...what’s next?
Next: Scala on .NET
• Work started again to have a native Scala compiler on
.NET with Visual Studio integration.
• Funded my Microsoft.
• Stay tuned...
Next: Reliability and Contracts
• Lost of things happening around contracts and
verification.
• Peter Müller: Scala verifier
• Viktor Kuncak: Scala code synthesis from constraints
• Our group: effects
• Common project: ProgLab.NET
Next: Concurrency
• Actor encapsulation
–
–
–
–
How to ensure that actors do not share memory?
Solved by capability-based uniqueness type system.
Implemented in compiler plugin.
Paper in ECOOP 2010.
• Software Transactional Memory
– CCSTM work in this conference
• Open question: What’s a good combination of actors and
transactions?
Next: Parallelism
• PPP Challenge: How to make use of multi-cores,
GPUs, clusters, clouds, to achieve better
performance for normal applications?
• First step in this direction: Parallel collections.
– will be integrated in the new collection API.
– based on persistent data structures with fast splits
and joins.
• Longer-term vision:
Embedded parallel DSLs.
• See keynote by
Kunle Olokutun tomorrow.
Next: The Green House
Great work on Scala is springing up
everywhere, no longer focussed
just at EPFL.
Q: How can we integrate contributions?
Tension: Want to get lots of exciting
new stuff contributed, yet maintain a
stable and reliable distribution.
Planned Solution: three-tiered:
Incubator
Hosts collaborative Scala
projects
Green house A “cutting edge” version of
the distribution with
experimental features.
Trunk
The stable distribution.
Thanks
• To the Scala team at EPFL:
Lukas Rytz, Hubert Plociniczak, Iulian Dragos, Miguel Garcia, Gilles
Dubochet, Philipp Haller, Aleksandar Prokopec, Antonio Cunei, Tiark Rompf,
Donna Malayeri, Phil Bagwell, Adriaan Moors, Ingo Maier.
• To our external committers and frequent contributors:
Paul Phillips, Miles Sabin, Ilya Sergey, Caoyuan Deng, James Matlik, Frank
Teubler, Kevin Wright, Manohar Jonnalagedda, Pedro Furlanetto, Johannes
Rudolph, Jason Zaugg, Seth Tisue, Ismael Juma, Mark Harrah, Colin Howe,
Mirko Stocker, Spiros Tzavellas, Matt Russell, David Taylor
• To the community at large for your support, suggestions,
encouragements.