Trustless Grid Computing
in ConCert
Robert Harper
Carnegie Mellon University
Acknowledgements
• Co-PI’s
Karl Crary, Frank Pfenning, Peter Lee.
• Support
NSF ITR program.
• Students (who do the real work)
Chang, Delap, Dreyer, Kliger , Magill, Moody,
Murphy, Petersen, Sarkar, Vanderwaart,
Watkins.
• Thank you to GALT Organizers for the
invitation!
5/31/2016
GALT Workshop, Edinburgh
2
Grid Computing
• “The network as computer.”
– Exploit idle resources on the network.
– Many ad hoc grids.
• SETI@HOME
• FOLDING@HOME
• But what is a general grid model?
– Application model?
– Trust model?
– Participation model?
5/31/2016
GALT Workshop, Edinburgh
3
The ConCert Grid Model
• One computer, many keyboards.
– Decentralized scheduling.
– Emphasis on code mobility.
• Policy-based participation.
– Declarative statement of participation criteria.
– Applications must prove compliance.
• Dependency-based scheduling.
– Arbitrary depth.
– And/or dependencies.
– Inspired by CILK/NOW.
5/31/2016
GALT Workshop, Edinburgh
8
The ConCert Network
Client
5/31/2016
Hosts
GALT Workshop, Edinburgh
9
Host Setup
Locator
Scheduler
Worker
5/31/2016
Peer-to-Peer
Discovery Protocol
Distributed Scheduler
Loader/Verifier/Runner
GALT Workshop, Edinburgh
10
Scheduler
• Work-stealing model.
– Who has work to do?
– Grab work, compute result, deliver to owner.
– Fully decentralized.
• Dependency-based scheduling.
– Supports depth > 1, don’t care and don’t know
parallelism.
– Well-founded dependencies only; no cycles.
– Cf. join calculus, JOCaml (more general).
• Maintain ready and waiting queues.
– Ready queue: available for “stealing”.
– Wait queue: awaiting satisfying assignment.
5/31/2016
GALT Workshop, Edinburgh
12
Chords
• The unit of work on the grid is a chord.
5/31/2016
GALT Workshop, Edinburgh
13
Chords
• Semantics:
– Idempotent: can always be re-run.
– Non-blocking: runs to completion (but may create
more chords).
– Communication via dependencies. Satisfying
assignment passed on activation.
• Dependencies:
– And/or dependencies on results of other chords.
• Certificate:
– Proof of compliance with host policy.
– Generated by a certifying compiler.
5/31/2016
GALT Workshop, Edinburgh
14
Moving Chords Around
A client submits work, broken into chords, to the local
conductor.
5/31/2016
GALT Workshop, Edinburgh
16
Moving Chords Around
Idle peers steal chords to work on.
Chords have destinations for their answers, shown by color.
5/31/2016
GALT Workshop, Edinburgh
17
Moving Chords Around
Some chords spawn new cords. They might depend on other
chords before they can run.
The destination of F and G is the green node, since they
will be used to fill H’s dependencies.
Moving Chords Around
When a chord finishes, the result is sent to its destination.
The client interprets and displays the results.
Simultaneously, unfinished chords continue to be stolen...
Moving Chords Around
When the green node has answers for F and G, H is then
ready to be stolen.
5/31/2016
GALT Workshop, Edinburgh
20
Grid Programming
• What is a good language for grid
applications?
– Functional language is natural.
– Manage chord creation, distribution, and
coordination.
– Permit binding to local resources.
– Compiler generation of safety certificates.
5/31/2016
GALT Workshop, Edinburgh
21
Two Foundational Questions
• What is an appropriate type system for
a grid programming language?
– Enforce mobility constraints.
– Clean type system to support development,
compilation, certification.
• What safety policies can we support?
– How to state policies?
– How to prove compliance?
– How to support multiple policies?
5/31/2016
GALT Workshop, Edinburgh
28
Modalities for Mobility
• Curry-Howard interpretation of modal logic.
– Cf. related ideas by Cardelli, Gordon, Walker, et al.
– Modalities enforce locality and mobility
constraints by type checking.
• Hosts are possible worlds.
– Each host provides an execution site for chords.
• Accessibility between possible worlds.
– A · B means that may move from A to B.
5/31/2016
GALT Workshop, Edinburgh
29
Modalities for Mobility
• Accessibility should be an equivalence:
– Reflexive: can stay here.
– Transitive: can move from host to host.
– Symmetric: can go back to source.
• This suggests looking at S5 modal logic.
– Appropriate for RST accessibility.
– Intuitionistic variant for computational
interpretation.
5/31/2016
GALT Workshop, Edinburgh
30
A Candidate Type System
• Necessity (¤ A): an A anywhere.
– Classifies mobile code of type A.
– Enforces marshalling and access restrictions.
– Runnable at any accessible site.
• Possibility (¦ A): an A somewhere.
– Classifies remote code of type A.
– Expresses resource locality.
– Can only depend on remote resources.
• Other modalities are imaginable.
– Walker: broadcast/multicast modality.
5/31/2016
GALT Workshop, Edinburgh
31
Modalities for Mobility
• Truth (local) typing judgment:
Possible (Remote)
Resources
True (Local)
Resources
5/31/2016
Valid (Mobile)
Resources
GALT Workshop, Edinburgh
32
Mobility as Necessity
• Validity (mobile) typing judgement:
• Mobile = does not use local resources.
5/31/2016
GALT Workshop, Edinburgh
33
Mobility as Necessity
• Box = marshal value and bindings.
• Values of boxed type are mobile code
available here that can run anywhere.
5/31/2016
GALT Workshop, Edinburgh
34
Mobility as Necessity
• Unboxing = extract and run mobile code.
• Implicit un-marshalling:
5/31/2016
GALT Workshop, Edinburgh
35
Locality as Possibility
• Possible (somewhere) typing judgement:
• What is here is somewhere:
5/31/2016
GALT Workshop, Edinburgh
36
Locality as Possibility
• Go to remote site, rendering local
resources possible.
• Can only use specified remote
resources!
5/31/2016
GALT Workshop, Edinburgh
37
Locality as Possibility
• Create a local proxy:
• Access it:
5/31/2016
GALT Workshop, Edinburgh
38
Joint Possibility
• Cannot consider only a single possibility
at a time!
– A and B both possible does not imply that
they are true at the same world.
• Could resort to explicit worlds.
– M : A @ w means M is of type A at w.
– Seems unnatural for our grid model in
which code moves spontaneously.
5/31/2016
GALT Workshop, Edinburgh
39
Joint Possibility
• Solution: take joint possibility as
primitive.
• Possibility context is clustered into
records:
• Possibility modality for a cluster: ¦fg
5/31/2016
GALT Workshop, Edinburgh
40
Joint Possibility
• Generalized “go to” rule:
5/31/2016
GALT Workshop, Edinburgh
41
Warning: Work in Progress
• Cut elimination for a sequent variant.
– Work in progress.
– Needs of proof suggested clustering.
• It really is S5.
– Relate to explicit worlds formulation.
– Cf Alex Simpson’s PhD work.
• Operational interpretation.
– Abstract machine for mobile code.
– Type safety proof for the semantics.
5/31/2016
GALT Workshop, Edinburgh
42
Policies and Certification
• Policy should specify what is permissible.
– Memory safety (no wild pointers)
– Control safety (no illegal instructions or jumps)
• Current approaches specify how to ensure
compliance.
– Fix a particular type system (or equivalent) and
certificate format, baked into TCB.
– For example, PCC or TAL certified code formats.
5/31/2016
GALT Workshop, Edinburgh
43
Foundational Certification
• This raises two issues:
– Flexibility: different type systems for
different problems.
– Robustness: what if the committed type
system is unsound?
• Moving the type system out of the TCB
solves both problems.
– Appel emphasizes robustness.
– We’re concerned with flexibility.
5/31/2016
GALT Workshop, Edinburgh
44
Foundational Safety
• Specify operational semantics of target
architecture.
– Fully realistic, e.g., IA-32 + OS + RTS.
– No unsafe transitions.
• Safety: target does not get stuck.
– Any type system must come with a proof of
progress and preservation.
– Experience shows that these proofs may be
mechanized fairly easily (using Twelf).
5/31/2016
GALT Workshop, Edinburgh
45
Foundational Certification
Certified Binary
of
Grid Application
5/31/2016
GALT Workshop, Edinburgh
46
Foundational Certification
Object code compiled as a DLL.
Compiled Machine
Code
5/31/2016
GALT Workshop, Edinburgh
47
Foundational Certification
Annotations facilitate type checking.
Typing Annotations
for Object Code
5/31/2016
GALT Workshop, Edinburgh
48
Foundational Certification
Type checking program (written in Twelf)
Type Checking
Program
5/31/2016
GALT Workshop, Edinburgh
49
Foundational Certification
Proof that “type checks ) safe” (i.e.,
partial correctness of checker)
Soundness Proof
for
Type Checker
5/31/2016
GALT Workshop, Edinburgh
50
Examples
• TALT
– Similar to TALx86, but more realistic and with a
safety proof.
– Safety proof is mechanically checked.
– Structured as a safety proof for an abstract
machine plus a simulation lemma of AM on target
architecture.
• TALT + Resource Bounds
– Goal: ensure that object code yields processor at
set intervals.
– Precludes denial of CPU service.
5/31/2016
GALT Workshop, Edinburgh
51
Summary
• Declarative approach to safe grids.
– Passive, policy-based participation model.
– Logic and proof technology for specifying
policies and proving compliance.
• Close interplay between systems
building and foundational theory.
– Type systems for mobile code.
– Type systems for various safety policies.
5/31/2016
GALT Workshop, Edinburgh
58
Thanks!
• Web site:
http://www.cs.cmu.edu/~concert.
• Demonstration available after talk.
• Any questions or comments?
5/31/2016
GALT Workshop, Edinburgh
59