next-generation programming platform, currently in development
help fund the project

Twitter . GitHub . RSS

A huge milestone - first distributed Unison program run successfully!

As of this PR, we got our first distributed program to run successfully. The program dynamically spawns two Unison nodes and ping-pongs a number back and forth between them until the count reaches 5. This exercises a huge amount of code - the BlockStore implementation that Sam has worked on, Arya’s parsing code, the node container code I wrote which handles spawning of new local nodes, the distributed programming API implementation, which is responsible for automatically transporting the computation back and forth between the nodes, and lots more:

Remote {
  n1 := Remote.spawn;
  n2 := Remote.spawn;
  let rec
    ping i = Remote { 
      i := n2 (i + 1); 
      if (i >= 5) (pure i) (pong i); 
    pong i = Remote { i := n1 (i + 1); ping i; }
  in ping 0;

Notes on syntax - Remote { ... } is an effect block, desugared much like a do block in Haskell, using calls to Remote.bind and Remote.pure. This isn’t specific to Remote, a Vector { ... } effect block will desugar to calls to Vector.bind and Vector.pure, and we could allow arbitrary, dynamic m { ... } there (haven’t bothered with this yet).

Why is this awesome???

This short program is deliberately missing a whole bunch of boring stuff. You don’t open up a socket or deal with communication channels in any explicit way, and you certainly don’t write manual parsing or serialization code on your Unison nodes (unless you are interacting with non-Unison services). You simply declare that you want the computation moved elsewhere, and the runtime makes it so. How… refreshing! Have a look at these two lines:

i := n1 (i + 1); 
if (i >= 5) (pure i) (pong i);

Here, we transport the value i + 1 to another node, n1 : Node. We are also transporting the continuation function i -> if (i >= 5) ... to n1, which will be applied on the result of i + 1 to continue the computation from there (which can terminate or bounce to the other node). The function or value being transported in this way could be a function with a million transitive dependencies; any that are missing will be synced as part of the inter-node protocol and cached for future use.

The other very nice thing here is that both nodes, and programs that spawn nodes are first-class values. You can stash nodes in a list, pass them to functions, pass them to functions which are passed to other nodes, and so on. In this trivial program, we’ve hardcoded a couple locally spawned nodes, but perhaps you can see how easy it would be to abstract over that, just using the ordinary means of abstraction and combination of the Unison language. Now imagine writing a single Unison program which abstracts over its node provisioning, and can therefore be run either with purely local nodes (for testing), or on a massive distributed cluster!

Lastly, observe that the Unison program describes its own node provisioning and deployment. You don’t write a program that describes just a single OS process, then use a bunch of ad hoc tools to provision machines, get your code to those machines, and run your programs. You describe, with a single, typed program, what computing resources are provisioned, and ‘deployment’ is just a trivial byproduct of the Unison inter-node protocol which syncs any needed definitions to the nodes you provision. In other words, provisioning, deployment, and orchestration can be talked about directly in Unison, in a first-class way, and you can abstract over details and build reusable bits of logic, just like you would with any other interesting logic you have in your programs. Nice!!

What’s next?

Now that all this is basically working, we’ll be focused on exploring what’s possible with these APIs. We have a lot of nice ingredients:

In case you missed it, here’s v0 of the persistent data API:

-- Simple local, persisted key-value storage

data Index k v

empty :: Remote (Index k v)
lookup :: k -> Index k v -> Remote (Maybe v)
insert :: k -> v -> Index k v -> Remote ()
delete :: k -> Index k v -> Remote ()

Those are the actual type signatures - all Unison values can be persisted in this way, including functions! Just like in the distributed programming API, you don’t deal with manual serialization or encoding to the database, or decoding on the other end.

Lastly, some other notable stuff in the PR that is of general interest:

More updates soon!

comments powered by Disqus