Nanyx
PlaygroundPackages

Hosts

What runs a program's entry point, which contexts it provides, and how to write one with foreign functions

A host is what runs a Nanyx program. It takes the program’s entry point, loads the contexts the program may use, such as Console, and runs it. Every application names its host, and the compiler checks that the entry point fits it.

A host is an ordinary Nanyx package. The compiler ships one, the console host nanyx.console, for command-line programs, and anyone can write another. Only a host can reach the world outside the program, through the foreign functions its embedder provides, so a library can do nothing the host’s contexts don’t allow.

Choosing a host

A package names its host in its manifest’s host field. A host the compiler ships is named at the compiler’s version:

(
  package = (name = "acme.tool", version = "0.1.0", sources = ["src/**/*.nyx"])
  host = (package = "nanyx.console", version = "0.28.0")
)

Any other host is found by its directory, relative to the manifest, as a dependency is, and has no version: host = (package = "acme.server", path = "../server"). nanyx new writes the console host into a new package’s manifest.

A package whose manifest names no host is a library. It can’t be run, and an @entry in it is an error (NYX0031). See Packages and projects.

nanyx run, nanyx build and nanyx check take --host to run a package under another host for one command: a host the compiler ships, --host nanyx.console, or the directory of one, --host ../server.

A single .nyx file runs under the console host unless a ::host directive at its top names another, in the same words a manifest would use (see Directives):

::host acme.server ../server

The entry point fits the host

The entry point is passed to the host’s run function, so it must have the type run takes. Under the console host that is a function with no parameters, which may require Async, Clock, Console, Env, FileSystem, Http and Random. An entry point of another shape is an error at its @entry that names the host and both types:

main.nyx:3:1: error NYX0237: The entry point 'main' doesn't fit the host 'nanyx.console/console.host'
  expected: [Async, Clock, Console, Env, FileSystem, Http, Random] () -> a
  found:    int -> int

A script’s statements are its entry point, and run under its host in the same way.

The console host

The console host, nanyx.console, runs command-line programs. It provides seven contexts:

  • Console reads standard input and writes standard output and standard error.
  • Env gives the program’s command-line arguments (args) and environment variables (variable), and ends it with an exit code (exit).
  • FileSystem reads and writes files, through the functions of the shipped package nanyx.fs (file, dir and path). nanyx run lets a program reach the current directory and, with --allow-dir, one other; a path outside them is #error(#permissionDenied).
  • Http makes HTTP requests, through the functions of the shipped package nanyx.http.core (http). Each call waits for its response.
  • Async lets a function wait. The calls of nanyx.fs and nanyx.http.core wait, so they require it, and so does every function that calls them.
  • Random gives random numbers, drawn from the operating system’s random source, through the functions of nanyx/random. random.seeded gives the same numbers on every run instead.
  • Clock gives the time of day, now(), in Unix milliseconds, and ticks(), milliseconds that never go backwards, for timing.

The host has a prelude: a module whose exports every module of the program sees without importing them. The console host’s brings in Async, Clock, Console, Env, FileSystem, Http, Random, print, println, eprint and eprintln, so a program uses them with no import:

match args()
  | [name, ..._] -> println("Hello, {name}!")
  | [] -> exit(2)

nanyx run greet.nyx Ada prints Hello, Ada!; with no arguments, the program ends with the exit code 2. Arguments for the program go after the file or package.

Console and Env are declared in a separate package, nanyx.console.core, which the compiler ships too. A library has no host, so it sees no host’s prelude: it lists nanyx.console.core in its dependencies, at the compiler’s version, and imports what it uses:

(
  package = (name = "acme.report", version = "0.1.0")
  dependencies = [(package = "nanyx.console.core", version = "0.28.0")]
)
export module greetings

import nanyx.console.core/console as (Console, println)

export def greet: [Console] string -> () = { name ->
  println("Hello, {name}!")
}

An application that depends on the library calls greet from its entry point, which the console host gives a Console.

Built as a component

nanyx build writes a program under the console host as a WebAssembly component, which wasmtime run (46 or later), or any runtime that implements WASI 0.3, runs as it is: wasmtime run out/greet.wasm Ada. nanyx run runs the program as it always has, and nanyx build --core writes a core module, as before.

A component is built from the host’s component root, console.host.wasi, which provides everything the console host does, on WASI 0.3. Its world is nanyx:console/command, which is wasi:cli/command@0.3.0 with wasi:http/client@0.3.0, and a component imports only what the program uses:

  • A program that makes no HTTP requests is a wasi:cli/command component, which any WASI 0.3 runtime runs.
  • One that does imports wasi:http/client too: wasmtime run -S http out/greet.wasm provides it.
  • A component reaches only the directories the runtime that runs it opens for it: wasmtime run --dir . out/greet.wasm opens the current directory, where a relative path is found.

The HTTP host

The HTTP host, nanyx.http, serves a program whose entry point is a request handler, a Request -> Response function. It provides Http, for requests the handler makes itself, with Async, so that the handler can wait on them, Random and Clock, and neither Console nor FileSystem. While one handler waits, the host answers other requests. Each request is answered by a fresh instance of the program, so nothing is shared between requests, and a handler that stops the program ends only its own request, which is answered with a 500.

@entry
def main: Request -> Response = { request ->
  match Request.path(request)
    | "/" -> Response.text(200, "Welcome!")
    | _ -> Response.text(404, "Not Found")
}

A package names it as its host with the port to serve on, host = (package = "nanyx.http", version = "0.28.0", settings = (port = 8080)), and nanyx run serves it there, on 3000 when no port is given. A host declares the settings it takes, each with its default, so a setting it doesn’t take is an error at the setting. See the HTTP server cookbook.

Served as a component

nanyx build writes a program under the HTTP host as a WebAssembly component of the world wasi:http/service@0.3.0, which wasmtime serve (Wasmtime 51 or later), or any runtime that implements the world, serves as it is: wasmtime serve -S cli out/app.wasm serves it on port 8080, with -S cli for the stdout and stderr the program writes dbg and its crashes to. nanyx run serves the program as it always has, and nanyx build --core writes a core module, as before.

A component is built from the host’s component root, http.host.wasi, which provides Http, Async, Random and Clock on WASI 0.3, as the core root does. Neither root provides Env: the world has no environment to read.

  • The runtime may reuse an instance. wasmtime serve gives an instance of the component up to 128 requests, one after another, and the program runs afresh for each, so nothing the program computes is kept from one request to the next. The instance’s memory is, though, and a request that crashes ends its instance, which the runtime answers with a 500. wasmtime serve --max-instance-reuse-count 1 gives each request an instance of its own, as nanyx run does.
  • An instance answers one request at a time. The runtime gives it no second request until it has written the first’s response, and starts other instances for requests that come in meanwhile.
  • A request whose method Request has no case for, such as CONNECT or TRACE, is answered with a 400, without calling the handler.

The playground host

The playground host, nanyx.playground, runs a console program in a web page, as the playground does. It stands in for the console host: it provides the same Console, from nanyx.console.core, whose output goes to the page, Random, drawn from a generator the page seeds, and Clock, the browser’s time, so a program that needs only those compiles under it unchanged. It provides nothing else, no Env, FileSystem, Http or Async, so a program that needs one of them is an error naming the host:

error NYX0205: 'file.read' requires the contexts 'Async' and 'FileSystem', which the host 'nanyx.playground/playground.host' does not provide

The compiler built for the browser runs a script under it unless the script’s ::host names another; elsewhere a script runs under it with ::host nanyx.playground. The package ships the JavaScript that runs such a program in a web worker and passes what it prints, and what its dbgs print, to the page. Reading a line waits for a line typed in the page, and gives #error(#endOfInput) once the input has ended.

Where dbg prints

dbg needs no context, so it works under every host, but where what it prints goes is the host’s to decide. A debug build that nanyx run runs gives each dbg it reaches to the runtime first, with where it is written and the value’s text:

  • The console host writes it to standard error, [main.nyx:12] 84, in order with what the program writes there itself.
  • The HTTP host writes it to the server’s log, with the request it came from, dbg (GET /hello): [src/main.nyx:5] "Ada", and never into the response.
  • A host of your own gets the console host’s: the line on standard error. So does a module written by nanyx build or nanyx wasm and run by another WebAssembly runtime, which writes it there itself.

A release build has no dbg to print.

Hosts and contexts

A host provides the contexts a program may require, and only those. A function that requires a context the host doesn’t provide can’t be called from the entry point, so a host that provides no FileSystem guarantees that neither the program nor any library it uses touches the file system. A library says what it needs in its types, [Console], and runs under every host that provides it.

Writing a host

A host is a package whose manifest says it is one, in its package’s host field: root names the module that exports run, and prelude, which may be left out, the module whose exports every module of a program it runs sees.

(
  package = (
    name = "acme.shout"
    version = "0.1.0"
    sources = ["src/**/*.nyx"]
    host = (root = "shout.host", prelude = "shout.prelude")
  )
)

Naming a package as a host when its manifest doesn’t declare it one is an error.

A host may also have a component root, which nanyx build builds a WASI 0.3 component from: component = (root = "shout.host.wasi", world = "wasi:cli/command@0.3.0"), beside root. It exports run as the root does, written against the WIT of the world it names, which the package keeps in its wit directory, with the packages that WIT uses under wit/deps. It provides the contexts the root does: one more or fewer is an error at its run (NYX0275). The host here provides one context, Log, which writes a line in capitals. The context is declared in a module of its own:

--| What a program under the shouting host can do.
export module shout.log

--| Writes a line, loudly.
export context Log = (line: string -> ())

The prelude re-exports it, so a program calls line without importing anything:

export module shout.prelude

export shout.log (Log)

The root module exports run, which takes the entry point, loads the contexts the host provides with use, and calls it. run’s parameter type is the entry shape the host accepts: here, a function with no parameters that may require Log. What the entry point returns is ignored.

export module shout.host

import (
  nanyx/host.memory
  nanyx/host.runtime
  nanyx/text
  shout.log as (Log)
)

@extern("wasi_snapshot_preview1", "fd_write")
def fdWrite: [HostRuntime] I32, I32, I32, I32 -> I32

def writeLine: [HostRuntime] string -> () = { text ->
  def loud = string.uppercase(text) ++ "!\n"
  def iovec = allocate(12)
  writeI32(iovec, addressOf(loud))
  writeI32(iovec + 4, byteLength(loud))
  ignore(fdWrite(1, iovec, 1, iovec + 8))
  keepAlive(loud)
  free(iovec)
}

export def run: [HostRuntime] ([Log] () -> a) -> () = { entry ->
  use Log(line = { text -> writeLine(text) })
  ignore(entry())
}

An application that names this host, host = (package = "acme.shout", path = "../shout"), gets Log and nothing else:

module main

@entry
def main = {
  line("hello")
}

It prints HELLO!. A call to println in it is an unknown function, since this host provides no Console.

HostRuntime

run and the functions it calls require HostRuntime, a context with no members that marks the host. The toolchain provides it only to the modules of the host’s own package (its root, its component root, and the modules they share) and of the host libraries it depends on; anywhere else, including the other packages the host depends on, a function that requires it, or calls one that does, is an error (NYX0238). Foreign functions and host memory require it, so only the host can call them, and a library can only reach the outside world through the contexts the host chose to provide.

Foreign functions

A foreign function is a function the WebAssembly module imports from its embedder: under nanyx run, a WASI call such as fd_write. It is declared with the @extern attribute, which names the import’s module and name, and a def with a type and no value:

@extern("wasi_snapshot_preview1", "proc_exit")
def procExit: [HostRuntime] I32 -> ()
  • Its type takes and returns only WebAssembly’s value types, written as the sized numbers I32, I64, F32 and F64, or () for no result. Strings, records and lists don’t cross the boundary: a host passes their addresses and lengths, in host memory. Any other type, a value after the type, or arguments other than two strings is an error (NYX0241).
  • It requires HostRuntime, and a type without it is an error (NYX0242). Any module may declare a foreign function, but only a host can call one.
  • Each import is declared once. Two foreign functions that name the same import with different types are an error (NYX0243).
  • The embedder provides it. nanyx run provides WASI preview 1’s functions. An import it doesn’t provide fails when the module is loaded, with an error that names the import.

A foreign function can be called, but naming one as a value, rather than calling it, can’t be compiled yet.

Host memory

A foreign function deals in addresses in the module’s linear memory: WASI’s fd_write reads the bytes it writes from a buffer, and args_get writes the arguments into one. nanyx/host.memory gives a host that access. Every function in it requires HostRuntime:

  • allocate and free a block of bytes;
  • readByte, readI32 and readI64, and writeByte, writeI32 and writeI64, at an address;
  • addressOf and byteLength, where a string’s bytes are and how many there are, and stringAt, a new string copied from an address and a length;
  • keepAlive, which keeps a string alive until it is called, for one whose address is written into memory a foreign function reads, as the iovec above is;
  • hostState, the address of 4096 bytes set aside for the host, all zero at the start, for what it keeps from one call to the next.

These are the only unchecked operations a host performs. A read or a write outside the module’s memory traps, but one inside it where the host has no block reads or overwrites the program’s own data.

Host libraries

Hosts share what they are written on through host libraries: packages whose manifest says hostLibrary = true in its package. A host library’s modules may require HostRuntime, as a host’s own do, when the build host’s package depends on it, so two hosts call the same foreign functions without each declaring them:

(
  package = (name = "acme.platform", version = "0.1.0", sources = ["src/**/*.nyx"], hostLibrary = true)
)

A program that depends on a host library can import it, but can’t call what requires HostRuntime there, since the program’s own code may not require it. The compiler ships one, nanyx.wasi, which the console host and the HTTP host are written on.

Packages the compiler ships

The compiler ships the packages every program needs beside itself, in a packages directory next to the nanyx executable, so a program builds and runs with nothing downloaded:

  • nanyx.console, the console host;
  • nanyx.console.core, its interface package, which declares Console and Env;
  • nanyx.fs, the interface package of the file system, which declares FileSystem, Path and the functions on them;
  • nanyx.http.core, the interface package of HTTP, which declares Http, Request, Response and the functions on them;
  • nanyx.http, the HTTP host;
  • nanyx.wasi, the host library the console and HTTP hosts are written on: WASI’s random source and clocks, the runtime’s nanyx:http, and the component model’s canonical ABI and WASI 0.3 interfaces their component roots use;
  • nanyx.playground, the playground host, which runs a console program in a web page;
  • nanyx.regex, for regular expressions.

A manifest or a directive names a package the compiler ships at the compiler’s version exactly; any other version is an error that names the one shipped (NYX0029). In a ::host or ::dependency directive it needs no version at all: ::dependency nanyx.console.core. The standard library, nanyx/..., is built into the compiler and is never listed as a dependency.