Obelisk

A deterministic workflow engine built on the WASM Component Model.

[!WARNING] Pre-release: Expect changes in CLI, gRPC, WIT, and database schema.

Obelisk in Action

The Stargazers app features:

• A webhook listening to GitHub star events.
• Activities for interacting with Turso DB, OpenAI, and GitHub.
• A workflow orchestrating the activities.

What's Included

• Obelisk Runtime: A single binary executing deterministic workflows, activities, and webhook endpoints, persisting steps in execution log using SQLite.
• Control Interfaces:
  • CLI: Manage executions via the obelisk command.
  • gRPC API: Programmatic interaction.
  • Web UI: View and submit function executions, execution logs, WIT definitions, and a time-traveling debugger showing backtraces and sources of recorded events.

Core Principles

• Replayable Workflows: Deterministic execution ensures reliable recovery, debugging, and auditing.
• Resilient Activities: Automatic retries on errors and timeouts, with persistent input and results.
• Schema-first design with end-to-end type safety: Uses WASM Component Model and WIT IDL for generating API bindings.

Use Cases

• Periodic Tasks: Automate periodic checks with complex logic using regular code.
• Background Jobs: Offload tasks with built-in error handling and retries.
• Batch jobs: Manage large-scale tasks like faas deployments.
• End-to-End Testing: Automate tests with detailed logs of each step.

Key Features

WASI Activities

• Activities must be idempotent (retriable). This contract must be fulfilled by the activity itself.
• Support for HTTP requests via WASI 0.2 HTTP client.
• Automatic retries on errors, timeouts, and panics (WASM traps).
• Persistent execution results.

Deterministic Workflows

• Runtime guaranteed determinism, fully replayable with persistent execution log and thus crash resilient.
• Automatic retries on failures.
• Support for spawning child executions with structured concurrency.
• Distributed sagas (planned).

WASI Webhook Endpoints

• Mounted as a URL path, serving HTTP traffic.
• Support for spawning child executions.

Work Stealing Executor

• Concurrency limits and customizable retry handling.

Installation

Supported Platforms

• Linux x64, arm64 (musl, glibc v2.35+, NixOS)
• MacOS x64, arm64

Pre-built Binary

curl -L --tlsv1.2 -sSf https://raw.githubusercontent.com/obeli-sk/obelisk/main/download.sh | bash

Or use cargo-binstall:

cargo binstall obelisk

Docker

docker run --net=host getobelisk/obelisk

From Source

Requires protoc.

cargo install --locked obelisk

Using Nix:

nix run github:obeli-sk/obelisk/latest

Getting Started

Check out the Getting Started Guide for details.

Start the Server

obelisk server run

CLI Usage

obelisk client component list
# Call fibonacci(10) activity from the workflow 500 times in series.
obelisk client execution submit testing:fibo-workflow/workflow.fiboa '[10, 500]' --follow

Web UI

Visit localhost:8080 to manage components, functions and execution history.

Contributing

This project has a roadmap and features are added in a certain order. Discuss features via GitHub Discussions before contributing. A Contributor License Agreement is required.

Development

Set up dependencies via Nix:

nix develop

Or manually install dependencies (see dev-deps.txt).

Run the program:

cargo run --release

Running Tests

./scripts/test.sh