MetewandReproducible benchmarks. Across languages.

Compare computational methods with explicit contracts, independent evaluation, and provenance for every result.

Use your own language

Run implementations as separate processes. Use the Python or R SDK, or implement the worker protocol in another language.

Define the comparison

Keep datasets, problem definitions, implementations, and execution policies explicit. Evaluate results with an independent worker.

Keep the evidence

Record content-addressed identities, execution measurements, and provenance. Inspect durable runs and export results for analysis.

From a manifest to an inspectable run

A benchmark combines a dataset, a problem, and one or more implementations. Metewand checks their contracts, expands the configurations into a deterministic plan, and runs workers through a versioned process protocol.

metewand check --manifest benchmark/metewand.yaml
metewand plan --manifest benchmark/metewand.yaml
metewand run --manifest benchmark/metewand.yaml --output-dir /tmp/metewand-run

The benchmark author defines what counts as a valid result and how to measure its quality. Metewand records the execution and its guarantees so you can trace an observation back to the inputs and policies that produced it.

Start with a small example

The getting started guide runs a complete benchmark from the repository. The examples cover Python and R workers, generated datasets, dataset-free problems, and parameter matrices.

Metewand is under active development. The current runtime supports trusted local, one-shot benchmarks. See the execution reference for supported behavior and the roadmap for planned capabilities.