Tutorials

Seven runnable notebooks that take you from reading LeMat-Synth data to producing it, and finally to changing what gets produced. Each one is self-contained, states its prerequisites and cost up front, and explains what every step is doing rather than just executing it.

They live in examples/notebooks/tutorials/ in the repository, and each one runs either locally or on Google Colab — the same file, no edited cells.

uv run jupyter lab examples/notebooks/tutorials/

Or open one on Colab and skip the install entirely:

•   1 — Explore the LeMat-Synth dataset
•   2 — Finding papers
•   3 — Batch extraction with the CLI
•   4 — Synthesis + performance from a paper
•   5 — Evaluating extraction quality
•   6 — Customising the ontology
•   7 — Building a custom case study

The tutorials#

The track says what a tutorial is for, not how hard it is: Use the data reads what is already published, Extract produces new data from papers, and Extend changes what “extracted” means — Tutorial 6 along the schema axis, Tutorial 7 along the domain axis. Tutorial 3 comes before Tutorial 4 because one CLI command is how most people will run this — Tutorial 4 opens the same pipeline up when you need to change it rather than run it.

Which one should I start with?#

Setting up .env#

Every tutorial that touches an LLM starts with the same setup, and none of them ever takes an API key as an argument. Keys live in one .env file at the repository root, are loaded into the process environment once per session, and LiteLLM reads them from there — so your keys never appear in notebook code, notebook output, or git history.

cp .env.example .env

Then edit .env — one key per line, no quotes, no spaces around =:

GEMINI_API_KEY=AIza...
ANTHROPIC_API_KEY=sk-ant-...
MISTRAL_API_KEY=...
HF_TOKEN=hf_...

.env is git-ignored, so it is never committed.

Using OpenRouter instead of per-provider keys#

Every tutorial that calls a model has a USE_OPENROUTER flag. Set it to True and all its LLM calls go through OpenRouter with a single OPENROUTER_API_KEY, using any model id from openrouter.ai/models:

USE_OPENROUTER = True

OPENROUTER_MODELS = {
    "material": "google/gemini-3-flash-preview",
    "synthesis": "google/gemini-3-flash-preview",
    "judge": "google/gemini-3-flash-preview",
    "linker": "google/gemini-3-flash-preview",
    "vlm": "anthropic/claude-sonnet-4.6",
}

Under the hood this builds the same SystemPrefixedLM the CLI uses, with api_base and the key passed explicitly — so system prompts and per-call cost tracking work exactly as they do on the direct path.

For the CLI (Tutorial 3) the equivalent is an openrouter/-prefixed model plus api_base:

lemat-synth extract paper.md \
    synthesis_model=openrouter/google/gemini-3-flash-preview \
    api_base=https://openrouter.ai/api/v1

How a key reaches a model#

1. load_dotenv() puts the values into os.environ.
2. get_llm_from_name(name) looks the name up in LLM_REGISTRY (src/llm_synthesis/utils/llms.py), which maps a friendly alias to a real LiteLLM model string — and, for OpenRouter models, names the environment variable holding the key.
3. LiteLLM reads the provider’s standard variable at call time.

Switching models is therefore a one-line change, and adding a provider means adding a LLM_REGISTRY entry plus a line in .env — never editing call sites. The lemat-synth CLI goes one step further and loads the repository-root .env itself, so you never have to export anything into your shell.

Notes on running them#

• Notebooks are outputs-free in git. nbstripout runs as a pre-commit hook, so committed notebooks carry no outputs. Your local runs will fill them in.
• No sample papers ship with the repository. data/ is git-ignored, but every tutorial still runs standalone: Tutorial 4 downloads its fixed example paper from arXiv, and Tutorials 3 and 5 write a small synthetic paper of their own. Tutorial 2 shows how to find papers of your own to point them at.
• The datasets are gated. Request access to LeMat-Synth and LeMat-Synth-Papers before running Tutorials 1 and 2, then hf auth login or set HF_TOKEN.
• On Colab, the first cell takes a few minutes. It shallow-clones the repository and installs it into the session; the clone is cached for the rest of the session. If an import fails right afterwards, use Runtime → Restart session and run the cell again. Anything a tutorial writes lands in the session’s copy of data/ and disappears when the runtime recycles — download what you want to keep.
• Rate limits. If a provider starts refusing calls, lower LLM_SYNTHESIS_MAX_CONCURRENT_LLM_CALLS (default 10) in .env.

Related documentation#

• Quickstart — the three-minute version
• CLI Reference — every command and config key
• Output Format — what the result files contain
• Python API — building pipelines in code
• Annotations — the ground-truth corpus used in Tutorial 5
• Troubleshooting — when something fails