nora pipelines

Pipelines are the Foundry surface: ingest, clean, chunk, enrich, index. The CLI mirrors the visual canvas: every node, every edge, every scheduled run is scriptable. The pipeline lifecycle is the biggest CLI surface after flows: plan on flows use for your Flow and separately reference pipelines by their IDs.

Command groups

CRUD: pipelines list/get/create/update/delete
Nodes: pipelines nodes add/list/update/remove
Edges: pipelines edges connect/list/update/delete
Schedule: pipelines schedule set/clear
Runs: pipelines dry-run, pipelines run file/document/folder
Programmatic invoke: pipelines invoke, pipelines rotate-secret
Blocks: pipelines blocks test/ingest-local
Palette introspection: pipelines palette, pipelines block
Publish: pipelines publish/rollback, pipelines publications list/live
Logs: pipelines logs list/get
Bulk: pipelines sync
Cloud ingest: pipelines ingest-cloud

Pipeline CRUD

List / get

nora pipelines list
nora pipelines get p_billing_ingest

list gives one row per pipeline. get prints full metadata plus nodes and edges.

Create

nora pipelines create \
  --name "Billing ingest" \
  --stages source,clean,chunk,enrich,output \
  --vector-store qdrant

Flags:

--name (required): display name.
--stages (required): comma list of stage names in order.
--id <id>: optional explicit ID; otherwise server mints.
--vector-store pgvector|qdrant: index backend for the output stage.

Update

nora pipelines update p_billing_ingest \
  --set-name "Billing ingest (v2)" \
  --set-vector-store qdrant

--set-name <n>, --set-stages <s,…>, --set-vector-store <backend>.

Delete

nora pipelines delete p_billing_ingest

Nodes and edges cascade. Live pipeline versions remain reachable by ID for a grace period.

Nodes

Nodes are the individual blocks in the pipeline canvas: each palette entry (src_local, chunk_markdown, embed_openai, etc.) becomes a node instance.

List

nora pipelines nodes list p_billing_ingest

Prints all nodes as JSON.

Add

nora pipelines nodes add p_billing_ingest \
  --palette-id chunk_markdown \
  --stage chunk \
  --title "Chunk markdown" \
  --icon scissors \
  --tone default \
  --order 2 \
  --config '{"max_tokens": 400, "overlap": 50}'

Flags:

--palette-id (required): the palette entry ID.
--stage (required): which pipeline stage.
--title, --icon, --tone (required): display.
--order (required): visual order within the stage.
--id <id>: explicit ID; server mints if omitted.
--sublabel <text>: small caption under title.
--config <json>: block-specific config.
--notes <text>: free-text notes.
--output-ports <json>: override the palette’s default output ports.

Update

nora pipelines nodes update n_chunk \
  --set-config '{"max_tokens": 600, "overlap": 100}' \
  --set-title "Chunk (longer)"

--set-* flags: --set-title, --set-sublabel, --clear-sublabel, --set-stage, --set-order, --set-config, --set-notes, --clear-notes, --set-output-ports.

Remove

nora pipelines nodes remove n_chunk

Edges

List / connect / update / delete

nora pipelines edges list p_billing_ingest
nora pipelines edges connect p_billing_ingest --from-node-id n_source --to-node-id n_chunk
nora pipelines edges update e_abc --set-label "raw markdown"
nora pipelines edges delete e_abc

Flags on connect: --from-node-id, --to-node-id (required), --id <id>, --from-port <port>, --label <text>.

Schedule

Set

nora pipelines schedule set p_billing_ingest \
  --cron "0 2 * * *" \
  --tz Asia/Seoul \
  --target billing \
  --sample 100

--cron <expr>: 5-field cron.
--tz <IANA>: timezone.
--target <tag>: the folder tag to process on each run.
--sample <n>: cap items per scheduled run.

Only flags passed change. Omit --cron to keep the current one, e.g. change only the target.

Clear

nora pipelines schedule clear p_billing_ingest

Wipes all schedule fields: pipeline becomes manual-only.

Running a pipeline

Dry-run against inline text

nora pipelines dry-run p_billing_ingest --input "This is a test document."
# Or from a file:
nora pipelines dry-run p_billing_ingest --input @sample.md
# Or from stdin:
cat sample.md | nora pipelines dry-run p_billing_ingest --input -

Runs the pipeline end-to-end against the given text without persisting output. Prints results per stage.

Run against a file

nora pipelines run file p_billing_ingest --file ./contract.pdf

Multipart-uploads the file and runs the pipeline.

Run against a stored document

nora pipelines run document p_billing_ingest --document-id doc_abc

Runs against a document already in the library.

Run against a folder

nora pipelines run folder p_billing_ingest --tag billing --max-documents 500

Runs against every document tagged billing (up to --max-documents).

Programmatic invoke

`invoke`: call the API trigger

# Simple: literal input
nora pipelines invoke p_billing_ingest --input "some text"

# JSON input from file
nora pipelines invoke p_billing_ingest --input @request.json

# With per-run variable overrides + idempotency
nora pipelines invoke p_billing_ingest \
  --input @request.json \
  --variables '{"target":"eu"}' \
  --idempotency-key inv_2026_09_01_a

# Iterate against DRAFT before publishing
nora pipelines invoke p_billing_ingest --input @sample.json --test

Programmatic equivalent of curl /foundry/:apiSlug against the public endpoint, minus the per-trigger auth (your workspace PAT already covers it). Async: returns runId + statusUrl immediately; the background executor drives the DAG. Flags:

--input <JSON|@file|->: payload. - reads stdin; @file reads a file; literal starting with {/[ is parsed as JSON, else wrapped as a JSON string.
--variables <JSON|@file|->: per-run flow variable overrides. Merged over pipeline defaults; unknown keys are ignored.
--idempotency-key <string>: dedupe key. Same key → same runId (no new execution).
--test: run the current DRAFT graph instead of the published snapshot. Skips the “publish first” gate. Use during iteration; publish before shipping externally.

`rotate-secret`: rotate an API trigger’s inbound token

nora pipelines rotate-secret p_billing_ingest --node n_api_trigger

Rotates one trg_http_api node’s apiAuthSecret. The DRAFT and (if the pipeline has a live publication) a new live version with only that trigger’s secret swapped both land atomically. External callers 401 the moment this returns. The plaintext token is printed to stdout once; save it immediately (secret manager, vault). There’s no “print again” path.

Single-block testing

nora pipelines blocks test \
  --palette-id chunk_markdown \
  --input "# Heading\n\nSome content..." \
  --config '{"max_tokens": 200}'

Test one palette block in isolation: no pipeline required. Useful when iterating on chunker or enricher config. For src_local nodes on a running pipeline:

nora pipelines blocks ingest-local n_source

Kicks off a local-filesystem scan.

Palette introspection

`palette`: list palette IDs

# Core stages only (default hides vertical items)
nora pipelines palette

# One stage
nora pipelines palette --stage normalize

# Deprecated but still accepted
nora pipelines palette --stage legacy

# Show every entry including medical / insurance / saas verticals
nora pipelines palette --profile generic

# Raw MCP response
nora pipelines palette --json

Every palette id the pipeline builder knows about: the six-stage groups plus the deprecated-but-alive legacyHidden entries. Use the id column as --palette-id on nora pipelines nodes add. Pair with pipelines block to see the config shape before you write a --config JSON. Flags:

--stage <stage>: narrow to one stage. Also accepts legacy.
--profile core|generic: core (default) hides medical / insurance / saas vertical items; generic shows everything.

`block`: schema for one palette id

nora pipelines block src_web
nora pipelines block norm_chunk --json

Prints the config-form schema the Inspector renders for one palette id: fields, defaults, options, presets. Use this to build a --config payload for nora pipelines nodes add without guessing keys.

Publishing pipelines

Pipelines version and publish like Flows.

Publish

nora pipelines publish p_billing_ingest \
  --summary "chunker: max_tokens 400 -> 600" \
  --author-label "paul"

Freezes DRAFT into a live version. Scheduled runs and downstream consumers use the live version.

Rollback

nora pipelines rollback p_billing_ingest 5

Flips the live pointer to version seq=5.

Publications

nora pipelines publications list p_billing_ingest
nora pipelines publications live p_billing_ingest

list shows history. live fetches the currently-live payload.

Logs

List runs

nora pipelines logs list p_billing_ingest \
  --status ok \
  --since 2026-07-01 \
  --limit 20

Filters:

--status ok|error
--search <text>: full-text over log payloads.
--since <ISO> / --until <ISO>
--run-kind <kind>: filter by which entry point.
--limit <n>: max rows.
--json: machine-readable.

Get one

nora pipelines logs get p_billing_ingest log_abc

Full per-stage detail for one run.

Bulk sync

nora pipelines sync p_billing_ingest --file ./pipelines/billing.json

Replaces the pipeline’s name, stages, nodes, and edges from a JSON file. Idempotent: safe to run in CI on every deploy.

Cloud ingest

nora pipelines ingest-cloud p_billing_ingest n_gdrive_source

Kicks off a manual bulk ingest for a src_cloud node (Google Drive, S3, etc.). Otherwise these run only on the schedule.

Recipes

Bootstrap a minimal pipeline in one script

P=$(nora pipelines create --name "Docs ingest" --stages source,chunk,enrich,output | jq -r '.id')

nora pipelines nodes add $P --palette-id src_local --stage source --title "Local" --icon folder --tone default --order 1
nora pipelines nodes add $P --palette-id chunk_markdown --stage chunk --title "Chunk MD" --icon scissors --tone default --order 1
nora pipelines nodes add $P --palette-id embed_openai --stage enrich --title "Embed" --icon spark --tone default --order 1
nora pipelines nodes add $P --palette-id out_pgvector --stage output --title "Index" --icon database --tone default --order 1

# Wire (need node IDs: capture from add)
# ...

nora pipelines publish $P --summary "initial"

CI: sync pipeline from git

nora pipelines sync p_docs_ingest --file pipelines/docs.json
nora pipelines publish p_docs_ingest --summary "$(git log -1 --pretty=%s)"

Get the most recent errored run

nora pipelines logs list p_billing_ingest --status error --limit 1 --json | jq '.[0]'

Turn off a schedule for maintenance

nora pipelines schedule clear p_billing_ingest
# Do maintenance...
nora pipelines schedule set p_billing_ingest --cron "0 2 * * *" --tz Asia/Seoul --target billing

​Command groups

​Pipeline CRUD

​List / get

​Create

​Update

​Delete

​Nodes

​List

​Add

​Update

​Remove

​Edges

​List / connect / update / delete

​Schedule

​Set

​Clear

​Running a pipeline

​Dry-run against inline text

​Run against a file

​Run against a stored document

​Run against a folder

​Programmatic invoke

​invoke: call the API trigger

​rotate-secret: rotate an API trigger’s inbound token

​Single-block testing

​Palette introspection

​palette: list palette IDs

​block: schema for one palette id

​Publishing pipelines

​Publish

​Rollback

​Publications

​Logs

​List runs

​Get one

​Bulk sync

​Cloud ingest

​Recipes

​Bootstrap a minimal pipeline in one script

​CI: sync pipeline from git

​Get the most recent errored run

​Turn off a schedule for maintenance

Command groups

Pipeline CRUD

List / get

Create

Update

Delete

Nodes

List

Add

Update

Remove

Edges

List / connect / update / delete

Schedule

Set

Clear

Running a pipeline

Dry-run against inline text

Run against a file

Run against a stored document

Run against a folder

Programmatic invoke

`invoke`: call the API trigger

`rotate-secret`: rotate an API trigger’s inbound token

Single-block testing

Palette introspection

`palette`: list palette IDs

`block`: schema for one palette id

Publishing pipelines

Publish

Rollback

Publications

Logs

List runs

Get one

Bulk sync

Cloud ingest

Recipes

Bootstrap a minimal pipeline in one script

CI: sync pipeline from git

Get the most recent errored run

Turn off a schedule for maintenance