DocumentationMotionCommons · Developer preview

Module and pipeline developer guide

Python authoring and trusted local execution implemented; managed dispatch planned

Write functions, organize source code, declare contracts and environments, compose workflows and publish exact versions programmatically.

Docs/Build extensions
Ask Claude

Module and pipeline developer guide

Page Markdown

The complete guide, ready to copy or save as a Markdown file.

Download .md ↓

Modules and workflows

A module describes one reusable computation: named input ports, output ports, settings, a runtime profile and an implementation. A pipeline connects exact published module versions. A workflow bundle carries that pipeline and its complete module definitions for local execution.

The same definitions serve the visual editor and Python tools. Code can live inside a module version or in an installed Python package. Saving or publishing registers a definition; running it is a separate operation.

Today, Python JSON workflows run through the trusted local SDK/CLI. Native iRehab capture and the source-mapped OpenCap Mesh workflow use separate local adapters. Managed builds, isolated hosted Python previews and general CPU/GPU graph dispatch remain future work.

Organization and Public module visibility

Modules belong to an organization and can be reused across its projects. The only visibility settings are Organization and Public. Existing modules keep their IDs, versions, source and pipeline references and default to Organization. Pipelines, dashboards, protocols and exercises retain project ownership.

Open Modules and choose the Organization or Public library. Each card names its publishing organization. Open a saved module and use Module ownership and visibility → Visibility → Save visibility. Visibility is independent of immutable published code; drafts stay organization-only even while the module is Public. Only Public published definitions, source and defaults are available anonymously. Patient data and recordings keep their existing access rules.

Organization staff can read the private library. Active researchers and organization admins can author; the creator, origin-study coordinator or organization admin may manage a module while still an organization author. Other organizations can pin a public published version, but cannot edit its owner record. Returning to Organization removes future public resolution and can leave foreign pipelines with an unavailable dependency. Already downloaded source cannot be recalled.

Programmatic authoring keeps the existing caller-scoped catalog actions. Organization module discovery uses GET /api/v1/organizations/{org}/modules?visibility=organization|public and exact module version lookup. POST /api/v1/organizations/{org}/module-visibility takes entry_id, visibility, expected_visibility_revision and a UUID request_key. Anonymous GET /api/v1/public/modules exposes published Public definitions only. Saving or sharing never runs code. The forward ownership migration is applied locally only; repository docs/MODULE_LIBRARY.md defines roles, rollout and limits.

Reusable model resources

Open Models beside Modules in a researcher workspace. Recommended supplies seven starter templates: YOLO26 detection/pose/segmentation, FastSAM segmentation, SAM 3D Body, the existing YOLO11 live detector family and SAM 2.1. Search names, tasks, goals, frameworks and checkpoint metadata. The local live adapter called FastSAM uses SAM 3D Body/MHR; it is different from the FastSAM segmentation model.

Register a recommended copy or a custom model version in your organization. A model records source, checkpoint, optional exact source commit and SHA-256, input/output descriptions, license and runtime requirements. Records have immutable UUIDs and unique organization/slug/version identities. Register another version when weights, code or dependencies change. Missing pins are shown explicitly; registration does not verify installation or execute a model.

Organization and Public are the only visibility settings. Organization is the default and shares metadata across authorized organization projects. Save visibility to Public explicitly to expose model metadata anonymously. Public does not redistribute weight files, accept upstream license terms or grant patient-data access. Never put secrets, signed URLs or patient information in model metadata.

Use GET /api/v1/model-templates, GET/POST /api/v1/organizations/{org}/models and GET /api/v1/organizations/{org}/models/{model_id}. Registration posts definition and a UUID request_key. POST /api/v1/organizations/{org}/model-visibility takes model_id, visibility, expected_revision and request_key. Anonymous GET /api/v1/public/models exposes only explicitly Public metadata. Reuse request keys for lost responses.

A model is an artifact resource; a module is code that uses it. Keep dependencies in reviewed runtime images. Model files now supports immutable download manifests for Hugging Face (exact artifact commit and filename), official HTTPS/GitHub release assets and HTTPS object storage. Each file declares a relative path, exact byte size and SHA-256; include required companion assets. The first file is the primary checkpoint. Attach sources once; use a new model version for corrections. Code and artifact revisions are separate.

An explicitly configured local worker downloads and verifies files before they are ready. The Models detail names the worker and shows Not downloaded, Downloading and verifying, Files ready, Provider access required or failure. Only organization administrators can start preparation. The operator approves organizations and source/redirect hosts; provider tokens remain in worker configuration. Completed files survive retries and cache hits work offline. Ready describes verified files, not inference compatibility. No model is automatically downloaded by registration or page navigation.

GET/POST /api/v1/organizations/{org}/models/{id}/artifacts reads or attaches a huggingbody/model-artifacts@1.0.0 manifest; attachment includes a UUID request_key. GET /preparation reads local status; POST /preparation names the configured worker_id. The trusted operator CLI hb model-files validate|prepare|resolve accepts an exported manifest; resolve returns verified local paths without network downloads. Remote enrollment, automatic pipeline model bindings, weight upload/training and inference compatibility probes remain planned. Existing native checkpoint choices are unchanged. Repository docs/MODEL_PREPARATION.md documents configuration, API, limits and tests.

Find exercises, protocols and dashboard templates

Exercises, Protocols and Dashboard templates use the same searchable table as Modules and Pipelines. Search the authorized loaded page by authored purpose, instructions, equipment, session/exercise names, dashboard panels or chart fields. Matching ignores case, accents and punctuation and requires all words as token prefixes. Format or Panel type filters, Draft/Published and name/newest sorting compose with search. Counts refer to saved versions; the existing catalog truncation notice remains.

Exercise cards show written instructions, equipment and independent video/skeleton-replay references. Protocol cards show saved session order, exercise steps and distinct exact exercise versions. Search uses exact references present on the authorized page; unavailable versions are marked rather than substituted with newer matching names. Dashboard cards show panel/plot/action counts and outlines derived from the actual saved order and dimensions. Outlines do not render patient values.

Exercise and protocol saves keep the committed version open, with compact overview and header actions. Save edits before publishing. Published versions and patient plans retain their exact pinned definitions. Back to the library restores filters and a suitable focus target. A successful write followed by a failed exact read offers Reload saved version; this retries the read without repeating creation.

Dashboard templates retain the separate authoring canvas, Assignments and Sharing views. Back to dashboards restores scoped query, panel/status filters and sorting after saving. This transient router state is scoped to user, organization, study and app; it is not stored in URL search text or localStorage. Project ownership and patient/media access rules are unchanged. Repository docs/CARE_LIBRARY.md records behavior, limits and acceptance tests.

Find and inspect reusable modules

In Modules, search names, categories, goals, descriptions, runtimes, contracts and setting names. Try walking speed, gait timing, ROM or joint mobility. Search ignores case, accents and punctuation and matches all entered words. Category buttons and Goal/Status filters narrow results; sort by name or newest version. Counts refer to saved versions. Search stays within the already-authorized current catalog page, with a warning if that page is truncated.

Module cards show their purpose, category, goal, version/status and input/output/setting counts. Open a card for Overview, Settings, Inputs & outputs and Implementation. Settings presents typed defaults directly; Edit settings schema exposes the detailed schema editor. Inputs & outputs presents readable contracts; Edit port contracts exposes structural changes. Implementation retains exact source links, native signatures and Python/package authoring.

Open code opens GitHub at the registered commit and exact function or class definition line for bundled iRehab, OpenCap Mesh, activity and gait modules. Implementation shows the source path and line. Line numbers come from that commit, so local edits cannot shift the target. Supporting files open at file level. Unknown source mappings open the commit tree; files absent from that revision are labeled local-only.

Save draft keeps the module open at the committed version. Save existing draft changes before publishing. Published versions remain read-only while tabs and source links remain available. Back to modules restores the in-memory filters and focus to the refreshed card; reload or changing workspace resets the search. Save edits before closing to retain them permanently. Add module collections now folds activity/gait import controls.

Category and goal labels are curated presentation metadata for known modules, not new execution parameters or clinical labels. Custom modules are searchable by their own name and description. No external service receives search text, and searching/inspecting makes no catalog writes. Full behavior and validation are documented in docs/PIPELINE_EDITOR.md in the repository.

Find and preview workflows

Pipelines now matches the Modules library: search workflow names, purposes and contained module names/goals, contracts, settings and runtimes. Search ignores case, accents and punctuation and matches all entered words. Workflow type and Draft/Published filters narrow the saved versions; sort by name or newest creation date.

Saved rows show stage names, module/connection counts and version status from the actual graph. Open a row for the existing workflow canvas. Back to workflows preserves search, filters and sorting in memory and restores focus after catalog refresh. Reloading or switching project resets discovery state. If the saved version falls outside the active filters, focus returns to search.

Workflow types describe available source/runtime metadata, not execution readiness. Missing exact module versions stay explicit; the library never substitutes a different version by name. Search is limited to the already-authorized current catalog page, and the existing truncation notice remains. Inspecting or searching performs no catalog writes.

Start from a workflow groups OpenCap Mesh, iRehab realtime and realtime + Biofeedback in compact cards below saved workflows. Explore first, then explicitly save a copy to your project. Publication still registers a definition; it does not run or deploy it.

Run the first workflow

From the MotionCommons repository root, prepare the locked development environment and run the checked-in synthetic example. This uses dimensionless scalar values, with no account or clinical data. Choose a new output path for every run.

The first node multiplies 7 by the default gain 2; the second applies an override of 3. The result contains outputs["second.output"] = 42, per-step outputs and provenance.

sh
uv sync --locked --extra dev
uv run hb block validate examples/workflow-blocks/module.json
uv run hb workflow validate examples/workflow-blocks/workflow.json
uv run hb workflow run examples/workflow-blocks/workflow.json --input examples/workflow-blocks/inputs.json --output result.json --runtime-profile python-local@0.1.0
uv run python -m json.tool result.json

Where the code lives

In the parent Fast-SAM repository, the MotionCommons application is apps/huggingbody. Paths below are relative to that application repository, unless explicitly labeled parent-repository paths.

Keep shared SDK validation and execution adapters in the platform. Keep a research algorithm in its owning module package or repository, wrapped by a small run function. A separate module repository is sufficient; adding a Git submodule is not a publication requirement.

Inline Python belongs to the catalog module definition and travels in exports. Package code remains in its source repository and installed wheel. A module records a GitHub repository and exact 40-character source commit; the current runner neither clones it nor installs it during execution.

text
src/huggingbody/
  product_contracts.py       # ModuleDefinition, PipelineDefinition, ports
  module_extensions.py       # settings, Python execution descriptors
  block_runner.py            # WorkflowBundle, validation, local execution
  block_worker.py            # separate process that calls contributed code
  cli.py                     # hb block / hb workflow commands
  product.py                 # caller-scoped catalog HTTP endpoints
  artifacts.py               # separate v2 file-module contracts
  irehab_realtime*.json       # native realtime stage descriptors
apps/web/app/
  module-extensions.tsx       # implementation/settings editors
  catalog-workbench.tsx       # catalog create/save/publish/revise
  pipeline-editor.tsx         # visual workflow studio
examples/workflow-blocks/     # runnable synthetic module and pipeline
examples/file-analysis/       # v2 arbitrary-file example
contracts/openapi.json        # generated HTTP schema

Structure a reusable module repository

Use this suggested layout for a package that contains one or several related blocks. The layout is a convention; the manifest entry point and installed distribution metadata determine what the runner calls.

Use normal Python packaging so the built wheel records the implementation files. Editable installations whose metadata omits those files are unsupported by the package adapter. Keep large models, recordings and generated results outside Git; record immutable model asset hashes in your scientific contract.

text
my-lab-blocks/
  pyproject.toml
  uv.lock
  src/my_lab_blocks/
    __init__.py
    metrics.py              # def run(inputs, parameters)
  modules/scalar-gain.json   # ModuleDefinition
  pipelines/gain.json       # workflow bundle or graph source
  examples/inputs.json       # synthetic inputs only
  tests/test_metrics.py      # known-answer and invalid-input tests
  README.md
  LICENSE

Naming and scientific conventions

Use snake_case for port names, setting keys and pipeline node IDs: markers, threshold_deg, inverse_kinematics. Current schemas require a lowercase letter followed by lowercase letters, digits or underscores, up to 40 characters. Node IDs and setting keys must be unique in their scope; input and output names are unique within each direction.

Contract identifiers have the form namespace/type@MAJOR.MINOR.PATCH, for example example/scalar@1.0.0. Namespace and type use lowercase letters, digits and hyphens, beginning with a letter. Wires require exact identifiers and matching data shapes. A filename or matching JSON shape alone does not establish scientific compatibility.

Runtime labels use profile-name@MAJOR.MINOR.PATCH, for example python-local@0.1.0. Human-facing names and labels may contain spaces. Distribution names and Python import names may differ: my-lab-blocks can provide my_lab_blocks.metrics:run.

Put units and time semantics in the contract and data, with readable field suffixes where useful: angle_deg, time_s, markers_mm. Declare axes, handedness, reference frame, clock origin, sampling, ordering and missing-value behavior. Never silently convert mm to m, degrees to radians or one joint convention to another.

Keep these identities separate: a catalog entry owns a history; its version UUID identifies an immutable published definition; its number is the displayed version; revision is the optimistic-lock counter of a draft. Contract semver, package version, source Git commit and workflow bundle schema version serve different purposes.

Create an inline module in Python

Save this as create_module.py and run it in the installed MotionCommons environment. It constructs and validates the same ModuleDefinition used by the catalog API, then writes module.json. Inline code is owned by this version, so repository and source_commit are empty or omitted.

The function signature is run(inputs, parameters). Both arguments are dictionaries. Return exactly the declared output keys, using JSON-compatible values. Imports must already exist in the runtime; the runner does not provide Roboflow baseline imports or accept its self-based signature.

python
from pathlib import Path
from huggingbody.product_contracts import ModuleDefinition

module = ModuleDefinition.model_validate({
    "kind": "module",
    "name": "Synthetic scalar gain",
    "description": "Dimensionless synthetic calculation",
    "runtime_profile": "python-local@0.1.0",
    "inputs": [{"name": "input", "contract": "example/scalar@1.0.0", "value_type": "number"}],
    "outputs": [{"name": "output", "contract": "example/scalar@1.0.0", "value_type": "number"}],
    "parameters": [{
        "name": "gain", "label": "Gain", "type": "number",
        "default": 2, "minimum": 0, "maximum": 5,
    }],
    "execution": {
        "kind": "inline_python", "timeout_seconds": 10,
        "code": 'def run(inputs, parameters):\n    return {"output": inputs["input"] * parameters["gain"]}\n',
    },
})
Path("module.json").write_text(module.model_dump_json(indent=2) + "\n")

Inputs outputs and settings

Each port declares name, contract and value_type. Supported shapes are json, number, integer, boolean, string, array and object. Omitted value_type means json for historical definitions. A number rejects booleans and numeric strings. The module must validate detailed contents, such as timestamp monotonicity and coordinate labels.

Settings support number, integer, boolean and string. Every setting has a typed default, with optional description, minimum, maximum and choices. Numeric bounds only apply to numeric settings; choices must match the setting type and include the default. The UI currently authors defaults/bounds; a manifest can additionally declare choices.

Node.parameters holds only its overrides. The runner merges these with module defaults before calling run. Unknown keys, incompatible types, invalid defaults, duplicate names, non-finite values and out-of-range overrides fail validation. Use huggingbody.module_extensions.resolve_parameters for the same rules in your own tooling.

Prefer deterministic calculations without hidden state. Each block invocation starts a fresh process, so module globals do not persist between nodes or runs. Make calibration, model versions and reference trajectories explicit inputs instead of relying on an earlier request.

Compose a pipeline programmatically

Run this after create_module.py, in the same directory. It creates a two-node acyclic graph and a self-contained WorkflowBundle. The synthetic UUIDs are local bundle identities; publishing a catalog pipeline requires replacing them with actual published module-version UUIDs in the same project.

An edge binds one source output to one target input. One output can feed several nodes; each target input accepts one binding. Unconnected inputs become external node.port inputs; unconsumed outputs become the workflow result. Positions are optional editor coordinates and do not affect execution.

python
import json
from pathlib import Path
from huggingbody.product_contracts import ModuleDefinition
from huggingbody.block_runner import WorkflowBundle

module = ModuleDefinition.model_validate_json(Path("module.json").read_bytes())
module_id = "f1000000-0000-4000-8000-000000000001"
bundle = WorkflowBundle.model_validate({
    "schema_version": "huggingbody/workflow-bundle@1.0.0",
    "modules": {module_id: module.model_dump(mode="json")},
    "pipeline": {
        "kind": "pipeline", "name": "Synthetic gain workflow",
        "nodes": [
            {"id": "first", "module_version_id": module_id},
            {"id": "second", "module_version_id": module_id, "parameters": {"gain": 3}},
        ],
        "edges": [{
            "source": "first", "source_port": "output",
            "target": "second", "target_port": "input",
        }],
    },
})
Path("workflow.json").write_text(bundle.model_dump_json(indent=2) + "\n")
Path("inputs.json").write_text(json.dumps({"first.input": 7}) + "\n")

Execute through the Python SDK

Save this as run_workflow.py next to workflow.json and inputs.json. Use execute_block(module, inputs, overrides, runtime_profile, lock_dir=None) for one block, or execute_workflow for a bundle. These functions execute trusted local code; ordinary Pydantic model validation only validates structure.

Use publish_result to create a complete result file atomically and refuse overwrites. The CLI offers the same validation and execution path. Native source descriptors with no Python adapter cannot run through this executor.

python
from pathlib import Path
from huggingbody.block_runner import (
    WorkflowBundle, execute_workflow, publish_result, read_json,
)

bundle = WorkflowBundle.model_validate(read_json(Path("workflow.json")))
result = execute_workflow(
    bundle, read_json(Path("inputs.json")),
    runtime_profile="python-local@0.1.0",
)
assert result["outputs"]["second.output"] == 42
publish_result(Path("result.json"), result)

Validate and execute through the CLI

hb block consumes a module definition; hb workflow consumes a workflow bundle, not a bare pipeline graph. The block input file uses its port names, such as {"input": 7}. The workflow input file uses unconnected node.port names, such as {"first.input": 7}. All external inputs are required.

validate checks contracts, settings and Python signature/syntax without importing contributed packages or executing code. Drafts may contain incomplete Python while being edited; validate before a release. Package presence and runtime compatibility are execution checks.

sh
uv run hb block validate module.json
uv run hb workflow validate workflow.json
uv run hb workflow run workflow.json --input inputs.json --output cli-result.json --runtime-profile python-local@0.1.0
# Run a package-backed workflow after preparing its environment:
# uv run hb workflow run workflow.json --input inputs.json --output package-result.json --runtime-profile my-runtime@1.0.0 --lock-dir locks

Connect an installed Python package

To use reusable source, retain the module ports/settings and set its repository, source_commit and execution descriptor as below. The package function uses the same run(inputs, parameters) signature. Replace every illustrative value with your actual release metadata; the zero commit and lock digest below are placeholders, not a release.

The runner checks the installed distribution version and the lock bytes. The entry-point source file must appear in that distribution’s installed file list. A GitHub function URL by itself is not executable configuration. No loader named load_blocks or Roboflow plugin dependency is required by MotionCommons.

json
{
  "repository": "https://github.com/example-lab/my-lab-blocks",
  "source_commit": "0000000000000000000000000000000000000000",
  "execution": {
    "kind": "python_package",
    "distribution": "my-lab-blocks",
    "version": "1.0.0",
    "entrypoint": "my_lab_blocks.metrics:run",
    "lock_sha256": "0000000000000000000000000000000000000000000000000000000000000000",
    "timeout_seconds": 10
  }
}

Environments and dependencies

Prepare a compatible environment before running a workflow. Keep pyproject.toml and its dependency lock with the package; install the wheel and dependencies during environment setup. Do not pip-install inside a function, patient job or API request. The runtime profile label is a declaration, not an automatic environment switch.

The current JSON runner uses the Python interpreter that launches hb. Every node must declare the same profile passed through --runtime-profile. It does not select a different virtualenv per node, attest the installed dependency set or download a runtime image.

For package blocks, SHA-256 the exact lock bytes and put a copy at locks/<digest>.lock. Pass --lock-dir locks. This proves which lock file was supplied; it does not prove every installed dependency matches that lock or that the installed wheel came from source_commit. Record the build and validate the prepared environment separately.

A pipeline may logically contain modules requiring different environments. The existing native iRehab adapter already coordinates specialized workers. Arbitrary multi-environment Python graph dispatch is future work; keep compatible Python stages together or write an explicit native/service adapter with a serializable contract.

python
from hashlib import sha256
from pathlib import Path
import subprocess

source_commit = subprocess.check_output(["git", "rev-parse", "HEAD"], text=True).strip()
lock_bytes = Path("uv.lock").read_bytes()
digest = sha256(lock_bytes).hexdigest()
Path("locks").mkdir(exist_ok=True)
Path("locks", digest + ".lock").write_bytes(lock_bytes)
# Set source_commit and execution.lock_sha256 in your module manifest.
# Install the tested wheel/dependencies in the chosen environment before hb run.

Register and publish through the API

Use a normal signed-in user access token and the organization/study UUIDs from GET /api/v1/workspace. The example API origin is your configured local API. Set X-HB-App-Space to irehab for iRehab, study for the study catalog, or the exact app version key for a custom app. Modules and pipelines must belong to the same owning project.

The example below creates a module draft, loads its server-owned entry ID/revision, publishes that exact version and reads it back. It performs real catalog writes when run. Treat HB_ACCESS_TOKEN as a secret; never put it in a manifest, Git file, notebook output or command URL. No service-role credential is needed.

Persist the operation body, including request_key, before sending it if you need restart-safe retries. Reuse the identical body/key after a lost response. A different logical operation gets a new UUID. Save and publish use expected_revision; a stale revision produces a conflict rather than overwriting another author.

python
import json
import os
from pathlib import Path
from urllib.request import Request, urlopen
from uuid import uuid4

origin = os.environ.get("HB_API_ORIGIN", "http://127.0.0.1:8000").rstrip("/")
org = os.environ["HB_ORGANIZATION_ID"]
study = os.environ["HB_STUDY_ID"]
base = f"{origin}/api/v1/organizations/{org}/studies/{study}"
headers = {
    "Authorization": "Bearer " + os.environ["HB_ACCESS_TOKEN"],
    "X-HB-App-Space": os.environ.get("HB_APP_SPACE", "irehab"),
    "Content-Type": "application/json",
}

def request(method, path, payload=None):
    data = None if payload is None else json.dumps(payload).encode()
    with urlopen(Request(base + path, data=data, headers=headers, method=method), timeout=30) as response:
        return json.load(response)

create_body = {
    "operation": "catalog",
    "body": {
        "action": "create", "request_key": str(uuid4()),
        "definition": json.loads(Path("module.json").read_text()),
    },
}
created = request("POST", "/actions", create_body)
draft = request("GET", "/catalog/" + created["id"])
publish_body = {
    "operation": "catalog",
    "body": {
        "action": "publish", "request_key": str(uuid4()),
        "entry_id": draft["entry_id"], "version_id": draft["id"],
        "expected_revision": draft["revision"],
    },
}
request("POST", "/actions", publish_body)
published = request("GET", "/catalog/" + draft["id"])
assert published["state"] == "published"
# published["id"] is the module_version_id for catalog pipeline nodes.

Register a pipeline with pinned modules

Continue with the request helper and published module from the preceding section. Replace local bundle UUIDs with published catalog IDs before registering the pipeline. Publishing the pipeline does not start a run.

For an existing draft, action save supplies entry_id, version_id, expected_revision and the new definition. For published code or graphs, action revise creates a new draft version; GET its returned ID before editing. Historical versions remain immutable. Catalog list responses are bounded, so use exact GET /catalog/{version_id} lookups for dependencies outside the current page.

To reconstruct a runnable bundle from a saved catalog pipeline, fetch each unique module_version_id in its nodes using the same scope/header and build modules[id] = returned_version["definition"]. Keep the catalog graph in pipeline. Do not substitute the newest module version.

python
from huggingbody.product_contracts import PipelineDefinition

local_graph = json.loads(Path("workflow.json").read_text())["pipeline"]
for node in local_graph["nodes"]:
    node["module_version_id"] = published["id"]
pipeline = PipelineDefinition.model_validate(local_graph)
create_pipeline_body = {
    "operation": "catalog",
    "body": {
        "action": "create", "request_key": str(uuid4()),
        "definition": pipeline.model_dump(mode="json"),
    },
}
created_pipeline = request("POST", "/actions", create_pipeline_body)
pipeline_draft = request("GET", "/catalog/" + created_pipeline["id"])
# Publish using pipeline_draft entry_id/id/revision and a NEW request_key,
# following the publish_body pattern above.

Choose JSON blocks or file modules

Use JSON blocks for finite, bounded values and small descriptors. They do not automatically stage referenced files, NumPy arrays, DataFrames, OpenSim objects, credentials or remote assets. Arrays must be ordinary JSON arrays; reject ambiguous units/content in the function.

Use the existing v2 file SDK for TRC, MOT, OSIM, CSV, video, binary arrays and other original bytes. Its separate extension.json uses manifest_version "2" and a run(inputs, parameters, output_dir) function returning declared artifact descriptors. The original v1 inline knee-summary template remains supported too.

A catalog ModuleDefinition is not a v1/v2 extension.json, and a JSON WorkflowBundle is not an artifact-run manifest. General automatic file staging between saved graph nodes is not implemented. Use the documented file runner and explicit composition for files; publish compatible bounded datasets for dashboard plotting.

sh
uv run hb init my-file-analysis --template files
uv run hb validate my-file-analysis
uv run python -m pytest my-file-analysis/test_main.py -q
uv run hb run my-file-analysis --input my-file-analysis/inputs.json --output new-file-result

How the native iRehab pipeline fits

The native chain is iPhone stream → FastSAM markers → OpenSim IK → visualization. Its optional biofeedback branch consumes markers plus solved IK state, then sends feedback to visualization. The 0.1.0 base and 0.2.0 feedback descriptors live in src/huggingbody/irehab_realtime.json and src/huggingbody/irehab_realtime_feedback.json.

Parent-repository implementations include core/hmr_opensim/pipeline/iphone_live.py, core/hmr_opensim/scripts/stream_fastsam_markers_iphone_live.py, core/hmr_opensim/pipeline/ik_live_subprocess.py and core/hmr_opensim/pipeline/reach_feedback.py. The shared browser viewer is in apps/huggingbody/packages/biomechanics. The authoritative native walkthrough is core/hmr_opensim/README.md.

The native entry point is ./run_irehab.sh from the parent repository. Reach biofeedback requires explicit REACH_FEEDBACK=1 plus the RT-stroke model, calibration and reference assets. Markers use millimetres and timestamps use seconds at the documented native boundary. Keep native OpenSim model/state handles in the owning worker; they are not portable JSON.

Logical blocks in the canvas do not require a separate process or environment per stage. The current adapter owns the persistent realtime workers and lifecycle. Editing, exporting or publishing its diagram does not rewire an active capture, and hb workflow run rejects its source-only native descriptors.

OpenCap Mesh: reusable native workflow

Open Pipelines → OpenCap Mesh · A to Z for a ten-phase recorded multi-camera workflow: trial intake, shared detector jobs, calibrated reconstruction, OpenSim scaling/IK, optional overlays, contact sheets and a 3D export. Modules → Browse templates and examples contains 35 exact-source native adapters, with upstream signatures and Open code links. Source remains pinned to opencap_mesh commit 808173f and nested HMR-OpenSim 886bca6.

Reconstruction preserves the upstream joint calibration/synchronization/cross-view re-solve loop, MHR fit, configured temporal filter and QC. Its internal functions are reusable through call_native in a prepared local Python environment. Native arguments include arrays, Paths and contexts; these ports are not portable JSON or automatically staged file jobs.

Export a workflow bundle and use python -m huggingbody.opencap_mesh plan with an explicit native repository, session/trial, output root, defaults file and native/OpenSim interpreters. Replace plan with run for trusted local execution. Remove optional output branches, or remove OpenSim and its dependent outputs for a mesh-only flow. Unsupported rewiring is rejected before processing.

The runner extracts pinned source into a temporary workspace with a private copy of the supplied defaults, so both native orchestration and the OpenSim child receive the same settings. Original recordings and prepared environments/models stay in place. No packages, weights, database access or remote machines are provisioned. QC failures remain failures; run records report observed-local-unlocked execution. Real-video/GPU/OpenSim accuracy has not been established by the synthetic adapter checks.

Docker images and workflow releases

Build images around compatible dependency environments and deployment units. Reuse a maintained base for scientific Python, OpenSim or a tested GPU/CUDA stack, then build a release image containing the module packages and locked dependencies it needs. Several compatible modules can share one image.

Keep workflow definitions separate from runtime images and large model/calibration assets. A release pins the workflow/module versions, exact image digests, settings and immutable input/model references. Changing graph connections or settings usually needs a new workflow version. Changing bundled package code, dependencies or native libraries needs a new image build. A dedicated all-in-one image is optional for a particular deployment.

Roboflow’s documented self-hosted model separates its Inference Server image, workflow JSON and cached model weights. A workflow update normally updates the definition rather than rebuilding the server image. MotionCommons follows this separation; its automatic image-build and machine-dispatch control plane is not enabled by the current local runner.

Select the machine for a flow

The intended admin experience is to register organization-owned machines, inspect their heartbeat, OS/architecture, CPU/GPU capabilities and allowed runtimes, then choose a compatible machine for a published flow. An offline or incompatible machine must be identified before dispatch; the platform must not silently move a patient run to another machine.

Use a small outbound agent on each enrolled machine. The admin sends the control plane a deployment/run request; that machine’s agent claims its assigned work and pulls the exact approved image digest from a registry. The user experiences this as sending the flow to the selected machine, without opening SSH or the Docker daemon to the web application.

The build service pushes an image to the registry. The machine pulls it over its own authorized connection, reusing cached layers. Private registry authentication belongs to machine configuration or a scoped credential provider, never to a flow manifest. Pin the platform/architecture as well as the digest, and retain the previous working release for rollback.

Machine enrollment, approved release activation and starting a run are separate administrative actions. Record the requesting user, target machine, release digest and resolved settings. Start, stop, heartbeat, cancellation and lease expiry must have explicit states. Reassigning an expired run creates a new attempt and must prevent a late old attempt from publishing.

This is the deployment contract to implement next. The current editor exports local workflows; it does not yet register remote machines, push images, enroll an agent or start arbitrary remote containers. The existing fixed inspection worker and native iRehab adapter retain their separate execution paths.

  1. Build and test the module package/image in a prepared build environment; publish the immutable image to an approved registry.
  2. Register and explicitly enroll the target machine with organization-scoped agent identity and declared capabilities.
  3. Choose the machine, published flow release and declared settings/input versions; validate compatibility and current permissions.
  4. Queue the immutable run specification. The selected machine’s outbound agent claims its own job, pulls/verifies the approved digest and stages authorized inputs.
  5. Run the appropriate container/native adapter, report progress and validate outputs before accepting a completed result. Retain audit/attempt history.

How workers obtain data and settings

Yes, use a worker/agent to obtain the assigned run and its inputs. Put data selection and algorithm settings in the flow; keep database access and credentials in the trusted coordinator. Contributed functions should receive named validated inputs and resolved settings, rather than querying arbitrary application tables.

Store domain metadata, permissions, input-version references and run/lease state in PostgreSQL. Store large recordings, model bundles and result files in authorized object storage. The trusted coordinator resolves an approved data binding under the requesting user’s study/project/patient access, freezes its input versions and grants the assigned attempt access only to those objects.

Flow configuration can name an authorized dataset/capture version, declared field mapping, analysis time interval or algorithm threshold. A reusable flow can expose these as external input ports and parameters. Bind actual study/session data at run creation; keep patient identifiers and private asset references out of public/shared flow templates.

Do not put a database URL, password, service-role key, bearer token, registry credential or arbitrary SQL in the graph. Do not give custom module containers access to the application database or Docker socket. Keep the machine agent outside the module container and keep its credentials out of the child process.

Resolve dynamic selectors such as latest authorized capture once when creating the run, then persist the exact IDs and hashes. Retrying a run uses the frozen input/configuration; new data creates a new run. Re-check access at claim and result acceptance, and apply lease/attempt identity to every result commit.

The existing fixed video-inspection coordinator already uses a leased job, stages bounded authorized originals, checks their sizes/hashes and commits a validated result. That is the pattern to extend. General saved-workflow data-source bindings and remote credential delivery remain to be implemented.

text
Flow template → declared input ports and settings
Run request → authorized input versions + settings + machine + release
Coordinator → permission checks + immutable snapshot + leased job
Machine agent → claim assigned job + pull pinned image + stage inputs
Module runtime → calculate with inputs and settings
Coordinator → validate outputs + accept result for that attempt

Realtime capture and persistent workers

The realtime iRehab flow needs a persistent adapter with explicit start/stop and reconnect behavior. A control-plane job starts the session/adapter; individual frames stay on the live stream path rather than becoming database jobs.

Keep iPhone capture native, FastSAM in its compatible GPU environment, OpenSim IK and reach feedback together where they share native state, and visualization in the browser. The deployment may coordinate several environments. A logical node does not require its own Docker container, and native OpenSim handles cannot be serialized between machines.

Batch Python workflows and persistent realtime sessions need distinct adapters, resource/lifetime limits and acceptance checks. A machine selector must advertise which adapter the machine supports. Until those adapters are explicitly connected, publishing the native diagram does not change or relocate a running capture.

Versioning provenance and access

Publish a new module version when code, dependencies or contract semantics change. Publish a new pipeline version to adopt it. An explicit conversion module should handle different units or contract versions. Tests should establish numerical tolerances; a matching dependency version does not guarantee identical GPU or solver results.

The local runner records definition/input/output hashes, resolved settings, selected runtime label and observed Python version. Inline runs include a code hash; package runs include distribution/version, lock hash and entry-point source hash. Runtime identity is observed-local-unlocked, not a signed image or verified source-build attestation.

Local results are independent files. The JSON runner does not automatically upload them into private study storage or register scientific run history. Existing authorized artifact/dataset/capture APIs own those workflows. Sharing a code definition grants no patient-data access.

The catalog keeps the caller’s verified identity, organization/study scope and project ownership. Publishing requires ordinary staff/author permissions; patients do not gain code-catalog access through a dashboard assignment. Contributed code receives no inherited database/cloud credentials from the local runner.

Limits and common errors

A JSON workflow supports up to 30 nodes, 60 edges, 12 input/output ports per module and 30 scalar settings. Custom code is bounded to 20,000 characters; string settings to 2,000 characters. Input JSON and each step output are bounded to 1,000,000 bytes. Declared block wall time is 1–60 seconds; a workflow has a 120-second deadline.

Trusted local workers support POSIX macOS/Linux, with CPU/file/log limits. They retain the OS user’s filesystem/network permissions and are not a sandbox. Use only trusted code in a prepared environment. Hosted isolation, arbitrary multi-environment dispatch and browser Run are separate integrations.

Runtime profile mismatch: prepare a compatible environment and use the module’s declared profile. Missing adapter: use the native adapter or add an actual Python implementation. Package failure: check the exact installed version, wheel file ownership and lock hash. Missing inputs: supply every external node.port key.

Invalid setting or incompatible wire: fix the schema, override, shape or contract rather than coercing data. Existing result path: choose a new destination. Draft conflict: reload the exact version and reconcile changes. Unauthorized project/study: use an account with access and the correct app-space header; never substitute an administrative token.

Test and release a module

  1. Write the function or adapter, manifest, dependency lock, README and synthetic fixtures in the owning repository.
  2. Test known answers, invalid inputs, units, missing data, parameter bounds and failure behavior. Keep patient data and secrets out of fixtures and Git.
  3. Validate the exact module and a small composed workflow; run them in the prepared target environment and inspect outputs and provenance.
  4. Record the source commit and package/lock/runtime identities. Register and publish module versions using the ordinary catalog API or editor.
  5. Create and publish a pipeline pinned to those module-version IDs. Export its bundle and retain the exact input/configuration for reanalysis.

OpenCap activity segmentation library

Nine native modules wrap walking, running, sit-to-stand, jump, TUG, toe stand, curls, arm ROM and Brooke segmentation from rdmagruder/opencap-processing at commit 67b576a4ed875727de5e854057487ccfbb3fa280. These segment a chosen task; they do not classify unknown activity. Algorithms remain in the pinned upstream checkout. Open code links to exact implementation function lines at the pinned commit.

In Modules, choose Create 9 activity segmentation modules to create editable caller-owned drafts. Review settings and publish selected versions before composing their exact IDs. Source signatures/defaults and native event mappings live in src/huggingbody/activity_segmentation.json; the adapter, worker and typed output models sit beside it. No submodule is required, and no dependencies are installed during a run.

The runtime profile opencap-activity-native@0.1.0 needs a separately prepared OpenSim, NumPy, SciPy, Pandas, Matplotlib and PyYAML environment. The local worker isolates upstream import-time OpenCap authentication and unused gait-VAE imports; no account or Torch is needed for segmentation. The CLI requires explicit source/interpreter/session paths. This is trusted local execution; managed images and generic graph dispatch are separate work.

Input contract opencap-processing/analysis-session@1.0.0 requires sessionMetadata.yaml, the matching scaled OpenSim model, trial MOT and marker TRC with native names and matching clocks. OpenCap Mesh output needs an explicit verified conversion; matching filenames alone are insufficient. Output huggingbody/activity-segmentation@1.0.0 retains raw native events, missing-flight sentinels, parameters and source/runtime provenance, plus chronological intervals with inclusive indices into the trimmed native series. Original time origins and interpolated event times are preserved.

Synthetic checks load all nine native classes offline, test all nine schema/binding/event shapes, a known toe-stand signal and a complete generated OpenSim curl session with settings overrides. Real-recording accuracy for each task and clinical validity remain unverified. Full setup, input layout, commands and limitations are in examples/activity-segmentation/README.md in the application repository.

sh
python -m huggingbody.activity_segmentation manifest
python -m huggingbody.activity_segmentation run --activity curls \
  --source-root /path/to/opencap-processing \
  --native-python /path/to/prepared/python \
  --session /path/to/session --trial trial_name \
  --output /path/to/new-segmentation.json

Reusable gait metric modules

Create 16 gait metric modules from Modules: stride length, bilateral step length, step length symmetry, gait speed, cadence, treadmill speed, step width, stance time, swing time, single support, double support, midswing dorsiflexion, midswing ankle-height difference, joint peak angle, joint range of motion and a bundled report of the 13 standard measures. Definitions and source signatures live in src/huggingbody/gait_metrics.json; the original methods remain in the same pinned ActivityAnalyses checkout.

Each module accepts the original analysis session and walking/running segmentation. Wire the segmenter output to its segmentation input and supply the same session. Output contract huggingbody/gait-metrics@1.0.0 retains native aggregates, per-cycle values/shapes/order, units, segmentation, input hashes and source/runtime provenance. Supplied events are reused rather than resegmented. The CLI can compute native segmentation when --segmentation is omitted. Saving a graph does not enable managed dispatch.

Single/double support are percentages of stride duration, not seconds. Stance/swing are seconds. Symmetry is the native R/L step-length ratio times 100, where equal steps yield 100. Gait speed uses native 3D COM displacement plus treadmill correction. Midswing ankle-height difference is not minimum toe clearance. Joint peak/ROM take a coordinate base name and stride/stance/swing phase, retaining the original half-open slice endpoints.

Choose walking or running, side, cycle limit, filter/trimming and gait_style (auto, overground or treadmill). Auto retains the original foot-velocity heuristic; explicit overground disables treadmill correction. Native walking/running load LaiUhlrich2022_scaled.osim. Segmentation settings, clocks, source and model/MOT/TRC hashes must match. Regenerate older segmentation reports without input hashes. Incompatible recordings, missing prerequisites and non-finite results fail explicitly.

Runtime profile opencap-gait-native@0.1.0 uses the same prepared OpenSim scientific interpreter and offline source loader. No OpenCap credentials, Torch/VAE installation or model download is requested. Learned DMU scores, other activity metrics and clinical classification need separate implementations. Full commands, units, setup and compatibility are in examples/gait-metrics/README.md. Synthetic OpenSim checks cover all 13 standard measures, peak/ROM, event reuse and the native walking segmenter; real-recording and running-trial accuracy remain unverified.

sh
python -m huggingbody.gait_metrics run --module report \
  --source-root /path/to/opencap-processing \
  --native-python /path/to/prepared/python \
  --session /path/to/session --trial walking_trial \
  --segmentation /path/to/segments.json \
  --output /path/to/new-gait-metrics.json

Relation to Roboflow

Roboflow documents inline dynamic Python definitions and reusable installed plugin blocks. MotionCommons follows that authoring distinction with its own SDK and schemas. Its function signature, workflow envelope and runtime behavior differ; Roboflow workflow JSON is not accepted by this runner.

Roboflow’s documented serverless container execution and block-state behavior should not be assumed here. MotionCommons currently starts a fresh local process per Python block invocation and retains native persistent state only inside its separate realtime adapter.

Accessibility

Make every MotionCommons page comfortable for you. Settings apply on this browser.

Text size

Your training, at your pace.

Reading & movement
Audio support

Spoken audio is unavailable in this browser. All instructions remain available as text.

Read this page speaks up to 1,800 characters of visible page text. Embedded dashboards have their own Read dashboard button.