ThearchyHelios
7794794278
- 题目（cases.py）：contribution 贡献恒为变化拆解（可加、比率、乘积）、另写 top_n（期望点名主要来源的前几段，缺省 1）；verdict 判定是波动基线、趋势判断、差异检验、可比性检查。参数同分析工具（不写输入、总数、下钻，不写阈值），文字参数可以带 {{ schedule.* }}；来源、方向、判定不许人填（HUMAN_VALUE_KEYS 加 segment、subjects、direction、verdict 等）；口径题、期望反问的题配不了
- 算期望（gold.py、judgements.py）：遮罩列、目录事实、现查表的日期范围和分析工具同一套取法；阈值用这次评测的分析设置快照（Context.analysis），试算时现读设置；期望 SQL 只返回两期时可以不写基期、本期；算不了的这一项照实记（评分时不能评分），参数和结果对不上的整题没有可用的值；截断的快照照旧拒收。作答前后两次期望、和上一次比的「数据变了」都不比运行当天、现查时刻、输入快照的工件（stable_sets）
- 评分（score.py、judgements.py，SCORER_VERSION 3）：判定码归到同一套判定类再比；答案引用了（判定语、具名的数、结果表的格、句末依据）判定对得上的分析结论是对且可追溯，引用它的那句却写反了不算；没引用但正文按说法表（engine/analysis_claims 同一个 scan、同一套绑定规则）写对了是对但没引用；判定、来源、方向对不上或写成相反、根本没做也没说是错。答案的分析结论从作答运行的台账读、引用从报告文档读，有判断词的句子一并读出；逐题 E1 引用分析结论时按它的输入查询比（下钻顺着上一层找）；判定语不再当成答案里的数；老结果的明细里没有这两项按空读
- 执行（runner.py、answer.py）：有分析期望的题集开评时拍一份分析设置，算期望、各臂作答的运行（Run.options 的 analysis）都按它，记进指纹（compare 的「分析设置」一项），重试沿用；别的题集的指纹、作答运行的选项不变。助手提示词的指纹加上分析方法（guide 的 id 和全文）、写作者的分析规则和说法表版本
- 总表（stats.py）：各期望的结论另按期望的种类各一份（outcomes_by_kind）
English | Simplified Chinese
AgentLab
A visual agent-orchestration workbench: define workflows on a canvas, watch execution node by node at run time, and put verifiable constraints on what comes out. Templates fix the skeleton, and agents are free only at the joints they are authorized for; the system fills the numbers in a report from snapshots, and every number opens to its provenance. The user interface is currently available in Simplified Chinese only.
The Research Assistant template running on Mock: finished nodes get a check mark, and running nodes show elapsed time and streaming output. The track along the bottom is a live timeline; the right column shows the run steps, driven by the same event stream.
Why use it
- Numbers you can check: all arithmetic happens in SQL and in caliber cards (controlled metric sets), and the narrative layer has no right to create numbers. The report-writer node writes only citation markers, the system fills in the numbers from snapshots, and every number opens all the way to the cell in the query result it came from.
- Constrained orchestration: the engine is compiled, so at run time the model can't change the graph structure; its freedom is confined to the inside of agent nodes and to branch decisions. Fully dynamic multi-agent collaboration stays in the exploration layer.
- Pinned, reproducible versions: formal runs accept only published immutable versions (pinned by graph hash). The complete evidence of every execution goes into a content-addressed artifact store and is sealed, and one click verifies that nothing was altered.
- Human control: approvals, joint sign-off, time limits, and Reject exits; nothing is ever approved automatically, and a process restart loses no state.
- Honest issuance: the issuance contract decides between three tiers, formal / degraded / withheld; when data is missing, the output is downgraded rather than forced through.
Main features
Orchestration and runs
- Canvas: 18 node types, drag and connect; live node highlighting, streaming output, and a timeline track at run time → Workflows
- Copilot: generate or rewrite a whole graph from one sentence; it looks up schemas on demand and checks its work before delivering → Copilot
- Human-in-the-loop: approvals, joint sign-off, time limits, Reject exits → Human nodes
- Agent guardrails: repeated calls, spending, and context each have a guard; the step count is only a backstop → Guardrails
- Runs and resuming: failed runs continue from the checkpoint; per-node duration, token usage, and cost → Runs
- Publishing and evidence gates: published and governed levels, gates G1–G5, pre-publish check and fixes → Publishing and versions
- Tools and the capability registry: built-in tools, custom tools, MCP, approval levels, registered endpoints → Tools
- Knowledge base, memory, and skills: hybrid retrieval, memory across runs, reusable methodology → Knowledge base
- Multi-model access: Anthropic, OpenAI, OpenAI-compatible services, and Mock → Model access
- Code sandbox: two isolation tiers, the OS sandbox (Seatbelt / bubblewrap) or microVM → Code sandbox
Data
- Ask Data: ask in natural language, and the system builds a graph, runs it, reviews it, and gives a conclusion → Ask Data
- Data sources and spreadsheet import: read-only access to business databases, Excel / CSV turned into tables you can query with SQL, cross-database merges → Data sources
- Data catalog: grain, business keys, valid-record conditions, code values, relationships; drafting, review, profiling, and a change log → Data catalog
- SQL checks: the catalog catches seven kinds of queries that compute wrong numbers → SQL checks
Evidence and issuance
- Clickable evidence: numbers, table and field names, and verbatim knowledge-base quotes all open to their provenance → Evidence and Issuance
- Three-tier issuance: caliber cards plus issuance contracts, checked by value or by citation → Issuance contracts
- Sealing and exports: one-click seal verification; self-contained report exports and verifiable reports → Exports
- Review suite: double-blind recomputation, adversarial review, approval inquiry, the what-if sandbox, run comparison → Review suite
Quality
- Regression evaluation: test suites, background evaluations, accuracy with intervals, comparisons and A/B, scheduled evaluations → Regression evaluation
- Caliber ledger: each number in an answer gets a caliber label, with issuance by caliber, reconciliation, and recall → Caliber ledger
Apps and integration
- Apps: entry forms, views, tasks, and actions derived from a published version → Apps
- Writing to external systems: confirmation cards, idempotency keys, no automatic resend when the result is unknown → Write to external systems
- External integration: inbound hooks, access keys, share links, embedding in portals → External integration
- Schedules, event triggers, and notifications → Schedules and notifications
Operations and security
- Identity and permissions: local accounts or OIDC, three roles plus app grants, masking boundaries → Identity, Permissions, and Security
- PostgreSQL and multiple processes: import, connection pools, PgBouncer, leases and takeover → PostgreSQL and Scaling
Quick start
You need conda (miniforge is enough), Node 20+, and pnpm; Docker isn't needed.
conda env create -f environment.yml # first time: create the agentlab environment and install dependencies
./scripts/dev.sh # after that: one command starts the frontend and backend
Open http://localhost:5273 . You can try everything without an API key: the built-in Mock provider runs the whole orchestration path end to end; to connect a real model, see Model access. The 9 built-in templates run as they are.
Login is off by default: anyone on the same network who can open the page can use every feature. Turn on login before multiple people use it; see Login modes. For more startup options, see Development.
Deployment
- Without Docker:
./scripts/prod.shbuilds the frontend and has the backend serve the pages too; one process by default, at http://127.0.0.1:8000 . - Offline environments: build a Docker offline bundle with
./scripts/package-docker.sh, copy it to a machine that has only Docker Engine, and run./deploy.sh.
See Deployment and deploy/README.md (written in Chinese). The app database defaults to SQLite with a single process; for multiple processes, switch to PostgreSQL (see PostgreSQL and Scaling). Read the Upgrade Notes before upgrading.
Documentation
| Document | What it covers |
|---|---|
| Using and building | |
| Ask Data and Copilot | What happens after you ask, output review, record retention, how Copilot builds graphs and looks things up |
| Workflows | The canvas, node types, passing data, human nodes and rejection, agent guardrails, subworkflows, publishing and gates, runs and resuming |
| Evidence and Issuance | Caliber cards, issuance contracts and tiers, citation markers, claim sentences, no data in the queried range, sealing and verification, report exports and verifiable reports, the review suite |
| Data Sources and Spreadsheet Import | Connecting databases, read-only guarantees, uploading spreadsheets and recipes, accumulating by period, cross-database merges, masked columns |
| Data Catalog | What the catalog records, drafting and review, export and import, data profiling, SQL checks, introspection scope for large databases, schema and value changes |
| Tools, MCP, and the Capability Registry | Kinds of tools, approval levels, capability governance, limits on write capabilities, registered endpoints, network rules |
| Apps | Creating apps, views and stamps, processes and tasks, writing to external systems, form submissions, personal apps, external integration |
| Evaluation and the Caliber Ledger | Test suites and evaluation results, caliber labels per number, issuance by caliber, reconciliation, distilling, recall |
| Schedules, Event Triggers, and Notifications | Schedules, event triggers, notification channels and email, subscribable events |
| Deployment and operations | |
| Deployment | prod.sh, the Docker offline bundle, the data directory, backups, upgrades |
| Configuration | Model access, the public URL, environment variables, UI settings, the sandbox backend and microVM |
| PostgreSQL and Scaling | Switching databases, importing, connection pools, PgBouncer, leases and takeover across processes, multiple machines, upgrades |
| Identity, Permissions, and Security | Login modes, roles and the permission table, masking boundaries, audit, the code sandbox, security boundaries |
| Upgrade Notes | How each update affects users and operators, and what to do when upgrading |
| Development | |
| Architecture and Design Decisions | The overall architecture, constrained dynamic orchestration, and the reasons behind key design decisions |
| Development | Development environment, directory layout, dependencies and the lock file, tests, frontend checks, documentation checks |
| FAQ | What to do about common errors and symptoms |
Development and testing
Before running backend tests, remove model keys from the environment; otherwise the tests might really call the API:
cd backend && env -u ANTHROPIC_API_KEY -u ANTHROPIC_BASE_URL -u ANTHROPIC_AUTH_TOKEN -u OPENAI_API_KEY \
python -m pytest -n auto
cd frontend && npx tsc -b
node scripts/check-all.mjs # frontend checks; start ./scripts/dev.sh first
For tests on PostgreSQL, the 43 frontend checks, and the documentation checks, see Development.
License
This project is source-available and is licensed under the PolyForm Noncommercial License 1.0.0.
Noncommercial Use
You may use, study, modify, experiment with, and develop upon this software for purposes permitted under the PolyForm Noncommercial License 1.0.0.
Typical permitted noncommercial uses include:
- personal research;
- learning and education;
- experimentation;
- evaluation and testing;
- hobby projects;
- noncommercial software development.
See the LICENSE file for the applicable license terms.
Commercial Use
Commercial use is not permitted without prior written authorization from Yilun JIANG.
If you intend to use this software in a commercial product, commercial service, SaaS offering, customer project, commercial deployment, or other commercial context, you must obtain a separate commercial license before doing so.
For commercial licensing, see COMMERCIAL-LICENSE.md or contact:
Yilun JIANG me@thearchyhelios.com
Copyright
Copyright (c) 2026 Yilun JIANG.
All rights reserved except as expressly granted under the applicable license.