全栈安全加固、性能提速与架构演进:连接池与沙箱止血、双路混合检索、多版本归档及前端分包治理

- 第一批(安全加固与生命周期止血):隔离测试宿主机密钥与 .env 泄漏;修复 SQLite 取消查询挂死、:memory: 连接池报错与 EngineCache 世代竞态;补齐运行与评测删除时的 Checkpointer 线程、Artifact 磁盘文件与无主附件回收;加固三套沙箱路径越界、进程组清理与 Seatbelt 数据目录隔离,封堵 SSRF IPv4-mapped IPv6 元数据绕过与响应体 OOM;修复推理模型多轮 reasoning_content、结构化输出 Token 漏计与缓存折扣计费
- 第二批(数据库/集群/问数提速与首屏分包):补齐核心表索引与 SQLite WAL 调优,列表接口按需 defer 大列,收敛启动扫描并打通跨进程 slot 槽位唤醒;优化跨源全局选表、哈希桶数字绑定、流式 CSV 导入及多方言分析算子;剥离首屏静态引入的 @xyflow/react 与全局样式(首屏主包降至 461.59 kB),补充流式 rAF 攒批、Store 精细订阅与表单无障碍绑定
- 第三批(架构演进与工程自动化闭环):记录库超挂接阈值历史版本自动增量归档至单库多表(解除 SQLite 10 库 ATTACH 硬上限);重构知识库为词面 BM25 与全集合归一化向量矩阵真·双路独立召回并支持游标分批重建;升级多方言感知 SQL 模板词法分析(E'...' / $$...$$ / q'[...]' / # / 嵌套注释)与 Decimal / bool / NaN 参数校验,消除 ledger 层对 api 路由层的反向依赖;拆分 3101 行 CaliberDrafts 页面为 7 个领域子模块,消除检查脚本本机路径硬编码并新增 GitHub Actions CI 门禁与三批回归测试套件
This commit is contained in:
ThearchyHelios
2026-11-13 14:51:36 +08:00
parent 09949fc8ac
commit eddeded2a4
106 changed files with 7178 additions and 3913 deletions
+8 -84
View File
@@ -1387,11 +1387,9 @@ async def _draft_validation_summary(session: AsyncSession, draft: CaliberDraft,
def _graph_entry_fields(graph: dict[str, Any]) -> list[dict[str, Any]]:
"""图里入口节点(input)的字段;草稿页和试跑用它,画布改过的入口字段不会被 payload 的旧快照盖住。"""
from app.apps.tree import node_map
from app.ledger.fieldbaseline import graph_entry_fields
start = next((node for node in node_map(graph).values() if node.get("type") == "input"), None)
return [dict(field) for field in ((start or {}).get("config") or {}).get("fields") or []
if isinstance(field, dict)]
return graph_entry_fields(graph)
def _allowed_tables(source: DataSource | None) -> set[str]:
@@ -1503,54 +1501,19 @@ async def _draft_graph_payload(session: AsyncSession, workflow: Workflow, versio
card_id 是这次要读的那张口径卡(写入时由 target.card_id 定);不给就按这张草稿的权威卡(draft_card)。
"""
from app.apps.tree import node_map
from app.ledger.fieldbaseline import project_graph
row = (await session.execute(select(WorkflowVersion).where(
WorkflowVersion.workflow_id == workflow.id,
WorkflowVersion.version == int(version)))).scalar_one_or_none()
if row is None or not isinstance(row.graph, dict):
return None
nodes = node_map(row.graph)
wanted, cards = draft_card(row.graph)
# 读的时候认不出唯一的权威卡(多张、契约也没说是哪张)就照旧显示第一张:显示不改任何东西,写入那条路上会拒
wanted = card_id or wanted or (cards[0]["id"] if cards else None)
card = nodes.get(wanted) if wanted else None
metrics = [metric for metric in ((card or {}).get("config") or {}).get("metrics") or []
if isinstance(metric, dict) and metric.get("id")]
fields = _graph_entry_fields(row.graph)
fetches: list[tuple[str, str]] = []
for node_id, node in nodes.items():
config = node.get("config") if isinstance(node.get("config"), dict) else {}
args = config.get("args") if isinstance(config.get("args"), dict) else {}
sql = args.get("sql") if isinstance(args, dict) else None
if node.get("type") == "tool" and isinstance(sql, str) and sql.strip():
fetches.append((str(node_id), sql))
groups: list[dict[str, Any]] = []
for node_id, sql in fetches:
pattern = re.compile(rf"nodes\.{re.escape(node_id)}(?![\w])")
from app.look.calibers import _metric_paths
ids = [str(metric["id"]) for metric in metrics
if any(pattern.search(path) for path in _metric_paths(metric) or [])]
groups.append({"sql": sql, "entry_fields": [dict(field) for field in fields], "metric_ids": ids})
assigned = {metric_id for group in groups for metric_id in group["metric_ids"]}
loose = [str(metric["id"]) for metric in metrics if str(metric["id"]) not in assigned]
if loose:
if groups:
groups[0]["metric_ids"].extend(loose)
else:
groups = [{"sql": "", "entry_fields": [dict(field) for field in fields], "metric_ids": loose}]
out = {
"version": version,
"metrics": metrics,
"entry_fields": [dict(field) for field in fields],
"sql": "\n\n".join(sql for _node, sql in fetches),
"groups": groups,
"blocks": len(fetches) or 1,
}
out = project_graph(row.graph, version, card_id=card_id)
if draft is not None and payload is not None:
out["trial"] = await _trial_view(session, payload.get("trial") or {}, version=version, graph=row.graph)
out["checks"] = await _draft_checks(session, draft, {**payload, "trial": out["trial"]}, row.graph, metrics, groups)
out["checks"] = await _draft_checks(
session, draft, {**payload, "trial": out["trial"]}, row.graph, out["metrics"], out["groups"],
)
return out
@@ -1577,46 +1540,7 @@ CHANGE_UNAVAILABLE = "change_unavailable"
DRAFT_CARD_GONE = "这份草稿的口径卡不见了(可能刚被删掉):请刷新页面再看一次"
def editable_cards(graph: dict[str, Any]) -> list[dict[str, str]]:
"""图里能写的一张张口径卡:[{id, title}],按节点出现的顺序。
能写的是源头口径卡(metrics 节点、没有 caliber_from):转抄别处的口径卡不是权威写入源(A01),改它等于在别处
另存一份定义。
"""
from app.apps.tree import node_map
return [{"id": node_id, "title": str(node.get("title") or node_id)}
for node_id, node in node_map(graph).items()
if node.get("type") == "metrics" and not (node.get("config") or {}).get("caliber_from")]
def contract_card(graph: dict[str, Any]) -> str | None:
"""下游契约声明从哪张卡取指标(metrics_from 只写了一张时就是它):多张卡时按它认这张草稿的权威卡。"""
for node in graph.get("nodes") or []:
config = ((node or {}).get("data") or {}).get("config") if isinstance(node, dict) else None
contract = config.get("contract") if isinstance(config, dict) else None
if not isinstance(contract, dict):
continue
sources = contract.get("metrics_from")
ids = [sources] if isinstance(sources, str) else [s for s in sources or [] if isinstance(s, str)]
if len(ids) == 1:
return ids[0]
return None
def draft_card(graph: dict[str, Any]) -> tuple[str | None, list[dict[str, str]]]:
"""这份草稿的权威口径卡,另交回全部能写的卡。
一张能写的卡时就是它;多张时认下游契约声明的那一张——不是「第一张」。都认不出来时交回 None:写的时候必须显式
给 target.card_id,读的时候页面也照实列候选。
"""
cards = editable_cards(graph)
if len(cards) == 1:
return cards[0]["id"], cards
referenced = contract_card(graph)
if referenced and any(card["id"] == referenced for card in cards):
return referenced, cards
return None, cards
from app.ledger.fieldbaseline import contract_card, draft_card, editable_cards
def _target_card(payload: "DraftPatchIn", workflow: Workflow) -> str | None: