全栈安全加固、性能提速与架构演进:连接池与沙箱止血、双路混合检索、多版本归档及前端分包治理

- 第一批(安全加固与生命周期止血):隔离测试宿主机密钥与 .env 泄漏;修复 SQLite 取消查询挂死、:memory: 连接池报错与 EngineCache 世代竞态;补齐运行与评测删除时的 Checkpointer 线程、Artifact 磁盘文件与无主附件回收;加固三套沙箱路径越界、进程组清理与 Seatbelt 数据目录隔离,封堵 SSRF IPv4-mapped IPv6 元数据绕过与响应体 OOM;修复推理模型多轮 reasoning_content、结构化输出 Token 漏计与缓存折扣计费
- 第二批(数据库/集群/问数提速与首屏分包):补齐核心表索引与 SQLite WAL 调优,列表接口按需 defer 大列,收敛启动扫描并打通跨进程 slot 槽位唤醒;优化跨源全局选表、哈希桶数字绑定、流式 CSV 导入及多方言分析算子;剥离首屏静态引入的 @xyflow/react 与全局样式(首屏主包降至 461.59 kB),补充流式 rAF 攒批、Store 精细订阅与表单无障碍绑定
- 第三批(架构演进与工程自动化闭环):记录库超挂接阈值历史版本自动增量归档至单库多表(解除 SQLite 10 库 ATTACH 硬上限);重构知识库为词面 BM25 与全集合归一化向量矩阵真·双路独立召回并支持游标分批重建;升级多方言感知 SQL 模板词法分析(E'...' / $$...$$ / q'[...]' / # / 嵌套注释)与 Decimal / bool / NaN 参数校验,消除 ledger 层对 api 路由层的反向依赖;拆分 3101 行 CaliberDrafts 页面为 7 个领域子模块,消除检查脚本本机路径硬编码并新增 GitHub Actions CI 门禁与三批回归测试套件
This commit is contained in:
ThearchyHelios
2026-11-13 14:51:36 +08:00
parent 09949fc8ac
commit eddeded2a4
106 changed files with 7178 additions and 3913 deletions
+40 -1
View File
@@ -11,7 +11,7 @@
from __future__ import annotations
import contextlib
from collections.abc import AsyncIterator
from collections.abc import AsyncIterator, Iterable
from pathlib import Path
from typing import Any
@@ -64,6 +64,12 @@ async def open_sqlite_saver(path: Path | str) -> AsyncIterator[BaseCheckpointSav
from langgraph.checkpoint.sqlite.aio import AsyncSqliteSaver
async with AsyncSqliteSaver.from_conn_string(str(path)) as saver:
conn = getattr(saver, "conn", None)
if conn is not None:
await conn.execute("PRAGMA journal_mode=WAL")
await conn.execute("PRAGMA synchronous=NORMAL")
await conn.execute("PRAGMA journal_size_limit=67108864")
await conn.execute("PRAGMA busy_timeout=5000")
yield saver
@@ -75,6 +81,39 @@ def open_saver() -> contextlib.AbstractAsyncContextManager[BaseCheckpointSaver]:
return open_sqlite_saver(settings.checkpoint_path)
async def _delete_on(saver: Any, ids: list[str]) -> None:
adelete = getattr(saver, "adelete_thread", None)
if not callable(adelete):
return
setup = getattr(saver, "setup", None)
if callable(setup):
with contextlib.suppress(Exception):
await setup()
for tid in ids:
with contextlib.suppress(Exception):
await adelete(tid)
async def delete_threads(thread_ids: Iterable[str]) -> None:
"""运行被彻底删除或评测明细被回收时,一并清理断点库里对应的 thread。
注意:正常运行结束(succeeded / cancelled)时不能删,因为 GET /api/runs/{id}/state 和 /checkpoints
还要回看历史断点;只有在运行记录本身从数据库删除之后才调这里。
"""
ids = sorted({str(t) for t in thread_ids if t})
if not ids:
return
from app.engine.runner import run_manager
saver = run_manager._checkpointer
if saver is not None:
await _delete_on(saver, ids)
return
if settings.checkpoint_conninfo or settings.checkpoint_path.is_file():
async with open_saver() as opened:
await _delete_on(opened, ids)
async def count(saver: Any, thread_id: str | None = None) -> tuple[int, int]:
"""(断点条数, pending writes 条数)。thread_id 为空数全部。copy-to-postgres 核对用,走公开接口。"""
config = {"configurable": {"thread_id": thread_id}} if thread_id else None