mirror of
https://github.com/ThearchyHelios/AgentLab.git
synced 2026-11-19 13:22:00 +00:00
ce71f7d5c1
复核第 1 轮(abb2707)找到四处:写作目录按证据面板宁可多遮的那一套认指标出处,SQL 认不准时没碰号码列的销售额也成了
「已遮罩」,正式运行引用它就判失败(相对基线回退);「紧挨着负号」连破折号「——」「—」和汉字「一」都算,「运费统一10元」
「合计——7,036.50元」原样导出变红(相对基线回退);紧挨着改字只堵住了数字、负号、小数点,接「万」、画删除线、从右往左
书写的字母、l / I / O 照样全绿,根子在总览把无法核对的项也给绿色;第二张口径卡转抄遮罩指标(nodes.<卡>.metrics[0].value、
vars.<assign_to>.metrics[0].value、nodes.<卡>.text)时号码照样进写作目录、文档证据目录和可验证报告。
取自遮罩格的指标只按确定的那一套认(写作目录、报告自查、可验证报告同一个判断)
- 写作目录(api/evidence.py 的 withheld_metrics)认指标出处时遮罩列只认确定取自遮罩列的(_InputMasks 的 sure:读得出来的
快照按单元格引用自查的 _cited_masks,读不出来的按数据源现在设的列名),和 [[v:…]] 引用同一格的自查同一个口径:WITH 写了
列名表、VALUES 当表、子查询里有没起别名的号码表达式、只装着分档标签的列,指标照常显示、照常引用;改了名字、套了函数的
号码列照旧「已遮罩」。证据面板、推演照旧宁可多遮。报告自查按目录里记的 masked 认,跟着同一个判断
- 可验证报告认老文档(证据目录里还写着原值,整份只给哈希)按同一个判断:api/bundle.py 先按证据面板的那一套认好每份查询
快照里确定取自遮罩列的列和这次运行查到过的遮罩值(certain_masks),交给 build_bundle(sure_columns、sure_secrets),
_Store.metric_word 的 sure 按它们认。认口径卡能不能随文件附带照旧宁可多遮(卡上记着输入的值),SQL 认不准的卡同基线只给 id
转抄上游口径卡的指标按上游那个指标认
- recompute.card_input 认出输入转抄了哪张卡的哪几个指标:nodes.<卡>.metrics[i].value、vars.<卡的 assign_to>.metrics[i]、
整个指标、输入、渲染算转抄那个指标;整张卡、清单文字、整份指标列表、下标不是字面量的同整份查询结果(值里有遮罩值才不给,
数指标个数的照给);只取名称、单位、口径名的不算
- 证据面板 / 写作目录(_InputMasks._copied)、可验证报告(_Store._copied_kind)都用同一个 _metric_masks / metric_word 往上追,
上游取自遮罩格,转抄的也不给;追了 CARD_DEPTH(8)张还没追到头(循环里抄自己上一轮的产出)的宁可多遮
- 写作目录读快照前把运行状态里每张口径卡的输入一并读好,不在事件循环里读盘
显示文字核对(verifier.js)
- 判不通过只剩紧挨着的数字、小数点,和真当负号用的三个(半角连字符减号、数学减号、全角减号)紧挨着数字开头、前面又不是
字母数字时(「SKU-12」「A-3」的连字符是编号的一部分)。破折号(——、—、–、―,小号的、竖排的)、波浪线、汉字「一」、
假名长音、制表符横线这些易混字当普通的字。连字符、不断行连字符、上下标负号、粗减号这类平时不当负号用的短横紧挨着数字
开头判无法核对(以前判不通过;基线也是无法核对)
- 判无法核对(带原因):后面接着量级单位(万、亿、千、百、兆、萬、億、%、‰、‱、k、K、M、B 及全角,中间隔着空格也算;
核对过的字以「万」「亿」结尾再接一个也算),核对过的字画着删除线或紧挨着画了删除线的字,紧挨着长得像数字的字母和数字
符号(l、I、O、上下标数字、圈码、罗马数字、汉字〇),从右往左书写的字母和方向控制字符(照旧)
- 总览(overallOf):一项无法核对都没有、每份报告都核对上才是绿色的「全部通过」;有无法核对的写中性的「可核对的项全部通过,
另有 N 项无法核对」,下面醒目地写明这个文件判断不了真伪,要在系统中核对封存(运行 ID、GET /api/runs/<id>/verify、定时任务
通知里的清单哈希前 12 位)。以前无法核对的项照样给绿色,伪造者只要让读者看到的那个数落进无法核对就行
- 交出 layoutOf 给 node 里逐条核排法;开发数据 39 份报告 1591 处、验收数据 58 份报告 1695 处引用按新规则扫过,没有一处判
不通过或无法核对
测试
- 新增 tests/test_report_masked_metric_sure.py(四种 SQL 认不准的销售额、分档标签照给照引、确定取自号码列的照旧遮、老文档
照常附带、写作目录和可验证报告逐个指标同一个判断)、tests/test_report_masked_card_copies.py(三种转抄写法在写作目录、文档、
自查、证据面板、可验证报告里都不给号码,没有报告的运行、老文档,转抄普通指标照给,认转抄的各种写法,追不到头的宁可多遮)、
tests/test_bundle_prose_export.py(「统一」「——」「—」「SKU-」「共…家」「–」真跑报告撰写节点、从接口导出,全部通过、总览绿)
- test_bundle_shown:负号只认三个、前面是汉字的照判不通过;像负号的短横、量级单位、删除线、长得像数字的字母各自判无法核对、
总览中性;正常写法原样导出全绿;总览只在一项无法核对都没有时是绿的
- 样本页新增 prose(写作者自己写在引用旁边的普通文字,原样导出全绿);check-bundle 补数学减号、紧挨着汉字加负号判不通过,
接万(隔空格)、接亿元、接 %、接 k、订单号接万、画删除线另写一个数、前面加小写 L、后面接大写 O、长得像负号的连字符判无法
核对,各自点名原因、总览中性并写明去系统里核对;有无法核对项的样本页(base、legacy、continued、missing-card、masked-card、
masked-source、nan)总览改断言中性,从右往左书写的字母那条不再断言总览是绿的
- 每处改动都做了「去掉实现、测试变红」:写作目录不用 sure、老文档不用 sure、sure 列不用、转抄认不出(三处各自)、追不到头
按核对不了、负号的字母数字例外、负号恢复成 \p{Pd} 全部、数学减号全角减号不算、量级单位、隔空格、以万结尾、删除线、像数字
的字母、像负号的短横、总览有无法核对照样绿
208 lines
12 KiB
Python
208 lines
12 KiB
Python
"""值取自遮罩格的口径卡指标只按「确定取自遮罩列」认:SQL 认不准、整份遮的快照里,没确定取自遮罩列的普通金额、计数、标签列,
|
||
指标照常显示、照常引用。
|
||
|
||
复核第 1 轮发现(相对基线的回退):写作目录按证据面板宁可多遮的那一套(_mask_sets 的 hidden)认指标的出处。SQL 认不准时
|
||
(WITH 写了列名表、VALUES 当表、子查询里有没起别名的表达式)整份结果的列都算遮罩列,完全没碰号码列的销售额也被写成
|
||
「已遮罩」,正式运行引用它就判失败,写作者也改不掉;同一格按 [[v:…]] 引用却不报违规(单元格引用的自查只认确定的那一套,
|
||
_cited_masks)。可验证报告认老文档时同样多算,证据目录里写着这类普通金额的老文档整份只给哈希。
|
||
|
||
现在写作目录、报告自查、可验证报告(含老文档只给哈希的判断)认指标取不取自遮罩格,都按单元格引用自查的那一套
|
||
(_cited_masks:_mask_sets 的 sure,再按血缘排除值确实不取自遮罩列的)。证据面板、推演照旧宁可多遮。
|
||
|
||
真跑的部分用虚构的 SQLite 收银库(号码是编的),写作者是 MockChatModel 照剧本写。
|
||
"""
|
||
from __future__ import annotations
|
||
|
||
import base64
|
||
import json
|
||
import sqlite3
|
||
|
||
import pytest
|
||
from httpx import ASGITransport, AsyncClient
|
||
from sqlalchemy import select
|
||
|
||
from app.core.artifact_store import read_bytes
|
||
from app.data.engine import engines
|
||
from app.db.base import SessionLocal
|
||
from app.db.models import DataSource
|
||
from app.engine.runner import run_manager
|
||
from app.main import app
|
||
from tests.test_bundle_shown import verdicts
|
||
from tests.test_report_masked_metrics import doc_of, events, finish, metric_lines, node, script
|
||
|
||
ROWS = [(1, "顾客甲", "19900001101", 1234.5), (2, "顾客乙", "19900001102", 2345.25), (3, "顾客丙", "19900001103", 3456.75)]
|
||
PHONES = [r[2] for r in ROWS]
|
||
GMV = {"id": "gmv", "name": "销售额", "unit": "元", "decimals": 2, "format": "thousands",
|
||
"expression": "cell(nodes.q, 0, 'gmv')"}
|
||
|
||
#: 没碰号码列的查询,SQL 却认不准结果里哪一列取自哪一列(证据面板、可验证报告的快照按整份遮):(SQL, 销售额的渲染)
|
||
PLAIN = [
|
||
# WITH 写了列名表
|
||
("WITH s(cid, amt) AS (SELECT id, amount FROM customers) SELECT SUM(amt) AS gmv FROM s", "7,036.50元"),
|
||
# VALUES 当参数表
|
||
("WITH params(lo) AS (VALUES (1000)) SELECT SUM(amount) AS gmv FROM customers, params WHERE amount >= lo",
|
||
"7,036.50元"),
|
||
("WITH v(id) AS (VALUES (1), (2)) SELECT SUM(c.amount) AS gmv FROM customers c JOIN v ON v.id = c.id",
|
||
"3,579.75元"),
|
||
# 子查询里有没起别名的号码表达式,外层只取金额
|
||
("SELECT id, amount AS gmv FROM (SELECT id, UPPER(phone), amount FROM customers) ORDER BY id", "1,234.50元"),
|
||
]
|
||
|
||
|
||
def leaks(value) -> list[str]:
|
||
raw = value if isinstance(value, str) else json.dumps(value, ensure_ascii=False, default=str)
|
||
return [p for p in PHONES if p in raw or p in raw.replace(",", "")]
|
||
|
||
|
||
@pytest.fixture
|
||
async def engine_up():
|
||
await run_manager.setup()
|
||
yield
|
||
await run_manager.shutdown()
|
||
|
||
|
||
@pytest.fixture
|
||
async def client(engine_up):
|
||
async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as c:
|
||
yield c
|
||
|
||
|
||
@pytest.fixture
|
||
async def pos(tmp_path, engine_up):
|
||
"""收银库:顾客的号码设了遮罩。"""
|
||
path = tmp_path / "pos.db"
|
||
db = sqlite3.connect(path)
|
||
db.executescript("CREATE TABLE customers (id INTEGER PRIMARY KEY, name TEXT, phone TEXT, amount REAL);")
|
||
db.executemany("INSERT INTO customers VALUES (?, ?, ?, ?)", ROWS)
|
||
db.commit()
|
||
db.close()
|
||
async with SessionLocal() as session:
|
||
row = (await session.execute(select(DataSource).where(DataSource.name == "n1pos"))).scalar_one_or_none()
|
||
if row is None:
|
||
row = DataSource(name="n1pos", kind="sqlite", database=str(path), readonly=True, enabled=True)
|
||
session.add(row)
|
||
row.database, row.kind, row.enabled, row.readonly = str(path), "sqlite", True, True
|
||
row.options = {"mask_columns": ["phone"]}
|
||
await session.commit()
|
||
source_id = row.id
|
||
await engines.invalidate(source_id)
|
||
yield path
|
||
await engines.invalidate(source_id)
|
||
|
||
|
||
def flow(sql: str, metrics: list[dict], **write) -> dict:
|
||
"""收银查询 → 口径卡 → 报告(违规处置默认判失败)→ 成果。"""
|
||
nodes = [
|
||
node("start", "input"),
|
||
node("q", "tool", tool="db_query__n1pos", args={"sql": sql}),
|
||
node("card", "metrics", caliber="收银口径", caliber_version="v1", metrics=metrics),
|
||
node("write", "report", instructions="写一句", **{"on_violation": "fail", **write}),
|
||
node("out", "output", fields=[{"name": "报告", "value": "{{ nodes.write.text }}"}]),
|
||
]
|
||
return {"nodes": nodes, "edges": [{"source": a["id"], "target": b["id"]} for a, b in zip(nodes, nodes[1:])]}
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 写作目录、报告自查
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
@pytest.mark.parametrize("sql, shown", PLAIN)
|
||
async def test_an_amount_from_a_query_whose_sql_cannot_be_traced_is_given_and_citable(monkeypatch, pos, sql, shown):
|
||
"""没碰号码列的销售额:写作目录照写它的值(以前写「已遮罩」),正式运行引用它照常通过(以前判失败),同一格按
|
||
[[v:…]] 引用同样不报违规——按指标引用和按格引用是同一个判断。"""
|
||
seen = script(monkeypatch, "本期销售额 [[m:gmv]],对照 [[v:Q1.r0.gmv]]。")
|
||
row = await finish(flow(sql, [GMV]))
|
||
assert row.status == "succeeded", row.error
|
||
lines = metric_lines(seen[0])
|
||
assert f"销售额 = {shown}" in lines and "已遮罩" not in lines, lines
|
||
[checked] = [e.data for e in await events(row.id) if e.type == "report.checked"]
|
||
assert checked["ok"] is True and checked["repairs"] == 0, checked
|
||
doc = await doc_of(row.id)
|
||
assert "masked" not in doc["catalog"]["m:gmv"] and f"本期销售额 {shown}" in doc["markdown"], doc["markdown"]
|
||
|
||
|
||
async def test_a_label_column_that_only_bands_the_masked_column_is_given_and_citable(monkeypatch, pos):
|
||
"""只装着分档标签的列(CASE 里写死的「已留」「未留」):值里没有号码,指标照给、照常引用。"""
|
||
seen = script(monkeypatch, "首位顾客 [[m:has]] 号码,对照 [[v:Q1.r0.has_phone]]。")
|
||
sql = "SELECT id, CASE WHEN phone IS NULL THEN '未留' ELSE '已留' END AS has_phone FROM customers ORDER BY id"
|
||
row = await finish(flow(sql, [{"id": "has", "name": "首位顾客留号", "format": "plain",
|
||
"expression": "cell(nodes.q, 0, 'has_phone')"}]))
|
||
assert row.status == "succeeded", row.error
|
||
assert "首位顾客留号 = 已留" in metric_lines(seen[0])
|
||
doc = await doc_of(row.id)
|
||
assert "masked" not in doc["catalog"]["m:has"] and "首位顾客 已留 号码" in doc["markdown"], doc["markdown"]
|
||
|
||
|
||
async def test_a_metric_whose_column_is_surely_derived_from_the_masked_one_is_still_withheld(monkeypatch, pos):
|
||
"""对照:SQL 里认得出取自号码列的结果列(套了函数、改了名字)照旧写「已遮罩」,同一张卡上的金额照给;引用它的正式运行
|
||
照旧判失败。"""
|
||
seen = script(monkeypatch, "首位顾客号码 [[m:p0]],消费 [[m:spend0]]。")
|
||
row = await finish(flow("SELECT id, UPPER(phone) AS p, amount FROM customers ORDER BY id", [
|
||
{"id": "p0", "name": "首位顾客号码", "format": "plain", "expression": "cell(nodes.q, 0, 'p')"},
|
||
{"id": "spend0", "name": "首位顾客消费", "unit": "元", "decimals": 2, "format": "thousands",
|
||
"expression": "cell(nodes.q, 0, 'amount')"}]))
|
||
lines = metric_lines(seen[0])
|
||
assert "首位顾客号码 = 已遮罩(取自设置了遮罩的列,不要引用)" in lines and "首位顾客消费 = 1,234.50元" in lines, lines
|
||
assert leaks(lines) == []
|
||
assert row.status == "failed" and "m:p0" in (row.error or "") and leaks(row.error) == [], row.error
|
||
|
||
|
||
# --------------------------------------------------------------------------
|
||
# 可验证报告:老文档只给哈希的判断
|
||
# --------------------------------------------------------------------------
|
||
|
||
|
||
async def test_an_old_doc_carrying_an_ordinary_amount_from_an_untraceable_query_is_embedded(client, monkeypatch, pos):
|
||
"""升级前写的文档(写作前不认遮罩,这里把那一步换成什么都不认):证据目录里写着没碰号码列的销售额。以前可验证报告按
|
||
宁可多遮的那一套认,整份文档只给哈希、页面说「有 1 份报告未能核对」;现在和写作目录同一个判断,文档照常附带、核对得上。
|
||
口径卡照旧只给 id(卡上记着输入的值,SQL 认不准的快照按整份遮,同基线),引用它的数字无法核对。"""
|
||
import app.api.evidence as evidence_api
|
||
|
||
async def nothing(*_args, **_kwargs):
|
||
return []
|
||
|
||
monkeypatch.setattr(evidence_api, "withheld_metrics", nothing)
|
||
script(monkeypatch, "本期销售额 [[m:gmv]]。")
|
||
row = await finish(flow(PLAIN[1][0], [GMV]))
|
||
assert row.status == "succeeded", row.error
|
||
resp = await client.get(f"/api/runs/{row.id}/bundle.html")
|
||
assert resp.status_code == 200, resp.text
|
||
from tests.test_bundle_masked_metrics import data_of
|
||
|
||
bundle = data_of(resp.text)
|
||
[doc] = [a for a in bundle["artifacts"] if a["kind"] == "report_doc"]
|
||
assert "b64" in doc and "masked" not in doc, doc
|
||
assert json.loads(base64.b64decode(doc["b64"]))["catalog"]["m:gmv"]["value"] == 7036.5
|
||
got = verdicts(bundle)
|
||
assert got["docs"] == [{"node": "write", "trusted": True}], got["docs"]
|
||
assert leaks(resp.text) == []
|
||
|
||
|
||
@pytest.mark.parametrize("sql, expected", [
|
||
*[(sql, None) for sql, _ in PLAIN],
|
||
("SELECT id, UPPER(phone) AS gmv FROM customers ORDER BY id", "masked"),
|
||
("SELECT id, phone AS gmv FROM customers ORDER BY id", "masked"),
|
||
("SELECT id, CASE WHEN phone IS NULL THEN '未留' ELSE '已留' END AS gmv FROM customers ORDER BY id", None),
|
||
])
|
||
async def test_the_bundle_judges_old_docs_like_the_report_writer(monkeypatch, pos, sql, expected):
|
||
"""可验证报告认老文档里的指标给不给(_Store.metric_word 的 sure),和报告撰写节点的写作目录(_metric_masks 的
|
||
sure、by_value=False)一样;证据面板照旧宁可多遮。"""
|
||
from app.api.evidence import MASKED, _mask_fingerprint, _Masks, _metric_in, _metric_masks, _sealed, certain_masks
|
||
from app.bundle.build import _Store
|
||
|
||
script(monkeypatch, "收银完成。")
|
||
row = await finish(flow(sql, [{**GMV, "format": "plain", "unit": "", "decimals": None}], on_violation="flag"))
|
||
assert row.status == "succeeded", row.error
|
||
sealed = await _sealed(row.id)
|
||
entry = next(e for e in sealed.entries if e.get("kind") == "metric_set")
|
||
card = json.loads(read_bytes(entry["artifact"], verify=False))
|
||
sure, secrets = await certain_masks(sealed, _Masks(sealed))
|
||
store = _Store(lambda a: read_bytes(a, verify=False), await _mask_fingerprint(), sealed.events,
|
||
sure_columns=sure, sure_secrets=secrets)
|
||
metric = _metric_in(card, "gmv")
|
||
_, word = await _metric_masks(metric, "card", entry["artifact"], sealed, _Masks(sealed), by_value=False, sure=True)
|
||
writer = {MASKED: "masked"}.get(word or "", word)
|
||
bundle = store.metric_word(entry["artifact"], metric, sealed.events, sure=True)
|
||
assert writer == bundle == expected, (writer, bundle)
|