数据剖析可以同时剖析几张表:剖析设置加「同时剖析的表数」(table_concurrency,1–3,缺省 1,即以前的一张做完再做下一张;上限 3 是给这个源上的分析留出连接池的临时连接),每张表的查询仍一条一条发,所以同一时刻对业务库最多这么多条查询。剖析任务的执行改成按设置开几个执行者,按任务里的顺序一张结束就开始下一张;进度、发过的查询各落在自己那张表上(不再记「正在做的那一张」)。每张表开始前照旧重读数据源:停用、关掉剖析或止损后不再开始新的表,正在做的几张照常做完(止损时剩下的项记止损的原因),都结束后再把没开始的记停下;止损的连续失败、超时比例按几张表的查询合起来算。停止、关停时正在做的几张各自收尾,做完的项照常写进目录,都收完才往上交(等的时候又被停止也等完);一张表让任务整体出错(快照被改动、数据源没了、记结果写库出错)时,另几张按停止收尾、原因写这一句,本进程续不上租约的不写。同时要估同一张表的行数时只估一次(按表加锁,先来的没估成后来的自己估)。同源已有剖析时的 409 说法从「第 x / y 张表」改成「已完成 x / y 张表」。界面:设置表单多一格(单位「张」,提示写明业务库同时要应付几条查询、和分析共用连接),风险说明、卡片悬停、开始前的醒目一句在大于 1 时写明「同时剖析 N 张表、每张表的查询一条一条地发」,预算多一行,这次最多用时按几轮算。测试:同一时刻在途查询不超过设置、每张表仍一条一条发(1/2/3 三种)、几张一起显示进行中并按顺序开始、同时停止各自写进目录、止损后不再开始新表、停用时正在做的做完、一张出错另几张按停止收尾、共享行数只估一次;check-catalog-profile 加一节(预算、估算、说法、两行进行中、表清单)。验收:后端全量 14979 通过;止损那条并发测试单独跑 30 遍、整个剖析任务测试文件并行跑 15 遍都没有偶发;tsc;check-catalog-profile 217 项、check-copy 28 项全过

This commit is contained in:
ThearchyHelios
2026-10-21 09:00:48 +08:00
parent 31f3af8bac
commit f774e42463
17 changed files with 511 additions and 106 deletions
+1 -1
View File
@@ -928,7 +928,7 @@ be brought back by the next drafting or inference.
| Review queue | Lists only inferred items, ordered by "how many times runs used the table × how much the item affects numbers × source": valid-record conditions, dedup rules, business dates, and relationships come first, business keys and measure types next, and Chinese names, meanings, and units last, with each item stating why it's ranked there; filter by category and table, and multi-select to confirm, reject, or undo review in bulk (optimistic locking per table, with conflicts flagged one by one); you can see which runs used a table, and how many items have been confirmed for frequently used tables, for all tables, and for the items that affect numbers |
| Table Copilot (「表助手」) | Chat with Copilot on the table details page to refine this one table: it can see the schema, the current state of the catalog (each item's source and status), relationships, profiling results, drafting instructions, and the columns left empty in the last drafting (masked columns: name only); it proposes changes in batches following your instructions, each with before, after, and basis (validated like the catalog: formulas aren't accepted, and column names must exist in the schema), and asks you back when it isn't sure; only on sources with profiling on does it run queries through the read-only guard to verify (at most 5 per request, with results masked first). Proposals accumulate in Pending changes (「待保存的修改」) and are saved in one go after confirmation (at most 200 items at a time), recorded as confirmed by a person, with each item's basis saved separately as that item's Basis at confirmation (「确认时的依据」) (kept apart from notes and visible in the table details; removed when undoing review returns the item to inferred). Conversations aren't stored on the server, and it changes only the catalog, never the business database |
| Export and import | Exports one JSON file (the catalog of all tables, each item's status, and drafting instructions) to import into another instance or data source: first preview which tables and columns match, and whether each item will be added, updated, or kept, with conflicts and skips listed one by one; items a person confirmed or rejected are carried over as they are, where the target already has a confirmed item with a different value, the target's is kept and recorded as a conflict, inferred items in the file only fill gaps, and columns not in the schema are skipped with the reason stated; import isn't possible while drafting or profiling is in progress. Drafting instructions are written only when the target has none |
| Data profiling | Off by default, enabled per data source. Runs as a background job, table by table: for each table you can see check progress, query count, elapsed time, and which item is being checked; closing the dialog doesn't stop it, and you can stop it or retry only the unfinished tables; on service restart, unfinished jobs are marked interrupted and don't resume by themselves. The budget is per table (by default at most 40 queries and 60 seconds per table), and when one table uses up its budget only that table stops, while the next one gets a full budget; there's no cap for the job as a whole, which relies on stop-loss rules and manual stopping: it stops after 3 consecutive failed queries, or when 7 of the last 10 queries time out, so as not to put load on the business database. Queries are sent one at a time, read-only, through the guard; large tables use statistics to estimate row counts, and masked columns aren't sampled; every query sent leaves an audit trail (with values replaced by "?", and the sampled key values never enter AgentLab's database). Sampling verifies inferred relationships, filling in coverage and cardinality, and the ones that hold up are promoted to verified; status columns get code-value candidates, with meanings left for a person to fill in. By default it profiles "tables that need profiling" (「需要剖析的表」): those never profiled, not finished last time, or that have since gained relationships awaiting verification or new code-value or date columns; on the same data source, drafting and profiling queue up serially, and the scope is set from the catalog as it stands when profiling starts, so you can simply click profile right after drafting finishes. The table list has a Profiling (「剖析」) column, and the table details show the last profiling result |
| Data profiling | Off by default, enabled per data source. Runs as a background job, table by table: for each table you can see check progress, query count, elapsed time, and which item is being checked; closing the dialog doesn't stop it, and you can stop it or retry only the unfinished tables; on service restart, unfinished jobs are marked interrupted and don't resume by themselves. The budget is per table (by default at most 40 queries and 60 seconds per table), and when one table uses up its budget only that table stops, while the next one gets a full budget; there's no cap for the job as a whole, which relies on stop-loss rules and manual stopping: it stops after 3 consecutive failed queries, or when 7 of the last 10 queries time out, so as not to put load on the business database. By default it profiles one table at a time; in the data source settings you can raise this to 2–3 tables at once (so at most that many queries hit the business database at any moment). Within each table, queries are sent one at a time, read-only, through the guard; large tables use statistics to estimate row counts, and masked columns aren't sampled; every query sent leaves an audit trail (with values replaced by "?", and the sampled key values never enter AgentLab's database). Sampling verifies inferred relationships, filling in coverage and cardinality, and the ones that hold up are promoted to verified; status columns get code-value candidates, with meanings left for a person to fill in. By default it profiles "tables that need profiling" (「需要剖析的表」): those never profiled, not finished last time, or that have since gained relationships awaiting verification or new code-value or date columns; on the same data source, drafting and profiling queue up serially, and the scope is set from the catalog as it stands when profiling starts, so you can simply click profile right after drafting finishes. The table list has a Profiling (「剖析」) column, and the table details show the last profiling result |
| Corrections in conversation | When a user says "status=9 means voided" (「status=9 表示作废」), Copilot produces a catalog change proposal listing before, after, and the reason; it's saved only after a person confirms, and the reason is recorded as the item's Basis at confirmation (「确认时的依据」); formulas aren't accepted and are left to caliber cards |
| Versions and reminders | The catalog is frozen into the evidence along with the schema snapshot, so you can look up which version every run used. Publishing records the catalog versions of the tables a template references, and a formal run gives a reminder when the catalog has changed; where an agent writes SQL only at run time, there's a separate reminder for "possibly involved" (「可能涉及」). It only reminds and never blocks the run |