| 123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244 |
- using System.Text.Json.Serialization;
- namespace Admin.NET.Plugin.AiDOP.DataPlatform.S0Dim;
- /// <summary>
- /// source / dim 两侧属性校验和的一次读数(<c>count</c> 与 <c>checksum</c> **成对**,缺一不可)。
- ///
- /// <para>由 <see cref="S0DimReconciler.ReadChecksumPairAsync"/> 唯一产出,
- /// 供两处共用:① <c>ReconcileAsync</c> 的 Layer 3 写后对账;
- /// ② <see cref="S0DimMaterializer"/> 的写前闸门。两者共用同一读取路径,
- /// 保证「写前判等」与「写后对账」不会各跑一套口径。</para>
- /// </summary>
- public sealed class S0DimChecksumPair
- {
- /// <summary>源侧参与校验和的行数(按租户)。</summary>
- public int SourceCount { get; set; }
- /// <summary>dim 侧参与校验和的行数(按租户)。</summary>
- public int DimCount { get; set; }
- /// <summary>
- /// 源侧校验和原值。<c>DECIMAL(40,0)</c> 的精确整数,**不外化为 JSON 数字** ——
- /// 20+ 位整数经 JSON/MySQL JSON 取出时会退化成 DOUBLE 丢精度,读者会看到一个错的数字。
- /// 对外一律走 <see cref="SourceChecksum"/> 字符串。
- /// </summary>
- [JsonIgnore]
- public decimal SourceChecksumValue { get; set; }
- /// <inheritdoc cref="SourceChecksumValue"/>
- [JsonIgnore]
- public decimal DimChecksumValue { get; set; }
- /// <summary>源侧校验和(十进制字符串,与 <see cref="S0DimReconcileResult.SourceChecksum"/> 同格式)。</summary>
- public string SourceChecksum => SourceChecksumValue.ToString("F0");
- /// <summary>dim 侧校验和(十进制字符串,与 <see cref="S0DimReconcileResult.DimChecksum"/> 同格式)。</summary>
- public string DimChecksum => DimChecksumValue.ToString("F0");
- /// <summary>
- /// 两侧完全一致。<b>count 与 checksum 必须同时相等</b> —— 判据与
- /// <c>S0DimReconciler</c> Layer 3 的失败条件逐字互补,不是「只比 checksum」。
- /// </summary>
- public bool IsMatch => IsMatchOf(SourceCount, SourceChecksumValue, DimCount, DimChecksumValue);
- /// <summary>
- /// 校验和判等的**唯一**判据(纯函数,可单测)。
- ///
- /// <para>🔴 <b>count 不可省略</b>:<c>SUM(RowHash)</c> 是可交换的加法,
- /// 理论上不同行集可能求和相同;<c>S0DimSqlBuilder.BuildSourceChecksumSql</c> 的注释
- /// 明确写了「必须与 COUNT 成对使用」。只比 checksum 会让闸门在极端情况下误判「无变化」
- /// 而跳过一次本该执行的刷新。</para>
- /// </summary>
- public static bool IsMatchOf(int sourceCount, decimal sourceChecksum, int dimCount, decimal dimChecksum)
- => sourceCount == dimCount && sourceChecksum == dimChecksum;
- }
- /// <summary>业务键集合三层等价的差集计数。全部为 0 才算通过。</summary>
- public sealed class S0DimBizKeyDiff
- {
- /// <summary>source 有而 stg 没有。</summary>
- public int SourceMinusStaging { get; set; }
- /// <summary>stg 有而 source 没有。</summary>
- public int StagingMinusSource { get; set; }
- /// <summary>stg 有而 dim 没有。</summary>
- public int StagingMinusDim { get; set; }
- /// <summary>dim 有而 stg 没有。</summary>
- public int DimMinusStaging { get; set; }
- /// <summary>四个方向是否全为 0。</summary>
- public bool IsEqual => SourceMinusStaging == 0 && StagingMinusSource == 0
- && StagingMinusDim == 0 && DimMinusStaging == 0;
- }
- /// <summary>某一层业务键的质量(重复 / 空值)。</summary>
- public sealed class S0DimBizKeyQuality
- {
- /// <summary>层名:source / staging / dim。</summary>
- public string Layer { get; set; } = "";
- /// <summary>总行数。</summary>
- public int Total { get; set; }
- /// <summary>去重后行数。</summary>
- public int Distinct { get; set; }
- /// <summary>空业务键行数(stg 侧出现即意味着 biz_key_expr 回落到了 source_row_id)。</summary>
- public int Blank { get; set; }
- /// <summary>无重复且无空值。</summary>
- public bool IsClean => Total == Distinct && Blank == 0;
- }
- /// <summary>单个维度的对账结果。</summary>
- public sealed class S0DimReconcileResult
- {
- /// <summary>维度 Key。</summary>
- public string Key { get; set; } = "";
- /// <summary>租户。</summary>
- public long TenantId { get; set; }
- /// <summary>本轮批次(只读对账时可能为 null)。</summary>
- public string? BatchId { get; set; }
- /// <summary>源表行数(按租户)。</summary>
- public int SourceCount { get; set; }
- /// <summary>staging 行数(四段谓词)。</summary>
- public int StagingCount { get; set; }
- /// <summary>dim 行数(按租户)。</summary>
- public int DimCount { get; set; }
- /// <summary>源表中无法归属租户的行(整条链路不可见,WARN)。</summary>
- public int SourceUntenanted { get; set; }
- /// <summary>dim 中不属于本批的残留行(FULL Replace 后必须为 0)。</summary>
- public int DimBatchImpurity { get; set; }
- /// <summary>业务键集合差集。</summary>
- public S0DimBizKeyDiff BizKeyDiff { get; set; } = new();
- /// <summary>三层业务键质量。</summary>
- public List<S0DimBizKeyQuality> BizKeyQuality { get; set; } = [];
- /// <summary>源侧属性校验和。</summary>
- public string SourceChecksum { get; set; } = "0";
- /// <summary>dim 侧属性校验和。</summary>
- public string DimChecksum { get; set; } = "0";
- /// <summary>dim 业务键重复组数。</summary>
- public int DimDuplicateBizKeys { get; set; }
- /// <summary>镜像唯一性违例组数(未声明时为 -1 表示不适用)。</summary>
- public int MirrorUniqueViolations { get; set; } = -1;
- /// <summary>
- /// 孤儿子行**真实总数**(WARN,不阻断;未声明父维度时为 0)。
- /// 与 <see cref="OrphanChildren"/> 不同:后者受 <c>OrphanSampleLimit</c> 截断,只是样本。
- /// </summary>
- public int OrphanChildCount { get; set; }
- /// <summary>孤儿子行业务键**样本**(最多 <c>OrphanSampleLimit</c> 条)。总数看 <see cref="OrphanChildCount"/>。</summary>
- public List<string> OrphanChildren { get; set; } = [];
- /// <summary>阻断级失败原因(为空表示通过)。</summary>
- public List<string> Failures { get; set; } = [];
- /// <summary>告警(不阻断)。</summary>
- public List<string> Warnings { get; set; } = [];
- /// <summary>无阻断级失败。</summary>
- public bool IsPass => Failures.Count == 0;
- }
- /// <summary>单个维度的物化结果。</summary>
- public sealed class S0DimMaterializeResult
- {
- /// <summary>本轮实际执行过刷新且写后对账通过。</summary>
- public const string StatusSuccess = "SUCCESS";
- /// <summary>本轮失败,dim 停在上一轮的完整快照。</summary>
- public const string StatusFailed = "FAILED";
- /// <summary>
- /// 本轮**验证过** source 与 dim 完全一致,因此**没有执行**刷新。
- ///
- /// <para>属「非失败」,但**不是** <see cref="StatusSuccess"/>:
- /// SUCCESS 表示「跑过并验证成功」,本状态表示「验证后确认无需跑」。
- /// 两者混同会让「自动刷新是否真的在工作」不可观测。</para>
- /// </summary>
- public const string StatusSkippedNoChange = "SKIPPED_NO_CHANGE";
- /// <summary>跳过原因:源与 dim 的 (count, checksum) 完全一致。</summary>
- public const string SkipReasonNoChange = "NO_CHANGE";
- /// <summary>维度 Key。</summary>
- public string Key { get; set; } = "";
- /// <summary>租户。</summary>
- public long TenantId { get; set; }
- /// <summary>批次。</summary>
- public string BatchId { get; set; } = "";
- /// <summary>purge 掉的 staging 行数。</summary>
- public int StagingPurged { get; set; }
- /// <summary>本批拉取到的源行数。</summary>
- public int SourcePulled { get; set; }
- /// <summary>本批写入 staging 的行数。</summary>
- public int StagingWritten { get; set; }
- /// <summary>写入 dim 的行数。</summary>
- public int DimRows { get; set; }
- /// <summary>对账结果。跳过时为 null —— 本轮没有写入,也就没有写后对账。</summary>
- public S0DimReconcileResult? Reconcile { get; set; }
- /// <summary>
- /// 写前闸门读到的 source / dim 校验和读数。闸门执行过即非 null(无论是否跳过),
- /// 外化进 <c>summary_json</c> 供事后复核「当时凭什么判定无变化」。
- /// </summary>
- public S0DimChecksumPair? Checksum { get; set; }
- /// <summary>跳过原因(仅 <see cref="StatusSkippedNoChange"/> 时非 null)。</summary>
- public string? SkipReason { get; set; }
- /// <summary><see cref="StatusSuccess"/> / <see cref="StatusSkippedNoChange"/> / <see cref="StatusFailed"/>。</summary>
- public string Status { get; set; } = StatusSuccess;
- /// <summary>失败信息。</summary>
- public string? Error { get; set; }
- }
- /// <summary>一次刷新(可含多个维度)的整体结果。</summary>
- public sealed class S0DimRefreshResult
- {
- /// <summary>批次(同时是 stg/dim 的 sync_batch_id 与 run log 的 batch_id)。</summary>
- public string BatchId { get; set; } = "";
- /// <summary>租户。</summary>
- public long TenantId { get; set; }
- /// <summary>
- /// SUCCESS / SUCCESS_WITH_WARNING / SKIPPED_NO_CHANGE / FAILED / CANCELED。
- /// <c>SKIPPED_NO_CHANGE</c> 仅在**全部**维度都被写前闸门跳过时出现。
- /// </summary>
- public string Status { get; set; } = "SUCCESS";
- /// <summary>逐维度结果。</summary>
- public List<S0DimMaterializeResult> Items { get; set; } = [];
- /// <summary>耗时(毫秒)。</summary>
- public int DurationMs { get; set; }
- /// <summary>run log 主键。</summary>
- public long RunLogId { get; set; }
- }
|