| 123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276 |
- using System.Text.RegularExpressions;
- namespace Admin.NET.Plugin.AiDOP.DataPlatform.MdpRebuild;
- public sealed record MdpRebuildScope(string ModuleCode, long TenantId, long FactoryId)
- {
- public static readonly HashSet<string> RebuildModules = new(StringComparer.OrdinalIgnoreCase)
- {
- "S1", "S2", "S3", "S4", "S5", "S6", "S7"
- };
- public static readonly HashSet<string> SnapshotModules = new(StringComparer.OrdinalIgnoreCase)
- {
- "S1", "S2", "S3", "S4", "S5", "S6", "S7", "S9"
- };
- public static MdpRebuildScope Create(string moduleCode, long tenantId, long factoryId)
- {
- var mc = NormalizeModule(moduleCode);
- if (!RebuildModules.Contains(mc))
- throw new InvalidOperationException($"模块 {mc} 不支持全量数据重算");
- if (tenantId <= 0)
- throw new InvalidOperationException($"{mc} 全量重算必须指定有效 tenantId,禁止 tenantId=0");
- if (factoryId <= 0)
- throw new InvalidOperationException($"{mc} 全量重算必须指定有效 factoryId");
- if (factoryId == tenantId)
- throw new InvalidOperationException($"{mc} 全量重算的 factoryId 不能等于 tenantId");
- return new MdpRebuildScope(mc, tenantId, factoryId);
- }
- public static string NormalizeModule(string? moduleCode)
- {
- var mc = (moduleCode ?? string.Empty).Trim().ToUpperInvariant();
- if (mc.Length == 0) throw new InvalidOperationException("moduleCode 不能为空");
- return mc;
- }
- public string LockName => $"{ModuleCode}_MDP_FULL:{TenantId}:{FactoryId}";
- public string ScopeKey => $"{ModuleCode}:{TenantId}:{FactoryId}";
- }
- public static class ModuleRebuildStatus
- {
- public const string Queued = "QUEUED";
- public const string Running = "RUNNING";
- public const string Success = "SUCCESS";
- public const string Failed = "FAILED";
- public const string Cancelled = "CANCELLED";
- }
- public static class ModuleRebuildStages
- {
- public const string Queued = "QUEUED";
- public const string AcquiringLock = "ACQUIRING_LOCK";
- public const string Preparing = "PREPARING";
- public const string Staging = "STAGING";
- public const string Standard = "STANDARD";
- public const string Dwd = "DWD";
- public const string Kpi = "KPI";
- public const string Atomic = "ATOMIC";
- public const string T8Inbound = "T8_INBOUND";
- public const string KpiPreparing = "KPI_PREPARING";
- public const string KpiCalculating = "KPI_CALCULATING";
- public const string Finalizing = "FINALIZING";
- public const string Success = "SUCCESS";
- public const string Failed = "FAILED";
- public const string Cancelled = "CANCELLED";
- public static int StageTotal(string moduleCode) => Normalize(moduleCode) switch
- {
- "S4" => 4,
- "S5" or "S6" or "S7" => 4,
- _ => 5
- };
- public static int ToStageIndex(string moduleCode, string stage)
- {
- var mc = Normalize(moduleCode);
- if (mc is "S5" or "S6" or "S7")
- {
- return stage switch
- {
- T8Inbound => 1,
- KpiPreparing => 2,
- KpiCalculating => 3,
- Atomic or Finalizing or Success => 4,
- _ => 0
- };
- }
- if (mc == "S4")
- {
- return stage switch
- {
- Staging => 1,
- Standard => 2,
- Dwd => 3,
- Kpi or Finalizing or Success => 4,
- _ => 0
- };
- }
- return stage switch
- {
- Staging => 1,
- Standard => 2,
- Dwd => 3,
- Kpi => 4,
- Atomic or Finalizing or Success => 5,
- _ => 0
- };
- }
- public static string ToChinese(string? stage) => stage switch
- {
- Queued => "排队中",
- AcquiringLock => "等待资源",
- Preparing => "准备运行环境",
- Staging => "拉取源数据",
- Standard => "标准化数据",
- Dwd => "生成 DWD 明细",
- Kpi => "重算 KPI",
- Atomic => "重建原子数据",
- T8Inbound => "T8 基表入站",
- KpiPreparing => "准备 KPI 计算",
- KpiCalculating => "计算 KPI",
- Finalizing => "收尾",
- Success => "已完成",
- Failed => "失败",
- Cancelled => "已取消",
- _ => stage ?? ""
- };
- public static string ConfirmTitle(string moduleCode) => $"确认重算 {Normalize(moduleCode)} 数据?";
- private static string Normalize(string? moduleCode) =>
- string.IsNullOrWhiteSpace(moduleCode) ? "" : moduleCode.Trim().ToUpperInvariant();
- }
- public sealed record ModuleProgressUpdate(
- string Stage,
- int StageIndex,
- int ProgressPercent,
- string Message,
- int? Rows = null,
- string? CompletedStage = null,
- string? DetailJson = null);
- public sealed class ModuleRebuildResult
- {
- public string BatchId { get; set; } = string.Empty;
- public long RunLogId { get; set; }
- public int StageRows { get; set; }
- public int StandardRows { get; set; }
- public int DwdRows { get; set; }
- public int KpiRows { get; set; }
- public int AtomicRows { get; set; }
- public string? DetailJson { get; set; }
- }
- public sealed class ModuleRebuildAlreadyRunningException : Exception
- {
- public ModuleRebuildAlreadyRunningException(string moduleCode)
- : base($"{moduleCode} 数据重算正在执行,请勿重复提交")
- {
- ModuleCode = moduleCode;
- }
- public string ModuleCode { get; }
- }
- public static class MdpSqlScope
- {
- private static readonly Regex FromAlias = new(
- @"\bFROM\s+[`.\w]+\s+(?:AS\s+)?([`\w]+)",
- RegexOptions.IgnoreCase | RegexOptions.Compiled);
- public static string InjectTenantFactory(string sql)
- => Inject(sql, includeFactory: true);
- public static string InjectTenantOnly(string sql)
- => Inject(sql, includeFactory: false);
- private static string Inject(string sql, bool includeFactory)
- {
- if (string.IsNullOrWhiteSpace(sql)) return sql;
- return Regex.Replace(sql, @"\bWHERE\b", m =>
- {
- var start = m.Index;
- var window = sql.Substring(start, Math.Min(240, sql.Length - start));
- if (window.Contains("@TenantId", StringComparison.Ordinal))
- return m.Value;
- var prefix = ResolveSourcePrefix(sql, start);
- var factoryFilter = includeFactory
- ? $"COALESCE(NULLIF({prefix}factory_id,0),1)=@FactoryId AND "
- : string.Empty;
- return $"WHERE {prefix}tenant_id=@TenantId AND {factoryFilter}";
- }, RegexOptions.IgnoreCase);
- }
- private static string ResolveSourcePrefix(string sql, int whereIndex)
- {
- var depth = 0;
- for (var i = whereIndex - 1; i >= 0; i--)
- {
- if (sql[i] == ')')
- {
- depth++;
- continue;
- }
- if (sql[i] == '(')
- {
- if (depth > 0) depth--;
- continue;
- }
- if (depth != 0 || i < 3) continue;
- if (!sql.AsSpan(i - 3, 4).Equals("FROM", StringComparison.OrdinalIgnoreCase)) continue;
- if (i > 3 && (char.IsLetterOrDigit(sql[i - 4]) || sql[i - 4] == '_')) continue;
- var match = FromAlias.Match(sql, i - 3);
- if (!match.Success || match.Index != i - 3) return string.Empty;
- var alias = match.Groups[1].Value.Trim('`');
- return alias is "LEFT" or "RIGHT" or "INNER" or "CROSS" or "JOIN" or "ON" or "AS" or "WHERE"
- ? string.Empty
- : alias + ".";
- }
- return string.Empty;
- }
- }
- public interface IModuleRebuildCapability
- {
- bool IsEnabled(string moduleCode);
- int MaxParallelScopes { get; }
- }
- public sealed class ModuleRebuildCapability : IModuleRebuildCapability, ISingleton
- {
- public bool IsEnabled(string moduleCode)
- {
- var mc = MdpRebuildScope.NormalizeModule(moduleCode);
- try
- {
- return Furion.App.GetConfig<bool?>($"AiDOP:MdpRebuild:Modules:{mc}:Enabled", true) ?? false;
- }
- catch
- {
- return false;
- }
- }
- public int MaxParallelScopes
- {
- get
- {
- try
- {
- return Math.Clamp(Furion.App.GetConfig<int?>("AiDOP:MdpRebuild:MaxParallelScopes", true) ?? 2, 1, 8);
- }
- catch
- {
- return 2;
- }
- }
- }
- }
- public sealed class AlwaysOnModuleRebuildCapability : IModuleRebuildCapability
- {
- public bool IsEnabled(string moduleCode) => true;
- public int MaxParallelScopes => 2;
- }
|