feat(syslog): 增加积压采样免采样窗口

- 新增 start-lag-ms 配置,默认15分钟内全量保留
- 在15分钟至1小时之间按10级逐步降低保留率
- 延迟达到1小时后使用普通10%、非标20%的最低保留率
- 校验起始延迟必须小于最低保留率阈值
- 状态接口新增 start_lag_ms 配置展示
- 将采样策略版本升级为 v2
- 更新 Docker Compose 和 v4 升级配置说明
- 补充采样时间边界及非法配置单元测试
This commit is contained in:
王绍宏
2026-09-12 23:43:56 +08:00
parent c868e3befd
commit 87fee6fe69
27 changed files with 1777 additions and 94 deletions
@@ -0,0 +1,49 @@
-- v4 达梦升级:创建 Kafka 积压分级采样的小时审计表。
-- 仅在 01 预检确认目标对象均不存在时执行一次,不要重复执行。
CREATE TABLE syslog_backlog_retention_hourly (
source_hour TIMESTAMP NOT NULL,
source_topic VARCHAR2(255) NOT NULL,
kafka_partition INT NOT NULL,
policy_version VARCHAR2(64) NOT NULL,
normal_retain_bps INT NOT NULL,
non_normal_retain_bps INT NOT NULL,
message_total BIGINT DEFAULT 0 NOT NULL,
alarm_rows_total BIGINT DEFAULT 0 NOT NULL,
alarm_rows_saved BIGINT DEFAULT 0 NOT NULL,
normal_rows_total BIGINT DEFAULT 0 NOT NULL,
normal_rows_saved BIGINT DEFAULT 0 NOT NULL,
normal_rows_dropped BIGINT DEFAULT 0 NOT NULL,
non_normal_rows_total BIGINT DEFAULT 0 NOT NULL,
non_normal_rows_saved BIGINT DEFAULT 0 NOT NULL,
non_normal_rows_dropped BIGINT DEFAULT 0 NOT NULL,
raw_total BIGINT DEFAULT 0 NOT NULL,
raw_saved BIGINT DEFAULT 0 NOT NULL,
raw_dropped BIGINT DEFAULT 0 NOT NULL,
min_lag_ms BIGINT DEFAULT 0 NOT NULL,
max_lag_ms BIGINT DEFAULT 0 NOT NULL,
created_at TIMESTAMP DEFAULT SYSDATE NOT NULL,
updated_at TIMESTAMP DEFAULT SYSDATE NOT NULL,
CONSTRAINT pk_syslog_backlog_ret_hourly PRIMARY KEY (
source_hour, source_topic, kafka_partition, policy_version,
normal_retain_bps, non_normal_retain_bps
),
CONSTRAINT ck_syslog_backlog_ret_bps CHECK (
normal_retain_bps BETWEEN 0 AND 10000
AND non_normal_retain_bps BETWEEN 0 AND 10000
)
);
CREATE INDEX idx_syslog_backlog_ret_updated
ON syslog_backlog_retention_hourly(updated_at);
COMMENT ON TABLE syslog_backlog_retention_hourly IS
'Kafka积压分级采样小时审计;按来源小时、Topic、Partition和策略聚合';
COMMENT ON COLUMN syslog_backlog_retention_hourly.source_hour IS
'syslog-serve生成的receive_time所在小时(Asia/Shanghai)';
COMMENT ON COLUMN syslog_backlog_retention_hourly.normal_retain_bps IS
'普通标准化记录该级实际保留万分比,10000表示100%';
COMMENT ON COLUMN syslog_backlog_retention_hourly.non_normal_retain_bps IS
'非标记录该级实际保留万分比,10000表示100%';
COMMIT;