nanChen/ai-security-xdr
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
206985a65edde751b4a2a93f30e621eb1fb8ce58
ai-security-xdr/haobang-security-xdr/logs/syslog-consumer.log
T

376 lines
46 KiB
Plaintext
Raw Normal View History

关联分析规则-数据降噪
2026-03-18 18:00:25 +08:00
2026-03-09 18:20:29.258 [main] INFO com.syslogApplication - Starting syslogApplication using Java 1.8.0_121 on LAPTOP-ARDUR3N0 with PID 31516 (E:\GIT_GOSAME\ai-security-xdr\haobang-security-xdr\syslog-consumer\target\classes started by chenc in E:\GIT_GOSAME\ai-security-xdr\haobang-security-xdr)
2026-03-09 18:20:29.258 [background-preinit] INFO o.h.validator.internal.util.Version - HV000001: Hibernate Validator 6.2.5.Final
2026-03-09 18:20:29.264 [main] INFO com.syslogApplication - No active profile set, falling back to 1 default profile: "default"
2026-03-09 18:20:32.501 [main] INFO o.s.d.r.c.RepositoryConfigurationDelegate - Multiple Spring Data modules found, entering strict repository configuration mode
2026-03-09 18:20:32.504 [main] INFO o.s.d.r.c.RepositoryConfigurationDelegate - Bootstrapping Spring Data Elasticsearch repositories in DEFAULT mode.
2026-03-09 18:20:33.247 [main] INFO o.s.d.r.c.RepositoryConfigurationDelegate - Finished Spring Data repository scanning in 735 ms. Found 1 Elasticsearch repository interfaces.
2026-03-09 18:20:33.255 [main] INFO o.s.d.r.c.RepositoryConfigurationDelegate - Multiple Spring Data modules found, entering strict repository configuration mode
2026-03-09 18:20:33.256 [main] INFO o.s.d.r.c.RepositoryConfigurationDelegate - Bootstrapping Spring Data Reactive Elasticsearch repositories in DEFAULT mode.
2026-03-09 18:20:33.435 [main] INFO o.s.d.r.c.RepositoryConfigurationExtensionSupport - Spring Data Reactive Elasticsearch - Could not safely identify store assignment for repository candidate interface com.common.service.AppLogRepository; If you want this repository to be a Reactive Elasticsearch repository, consider annotating your entities with one of these annotations: org.springframework.data.elasticsearch.annotations.Document (preferred), or consider extending one of the following types with your repository: org.springframework.data.elasticsearch.repository.ReactiveElasticsearchRepository
2026-03-09 18:20:33.435 [main] INFO o.s.d.r.c.RepositoryConfigurationDelegate - Finished Spring Data repository scanning in 179 ms. Found 0 Reactive Elasticsearch repository interfaces.
2026-03-09 18:20:33.460 [main] INFO o.s.d.r.c.RepositoryConfigurationDelegate - Multiple Spring Data modules found, entering strict repository configuration mode
2026-03-09 18:20:33.461 [main] INFO o.s.d.r.c.RepositoryConfigurationDelegate - Bootstrapping Spring Data Redis repositories in DEFAULT mode.
2026-03-09 18:20:33.643 [main] INFO o.s.d.r.c.RepositoryConfigurationExtensionSupport - Spring Data Redis - Could not safely identify store assignment for repository candidate interface com.common.service.AppLogRepository; If you want this repository to be a Redis repository, consider annotating your entities with one of these annotations: org.springframework.data.redis.core.RedisHash (preferred), or consider extending one of the following types with your repository: org.springframework.data.keyvalue.repository.KeyValueRepository
2026-03-09 18:20:33.643 [main] INFO o.s.d.r.c.RepositoryConfigurationDelegate - Finished Spring Data repository scanning in 167 ms. Found 0 Redis repository interfaces.
2026-03-09 18:20:34.518 [main] INFO o.s.b.w.e.tomcat.TomcatWebServer - Tomcat initialized with port(s): 8089 (http)
2026-03-09 18:20:34.530 [main] INFO o.a.coyote.http11.Http11NioProtocol - Initializing ProtocolHandler ["http-nio-8089"]
2026-03-09 18:20:34.531 [main] INFO o.a.catalina.core.StandardService - Starting service [Tomcat]
2026-03-09 18:20:34.531 [main] INFO o.a.catalina.core.StandardEngine - Starting Servlet engine: [Apache Tomcat/9.0.65]
2026-03-09 18:20:34.885 [main] INFO o.a.c.c.C.[.[.[/xdrservice] - Initializing Spring embedded WebApplicationContext
2026-03-09 18:20:34.885 [main] INFO o.s.b.w.s.c.ServletWebServerApplicationContext - Root WebApplicationContext: initialization completed in 5554 ms
2026-03-09 18:20:34.950 [main] INFO o.s.b.f.a.AutowiredAnnotationBeanPostProcessor - Autowired annotation is not supported on static fields: private static com.common.service.DmColumnService com.syslogApplication.dmColumnService
2026-03-09 18:20:37.863 [main] INFO com.influx.InfluxDBClient - InfluxDB connection successful: ready for queries and writes
2026-03-09 18:20:38.381 [main] WARN c.b.m.core.injector.AbstractMethod - [com.common.mapper.DeviceCollectTaskMapper.insert] Has been loaded by XML or SqlProvider or Mybatis's Annotation, so ignoring this injection for [class com.baomidou.mybatisplus.core.injector.methods.Insert]
2026-03-09 18:20:38.394 [main] WARN c.b.m.core.injector.AbstractMethod - [com.common.mapper.DeviceCollectTaskMapper.update] Has been loaded by XML or SqlProvider or Mybatis's Annotation, so ignoring this injection for [class com.baomidou.mybatisplus.core.injector.methods.Update]
2026-03-09 18:20:38.410 [main] WARN c.b.m.core.injector.AbstractMethod - [com.common.mapper.DeviceCollectTaskMapper.deleteById] Has been loaded by XML or SqlProvider or Mybatis's Annotation, so ignoring this injection for [class com.baomidou.mybatisplus.core.injector.methods.DeleteById]
2026-03-09 18:20:38.414 [main] WARN c.b.m.core.injector.AbstractMethod - [com.common.mapper.DeviceCollectTaskMapper.selectById] Has been loaded by XML or SqlProvider or Mybatis's Annotation, so ignoring this injection for [class com.baomidou.mybatisplus.core.injector.methods.SelectById]
2026-03-09 18:20:38.469 [main] ERROR c.b.m.core.MybatisConfiguration - mapper[com.common.mapper.SecExceptionAlgorithmMapper.findById] is ignored, because it exists, maybe from xml file
2026-03-09 18:20:44.376 [main] INFO c.c.s.RealtimeAnalysisScheduler - ========== 初始化实时分析调度器 ==========
2026-03-09 18:20:44.398 [main] INFO com.zaxxer.hikari.HikariDataSource - HikariPool-SyslogConsumer - Starting...
2026-03-09 18:20:45.062 [main] INFO com.zaxxer.hikari.HikariDataSource - HikariPool-SyslogConsumer - Start completed.
2026-03-09 18:20:45.249 [main] INFO c.c.s.RealtimeAnalysisScheduler - 查询到 0 个实时分析规则
2026-03-09 18:20:45.250 [main] INFO c.c.s.RealtimeAnalysisScheduler - ========== 实时分析调度器初始化完成 ==========
2026-03-09 18:20:45.256 [main] INFO o.s.b.f.a.AutowiredAnnotationBeanPostProcessor - Autowired annotation is not supported on static fields: public static com.common.service.DeviceDeviceService com.common.service.AccessLogAlertService.deviceDeviceService
2026-03-09 18:20:45.296 [main] INFO c.c.service.AccessLogAlertService - 初始化AccessLogAlertService，上次处理时间: 2026-03-09T18:19:45.296
2026-03-09 18:20:45.457 [main] INFO c.c.service.AccessLogAlertService - 加载了 1 个启用的算法配置
2026-03-09 18:20:46.497 [main] INFO com.influx.InfluxDBClient - InfluxDB connection successful: ready for queries and writes
2026-03-09 18:20:46.694 [main] INFO com.common.util.MyBatisUtil - MyBatis 初始化成功
2026-03-09 18:20:47.630 [main] INFO org.quartz.impl.StdSchedulerFactory - Using default implementation for ThreadExecutor
2026-03-09 18:20:47.642 [main] INFO o.quartz.core.SchedulerSignalerImpl - Initialized Scheduler Signaller of type: class org.quartz.core.SchedulerSignalerImpl
2026-03-09 18:20:47.643 [main] INFO org.quartz.core.QuartzScheduler - Quartz Scheduler v.2.3.2 created.
2026-03-09 18:20:47.644 [main] INFO org.quartz.simpl.RAMJobStore - RAMJobStore initialized.
2026-03-09 18:20:47.644 [main] INFO org.quartz.core.QuartzScheduler - Scheduler meta-data: Quartz Scheduler (v2.3.2) 'quartzScheduler' with instanceId 'NON_CLUSTERED'
初次提交代码
2026-01-11 15:33:22 +08:00
Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally.
NOT STARTED.
Currently in standby mode.
Number of jobs executed: 0
Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 10 threads.
Using job-store 'org.quartz.simpl.RAMJobStore' - which does not support persistence. and is not clustered.
关联分析规则-数据降噪
2026-03-18 18:00:25 +08:00
2026-03-09 18:20:47.644 [main] INFO org.quartz.impl.StdSchedulerFactory - Quartz scheduler 'quartzScheduler' initialized from an externally provided properties instance.
2026-03-09 18:20:47.644 [main] INFO org.quartz.impl.StdSchedulerFactory - Quartz scheduler version: 2.3.2
2026-03-09 18:20:47.645 [main] INFO org.quartz.core.QuartzScheduler - JobFactory set to: org.springframework.scheduling.quartz.SpringBeanJobFactory@25297d52
2026-03-09 18:20:47.838 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka version: 3.4.0
2026-03-09 18:20:47.838 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka commitId: 2e1947d240607d53
2026-03-09 18:20:47.838 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka startTimeMs: 1773051647836
2026-03-09 18:20:47.859 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka version: 3.4.0
2026-03-09 18:20:47.859 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka commitId: 2e1947d240607d53
2026-03-09 18:20:47.859 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka startTimeMs: 1773051647859
2026-03-09 18:20:47.861 [main] INFO o.a.coyote.http11.Http11NioProtocol - Starting ProtocolHandler ["http-nio-8089"]
2026-03-09 18:20:47.878 [main] INFO o.s.b.w.e.tomcat.TomcatWebServer - Tomcat started on port(s): 8089 (http) with context path '/xdrservice'
2026-03-09 18:20:47.879 [main] INFO o.s.s.quartz.SchedulerFactoryBean - Starting Quartz Scheduler now
2026-03-09 18:20:47.880 [main] INFO org.quartz.core.QuartzScheduler - Scheduler quartzScheduler_$_NON_CLUSTERED started.
2026-03-09 18:20:47.897 [main] INFO com.syslogApplication - Started syslogApplication in 19.043 seconds (JVM running for 24.576)
2026-03-09 18:20:48.685 [org.springframework.kafka.KafkaListenerEndpointContainer#0-1-C-1] INFO o.s.k.l.KafkaMessageListenerContainer - test-group-app: partitions assigned: []
2026-03-09 18:20:48.753 [org.springframework.kafka.KafkaListenerEndpointContainer#0-0-C-1] INFO o.s.k.l.KafkaMessageListenerContainer - test-group-app: partitions assigned: [test-topic-0]
2026-03-09 18:21:00.012 [scheduling-1] INFO com.common.schedule.ETLOrchestrator - ETL任务开始执行，开始时间：2026-03-09 18:15:00,结束时间：2026-03-09 18:20:00
2026-03-09 18:21:00.017 [scheduling-1] INFO com.common.service.DataExtractor - 开始处理告警类型指定时间范围内数据，时间范围: 2026-03-09T18:15 - 2026-03-09T18:20
2026-03-09 18:21:00.017 [log-processor-1] INFO c.c.service.AccessLogAlertService - 开始执行访问日志告警处理任务
2026-03-09 18:21:00.017 [scheduling-5] INFO c.c.service.AccessLogAlertService - 开始执行访问日志告警处理任务
2026-03-09 18:21:00.099 [scheduling-6] INFO c.c.service.DeviceStatsUpdateService - 开始执行设备统计更新任务...
2026-03-09 18:21:00.186 [scheduling-6] INFO c.c.service.DeviceStatsUpdateService - 设备统计更新完成，处理设备数：0，耗时：87ms
2026-03-09 18:21:00.186 [scheduling-6] INFO c.c.service.DeviceStatsUpdateService - 开始执行设备采集探针任务时间更新，时间: 2026-03-09T18:21:00.186
2026-03-09 18:21:00.191 [scheduling-6] INFO c.c.s.DeviceCollectTaskUpdateService - 开始批量更新设备采集任务时间，当前时间: 2026-03-09T18:21:00.191
2026-03-09 18:21:00.243 [log-processor-1] INFO c.c.service.AccessLogAlertService - 加载了 1 个启用的算法配置
2026-03-09 18:21:00.243 [scheduling-5] INFO c.c.service.AccessLogAlertService - 加载了 1 个启用的算法配置
2026-03-09 18:21:00.250 [scheduling-1] INFO com.common.service.DataExtractor - 指定时间范围分组数据量: 0 组
2026-03-09 18:21:00.250 [scheduling-1] INFO com.common.service.DataExtractor - 没有需要处理的数据
2026-03-09 18:21:00.250 [scheduling-1] INFO com.common.schedule.ETLOrchestrator - 定时ETL任务执行完成，耗时: 0 秒
2026-03-09 18:21:00.250 [scheduling-1] INFO c.c.s.NormalizeRuleHitTimeService - 开始执行泛化规则命中时间更新任务，时间：2026-03-09T18:21:00.250
2026-03-09 18:21:00.672 [scheduling-6] INFO c.c.s.DeviceCollectTaskUpdateService - 批量更新完成，总计: 48，已更新: 1
2026-03-09 18:21:00.672 [scheduling-6] INFO c.c.service.DeviceStatsUpdateService - 设备采集探针任务时间更新完成，耗时: 484ms
2026-03-09 18:21:00.833 [scheduling-5] INFO c.c.service.AccessLogAlertService - 没有发现新的日志数据，上次处理时间: 2026-03-09T18:19:45.296
2026-03-09 18:21:00.833 [log-processor-1] INFO c.c.service.AccessLogAlertService - 没有发现新的日志数据，上次处理时间: 2026-03-09T18:19:45.296
2026-03-09 18:21:00.915 [scheduling-1] INFO c.c.s.NormalizeRuleHitTimeService - 从 syslog_normal_data 表统计到 0 条规则命中记录
2026-03-09 18:21:00.915 [scheduling-1] INFO c.c.s.NormalizeRuleHitTimeService - 从 syslog_normal_alarm 表统计到 0 条规则命中记录
2026-03-09 18:21:00.915 [scheduling-1] INFO c.c.s.NormalizeRuleHitTimeService - 合并后需要更新的规则数量：0
2026-03-09 18:21:01.069 [scheduling-1] INFO c.c.s.NormalizeRuleHitTimeService - 当前启用状态的规则数量：173
2026-03-09 18:21:01.069 [scheduling-1] INFO c.c.s.NormalizeRuleHitTimeService - 开始批量更新，规则总数：173，分批数：1
2026-03-09 18:21:01.070 [scheduling-1] INFO c.c.s.NormalizeRuleHitTimeService - 泛化规则命中时间更新任务完成，更新规则数：0，耗时：820ms
2026-03-09 18:21:32.055 [http-nio-8089-exec-1] INFO o.s.web.servlet.DispatcherServlet - Initializing Servlet 'dispatcherServlet'
2026-03-09 18:21:32.060 [http-nio-8089-exec-1] INFO o.s.web.servlet.DispatcherServlet - Completed initialization in 5 ms
2026-03-09 18:21:32.233 [http-nio-8089-exec-1] INFO com.controllers.SyslogPushController - 收到syslog发送请求: SyslogRequest{ip='192.168.1.19', port=514, logContent='<0> 2026-01-12T14:37:53+08:00 ubuntu log_forward[3419]: {"flow_id": 1028204815001825, "serial_num": "CJFBT92", "src_ip": "120.238.245.132", "src_port": 60838, "dest_ip": "211.136.192.6", "dest_port": 53, "proto": "UDP", "app_proto": "dns", "direction": "CTS", "attacker_ip": "120.238.245.132", "victim_ip": "211.136.192.6", "rule_id": "0x20001e", "rule_name": "发现带外域名DNS请求行为", "attack_type": "网络嗅探", "severity": "1", "bulletin": "确认受害者以及其他信息，及时清除恶意链接", "detail_info": "发现主机正在请求DNSLOG服务器地址", "vuln_type": "网络嗅探", "vuln_desc": "发现主机正在请求DNSLOG服务器地址", "vuln_harm": "发现主机正在请求DNSLOG服务器地址", "tags": "dnslog", "cnnvd_id": null, "cve_id": null, "killchain": "侦查跟踪", "enable": "启用", "attack_result": "企图", "attack_method": "远程", "site_app": null, "code_language": "通用", "att_ck": "TA0002", "timestamp": "2026-01-12T14:37:53.588+0800", "custom": "{}", "feature_field": "", "feature_payload": "", "": null, "payload": "SQkBAAABAAAAAAAAB3BvbGxpbmcHb2FzdGlmeQNjb20AAAEAAQ==", "packet_size": 37, "pcap_file": ""}', protocol='TCP', facility='USER', severity='INFO'}
2026-03-09 18:21:32.234 [http-nio-8089-exec-1] INFO com.common.service.SyslogService - 开始发送syslog消息: IP=192.168.1.19, Port=514
2026-03-09 18:21:32.235 [http-nio-8089-exec-1] INFO com.common.service.SyslogService - TCP Syslog消息发送成功: 192.168.1.19:514
2026-03-09 18:21:32.235 [http-nio-8089-exec-1] INFO com.controllers.SyslogPushController - Syslog消息发送成功: IP=192.168.1.19, Port=514
2026-03-09 18:21:34.502 [org.springframework.kafka.KafkaListenerEndpointContainer#0-0-C-1] INFO c.Modules.NormalData.SysLogProcessor - 开始处理批次消息，数量: 1
2026-03-09 18:21:34.502 [log-processor-2] INFO c.Modules.NormalData.SysLogProcessor - 收到syslogmessage：[receive_time=20260309182133303 device_id=103 device_name=公司开发内部测试探针 vendor=null data_type=json device_collect_id=1]<0> 2026-01-12T14:37:53+08:00 ubuntu log_forward[3419]: {"flow_id": 1028204815001825, "serial_num": "CJFBT92", "src_ip": "120.238.245.132", "src_port": 60838, "dest_ip": "211.136.192.6", "dest_port": 53, "proto": "UDP", "app_proto": "dns", "direction": "CTS", "attacker_ip": "120.238.245.132", "victim_ip": "211.136.192.6", "rule_id": "0x20001e", "rule_name": "???????????DNS???????", "attack_type": "???????", "severity": "1", "bulletin": "??????????????????????????????????", "detail_info": "????????????????DNSLOG?????????", "vuln_type": "???????", "vuln_desc": "????????????????DNSLOG?????????", "vuln_harm": "????????????????DNSLOG?????????", "tags": "dnslog", "cnnvd_id": null, "cve_id": null, "killchain": "??????", "enable": "????", "attack_result": "???", "attack_method": "???", "site_app": null, "code_language": "???", "att_ck": "TA0002", "timestamp": "2026-01-12T14:37:53.588+0800", "custom": "{}", "feature_field": "", "feature_payload": "", "": null, "payload": "SQkBAAABAAAAAAAAB3BvbGxpbmcHb2FzdGlmeQNjb20AAAEAAQ==", "packet_size": 37, "pcap_file": ""}
2026-03-09 18:21:40.696 [log-processor-2] ERROR c.M.NormalData.LogNormalProcessor - OrginalColumnMap 对象获取为空
2026-03-09 18:21:41.051 [log-processor-2] ERROR c.M.NormalData.LogNormalProcessor - OrginalColumnMap 对象获取为空
2026-03-09 18:21:41.062 [log-processor-2] ERROR c.M.NormalData.LogNormalProcessor - OrginalColumnMap 对象获取为空
2026-03-09 18:21:41.153 [log-processor-2] WARN c.c.service.LogDataFilterService - 泛化规则-数据过滤规则为空，默认不处理!
2026-03-09 18:21:41.611 [log-processor-2] ERROR c.c.service.LogDataFilterService - 解析过滤规则失败或filters_params为空: null
2026-03-09 18:21:41.797 [org.springframework.kafka.KafkaListenerEndpointContainer#0-0-C-1] INFO c.Modules.NormalData.SysLogProcessor - 批次处理完成，总数: 1
2026-03-09 18:22:00.006 [scheduling-1] INFO c.c.service.AccessLogAlertService - 开始执行访问日志告警处理任务
2026-03-09 18:22:00.006 [scheduling-4] INFO c.c.service.DeviceStatsUpdateService - 开始执行设备统计更新任务...
2026-03-09 18:22:00.007 [log-processor-3] INFO c.c.service.AccessLogAlertService - 开始执行访问日志告警处理任务
2026-03-09 18:22:00.168 [scheduling-4] INFO c.c.service.DeviceStatsUpdateService - 设备统计更新完成，处理设备数：1，耗时：162ms
2026-03-09 18:22:00.168 [scheduling-4] INFO c.c.service.DeviceStatsUpdateService - 开始执行设备采集探针任务时间更新，时间: 2026-03-09T18:22:00.168
2026-03-09 18:22:00.168 [scheduling-4] INFO c.c.s.DeviceCollectTaskUpdateService - 开始批量更新设备采集任务时间，当前时间: 2026-03-09T18:22:00.168
2026-03-09 18:22:00.236 [log-processor-3] INFO c.c.service.AccessLogAlertService - 加载了 1 个启用的算法配置
2026-03-09 18:22:00.238 [scheduling-1] INFO c.c.service.AccessLogAlertService - 加载了 1 个启用的算法配置
2026-03-09 18:22:00.602 [scheduling-4] INFO c.c.s.DeviceCollectTaskUpdateService - 批量更新完成，总计: 48，已更新: 1
2026-03-09 18:22:00.602 [scheduling-4] INFO c.c.service.DeviceStatsUpdateService - 设备采集探针任务时间更新完成，耗时: 434ms
2026-03-09 18:22:00.638 [scheduling-1] INFO c.c.service.AccessLogAlertService - 获取到 1 条新的日志数据，时间范围: 2026-03-09T18:19:45.296 到 2026-03-09T18:22:00.238
2026-03-09 18:22:00.638 [scheduling-1] INFO c.c.service.AccessLogAlertService - 开始处理算法: 测试算法3 (ID: 2004083121877696514)
2026-03-09 18:22:00.720 [scheduling-1] INFO c.c.service.AccessLogAlertService - 算法 测试算法3 未检测到告警
2026-03-09 18:22:00.722 [scheduling-1] INFO c.c.service.AccessLogAlertService - 访问日志告警处理任务完成，下次将从 2026-03-09T18:22:00.238 开始处理
2026-03-09 18:22:00.785 [log-processor-3] INFO c.c.service.AccessLogAlertService - 获取到 1 条新的日志数据，时间范围: 2026-03-09T18:22:00.238 到 2026-03-09T18:22:00.236
2026-03-09 18:22:00.785 [log-processor-3] INFO c.c.service.AccessLogAlertService - 开始处理算法: 测试算法3 (ID: 2004083121877696514)
2026-03-09 18:22:01.137 [log-processor-3] INFO c.c.service.AccessLogAlertService - 算法 测试算法3 未检测到告警
2026-03-09 18:22:01.137 [log-processor-3] INFO c.c.service.AccessLogAlertService - 访问日志告警处理任务完成，下次将从 2026-03-09T18:22:00.236 开始处理
2026-03-09 18:23:00.003 [scheduling-6] INFO c.c.service.AccessLogAlertService - 开始执行访问日志告警处理任务
2026-03-09 18:23:00.003 [log-processor-4] INFO c.c.service.AccessLogAlertService - 开始执行访问日志告警处理任务
2026-03-09 18:23:00.084 [scheduling-7] INFO c.c.service.DeviceStatsUpdateService - 开始执行设备统计更新任务...
2026-03-09 18:23:00.235 [scheduling-6] INFO c.c.service.AccessLogAlertService - 加载了 1 个启用的算法配置
2026-03-09 18:23:00.235 [scheduling-7] INFO c.c.service.DeviceStatsUpdateService - 设备统计更新完成，处理设备数：1，耗时：151ms
2026-03-09 18:23:00.235 [log-processor-4] INFO c.c.service.AccessLogAlertService - 加载了 1 个启用的算法配置
2026-03-09 18:23:00.235 [scheduling-7] INFO c.c.service.DeviceStatsUpdateService - 开始执行设备采集探针任务时间更新，时间: 2026-03-09T18:23:00.235
2026-03-09 18:23:00.236 [scheduling-7] INFO c.c.s.DeviceCollectTaskUpdateService - 开始批量更新设备采集任务时间，当前时间: 2026-03-09T18:23:00.236
2026-03-09 18:23:00.444 [log-processor-4] INFO c.c.service.AccessLogAlertService - 没有发现新的日志数据，上次处理时间: 2026-03-09T18:22:00.236
2026-03-09 18:23:00.452 [scheduling-6] INFO c.c.service.AccessLogAlertService - 没有发现新的日志数据，上次处理时间: 2026-03-09T18:22:00.236
2026-03-09 18:23:00.684 [scheduling-7] INFO c.c.s.DeviceCollectTaskUpdateService - 批量更新完成，总计: 48，已更新: 1
2026-03-09 18:23:00.684 [scheduling-7] INFO c.c.service.DeviceStatsUpdateService - 设备采集探针任务时间更新完成，耗时: 448ms
2026-03-09 18:23:01.145 [scheduling-2] INFO c.c.s.RealtimeAnalysisScheduler - 执行规则: ruleId=4e134d65-1170-4d20-ab48-77f3fee6a765, ruleName=告警降噪分析规则-测试-V2, nextTime=2026-03-05T19:12, now=2026-03-09T18:23:00.971
2026-03-09 18:23:01.145 [scheduling-2] INFO c.c.s.impl.AnalysisRuleServiceImpl - 执行实时分析规则: ruleId=4e134d65-1170-4d20-ab48-77f3fee6a765
2026-03-09 18:23:01.608 [scheduling-2] INFO c.c.s.impl.RealtimeAnalysisEngine - 滚动窗口查询范围: 窗口大小=5m，查询时间范围=[2026-03-09 18:18:00, 2026-03-09 18:23:00]
2026-03-09 18:23:01.608 [scheduling-2] INFO c.c.s.impl.RealtimeAnalysisEngine - 开始执行实时规则: ruleId=4e134d65-1170-4d20-ab48-77f3fee6a765, ruleName=告警降噪分析规则-测试-V2, batchNo=20260309182301302, windowType=tumble, dataStartTime=2026-03-09 18:18:00, dataEndTime=2026-03-09 18:23:00
2026-03-09 18:23:03.009 [scheduling-2] INFO c.c.s.impl.RealtimeAnalysisEngine - 生成的SQL: SELECT src_ip AS attack_ip,
dest_ip AS victim_ip,
origin_event_name AS alarm_name,
ARRAY_AGG(DISTINCT src_port) AS attack_port,
ARRAY_AGG(DISTINCT dest_port) AS victim_port,
MAX(event_level) AS alarm_level,
MODE() WITHIN GROUP (ORDER BY dest_domain) AS dns_info,
MODE() WITHIN GROUP (ORDER BY origin_event_type) AS alarm_type,
COUNT(dest_ip) AS log_count,
MAX(attack_result) AS attack_result,
ARRAY_AGG(DISTINCT http_req_header) AS http_req_header,
ARRAY_AGG(DISTINCT http_req_body) AS http_req_body,
ARRAY_AGG(DISTINCT http_resp_header) AS http_resp_header,
ARRAY_AGG(DISTINCT http_resp_body) AS http_resp_body,
ARRAY_AGG(DISTINCT http_url) AS victim_web_url,
ARRAY_AGG(DISTINCT id) AS origin_log_ids,
MIN(log_time) AS log_start_at,
MAX(log_time) AS log_end_at,
ARRAY_AGG(DISTINCT device_id) AS device_id,
ARRAY_AGG(DISTINCT payload) AS payload,
TUMBLE(log_time, INTERVAL '5 MINUTE') AS window_time
FROM syslog_normal_alarm AS t
WHERE log_time >= '2026-03-09 18:18:00' AND log_time < '2026-03-09 18:23:00' AND src_ip != '127.0.0.1' AND event_level >= 1
GROUP BY src_ip, dest_ip, origin_event_name, TUMBLE(log_time, INTERVAL '5 MINUTE')
初次提交代码
2026-01-11 15:33:22 +08:00
关联分析规则-数据降噪
2026-03-18 18:00:25 +08:00
2026-03-09 18:23:03.655 [scheduling-2] INFO c.c.s.impl.RealtimeAnalysisEngine - 规则执行成功: ruleId=4e134d65-1170-4d20-ab48-77f3fee6a765, processedCount=1, alarmCount=1
2026-03-09 18:23:03.970 [scheduling-2] INFO c.c.s.i.RuleExecutionTimeServiceImpl - 更新规则下次执行时间，ruleId=4e134d65-1170-4d20-ab48-77f3fee6a765, ruleName=告警降噪分析规则-测试-V2, windowType=tumble, nextExecuteTime=2026-03-09 18:28:00
2026-03-09 18:23:03.970 [scheduling-2] INFO c.c.s.RealtimeAnalysisScheduler - 本次调度执行规则数: 1, 跳过规则数: 0
2026-03-09 18:24:00.001 [scheduling-5] INFO c.c.service.AccessLogAlertService - 开始执行访问日志告警处理任务
2026-03-09 18:24:00.001 [log-processor-5] INFO c.c.service.AccessLogAlertService - 开始执行访问日志告警处理任务
2026-03-09 18:24:00.077 [scheduling-9] INFO c.c.service.DeviceStatsUpdateService - 开始执行设备统计更新任务...
2026-03-09 18:24:00.226 [scheduling-5] INFO c.c.service.AccessLogAlertService - 加载了 1 个启用的算法配置
2026-03-09 18:24:00.229 [scheduling-9] INFO c.c.service.DeviceStatsUpdateService - 设备统计更新完成，处理设备数：1，耗时：152ms
2026-03-09 18:24:00.229 [log-processor-5] INFO c.c.service.AccessLogAlertService - 加载了 1 个启用的算法配置
2026-03-09 18:24:00.229 [scheduling-9] INFO c.c.service.DeviceStatsUpdateService - 开始执行设备采集探针任务时间更新，时间: 2026-03-09T18:24:00.229
2026-03-09 18:24:00.229 [scheduling-9] INFO c.c.s.DeviceCollectTaskUpdateService - 开始批量更新设备采集任务时间，当前时间: 2026-03-09T18:24:00.229
2026-03-09 18:24:00.419 [log-processor-5] INFO c.c.service.AccessLogAlertService - 没有发现新的日志数据，上次处理时间: 2026-03-09T18:22:00.236
2026-03-09 18:24:00.423 [scheduling-5] INFO c.c.service.AccessLogAlertService - 没有发现新的日志数据，上次处理时间: 2026-03-09T18:22:00.236
2026-03-09 18:24:00.673 [scheduling-9] INFO c.c.s.DeviceCollectTaskUpdateService - 批量更新完成，总计: 48，已更新: 1
2026-03-09 18:24:00.673 [scheduling-9] INFO c.c.service.DeviceStatsUpdateService - 设备采集探针任务时间更新完成，耗时: 444ms
2026-03-09 18:25:00.003 [scheduling-3] INFO c.c.service.AccessLogAlertService - 开始执行访问日志告警处理任务
2026-03-09 18:25:00.003 [log-processor-6] INFO c.c.service.AccessLogAlertService - 开始执行访问日志告警处理任务
2026-03-09 18:25:00.079 [scheduling-2] INFO c.c.service.DeviceStatsUpdateService - 开始执行设备统计更新任务...
2026-03-09 18:25:00.230 [log-processor-6] INFO c.c.service.AccessLogAlertService - 加载了 1 个启用的算法配置
2026-03-09 18:25:00.230 [scheduling-2] INFO c.c.service.DeviceStatsUpdateService - 设备统计更新完成，处理设备数：1，耗时：151ms
2026-03-09 18:25:00.230 [scheduling-3] INFO c.c.service.AccessLogAlertService - 加载了 1 个启用的算法配置
2026-03-09 18:25:00.230 [scheduling-2] INFO c.c.service.DeviceStatsUpdateService - 开始执行设备采集探针任务时间更新，时间: 2026-03-09T18:25:00.230
2026-03-09 18:25:00.230 [scheduling-2] INFO c.c.s.DeviceCollectTaskUpdateService - 开始批量更新设备采集任务时间，当前时间: 2026-03-09T18:25:00.230
2026-03-09 18:25:00.420 [log-processor-6] INFO c.c.service.AccessLogAlertService - 没有发现新的日志数据，上次处理时间: 2026-03-09T18:22:00.236
2026-03-09 18:25:00.420 [scheduling-3] INFO c.c.service.AccessLogAlertService - 没有发现新的日志数据，上次处理时间: 2026-03-09T18:22:00.236
2026-03-09 18:25:00.667 [scheduling-2] INFO c.c.s.DeviceCollectTaskUpdateService - 批量更新完成，总计: 48，已更新: 1
2026-03-09 18:25:00.667 [scheduling-2] INFO c.c.service.DeviceStatsUpdateService - 设备采集探针任务时间更新完成，耗时: 437ms
2026-03-09 18:26:00.003 [scheduling-2] INFO c.c.service.AccessLogAlertService - 开始执行访问日志告警处理任务
2026-03-09 18:26:00.003 [scheduling-7] INFO com.common.schedule.ETLOrchestrator - ETL任务开始执行，开始时间：2026-03-09 18:20:00,结束时间：2026-03-09 18:25:00
2026-03-09 18:26:00.003 [scheduling-7] INFO com.common.service.DataExtractor - 开始处理告警类型指定时间范围内数据，时间范围: 2026-03-09T18:20 - 2026-03-09T18:25
2026-03-09 18:26:00.003 [log-processor-7] INFO c.c.service.AccessLogAlertService - 开始执行访问日志告警处理任务
2026-03-09 18:26:00.080 [scheduling-10] INFO c.c.service.DeviceStatsUpdateService - 开始执行设备统计更新任务...
2026-03-09 18:26:00.229 [log-processor-7] INFO c.c.service.AccessLogAlertService - 加载了 1 个启用的算法配置
2026-03-09 18:26:00.229 [scheduling-7] INFO com.common.service.DataExtractor - 指定时间范围分组数据量: 1 组
2026-03-09 18:26:00.229 [scheduling-10] INFO c.c.service.DeviceStatsUpdateService - 设备统计更新完成，处理设备数：1，耗时：149ms
2026-03-09 18:26:00.229 [scheduling-2] INFO c.c.service.AccessLogAlertService - 加载了 1 个启用的算法配置
2026-03-09 18:26:00.229 [scheduling-10] INFO c.c.service.DeviceStatsUpdateService - 开始执行设备采集探针任务时间更新，时间: 2026-03-09T18:26:00.229
2026-03-09 18:26:00.229 [scheduling-10] INFO c.c.s.DeviceCollectTaskUpdateService - 开始批量更新设备采集任务时间，当前时间: 2026-03-09T18:26:00.229
2026-03-09 18:26:00.420 [scheduling-2] INFO c.c.service.AccessLogAlertService - 没有发现新的日志数据，上次处理时间: 2026-03-09T18:22:00.236
2026-03-09 18:26:00.424 [log-processor-7] INFO c.c.service.AccessLogAlertService - 没有发现新的日志数据，上次处理时间: 2026-03-09T18:22:00.236
2026-03-09 18:26:00.656 [scheduling-7] INFO com.common.service.DataLoader - 告警数据入库完成，成功: 1 条，总数: 1 条
2026-03-09 18:26:00.668 [scheduling-10] INFO c.c.s.DeviceCollectTaskUpdateService - 批量更新完成，总计: 48，已更新: 1
2026-03-09 18:26:00.668 [scheduling-10] INFO c.c.service.DeviceStatsUpdateService - 设备采集探针任务时间更新完成，耗时: 439ms
2026-03-09 18:26:00.737 [scheduling-7] INFO com.common.service.DataExtractor - 分组数据处理进度: 1/1 (100.00%)
2026-03-09 18:26:00.737 [scheduling-7] INFO com.common.service.DataExtractor - 分组数据处理完成，共处理 1 组数据
2026-03-09 18:26:00.737 [scheduling-7] INFO com.common.schedule.ETLOrchestrator - 定时ETL任务执行完成，耗时: 0 秒
2026-03-09 18:26:00.737 [scheduling-7] INFO c.c.s.NormalizeRuleHitTimeService - 开始执行泛化规则命中时间更新任务，时间：2026-03-09T18:26:00.737
2026-03-09 18:26:01.294 [scheduling-7] INFO c.c.s.NormalizeRuleHitTimeService - 从 syslog_normal_data 表统计到 1 条规则命中记录
2026-03-09 18:26:01.294 [scheduling-7] INFO c.c.s.NormalizeRuleHitTimeService - 从 syslog_normal_alarm 表统计到 1 条规则命中记录
2026-03-09 18:26:01.294 [scheduling-7] INFO c.c.s.NormalizeRuleHitTimeService - 合并后需要更新的规则数量：2
2026-03-09 18:26:01.444 [scheduling-7] INFO c.c.s.NormalizeRuleHitTimeService - 当前启用状态的规则数量：173
2026-03-09 18:26:01.444 [scheduling-7] INFO c.c.s.NormalizeRuleHitTimeService - 开始批量更新，规则总数：173，分批数：1
2026-03-09 18:26:01.761 [scheduling-7] INFO c.c.s.NormalizeRuleHitTimeService - 泛化规则命中时间更新任务完成，更新规则数：2，耗时：1024ms
2026-03-09 18:27:00.005 [scheduling-6] INFO c.c.service.AccessLogAlertService - 开始执行访问日志告警处理任务
2026-03-09 18:27:00.005 [log-processor-8] INFO c.c.service.AccessLogAlertService - 开始执行访问日志告警处理任务
2026-03-09 18:27:00.078 [scheduling-5] INFO c.c.service.DeviceStatsUpdateService - 开始执行设备统计更新任务...
2026-03-09 18:27:00.233 [scheduling-6] INFO c.c.service.AccessLogAlertService - 加载了 1 个启用的算法配置
2026-03-09 18:27:00.234 [scheduling-5] INFO c.c.service.DeviceStatsUpdateService - 设备统计更新完成，处理设备数：1，耗时：156ms
2026-03-09 18:27:00.234 [scheduling-5] INFO c.c.service.DeviceStatsUpdateService - 开始执行设备采集探针任务时间更新，时间: 2026-03-09T18:27:00.234
2026-03-09 18:27:00.234 [scheduling-5] INFO c.c.s.DeviceCollectTaskUpdateService - 开始批量更新设备采集任务时间，当前时间: 2026-03-09T18:27:00.234
2026-03-09 18:27:00.480 [scheduling-6] INFO c.c.service.AccessLogAlertService - 没有发现新的日志数据，上次处理时间: 2026-03-09T18:22:00.236
2026-03-09 18:27:00.494 [log-processor-8] INFO c.c.service.AccessLogAlertService - 加载了 1 个启用的算法配置
2026-03-09 18:27:00.704 [scheduling-5] INFO c.c.s.DeviceCollectTaskUpdateService - 批量更新完成，总计: 48，已更新: 1
2026-03-09 18:27:00.704 [scheduling-5] INFO c.c.service.DeviceStatsUpdateService - 设备采集探针任务时间更新完成，耗时: 470ms
2026-03-09 18:27:00.755 [log-processor-8] INFO c.c.service.AccessLogAlertService - 没有发现新的日志数据，上次处理时间: 2026-03-09T18:22:00.236
2026-03-09 18:28:00.004 [scheduling-3] INFO c.c.service.AccessLogAlertService - 开始执行访问日志告警处理任务
2026-03-09 18:28:00.004 [log-processor-9] INFO c.c.service.AccessLogAlertService - 开始执行访问日志告警处理任务
2026-03-09 18:28:00.081 [scheduling-10] INFO c.c.service.DeviceStatsUpdateService - 开始执行设备统计更新任务...
2026-03-09 18:28:00.231 [scheduling-10] INFO c.c.service.DeviceStatsUpdateService - 设备统计更新完成，处理设备数：1，耗时：150ms
2026-03-09 18:28:00.231 [log-processor-9] INFO c.c.service.AccessLogAlertService - 加载了 1 个启用的算法配置
2026-03-09 18:28:00.231 [scheduling-10] INFO c.c.service.DeviceStatsUpdateService - 开始执行设备采集探针任务时间更新，时间: 2026-03-09T18:28:00.231
2026-03-09 18:28:00.231 [scheduling-10] INFO c.c.s.DeviceCollectTaskUpdateService - 开始批量更新设备采集任务时间，当前时间: 2026-03-09T18:28:00.231
2026-03-09 18:28:00.231 [scheduling-3] INFO c.c.service.AccessLogAlertService - 加载了 1 个启用的算法配置
2026-03-09 18:28:00.429 [log-processor-9] INFO c.c.service.AccessLogAlertService - 没有发现新的日志数据，上次处理时间: 2026-03-09T18:22:00.236
2026-03-09 18:28:00.529 [scheduling-3] INFO c.c.service.AccessLogAlertService - 没有发现新的日志数据，上次处理时间: 2026-03-09T18:22:00.236
2026-03-09 18:28:00.637 [scheduling-10] INFO c.c.s.DeviceCollectTaskUpdateService - 批量更新完成，总计: 48，已更新: 1
2026-03-09 18:28:00.638 [scheduling-10] INFO c.c.service.DeviceStatsUpdateService - 设备采集探针任务时间更新完成，耗时: 406ms
2026-03-09 18:28:00.865 [scheduling-8] INFO c.c.s.RealtimeAnalysisScheduler - 执行规则: ruleId=4e134d65-1170-4d20-ab48-77f3fee6a765, ruleName=告警降噪分析规则-测试-V2, nextTime=2026-03-09T18:28, now=2026-03-09T18:28:00.711
2026-03-09 18:28:00.865 [scheduling-8] INFO c.c.s.impl.AnalysisRuleServiceImpl - 执行实时分析规则: ruleId=4e134d65-1170-4d20-ab48-77f3fee6a765
2026-03-09 18:28:01.335 [scheduling-8] INFO c.c.s.impl.RealtimeAnalysisEngine - 滚动窗口查询范围: 窗口大小=5m，查询时间范围=[2026-03-09 18:23:00, 2026-03-09 18:28:00]
2026-03-09 18:28:01.335 [scheduling-8] INFO c.c.s.impl.RealtimeAnalysisEngine - 开始执行实时规则: ruleId=4e134d65-1170-4d20-ab48-77f3fee6a765, ruleName=告警降噪分析规则-测试-V2, batchNo=20260309182801024, windowType=tumble, dataStartTime=2026-03-09 18:23:00, dataEndTime=2026-03-09 18:28:00
2026-03-09 18:28:02.580 [scheduling-8] INFO c.c.s.impl.RealtimeAnalysisEngine - 生成的SQL: SELECT src_ip AS attack_ip,
dest_ip AS victim_ip,
origin_event_name AS alarm_name,
ARRAY_AGG(DISTINCT src_port) AS attack_port,
ARRAY_AGG(DISTINCT dest_port) AS victim_port,
MAX(event_level) AS alarm_level,
MODE() WITHIN GROUP (ORDER BY dest_domain) AS dns_info,
MODE() WITHIN GROUP (ORDER BY origin_event_type) AS alarm_type,
COUNT(dest_ip) AS log_count,
MAX(attack_result) AS attack_result,
ARRAY_AGG(DISTINCT http_req_header) AS http_req_header,
ARRAY_AGG(DISTINCT http_req_body) AS http_req_body,
ARRAY_AGG(DISTINCT http_resp_header) AS http_resp_header,
ARRAY_AGG(DISTINCT http_resp_body) AS http_resp_body,
ARRAY_AGG(DISTINCT http_url) AS victim_web_url,
ARRAY_AGG(DISTINCT id) AS origin_log_ids,
MIN(log_time) AS log_start_at,
MAX(log_time) AS log_end_at,
ARRAY_AGG(DISTINCT device_id) AS device_id,
ARRAY_AGG(DISTINCT payload) AS payload,
TUMBLE(log_time, INTERVAL '5 MINUTE') AS window_time
FROM syslog_normal_alarm AS t
WHERE log_time >= '2026-03-09 18:23:00' AND log_time < '2026-03-09 18:28:00' AND src_ip != '127.0.0.1' AND event_level >= 1
GROUP BY src_ip, dest_ip, origin_event_name, TUMBLE(log_time, INTERVAL '5 MINUTE')
初次提交代码
2026-01-11 15:33:22 +08:00
关联分析规则-数据降噪
2026-03-18 18:00:25 +08:00
2026-03-09 18:28:03.047 [scheduling-8] INFO c.c.s.impl.RealtimeAnalysisEngine - 规则执行成功: ruleId=4e134d65-1170-4d20-ab48-77f3fee6a765, processedCount=0, alarmCount=0
2026-03-09 18:28:03.362 [scheduling-8] INFO c.c.s.i.RuleExecutionTimeServiceImpl - 更新规则下次执行时间，ruleId=4e134d65-1170-4d20-ab48-77f3fee6a765, ruleName=告警降噪分析规则-测试-V2, windowType=tumble, nextExecuteTime=2026-03-09 18:33:00
2026-03-09 18:28:03.362 [scheduling-8] INFO c.c.s.RealtimeAnalysisScheduler - 本次调度执行规则数: 1, 跳过规则数: 0
2026-03-09 18:29:00.006 [scheduling-5] INFO c.c.service.AccessLogAlertService - 开始执行访问日志告警处理任务
2026-03-09 18:29:00.006 [log-processor-10] INFO c.c.service.AccessLogAlertService - 开始执行访问日志告警处理任务
2026-03-09 18:29:00.081 [scheduling-2] INFO c.c.service.DeviceStatsUpdateService - 开始执行设备统计更新任务...
2026-03-09 18:29:00.236 [scheduling-5] INFO c.c.service.AccessLogAlertService - 加载了 1 个启用的算法配置
2026-03-09 18:29:00.236 [log-processor-10] INFO c.c.service.AccessLogAlertService - 加载了 1 个启用的算法配置
2026-03-09 18:29:00.240 [scheduling-2] INFO c.c.service.DeviceStatsUpdateService - 设备统计更新完成，处理设备数：1，耗时：159ms
2026-03-09 18:29:00.240 [scheduling-2] INFO c.c.service.DeviceStatsUpdateService - 开始执行设备采集探针任务时间更新，时间: 2026-03-09T18:29:00.240
2026-03-09 18:29:00.240 [scheduling-2] INFO c.c.s.DeviceCollectTaskUpdateService - 开始批量更新设备采集任务时间，当前时间: 2026-03-09T18:29:00.240
2026-03-09 18:29:00.487 [scheduling-5] INFO c.c.service.AccessLogAlertService - 没有发现新的日志数据，上次处理时间: 2026-03-09T18:22:00.236
2026-03-09 18:29:00.488 [log-processor-10] INFO c.c.service.AccessLogAlertService - 没有发现新的日志数据，上次处理时间: 2026-03-09T18:22:00.236
2026-03-09 18:29:00.702 [scheduling-2] INFO c.c.s.DeviceCollectTaskUpdateService - 批量更新完成，总计: 48，已更新: 1
2026-03-09 18:29:00.702 [scheduling-2] INFO c.c.service.DeviceStatsUpdateService - 设备采集探针任务时间更新完成，耗时: 462ms
2026-03-09 18:30:00.005 [scheduling-2] INFO c.c.service.AccessLogAlertService - 开始执行访问日志告警处理任务
2026-03-09 18:30:00.005 [log-processor-1] INFO c.c.service.AccessLogAlertService - 开始执行访问日志告警处理任务
2026-03-09 18:30:00.081 [scheduling-8] INFO c.c.service.DeviceStatsUpdateService - 开始执行设备统计更新任务...
2026-03-09 18:30:00.233 [scheduling-2] INFO c.c.service.AccessLogAlertService - 加载了 1 个启用的算法配置
2026-03-09 18:30:00.233 [log-processor-1] INFO c.c.service.AccessLogAlertService - 加载了 1 个启用的算法配置
2026-03-09 18:30:00.235 [scheduling-8] INFO c.c.service.DeviceStatsUpdateService - 设备统计更新完成，处理设备数：1，耗时：154ms
2026-03-09 18:30:00.235 [scheduling-8] INFO c.c.service.DeviceStatsUpdateService - 开始执行设备采集探针任务时间更新，时间: 2026-03-09T18:30:00.235
2026-03-09 18:30:00.235 [scheduling-8] INFO c.c.s.DeviceCollectTaskUpdateService - 开始批量更新设备采集任务时间，当前时间: 2026-03-09T18:30:00.235
2026-03-09 18:30:00.430 [log-processor-1] INFO c.c.service.AccessLogAlertService - 没有发现新的日志数据，上次处理时间: 2026-03-09T18:22:00.236
2026-03-09 18:30:00.501 [scheduling-2] INFO c.c.service.AccessLogAlertService - 没有发现新的日志数据，上次处理时间: 2026-03-09T18:22:00.236
2026-03-09 18:30:00.639 [scheduling-8] INFO c.c.s.DeviceCollectTaskUpdateService - 批量更新完成，总计: 48，已更新: 1
2026-03-09 18:30:00.639 [scheduling-8] INFO c.c.service.DeviceStatsUpdateService - 设备采集探针任务时间更新完成，耗时: 404ms
2026-03-09 18:31:00.006 [scheduling-3] INFO c.c.service.AccessLogAlertService - 开始执行访问日志告警处理任务
2026-03-09 18:31:00.006 [log-processor-2] INFO c.c.service.AccessLogAlertService - 开始执行访问日志告警处理任务
2026-03-09 18:31:00.006 [scheduling-8] INFO com.common.schedule.ETLOrchestrator - ETL任务开始执行，开始时间：2026-03-09 18:25:00,结束时间：2026-03-09 18:30:00
2026-03-09 18:31:00.006 [scheduling-8] INFO com.common.service.DataExtractor - 开始处理告警类型指定时间范围内数据，时间范围: 2026-03-09T18:25 - 2026-03-09T18:30
2026-03-09 18:31:00.084 [scheduling-10] INFO c.c.service.DeviceStatsUpdateService - 开始执行设备统计更新任务...
2026-03-09 18:31:00.235 [scheduling-8] INFO com.common.service.DataExtractor - 指定时间范围分组数据量: 0 组
2026-03-09 18:31:00.235 [log-processor-2] INFO c.c.service.AccessLogAlertService - 加载了 1 个启用的算法配置
2026-03-09 18:31:00.235 [scheduling-3] INFO c.c.service.AccessLogAlertService - 加载了 1 个启用的算法配置
2026-03-09 18:31:00.235 [scheduling-8] INFO com.common.service.DataExtractor - 没有需要处理的数据
2026-03-09 18:31:00.235 [scheduling-8] INFO com.common.schedule.ETLOrchestrator - 定时ETL任务执行完成，耗时: 0 秒
2026-03-09 18:31:00.235 [scheduling-8] INFO c.c.s.NormalizeRuleHitTimeService - 开始执行泛化规则命中时间更新任务，时间：2026-03-09T18:31:00.235
2026-03-09 18:31:00.236 [scheduling-10] INFO c.c.service.DeviceStatsUpdateService - 设备统计更新完成，处理设备数：1，耗时：152ms
2026-03-09 18:31:00.236 [scheduling-10] INFO c.c.service.DeviceStatsUpdateService - 开始执行设备采集探针任务时间更新，时间: 2026-03-09T18:31:00.236
2026-03-09 18:31:00.236 [scheduling-10] INFO c.c.s.DeviceCollectTaskUpdateService - 开始批量更新设备采集任务时间，当前时间: 2026-03-09T18:31:00.236
2026-03-09 18:31:00.515 [scheduling-3] INFO c.c.service.AccessLogAlertService - 没有发现新的日志数据，上次处理时间: 2026-03-09T18:22:00.236
2026-03-09 18:31:00.519 [log-processor-2] INFO c.c.service.AccessLogAlertService - 没有发现新的日志数据，上次处理时间: 2026-03-09T18:22:00.236
2026-03-09 18:31:00.629 [scheduling-10] INFO c.c.s.DeviceCollectTaskUpdateService - 批量更新完成，总计: 48，已更新: 1
2026-03-09 18:31:00.629 [scheduling-10] INFO c.c.service.DeviceStatsUpdateService - 设备采集探针任务时间更新完成，耗时: 393ms
2026-03-09 18:31:00.758 [scheduling-8] INFO c.c.s.NormalizeRuleHitTimeService - 从 syslog_normal_data 表统计到 1 条规则命中记录
2026-03-09 18:31:00.758 [scheduling-8] INFO c.c.s.NormalizeRuleHitTimeService - 从 syslog_normal_alarm 表统计到 1 条规则命中记录
2026-03-09 18:31:00.758 [scheduling-8] INFO c.c.s.NormalizeRuleHitTimeService - 合并后需要更新的规则数量：2
2026-03-09 18:31:00.910 [scheduling-8] INFO c.c.s.NormalizeRuleHitTimeService - 当前启用状态的规则数量：173
2026-03-09 18:31:00.910 [scheduling-8] INFO c.c.s.NormalizeRuleHitTimeService - 开始批量更新，规则总数：173，分批数：1
2026-03-09 18:31:00.910 [scheduling-8] INFO c.c.s.NormalizeRuleHitTimeService - 泛化规则命中时间更新任务完成，更新规则数：0，耗时：675ms
2026-03-09 18:32:00.001 [scheduling-8] INFO c.c.service.AccessLogAlertService - 开始执行访问日志告警处理任务
2026-03-09 18:32:00.001 [log-processor-3] INFO c.c.service.AccessLogAlertService - 开始执行访问日志告警处理任务
2026-03-09 18:32:00.077 [scheduling-1] INFO c.c.service.DeviceStatsUpdateService - 开始执行设备统计更新任务...
2026-03-09 18:32:00.226 [scheduling-8] INFO c.c.service.AccessLogAlertService - 加载了 1 个启用的算法配置
2026-03-09 18:32:00.226 [log-processor-3] INFO c.c.service.AccessLogAlertService - 加载了 1 个启用的算法配置
2026-03-09 18:32:00.232 [scheduling-1] INFO c.c.service.DeviceStatsUpdateService - 设备统计更新完成，处理设备数：1，耗时：155ms
2026-03-09 18:32:00.232 [scheduling-1] INFO c.c.service.DeviceStatsUpdateService - 开始执行设备采集探针任务时间更新，时间: 2026-03-09T18:32:00.232
2026-03-09 18:32:00.233 [scheduling-1] INFO c.c.s.DeviceCollectTaskUpdateService - 开始批量更新设备采集任务时间，当前时间: 2026-03-09T18:32:00.233
2026-03-09 18:32:00.461 [scheduling-8] INFO c.c.service.AccessLogAlertService - 没有发现新的日志数据，上次处理时间: 2026-03-09T18:22:00.236
2026-03-09 18:32:00.505 [log-processor-3] INFO c.c.service.AccessLogAlertService - 没有发现新的日志数据，上次处理时间: 2026-03-09T18:22:00.236
2026-03-09 18:32:00.640 [scheduling-1] INFO c.c.s.DeviceCollectTaskUpdateService - 批量更新完成，总计: 48，已更新: 1
2026-03-09 18:32:00.640 [scheduling-1] INFO c.c.service.DeviceStatsUpdateService - 设备采集探针任务时间更新完成，耗时: 408ms
2026-03-09 18:33:00.002 [scheduling-10] INFO c.c.service.AccessLogAlertService - 开始执行访问日志告警处理任务
2026-03-09 18:33:00.002 [log-processor-4] INFO c.c.service.AccessLogAlertService - 开始执行访问日志告警处理任务
2026-03-09 18:33:00.078 [scheduling-3] INFO c.c.service.DeviceStatsUpdateService - 开始执行设备统计更新任务...
2026-03-09 18:33:00.228 [log-processor-4] INFO c.c.service.AccessLogAlertService - 加载了 1 个启用的算法配置
2026-03-09 18:33:00.228 [scheduling-10] INFO c.c.service.AccessLogAlertService - 加载了 1 个启用的算法配置
2026-03-09 18:33:00.230 [scheduling-6] INFO c.c.s.RealtimeAnalysisScheduler - 执行规则: ruleId=4e134d65-1170-4d20-ab48-77f3fee6a765, ruleName=告警降噪分析规则-测试-V2, nextTime=2026-03-09T18:33, now=2026-03-09T18:33:00.002
2026-03-09 18:33:00.230 [scheduling-6] INFO c.c.s.impl.AnalysisRuleServiceImpl - 执行实时分析规则: ruleId=4e134d65-1170-4d20-ab48-77f3fee6a765
2026-03-09 18:33:00.232 [scheduling-3] INFO c.c.service.DeviceStatsUpdateService - 设备统计更新完成，处理设备数：1，耗时：154ms
2026-03-09 18:33:00.232 [scheduling-3] INFO c.c.service.DeviceStatsUpdateService - 开始执行设备采集探针任务时间更新，时间: 2026-03-09T18:33:00.232
2026-03-09 18:33:00.232 [scheduling-3] INFO c.c.s.DeviceCollectTaskUpdateService - 开始批量更新设备采集任务时间，当前时间: 2026-03-09T18:33:00.232
2026-03-09 18:33:00.426 [scheduling-10] INFO c.c.service.AccessLogAlertService - 没有发现新的日志数据，上次处理时间: 2026-03-09T18:22:00.236
2026-03-09 18:33:00.494 [log-processor-4] INFO c.c.service.AccessLogAlertService - 没有发现新的日志数据，上次处理时间: 2026-03-09T18:22:00.236
2026-03-09 18:33:00.634 [scheduling-3] INFO c.c.s.DeviceCollectTaskUpdateService - 批量更新完成，总计: 48，已更新: 1
2026-03-09 18:33:00.634 [scheduling-3] INFO c.c.service.DeviceStatsUpdateService - 设备采集探针任务时间更新完成，耗时: 402ms
2026-03-09 18:33:00.688 [scheduling-6] INFO c.c.s.impl.RealtimeAnalysisEngine - 滚动窗口查询范围: 窗口大小=5m，查询时间范围=[2026-03-09 18:28:00, 2026-03-09 18:33:00]
2026-03-09 18:33:00.688 [scheduling-6] INFO c.c.s.impl.RealtimeAnalysisEngine - 开始执行实时规则: ruleId=4e134d65-1170-4d20-ab48-77f3fee6a765, ruleName=告警降噪分析规则-测试-V2, batchNo=20260309183300381, windowType=tumble, dataStartTime=2026-03-09 18:28:00, dataEndTime=2026-03-09 18:33:00
2026-03-09 18:33:01.943 [scheduling-6] INFO c.c.s.impl.RealtimeAnalysisEngine - 生成的SQL: SELECT src_ip AS attack_ip,
dest_ip AS victim_ip,
origin_event_name AS alarm_name,
ARRAY_AGG(DISTINCT src_port) AS attack_port,
ARRAY_AGG(DISTINCT dest_port) AS victim_port,
MAX(event_level) AS alarm_level,
MODE() WITHIN GROUP (ORDER BY dest_domain) AS dns_info,
MODE() WITHIN GROUP (ORDER BY origin_event_type) AS alarm_type,
COUNT(dest_ip) AS log_count,
MAX(attack_result) AS attack_result,
ARRAY_AGG(DISTINCT http_req_header) AS http_req_header,
ARRAY_AGG(DISTINCT http_req_body) AS http_req_body,
ARRAY_AGG(DISTINCT http_resp_header) AS http_resp_header,
ARRAY_AGG(DISTINCT http_resp_body) AS http_resp_body,
ARRAY_AGG(DISTINCT http_url) AS victim_web_url,
ARRAY_AGG(DISTINCT id) AS origin_log_ids,
MIN(log_time) AS log_start_at,
MAX(log_time) AS log_end_at,
ARRAY_AGG(DISTINCT device_id) AS device_id,
ARRAY_AGG(DISTINCT payload) AS payload,
TUMBLE(log_time, INTERVAL '5 MINUTE') AS window_time
FROM syslog_normal_alarm AS t
WHERE log_time >= '2026-03-09 18:28:00' AND log_time < '2026-03-09 18:33:00' AND src_ip != '127.0.0.1' AND event_level >= 1
GROUP BY src_ip, dest_ip, origin_event_name, TUMBLE(log_time, INTERVAL '5 MINUTE')
初次提交代码
2026-01-11 15:33:22 +08:00
关联分析规则-数据降噪
2026-03-18 18:00:25 +08:00
2026-03-09 18:33:02.410 [scheduling-6] INFO c.c.s.impl.RealtimeAnalysisEngine - 规则执行成功: ruleId=4e134d65-1170-4d20-ab48-77f3fee6a765, processedCount=0, alarmCount=0
2026-03-09 18:33:02.717 [scheduling-6] INFO c.c.s.i.RuleExecutionTimeServiceImpl - 更新规则下次执行时间，ruleId=4e134d65-1170-4d20-ab48-77f3fee6a765, ruleName=告警降噪分析规则-测试-V2, windowType=tumble, nextExecuteTime=2026-03-09 18:38:00
2026-03-09 18:33:02.718 [scheduling-6] INFO c.c.s.RealtimeAnalysisScheduler - 本次调度执行规则数: 1, 跳过规则数: 0
Reference in New Issue Copy Permalink