南京城交院 Automated traffic incidents detection 东南大学交通学院 陈淑燕 教授 博导 School of Transportation, Southeast University Dr. Prof. Shuyan Chen 1. Definition of Traffic Incident and Significance of Incident Detection 一、交通事件定义及事件检测意义 3. Algorithms of Automatic Incident Detection 二、交通事件自动检测算法 4. Automatic Incident Detection on Urban Roads 三、城市道路交通事件自动检测 5. Future Work 四、未来的工作 一、交通事件定义及事件检测意义 Definition of Traffic Incident and Significance of Incident Detection 定义:交通事件是指导致道路通 行能力下降或交通需求不正常升 高的非周期性发生的情况。 车辆抛锚 vehicle broke down Definition:Traffic incidents are defined as unusual events that reduce the capacity of network links to carry traffic, or 抛洒物 spilled loads raise the traffic demand abnormally. 交通事故 collision, crash 分类:可预测、不可预测。 Type:predicted, unpredicted 可预测事件 不可预测事件 predicted unpredicted 道路养护 事故 road maintenance 道路修筑 车辆抛锚 road construction accident Vehicle broke down severe weather (rain、snow、 ice、fog) collapse of bridges or road 大型活动(体育比赛、 恶劣天气(雨、雪、 游行、音乐会等) 冰、雾) 桥梁或道路坍塌 货物散落 Large-scale activities (Sports games 、 Parade 、concert etc.) spilled loads 不利影响: 交通事件造成巨大的经济损失和人员伤亡 交通事件是导致道路拥堵的重要原因之一 交通事件导致二次事故,降低道路安全水平 Adverse effect Incident cause economic losses, and personal injuries; Main reason which cause traffic congestion; Cause the second incidents, reduce the road safety level; 1 近年来 • 20%-50%的交 通事故是由于受 的统计 到已发生交通事 数据 件(称为一次事件) 的影响而造成的, 2 • 50%以上的二次 • 在城市道路上, 事故是在一次事 交通事故和车辆 件发生后10分钟 抛锚导致的交通 内发生的 拥堵占全部拥堵 20%,而且这类 这些随后发生的 拥堵持续的时间 事故被称为二次 更长 事故。 Statistical data 20%-50% second accidents were caused by the first incidents 3 50% second accidents happened within 10 minutes after the first incidents On urban road, traffic jam caused by accidents and broke down account to 20%, and last much time. 尽早发现道路上的突发事件,便 Traffic incident detection can help the 于管理部门及时做出事件响应, manage department take actions to 将损失降到最低。 clear incident and reduce the losses. 如果提前1分钟发现并清理交通 事件,至少可减少4~5分钟的延 误。 early 1 minute early detected reduce 4 to 5 minutes delay. 各国交通工程专家纷纷积极研究和开发运行成本低,能够全天候、全程发挥 作用,检测率高的自动事件检测技术。 Experts have been studied and developed Automatic Incident Detection techniques (AID) which cost less, work 24hours with high detection rate. 现代化交通监控中心 1.3 交通事件自动检测系统 Automatic Incident Detection Systems System Function 系统功能 接收从交通信息采集系统传来的数据 判断有无事件发生 发出事件警报 生成事件处理策略 实时交通 感知 AID算法 是 Receives real time information from sensors; Makes a decision whether an incident happening or not; If so, give the alarms, and formulating effective response strategy; 事件警报 事件清理 否 Real data AID Give alarm Clear incident 1.3 交通事件自动检测系统 Automatic Incident Detection Systems 系统组成 System structure 实时交通信息采集子系统 / 检测器 Real time collecting sub-system/ sensors 事件自动检测算法子系统(核心部分) Incident detection algorithms ( core ) 数据管理子系统 Data management sub-system WEB系统 修改配置/显示数据 数采任 务管理 数采点 管理 系统管理 参数配 置管理 路线路 段管理 数据日志管理 数据库 获取配置/输出结果 数据处理系统 获取数据/输出结果 参数获取 实时数据处理 数据输出 JMS 获取配置/上传数据 终端采集系统 采集参数获取 数据采集 数据上传 数据源 采集数据 终端1 终端2 …… Algorithms of Automatic Incident Detection 检测率(Detection Rate,DR): 算法检测的实际事件总数 DR 100% 所有发生的实际事件总数. DR number of incident cases detected total number of incident cases 误报率(False Alarm Rate,FAR): FAR 误报的交通事件总数 100% 给定时间段内所有决策次数. number of false alarmcases FAR totalnumber of input instances 平均检测时间(Mean Time To Detection,MTTD): 1 n MTTD 算法检测到事件i的时间 事件i实际发生的时间 n i 1 1 n MTTD the timethealarmis initiated- thestart of theith incident n i 1 AID算法大致可分为: AID algorithms including : 基于模式识别的算法(比较算法) pattern recognition 基于统计技术的算法 statistics techniques 基于交通流模型的算法 traffic flow theory 基于时间序列的算法 基于数据挖掘的算法 …… time series and filtering analysis Data mining … 决策树方法 Decision Tree Learning 粗集分类 Rough Sets Classification 偏最小二乘回归法 Partial Least Squares Regression 归纳逻辑法 Inductive logic programming 基于集成学习的方法 Ensemble learning 偏最小二乘神经网络 Partial Least Squares Neural Network 神经网络集成技术 Neural Network Ensemble,NNE 支持向量机及集成技术 Support Vector Machine Ensemble, SVM 基于随机森林的交通事件检测算法 Random Forest 基于SMOTE的事件自动检测算法 基于SVDD算法的事件检测算法 Synthetic Minority Over-Sampling Technique,SMOTE Support Vector Domain Description 基于决策树的AID算法( Decision Tree Learning Based AID Algorithm) 动机 Motivation 事件检测可以视为二分类问题 incident detection can be viewed as a 决策树是一个分类模型 classification problem with 分类规则易解释 交通工程师易理解 不同变量的相对重要性 two desired output classes Decision tree is a classifier easily interpret the rules, easy understood by traffic engineers bring to light the relative importance of different variables 基于决策树的AID算法( Decision Tree Learning Based AID Algorithm) Tab. Comparison of three AID algorithms be st Algorithms DR (%) FAR (%) MTTD CR (%) (minutes) BPNN 86.00 2.08 4.63 84.40 RBFNN 90.00 8.40 3.27 82.23 C4.5 90.00 2.72 4.44 83.43 1 1 0.8 true positive detection rate 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 false alarm rate 0.8 0.4 0.2 c4.5 BP RBF 0 0.6 0 1 Fig. detection rate versus false alarm rate c4.5 BP RBF 0 0.2 0.4 0.6 false positive 0.8 1 Fig. Comparison among ROC curves 1. 基于决策树的AID算法( Decision Tree Learning Based AID Algorithm) 结论 Conclusions 检测效果与神经网络相当 Decision trees learning based AID algorithm 属性离散化可以 can provide comparable performance to neural 提高检测率 但误报率上升 实例选择可以提高检测效果 缺点:移植性较差 network Attributes discretizing improve DR, but FAR deteriorated Instance selection can enhance the performance Poor transferability 原理 Ideas 不可分辨性 Indiscernibility Rough set approximations Classification rules 粗集近似集 分类规则 算法步骤 属性离散化处理 Steps Discretize attributes . 属性削减 Attributes Reduction 决策规则生成 Generate a set of decision rules. 规则削减 判断交通状态 Rule reduction Identify traffic states for a testing set. 模拟数据实验结果 RS Experiment with simulated data DR(%) FAR (%) No rule choose 98.67 15.27 2.32 77.13 Rule choose 97.33 3.53 2.89 73.52 99.33 25.83 2.09 76.03 SVM MTTD (minutes) CR (%) 实测数据实验结果 Experiment with I-880 data RS DR (%) FAR (%) No rule choose 86.96 1.91 3.15 94.54 Rule choose 95.65 2.65 2.95 94.15 78.26 1.63 3.50 96.31 SVM MTTD CR (%) (minutes) 3. 偏最小二乘神经网络算法( Partial Least Squares Neural Network Based AID Algorithm) 神经网络 Neural Network 处理非线性问题有优势; Architecture and parameter setting for NN; 结构和参数难以设置; The performance of NN relates highly to the 性能依赖于使用者的经验; experience of the users. 最小二乘算法 PLSR 不擅长处理非线性问题; Inappropriate for constructing nonlinear prediction 不适用于不平衡数据; 结合两者? 用偏最小二乘算法确定神经网络的结构和 权值 equations; Performs poor when used on the imbalanced data; Combine? Use PLSR to optimize the structure and parameter settings for NN 3. 偏最小二乘神经网络算法(Partial Least Squares Neural Network Based AID Algorithm) PLSR X t1 p1T t2 p2T Y q1t1 q2t 2 tr prT Er qr tr Fr Combine PLSR and NN p1 x1 x2 p2 pr xm Input layer t1 q 1 t2 tr q2 y Output layer qr Hidden layer 结构 neural network 1~m combine 神经网络1 original data 原始数据 神经网络2 …… 神经网络m 集 成 各 网 络 输 出 分类结果 神经网络的生成 :Bagging、Boosting M f ( x)) majority voting: f e ( x) sgn( 输出结果的集成: k k 1 M weighted voting: f e ( x) sgn( wk f k ( x)) k 1 4. 基于神经网络集成的AID算法(Neural Network Ensemble Based AID Algorithm) 结论(Conclusion) it is really attractive to user who lack experience of neural network design ensembles is better than individual networks in most cases 1 1 0.8 0.8 true positive rate true positive rate take the advantage of possibility to combine the members member ensemble 0.6 0.4 0.2 0 member ensemble 0.6 0.4 0.2 0 0.1 0.2 0.3 0.4 0.5 0.6 false positive rate 0.7 0.8 0.9 Fig.2 Comparison of ROC curve between the ensembles and one of its members 1 0 0 0.1 0.2 0.3 0.4 0.5 0.6 false positive rate 0.7 0.8 0.9 1 Fig. 2 Comparison of ROC curve between ensembles and one of its members. 随机森林是一个包含多个决策树 Random forests are an ensemble 的分类器,让所有树参加分类, learning method for classification 最终分类结果,根据一定的规则 (and regression) that operate by 如多数表决等,由决策树输出的 constructing a multitude of decision 众数而定。 trees at training time and outputting the class that is the mode of the classes output by individual trees. Tree 1 Tree 2 Data set …… Tree m majority vote class 基于SMOTE算法的交通事件自动检测算法(SMOTE-SVM-AID) 利用SMOTE(Synthetic Minority Over-Sampling Technique )算法对少数类的事件数据集进 行过抽样预处理,降低数据集的 不平衡性; 再以支持向量机SVM(Support The SMOTE (Synthetic Minority Over-sampling Technique) algorithm was used to reconstruct and balance the training set ; Then the SVM (Support Vector Vector Machine)作为分类器对 Machine) was adopted as the 处理后的交通流数据进行分类, classifier to detect traffic incidents 从而实现事件检测。 . 6. 面向不平衡数据集的AID算法(Imbalanced Datasets Based AID Algorithm) 基于SMOTE算法的交通事件自动检测算法(SMOTE-SVM-AID) Tab. experimental result FAR MTTD (%) (min) Training set DR(%) Original set 85.71 3.31 3.28 91.43 3.95 2..39 91.43 4.08 2.16 Reconstruct ed set 1 Reconstruct ed set 2 结果表明,基于SMOTE-SVM-AID 算法可以提高检测率,减少平均 检测时间. SMOTE-SVM can improve the detection rate and reduce the mean time to detection. 三、城市道路交通事件检测 Automatic Incident Detection on Urban Roads 大多数AID算法都是针对高速公路或城 Most researches are on freeway, few 市快速路,对于城市道路的交通事件检 research about incident detection on 测研究较少。 urban road. 原因 城市道路交通流构成复杂,交通流 局部不守恒性,交通流量潮汐性, 增加了城市交通事件检测难度。 人工判别快速、准确 发展趋势 城市路网规模的扩大,道路交通需 求的增长,人工方法已无法满足交 通管理的需求,需要研究城市道路 交通事件自动检测技术。 Reason traffic flow is more complex on urban roads; artificial discrimination is more Convenient; Tendency With the development of city scale and improvement of traffic demand, artificial method has been failed to meet the demand of traffic management, AID on urban road is needed. 城市道路接入点多,车辆频繁进出,导致路段上的交通流不 具有守恒性; 由于交叉口渠化,转向车辆在进入交叉口前变换车道; 在信控交叉口,由于受交通信号控制阻滞; 车辆速度不稳定; 高峰时刻路段车辆排队等候通行。 城市道路交通流特点增加了交通事件判断难度! TOO many access point, vehicles come and go frequently, traffic flow is non-conservation; vehicles change lanes when approaching the intersection; vehicles block at signal control intersection unstable spee; queue up at rush hours Increased the difficult to detect incident on urban roads! 由于城市道路交通流与高速公路的差 The successfully applied AID 异性,高速公路上成功应用的AID算法 algorithm in highways is not suitable 已不适合城市道路,不能简单地移植 to urban roads due to different traffic 现有AID算法用于城市道路交通事件自 characteristics, the existing AID 动检测,应根据城市道路交通流特点 algorithms should be modified 进行算法改进。 according to the traffic characteristics 注意的问题 检测算法固有的缺陷 交通流数据不均衡 of urban roads. Notes Inherent deficiencies of the detection algorithms; 城市道路交通流特殊性 imbalanced datasets 表征城市道路交通状态的特征参数 traffic special characteristics on urban 选取 roads; characteristic parameters selection; 使用差分方法消除信号控制引入的周期性噪声 协同行程时间,用参数差分流量进行城市路段交通事件检测; 交叉口的交通事件通过视频监控判断的方式检测。 用差分流量代替流量可以有效抑制交通信号对于交通流的周期性影响。 Difference method to eliminate the periodic noise introduced by signal control the differential flow combined with travel time to detect incidents on road sections. Detect incident at intersection By video that the noise introduced by signal control can be well restrain when the differential flow instead of flow was chosen to be the variable. 多源数据融合 把基于浮动车数据和基于感应 Multi-source data fusion Combine the results of incident 线圈数据的事件检测结果进行 detection based on floating car data 决策级融合,以此判断事件发 and the inductive loop detector, to 生与否。 detect incident on urban roads; 该算法包括三个模块: ①感应线圈数据算法模块 ②浮动车数据算法模块 ③决策融合模块 Including 3 modules ①Detection algorithm module based on the inductive loop detection data; ②Detection algorithm module based on Detection algorithm module based on ③Module of decision fusion) Future Work 基于多源数据融合的AID算法 将新技术(物联网、手机定位等 )应用于交通信息采集 对现有多源交通信息(感应线圈 、视频、微波等获取)进行融合 (数据层融合,算法层融合,决 策层融合) 解决交通流数据不平衡问题 提高算法移植性 Multisource data fusion based AID algorithms Information collection (mobile, the internet of things); Data fusion including Data layer fusion, Algorithm layer fusion, Decision layer fusion; Imbalanced traffic datasets Transferability problem 空间 提出道路交通事件影响范围确定方法 ,为交通疏导措施的实施提供依据, 最大程度降低交通事件本身以及实施 交通疏导措施后对周边路网造成的影 Space Determine the area where the traffic flow are affected by incidents. 响。 时间 建立事件持续时间预测模型,为事件 Time Build the incident 发生后的有效诱导交通、及时疏导交 duration prediction 通、尽快恢复交通正常运行提供准确 models for traffic 信息。 guidance, incident cleanup, in order to recover the traffic flow. 事件检测只是事件管理的第一步,迅速进行事件清理可将事件带来的 不利影响降至最低,使交通流恢复正常. 构建城市交通事件疏导案例库 运用聚类、实例学习、决策树等数据挖掘技术研究交通事件清理 对策自动生成的技术和方法; Formulate strategies quickly to minimize the effect of incidents, and recover traffic flow as early as possible. Build case base of urban traffic events cleaning Use data mining techniques such as clustering, learning from examples, decision tree to study technology and methods of automatically generated traffic incident clean-up measures. THANK YOU !