Environmental Information and Multi-sensor Data Fusion Based Performance Estimations for Smart Cars | Consortium for Embedded Systems | SIU

Abstract: The rapid growth of on-vehicle multi-sensor inputs along with off-vehicle data streams provides an opportunity for innovation in real-time decision making. The development of new advanced sensors is not sufficient enough without the utilization of enhanced signal processing techniques such as the data fusion methods. Multi-sensor data fusion (MSDF) is the process of combining or integrating measured or preprocessed data or information originating from different active or passive sensors. Besides the combination of sensors towards an automotive multi-sensor system, complex signal processing and sensor data fusion strategies are of remarkable importance for the availability and robustness of the overall system. As vehicles become autonomous with personal data platform concepts such as intelligent agents and hypervisors are critical for a robust foundation. A lean design becomes the spring-board for introducing future connectivity and mobility features in a scalable manner. In this project, we aim to develop and explore state-of-art data fusion techniques for decision making for automotive applications. Specifically we will combine the benefits offered by car’s increased connectivity (e.g. internet, cloud services) with on-vehicle sensor information for providing detailed information about the state of the car and the environment.

Problem: In modern cars, data provided by sensors and agents is always affected by some level of impreciseness as well as uncertainty in the measurements and in many cases this information can also be controversial. Also, as the number of information sources increases the amount of data to be processed increases as well. For that reason, data fusion strategies are of remarkable importance for the availability and robustness of the overall system. With the increasing number of information sources available in today's cars, estimation of friction coefficient is a challenge. Besides using only on-vehicle information, the car can also utilize information obtained by different sources (e.g. other vehicles, internet).

Rationale / Approach: In this project, we couple the concept of multi-agent systems with data fusion techniques in order to develop a distributed adaptive and reliable decision making system for automotive applications. This project will focus on multi-sensor data fusion techniques over a multi-agent system that has been designed to provide service flexibility for modern cars. The most important thing offered by agents is connectivity. The increased connectivity offered by autonomous intelligent agents provide more sources of information. In this project we propose an MSDF methodology for processing multi-sensor data from different types of sensors and information inputs. Specifically, the proposed methodology will: (i) Collect data and inputs from on-vehicle sensors and environmental sources; (ii) Merge direct estimations with information from multiple on vehicle sensors as well as environmental information; (iii) Explore and address static and dynamic changes both on the vehicle and the environment.; and (iv) Explore adaptive learning (historical data) methodologies and compare data and model driven techniques.

Novelty: The innovative aspects of the project are many-fold: (i) Multi-sensor data fusion techniques will explore static and dynamic changes both on the vehicle and the environment; (ii) Employ reflective and deliberative information gathering; (iii) The estimation of friction coefficient will be used as a use case. Specifically the project will investigate how the estimation of coefficient of friction depends and varies on different information sources (sensors, vehicle-to-vehicle, information from cloud, historical data); and (iv) Propose an MSDF framework for fusing data from different types of sensors and inputs. Explore adaptive learning methodologies and compare data and model driven techniques.

Potential Member Company Benefits: The project will focus on the integration and analysis of information from multiple sensors including static and dynamic changes (vehicle and the environment). These techniques will be employed over a multi-agent system especially designed for on-vehicle communication. The benefits are: (i) new techniques for real-time data fusion will be developed, tested and evaluated; (ii) The use case for the proposed project is the estimation of coefficient of friction; (iii) The concept of multi-sensor data fusion on top of a multi-agent system will be evaluated.

Deliverables for the proposed year: (i) Comprehensive report on the developed techniques including details about the implemented algorithms and tools. The report will also include the results regarding the selected use case. (ii) A simulation demo with a lot of different scenarios that cover a great variety of results from the selected use case.

Milestones for the proposed year: Q1: Review of data fusion method and extraction of friction models. Q2: Design and implementation of the data fusion techniques targeting coefficient of friction. Q3: Evaluation of estimator performance in simulations. Q4: Evaluation of the estimator performance with experimental data