Ifsttar PhD subject

French version
Detailed form :

Title : Fine characterisation of transport environment using image processing: contribution to multi-source localisation of autonomous vehicle

Main host Laboratory - Referent Advisor   -

Director of the main host Laboratory   -

PhD Speciality traitement des images, vision par ordinateur

Axis of the performance contract 1 - COP2017 - Efficient transport and safe travel

Main location Lille-Villeneuve d'Ascq

Doctoral affiliation UNIVERSITE DES SCIENCES ET TECHNOLOGIE DE LILLE 1

PhD school SCIENCES POUR L'INGENIEUR (SPI)

Planned PhD supervisor MARAIS Juliette  -  Université Gustave Eiffel  -  COSYS - LEOST

Planned PhD co-supervisor EL BADAOUI EL NAJJAR Maan  -  Université de Lille  -  UMR CRISTAL

Planned financing Contrat doctoral  - Ifsttar

Abstract

Autonomous train is a key issue for future transport. In Australia, the global mining group Rio Tinto has successfully completed the first fully autonomous rail journey (without drivers) for nearly 100 kms. In France, SNCF is planning first experiments on an autonomous freight train from 2019. To address it, the future autonomous train will be equipped with a multitude of sensors and associated intelligence. The use of optical sensors and cameras is already part of the equipment of road autonomous vehicle, especially for the detection of moving object, or for parking assistance. But they are also considered as part of the future autonomous train for the detection of signalling along the tracks or for obstacles detection: presence of object and people on the tracks. Although we clearly see the contribution of this type of sensor for perception needs, it also represents a key source of information to feed the geolocation solution to be developed for the future autonomous train.

This Ph.D. topic is devoted to this new application with the aim to characterize as finely as possible the transport environment by developing new image processing techniques. The objective is to provide a new safe source of information, extracted from the image, and that can be exploited such as others heterogeneous sensors (lidar, odometer, gyrometer, inertial unit) in a fault-tolerant multi-source fusion system for the continuous and safe geolocalisation of autonomous vehicle. In this thesis, we will seek to: i) Detect and recognize visual landmarks. For that, we will investigate (deep) learning techniques already used for the recognition of road/railway signs [Marmo 2006, IRT SystemX] and matching techniques between real images acquired by the vehicle and those derived from the 3D cartography [Adjrad 2015, Dawood 2016]. We will also explore tree-based recognition techniques [Nister 2006; Schindler 2007] and will adapt them to take into account the temporal aspect and the stress related to our application (fast illumination changes, processing time, variability of points of interests); ii) Characterize as finely as possible the environment surrounding the vehicle by developing new adaptive methods of segmentation/classification of colour/texture information to detect defaults/faults of others sensors [Meurie 2010; Marais 2014]. Literature offers in this domain, a classification of the regions of the image into two classes "sky" VS "no-sky" without any distinction of the type of obstacle. The prime development envisaged and based on an adaptive combination of colour and texture information will allow to refine the characterisation of the objects of interest (vegetation, sky, infrastructure, etc); iii) The addition of a new sensor in the overall system must not lead to new faults and must provide a robust information. For that, we will define quality criteria and confidence criterion of information provided by the image processing approach for integration in a fault-tolerance fusion system [Al Hage 2015, Al Hage 2017]. In this context, the objective consists in developing multi-source fusion approaches based on state estimators/observers with a layer of detection and identification of sensors faults coupled with a step of processing of measurements, signals or observations from sensors. The use of data with a Top-Down approach (Information-Driven techniques) allows, by using informational metrics, to select relevant data and facilitates the distinction between "default" VS "no-default". Special attention will be devoted to the development of methods that allow the detection and isolation of sensor faults but also the identification of the latter in order to reinject the corrected measurement in the fault-tolerant multi-source fusion procedure applied in the context of localization.

This Ph.D. thesis will be co-supervised by researchers from the French institute of science and technology for transport, spatial planning, development and networks (IFSTTAR) and CRIStAL UMR CNRS 9189 (University of Lille 1, France). The IFSTTAR-COSYS-LEOST and CRISTAL-CI2S-DICOT laboratories have multi-vehicle platforms equipped with several cameras, optical sensors, GNSS receivers and other location sensors. The CRISTAL-CI2S-DICOT laboratory has a 3D cartography of several experimental sites on the campus of Lille 1 and the IFSTTAR-COSYS-LEOST benefits from image/video datasets of transport environment (road/rail) acquired in different projects. An approach to test and validate, on real data, the performance of the strategy developed during the thesis, is therefore planned.

The thesis proposed represents a straight continuation of works carried out during the "Contrat d'objectifs et de performance" of IFSTTAR [Marais 2015], tackles the research aspects that will be developed in the unifying projet " mobilités & transitions numériques" and is a part of the major challenge for the transport domain, the future autonomous train. It is also complementary to the research works developed in the framework of the ELSAT-SMARTIES Regional project (Smart, fail-safe communication and positioning system) and the ERSAT GGC European project. Finally, it is also aims to reinforce the collaboration with the DiCOT team of the CI2S group of the CRIStAL laboratory of the University of Lille 1, partner of the ELSAT-SMARTIES project.

This Ph.D. thesis will deal with the disciplines of: image processing, signal processing, data fusion and localization. It will take place at IFSTTAR-COSYS-LEOST (Lille-Villeneuve d'Ascq site) with strong interactions with the DiCOT team of the CI2S group of the CRIStAL laboratory of the University of Lille 1. It will be supervised by C. Meurie (Researcher at IFSTTAR-COSYS-LEOST) - cyril.meurie@ifsttar.fr - and co-directed by J. Marais (Researcher with the habilitation to conduct researches at IFSTTAR-COSYS-LEOST) - juliette.marais@ifsttar.fr - and Mr El Badaoui El Najjar (Professor at UMR CRIStAL / University of Lille 1) - maan.e-elnajjar@univ_lille1.fr -. It is envisaged to publish the research work in three international conferences in the field of image processing (ICIP, ICISP, ICIAP, MICCAI, ECCV) and/or Intelligent Transport Systems (ITSC, IV, ION), and at least in a popular international journal (IEEE on ITS, Transportation Research Part C, Experts systems with applications).

Candidate profile: Master's degree or equivalent. The candidate must have strong skills in mathematics, computer vision, image processing and software development. He (She) should have excellent working knowledge in object-oriented programming (C++) and Python language, and in the use of OpenCv library. The (a) candidate will have to send in electronic format: a detailed CV, the grades and rankings obtained in Master (and previous course if possible), a reasoned opinion of the Head of Master, a scientific annex that will be established after contacting the supervisors indicated above.

References:

[Adjrad, 2015] Adjrad, M., & Groves, P. D. (2015). Enhancing Conventional GNSS Positioning with 3D Mapping without Accurate Prior Knowledge. ION GNSS+ 2015.
[Al Hage, 2015] Al Hage J., N. Aït Tmazirte, M. E. El Najjar and D. Pomorski, “Fault tolerant fusion approach based on information theory applied on GNSS localization” , In proc. of 18th international conference on information Fusion , Fusion 2015, Washington DC; 07/2015
[Al Hage, 2017] Al Hage J., Maan E. El Najjar “Improved Outdoor Localization Based on Weighted Kullback-Leibler Divergence for Measurements Diagnosis”, IEEE Intelligent Transportation Systems Magazine (à paraître).
[Dawood 2016] Maya Dawood, Cindy Cappelle, Maan E. El Najjar, Jing Peng, Mohamad Khalil, Denis Pomorski “Virtual 3D city model as a priori information source of vehicle ego-localization system”, Journal of Transportation Research Part C: Emerging Technologies, Elsevier Publisher 2016 (IF: 3.805).
[Marais, 2014] J. Marais, C. Meurie, D. Attia, Y. Ruichek, A. Flancquart, "Toward accurate localization in guided transport: combining GNSS data and imaging information", Transportation Research Part C: Emerging Technologies (5-Year IF 2011: 2.284), Vol. 43(2), pp. 188–197, June 2014
[Marais, 2015] J. Marais, D. Bétaille & al, Algorithmes embarqués de localisation outdoor pour les véhicules en milieu urbain et algorithme embarqué de fusion GNSS et carte numérique, Livrable du COP, décembre 2015
[Marmo, 2006] Marmo, R., Lombardi, L., Gagliardi, B., “Railway sign detection and classification », In proc. Of IEEE Intelligent Transportation Systems Conference, pp. 1358-1363, 2006
[Meurie, 2010] C. Meurie, Y. Ruichek, A. Cohen, J. Marais, "An hybrid an adaptive segmentation method using color and textural information", SPIE Electronic Imaging 2010 - Image Processing: Machine Vision Applications III, SPIE Vol. 7538, pp. 11, USA, January 2010.
[Nister, 2006] D. Nister, H. Stewenius, "Scalable recognition with a vocabulary tree", In proc. of International Conference on Computer Vision and Pattern Recognition, 2006.
[Schindler, 2007] G. Schindler, M. Brown and R. Szeliski., "City-Scale Location Recognition", In proc. of International Conference on Computer Vision and Pattern Recognition, 2007.

Keywords : images processing, images segmentation, object recognition, signal processing, data fusion, localization

Main host Laboratory - Referent Advisor	-
Director of the main host Laboratory	-
PhD Speciality	traitement des images, vision par ordinateur
Axis of the performance contract	1 - COP2017 - Efficient transport and safe travel
Main location	Lille-Villeneuve d'Ascq
Doctoral affiliation	UNIVERSITE DES SCIENCES ET TECHNOLOGIE DE LILLE 1
PhD school	SCIENCES POUR L'INGENIEUR (SPI)
Planned PhD supervisor	MARAIS Juliette - Université Gustave Eiffel - COSYS - LEOST
Planned PhD co-supervisor	EL BADAOUI EL NAJJAR Maan - Université de Lille - UMR CRISTAL
Planned financing	Contrat doctoral - Ifsttar
Abstract Autonomous train is a key issue for future transport. In Australia, the global mining group Rio Tinto has successfully completed the first fully autonomous rail journey (without drivers) for nearly 100 kms. In France, SNCF is planning first experiments on an autonomous freight train from 2019. To address it, the future autonomous train will be equipped with a multitude of sensors and associated intelligence. The use of optical sensors and cameras is already part of the equipment of road autonomous vehicle, especially for the detection of moving object, or for parking assistance. But they are also considered as part of the future autonomous train for the detection of signalling along the tracks or for obstacles detection: presence of object and people on the tracks. Although we clearly see the contribution of this type of sensor for perception needs, it also represents a key source of information to feed the geolocation solution to be developed for the future autonomous train. This Ph.D. topic is devoted to this new application with the aim to characterize as finely as possible the transport environment by developing new image processing techniques. The objective is to provide a new safe source of information, extracted from the image, and that can be exploited such as others heterogeneous sensors (lidar, odometer, gyrometer, inertial unit) in a fault-tolerant multi-source fusion system for the continuous and safe geolocalisation of autonomous vehicle. In this thesis, we will seek to: i) Detect and recognize visual landmarks. For that, we will investigate (deep) learning techniques already used for the recognition of road/railway signs [Marmo 2006, IRT SystemX] and matching techniques between real images acquired by the vehicle and those derived from the 3D cartography [Adjrad 2015, Dawood 2016]. We will also explore tree-based recognition techniques [Nister 2006; Schindler 2007] and will adapt them to take into account the temporal aspect and the stress related to our application (fast illumination changes, processing time, variability of points of interests); ii) Characterize as finely as possible the environment surrounding the vehicle by developing new adaptive methods of segmentation/classification of colour/texture information to detect defaults/faults of others sensors [Meurie 2010; Marais 2014]. Literature offers in this domain, a classification of the regions of the image into two classes "sky" VS "no-sky" without any distinction of the type of obstacle. The prime development envisaged and based on an adaptive combination of colour and texture information will allow to refine the characterisation of the objects of interest (vegetation, sky, infrastructure, etc); iii) The addition of a new sensor in the overall system must not lead to new faults and must provide a robust information. For that, we will define quality criteria and confidence criterion of information provided by the image processing approach for integration in a fault-tolerance fusion system [Al Hage 2015, Al Hage 2017]. In this context, the objective consists in developing multi-source fusion approaches based on state estimators/observers with a layer of detection and identification of sensors faults coupled with a step of processing of measurements, signals or observations from sensors. The use of data with a Top-Down approach (Information-Driven techniques) allows, by using informational metrics, to select relevant data and facilitates the distinction between "default" VS "no-default". Special attention will be devoted to the development of methods that allow the detection and isolation of sensor faults but also the identification of the latter in order to reinject the corrected measurement in the fault-tolerant multi-source fusion procedure applied in the context of localization. This Ph.D. thesis will be co-supervised by researchers from the French institute of science and technology for transport, spatial planning, development and networks (IFSTTAR) and CRIStAL UMR CNRS 9189 (University of Lille 1, France). The IFSTTAR-COSYS-LEOST and CRISTAL-CI2S-DICOT laboratories have multi-vehicle platforms equipped with several cameras, optical sensors, GNSS receivers and other location sensors. The CRISTAL-CI2S-DICOT laboratory has a 3D cartography of several experimental sites on the campus of Lille 1 and the IFSTTAR-COSYS-LEOST benefits from image/video datasets of transport environment (road/rail) acquired in different projects. An approach to test and validate, on real data, the performance of the strategy developed during the thesis, is therefore planned. The thesis proposed represents a straight continuation of works carried out during the "Contrat d'objectifs et de performance" of IFSTTAR [Marais 2015], tackles the research aspects that will be developed in the unifying projet " mobilités & transitions numériques" and is a part of the major challenge for the transport domain, the future autonomous train. It is also complementary to the research works developed in the framework of the ELSAT-SMARTIES Regional project (Smart, fail-safe communication and positioning system) and the ERSAT GGC European project. Finally, it is also aims to reinforce the collaboration with the DiCOT team of the CI2S group of the CRIStAL laboratory of the University of Lille 1, partner of the ELSAT-SMARTIES project. This Ph.D. thesis will deal with the disciplines of: image processing, signal processing, data fusion and localization. It will take place at IFSTTAR-COSYS-LEOST (Lille-Villeneuve d'Ascq site) with strong interactions with the DiCOT team of the CI2S group of the CRIStAL laboratory of the University of Lille 1. It will be supervised by C. Meurie (Researcher at IFSTTAR-COSYS-LEOST) - cyril.meurie@ifsttar.fr - and co-directed by J. Marais (Researcher with the habilitation to conduct researches at IFSTTAR-COSYS-LEOST) - juliette.marais@ifsttar.fr - and Mr El Badaoui El Najjar (Professor at UMR CRIStAL / University of Lille 1) - maan.e-elnajjar@univ_lille1.fr -. It is envisaged to publish the research work in three international conferences in the field of image processing (ICIP, ICISP, ICIAP, MICCAI, ECCV) and/or Intelligent Transport Systems (ITSC, IV, ION), and at least in a popular international journal (IEEE on ITS, Transportation Research Part C, Experts systems with applications). Candidate profile: Master's degree or equivalent. The candidate must have strong skills in mathematics, computer vision, image processing and software development. He (She) should have excellent working knowledge in object-oriented programming (C++) and Python language, and in the use of OpenCv library. The (a) candidate will have to send in electronic format: a detailed CV, the grades and rankings obtained in Master (and previous course if possible), a reasoned opinion of the Head of Master, a scientific annex that will be established after contacting the supervisors indicated above. References: [Adjrad, 2015] Adjrad, M., & Groves, P. D. (2015). Enhancing Conventional GNSS Positioning with 3D Mapping without Accurate Prior Knowledge. ION GNSS+ 2015. [Al Hage, 2015] Al Hage J., N. Aït Tmazirte, M. E. El Najjar and D. Pomorski, “Fault tolerant fusion approach based on information theory applied on GNSS localization” , In proc. of 18th international conference on information Fusion , Fusion 2015, Washington DC; 07/2015 [Al Hage, 2017] Al Hage J., Maan E. El Najjar “Improved Outdoor Localization Based on Weighted Kullback-Leibler Divergence for Measurements Diagnosis”, IEEE Intelligent Transportation Systems Magazine (à paraître). [Dawood 2016] Maya Dawood, Cindy Cappelle, Maan E. El Najjar, Jing Peng, Mohamad Khalil, Denis Pomorski “Virtual 3D city model as a priori information source of vehicle ego-localization system”, Journal of Transportation Research Part C: Emerging Technologies, Elsevier Publisher 2016 (IF: 3.805). [Marais, 2014] J. Marais, C. Meurie, D. Attia, Y. Ruichek, A. Flancquart, "Toward accurate localization in guided transport: combining GNSS data and imaging information", Transportation Research Part C: Emerging Technologies (5-Year IF 2011: 2.284), Vol. 43(2), pp. 188–197, June 2014 [Marais, 2015] J. Marais, D. Bétaille & al, Algorithmes embarqués de localisation outdoor pour les véhicules en milieu urbain et algorithme embarqué de fusion GNSS et carte numérique, Livrable du COP, décembre 2015 [Marmo, 2006] Marmo, R., Lombardi, L., Gagliardi, B., “Railway sign detection and classification », In proc. Of IEEE Intelligent Transportation Systems Conference, pp. 1358-1363, 2006 [Meurie, 2010] C. Meurie, Y. Ruichek, A. Cohen, J. Marais, "An hybrid an adaptive segmentation method using color and textural information", SPIE Electronic Imaging 2010 - Image Processing: Machine Vision Applications III, SPIE Vol. 7538, pp. 11, USA, January 2010. [Nister, 2006] D. Nister, H. Stewenius, "Scalable recognition with a vocabulary tree", In proc. of International Conference on Computer Vision and Pattern Recognition, 2006. [Schindler, 2007] G. Schindler, M. Brown and R. Szeliski., "City-Scale Location Recognition", In proc. of International Conference on Computer Vision and Pattern Recognition, 2007.
Keywords :	images processing, images segmentation, object recognition, signal processing, data fusion, localization

List of topics

Applications closed