Thèses sur le sujet « Autonomous Driving »

Autonomous Driving and Advance Driver Assistance Systems (ADAS) are revolutionizing the way we drive and the future of mobility. Among ADAS, Traffic Sign Classification is an important technique which assists the driver to easily interpret traffic signs on the road. In this thesis, we used the powerful combination of Image Processing and Deep Learning to pre-process and classify the traffic signs. Recent studies in Deep Learning show us how good a Convolutional Neural Network (CNN) is for image classification and there are several state-of-the-art models with classification accuracies over 99 % existing out there. This shaped our thesis to focus more on tackling the current challenges and some open-research cases. We focussed more on performance tuning by modifying the existing architectures with a trade-off between computations and accuracies. Our research areas include enhancement in low light/noisy conditions by adding Recurrent Neural Network (RNN) connections, and contribution to a universal-regional dataset with Generative Adversarial Networks (GANs). The results obtained on the test data are comparable to the state-of-the-art models and we reached accuracies above 98% after performance evaluation in different frameworks

Mestrado em Engenharia Mecânica
Foram simulados numericamente jogos de recursos públicos em redes usando algoritmo de Monte Carlo. Foram usadas redes regulares unidimensionais em anel, redes regulares bidimensionais (rede quadrada) e redes scale-free. São apresentados os métodos seguidos, a teoria e os algoritmos usados. Estes jogos apresentam uma transição de fase entre uma fase dominada por oportunistas de uma fase dominada por cooperadores em função de um parâmetro de rendimento das contribuições. Foi encontrado um intervalo, dependente do número médio de vizinhos, para o qual a fracção de configurações sobreviventes tende para 1 quando o tamanho da rede aumenta. Foi também encontrada uma dependência no valor de parâmetro crítico de transição no número médio de vizinhos para as configurações sobreviventes. Esses efeitos foram observados em todos os tipos de rede estudados neste trabalho. ABSTRACT: Public goods games were numerically simulated in networks using Monte Carlo Algorithm. Regular one-dimensional ring networks, regular two-dimensional lattice networks and scale-free networks had been used. The methods followed, the theory and the algorithms used are presented. This games have a phase transition between one phase dominated by defectors from one dominated by cooperators in function of the value of efficiency from the contributions. It was found an interval, dependent on the average number of neighbors, where the fraction of surviving configurations tens to 1 when the size of the network increases. It was found dependence in the critical value of transition value with the average number of neighbors. Both effects were observed in all types of networks studied in this work.

L’objectiu d’aquesta tesi és estudiar algoritmes de reconstrucció 3D adequats per a la conducció autònoma. Per fer-ho, necessitem implementacions i representacions ràpides de l’entorn 3D que tinguin en compte la informació geomètrica i semàntica. L’ús de paral·lelització CUDA i GPU permet aprofitar maquinari flexible i programable d’alt rendiment per complir els requisits de temps exigents. La tesi presenta tres contribucions principals. En primer lloc, descrivim la paral·lelització del conegut algorisme d’estèreo basat en el Semi-Global Matching (SGM), que estima la profunditat a partir de dues imatges estèreo. Desplegem un disseny de paral·lelització eficient que funciona a les GPU de baix consum energètic i aconsegueix un rendiment en temps real. Com a segona contribució, presentem una millora del model de representació 3D anomenat Stixel World, que representa les superfícies inclinades. L’extensió del model ajuda a representar escenes reals que fallen sota els supòsits anteriors i, mitjançant una regularització eficient del model, manté la mateixa precisió que el model anterior. També proposem una estratègia algorítmica per accelerar el procés, que redueix la quantitat de combinacions Stixel provades. Finalment, expliquem les nostres estratègies de paral·lelització per a l’algorisme de segmentació de Stixel. Proposem una estratègia de paral·lelització que s’adapti a l’arquitectura massivament paral·lela de les GPU. També estudiem les diferents tècniques d’acceleració disponibles per a Stixels i com es poden implementar de manera eficient per a aquesta arquitectura. A més, el nostre enfocament redueix la complexitat computacional de l’algorisme mitjançant la reformulació del model.
El objetivo de esta tesis es estudiar algoritmos de reconstrucción 3D aptos para la conducción autónoma. Para ello, necesitamos implementaciones y representaciones rápidas del entorno 3D que tengan en cuenta la información geométrica y semántica. El uso de la paralelización de CUDA y GPU permite aprovechar el hardware de alto rendimiento flexible y programable para cumplir con los estrictos requisitos de tiempo. La tesis presenta tres contribuciones principales. Primero, describimos la paralelización del conocido algoritmo de estéreo basado en Semi-Global Matching (SGM), que estima la profundidad de dos imágenes estéreo. Implementamos un diseño de paralelización eficiente que se ejecuta sobre GPU de bajo consumo de energía y logra un rendimiento en tiempo real. Como segunda contribución, presentamos una mejora del modelo de representación 3D llamado Stixel World que da cuenta de las superficies inclinadas. La extensión del modelo ayuda a representar escenas reales que fallan bajo los supuestos anteriores y, mediante una regularización eficiente del modelo, mantiene la misma precisión del modelo anterior. También proponemos una estrategia algorítmica para acelerar el proceso, lo que reduce la cantidad de combinaciones de Stixel probadas. Finalmente, explicamos nuestras estrategias de paralelización para el algoritmo de segmentación de Stixel. Proponemos una estrategia de paralelización que se adapta a la arquitectura masivamente paralela de las GPU. También estudiamos las diferentes técnicas de aceleración disponibles para Stixels y cómo se pueden implementar de manera eficiente para esta arquitectura. Además, nuestro enfoque reduce la complejidad computacional del algoritmo al reformular el modelo.
The objective of this thesis is to study 3D reconstruction algorithms suitable for autonomous driving. In order to do so, we need fast implementations and representations of the 3D environment that take into account geometric and semantic information. The use of CUDA and GPU parallelization allows to leverage flexible and programmable high performance hardware to fulfill the strong time requirements. The thesis presents three main contributions. First, we describe the parallelization of the well-known stereo matching algorithm based on Semi-Global Matching (SGM), which estimates depth from two stereo images. We deploy an efficient parallelization design that runs on top of low-energy consumption GPUs and achieves real-time performance. As our second contribution, we present an improvement of the 3D representation model called the Stixel World that accounts for slanted surfaces. The extension of the model helps representing real scenes that fail under the previous assumptions, and, by an efficient model regularization, keeps the same accuracy of the previous model. We also propose an algorithmic strategy to speed up the process, which reduces the amount of Stixel combinations tested by the dynamic programming approach. Finally, we explain our parallelization strategies for the Stixel segmentation algorithm. We propose a parallelization strategy that fits the massively parallel architecture of GPUs. We also study the different speed up techniques available for Stixels and how they can be implemented efficiently for this architecture. Additionally, our approach reduces the computational complexity of the algorithm by reformulating the measurement depth model, relying on the confidence of the depth estimation and the identification of invalid values to handle outliers.

Autonomous driving, or self-driving, is the ability of a vehicle to drive itself without human input. To achieve this, the vehicle uses mechanical and electronic parts, sensors, actuators and an on-board computer. The on-board computer runs sophisticated software which allows the vehicle to perceive and understand its environment based on sensor input, localise itself in that environment and plan the optimal route from point A to point B. Autonomous driving is no longer a thing of the future, and to develop autonomous driving solutions is a highly valuable skill in today’s software engineering field. Robot Operating System (ROS) is a meta-operating system that simplifies the process of robotics programming. This master’s thesis aims to demonstrate how ROS could be used to develop autonomous driving software by analysing autonomous driving problems, examining existing solutions and developing a prototype vehicle using ROS. This thesis provides an overview of autonomous driving and usage of ROS in the development of autonomous driving, then elaborates on the benefits and challenges of using ROS for autonomous car development. The research methods used in this master’s thesis are design science research (DSR) and a literature review. An artefact is developed and evaluated—a remote-controlled (RC) car equipped with Raspberry Pi 3 board as the on-board computer, an Arduino Uno board, Teensy LC board, a set of sensors and ROS-based software. The thesis is supported by the author’s employer, automotive software company called Elektrobit. By following the steps described in this thesis, it is possible to develop an autonomous driving RC car which runs on ROS. Additionally, this thesis shows why ROS provides good solutions for the autonomous driving issues. It points to the benefits of ROS: open sourced, peer-to-peer, network-based meta-operating system with ready-made components for autonomous driving, and highlights some of the challenges of ROS: security issues and single point of failure.

Thesis: S.M., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2018.
This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.
Cataloged from student-submitted PDF version of thesis.
Includes bibliographical references (pages 77-83).
The safe, successful deployment of autonomous systems under real-world conditions, in part, hinges upon providing rigorous performance and safety guarantees. This thesis considers the problem of establishing and verifying the safety of autonomous systems. To this end, we present a novel framework for the synthesis of safety constraints for autonomous systems, so-called safety contracts, that can be applied to and used by a wide set of real-world systems by acting as a design requirement for the controller implementation of the system. The contracts consider a large variety of road models, guarantee that the controlled system will remain safe with respect to probabilistic models of traffic behavior, and ensure that it will follow the rules of the road. We generate contracts using reachability analysis in a reach-avoid problem under consideration of dynamic obstacles, i.e., other traffic participants. Contracts are then derived directly from the reachable sets. By decomposing large road networks into local road geometries and defining assume-guarantee contracts between local geometries, we enable computational tractability over large spatial domains. To efficiently account for the behavior of other traffic participants, we iteratively alternate between falsification to generate new traffic scenarios that violate the safety contract and reachable set computation to update the safety contract. These counterexamples to collision-free behavior are found by solving a gradient-based trajectory optimization problem. We demonstrate the practical effectiveness of the proposed methods in a set of experiments involving the Manhattan road network as well as interacting multi-car traffic scenarios.
by Lucas Liebenwein.
S.M.

The thesis develops a trajectory planner which operates in a formula racing scenario. The proposed trajectory planner gives time-optimal off-road trajectory planning solutions and generates sequences of control signals for the vehicle to follow the trajectory. Outputs of the trajectory planner are time-optimal trajectory, steering angle, resultant force of brake and throttle. The trajectory planner is designed to have two modes, the Exploring mode which is based on Rapidly-exploring Random Tree (RRT), and the optimization mode which is built upon optimal Rapidly-exploring random tree (RRT*). The exploring mode can generate a valid and safe trajectory in real  time, but the solution is not optimal; optimization mode gives optimal trajectories but it can only do offline planning. The system structure makes it possible that the exploring mode keeps running while the optimization mode runs in the background; once the optimization process is complete, the vehicle could then follow the optimal trajectory. A local trajectory planning method which considers the constraint of off-road vehicle dynamics is designed and integrated in the planner. System performance is evaluated by simulations on a racing track. Many aspects including time optimality, vehicle stability have been taken into account. A new, fast, local steering method has been proposed; the method generates trajectory based on random input. By a more efficient implementation of the planner in the future, for example by using parallel computing, the optimization mode is promising for real-time trajectory generation.
Detta examensarbete utvecklar en trajektoriegenerering som baseras på ett Formula racing scenario. Den föreslagna trajektoriegenerering ger en tidsoptimal off-road-lösning och medför sekventiella kontrollsignaler till fordonen att följa trajektorien. Utsignaler från trajektoriegenerering är den tidsoptimal trajektorien, styrvinkel och motsvarande kraft for broms och gaspådrag. Trajektoriegenerering är designad med två funktioner, utforskningsläget baserat på Rapidly-exploring Random Tree (RRT) och optimeringsläget som baseras på det optimala Rapidly-exploring Random Tree (RRT). Utforskningsläget genererar en tillåten och säker trajektorie i realtid, men den är inte optimal; Optimeringsläget ger den optimala trajektorien men kan inte bli genererad i realtid. Systemstrukturen gör det möjligt att ha utforskningsläget körandes, medans optimeringsläget körs i bakgrunden; när optimeringsprocessen är färdig kan fordonet följa den optimala trajektorien. En lokal trajektoriegenererings-metod tar med dynamiska off-road krav för fordonet när planeringen görs. Prestandan av systemet är avgjort på en simulerat racerbana.Många aspekter såsom tidoptima och fordonsstabilitet har tagits med i beräkningen. En ny styrmetod har föreslagits; där metoden snabbt genererar en trajektorie baserad på slumpmässiga insignaler. Med mer effektiv framtida implementering av planeringen, exempelvis parallellberäkning, blir optimeringsläget en lovande trajektoriegenerering i realtid.

Dans cette thèse, nous abordons les défis de la rareté des annotations et la fusion de données hétérogènes tels que les nuages de points 3D et images 2D. D’abord, nous adoptons une stratégie de conduite de bout en bout où un réseau de neurones est entraîné pour directement traduire l'entrée capteur (image caméra) en contrôles-commandes, ce qui rend cette approche indépendante des annotations dans le domaine visuel. Nous utilisons l’apprentissage par renforcement profond où l'algorithme apprend de la récompense, obtenue par interaction avec un simulateur réaliste. Nous proposons de nouvelles stratégies d'entraînement et fonctions de récompense pour une meilleure conduite et une convergence plus rapide. Cependant, le temps d’apprentissage reste élevé. C'est pourquoi nous nous concentrons sur la perception dans le reste de cette thèse pour étudier la fusion de nuage de points et d'images. Nous proposons deux méthodes différentes pour la fusion 2D-3D. Premièrement, nous projetons des nuages de points LiDAR 3D dans l’espace image 2D, résultant en des cartes de profondeur éparses. Nous proposons une nouvelle architecture encodeur-décodeur qui fusionne les informations de l’image et la profondeur pour la tâche de complétion de carte de profondeur, améliorant ainsi la résolution du nuage de points projeté dans l'espace image. Deuxièmement, nous fusionnons directement dans l'espace 3D pour éviter la perte d'informations dû à la projection. Pour cela, nous calculons les caractéristiques d’image issues de plusieurs vues avec un CNN 2D, puis nous les projetons dans un nuage de points 3D global pour les fusionner avec l’information 3D. Par la suite, ce nuage de point enrichi sert d'entrée à un réseau "point-based" dont la tâche est l'inférence de la sémantique 3D par point. Sur la base de ce travail, nous introduisons la nouvelle tâche d'adaptation de domaine non supervisée inter-modalités où on a accès à des données multi-capteurs dans une base de données source annotée et une base cible non annotée. Nous proposons une méthode d’apprentissage inter-modalités 2D-3D via une imitation mutuelle entre les réseaux d'images et de nuages de points pour résoudre l’écart de domaine source-cible. Nous montrons en outre que notre méthode est complémentaire à la technique unimodale existante dite de pseudo-labeling
In this thesis, we address the challenges of label scarcity and fusion of heterogeneous 3D point clouds and 2D images. We adopt the strategy of end-to-end race driving where a neural network is trained to directly map sensor input (camera image) to control output, which makes this strategy independent from annotations in the visual domain. We employ deep reinforcement learning where the algorithm learns from reward by interaction with a realistic simulator. We propose new training strategies and reward functions for better driving and faster convergence. However, training time is still very long which is why we focus on perception to study point cloud and image fusion in the remainder of this thesis. We propose two different methods for 2D-3D fusion. First, we project 3D LiDAR point clouds into 2D image space, resulting in sparse depth maps. We propose a novel encoder-decoder architecture to fuse dense RGB and sparse depth for the task of depth completion that enhances point cloud resolution to image level. Second, we fuse directly in 3D space to prevent information loss through projection. Therefore, we compute image features with a 2D CNN of multiple views and then lift them all to a global 3D point cloud for fusion, followed by a point-based network to predict 3D semantic labels. Building on this work, we introduce the more difficult novel task of cross-modal unsupervised domain adaptation, where one is provided with multi-modal data in a labeled source and an unlabeled target dataset. We propose to perform 2D-3D cross-modal learning via mutual mimicking between image and point cloud networks to address the source-target domain shift. We further showcase that our method is complementary to the existing uni-modal technique of pseudo-labeling

Mestrado em Engenharia de Computadores e Telemática
Condução autónoma constitui a deslocação de um agente, robô ou veículo, de um qualquer ponto no espaço para um outro, sem qualquer intervenção humana, por forma a atingir objectivos pré-estabelecidos. Para conduzir de forma autónoma, usando planeamento de trajectória, é crucial que o agente consiga representar abstractamente tanto o conhecimento a priori acerca do mundo, como a informação que este vai adquirindo à medida que avança. Para alcançar este propósito, desenvolveu-se um sistema para ser usado na pista da Competição de Condução Autónoma do Festival Nacional de Robótica. Este sistema caracteriza-se por ser flexível e modular. Tais características permitem não são a adição componentes na pista acima referida, mas também a fácil expansão do suporte a outros tipos de pistas ou circuitos. Concluiu-se, pois, que o modelo de representação mais adequado para o sistema que se pretendia desenvolver seria um modelo híbrido, na medida em que, ao nível global tal representação seria topológica e ao nível local métrica. Ou seja, dividindo a pista em secções, estas são a base para a representação topológica, sendo depois cada secção mapeada internamente de forma métrica. Ao integrar o trabalho desta dissertação com o sistema global lograva-se alcançar um sistema de Condução Autónoma susceptível de planear a curto e médio prazo, com vista a melhorar o desempenho dos robôs usados no projecto, relativamente à solução anteriormente usada, que era baseada num sistema reactivo com alguma memória e noção de estado, mas sem planeamento de trajectória. ABSTRACT: Autonomous driving is the movement of an agent, robot or vehicle, from some point in space to another one, without any human intervention, in order to achieve predetermined goals. To drive autonomously using trajectory planning, it is vital to have an abstraction of the knowledge about the world, be it a priori or information that the agent acquires during the driving. For this, we developed a system capable of abstractly represent, not only the track for the Autonomous Driving Competition of the Portuguese Robotics Open, but also, tracks with similar characteristics. The system was developed in a exible and modular manner, in order to allow the addition of new elements to the stated track and the easy expansion to support other types of tracks and circuits. The conclusion was that the most appropriate representation model for the system we were trying to develop was an hybrid model, in that, at a global level the representation would be topological and at a local level it would be metrical. In other words, dividing the track into sections, these are the basis for the topological representation, being each of the sections then mapped internally using a metrical representation. Integrating the work of this dissertation in the global system, one hoped to achieve a Autonomous Driving system capable of short and medium term planning, with the goal of improve the performance of the ROTA project robots, comparatively with the previous solution, which was based in a reactive system with some memory and to some degree stateful.

Mestrado em Engenharia de Computadores e Telemática
Visão por computador e fusão sensorial são temas relativamente recentes, no entanto largamente adoptados no desenvolvimento de robôs autónomos que exigem adaptabilidade ao seu ambiente envolvente. Esta dissertação foca-se numa abordagem a estes dois temas para alcançar percepção no contexto de condução autónoma. O uso de câmaras para atingir este fim é um processo bastante complexo. Ao contrário dos meios sensoriais clássicos que fornecem sempre o mesmo tipo de informação precisa e atingida de forma determinística, as sucessivas imagens adquiridas por uma câmara estão repletas da mais variada informação e toda esta ambígua e extremamente difícil de extrair. A utilização de câmaras como meio sensorial em robótica é o mais próximo que chegamos na semelhança com aquele que é o de maior importância no processo de percepção humana, o sistema de visão. Visão por computador é uma disciplina científica que engloba àreas como: processamento de sinal, inteligência artificial, matemática, teoria de controlo, neurobiologia e física. A plataforma de suporte ao estudo desenvolvido no âmbito desta dissertação é o ROTA (RObô Triciclo Autónomo) e todos os elementos que consistem o seu ambiente. No contexto deste, são descritas abordagens que foram introduzidas com fim de desenvolver soluções para todos os desafios que o robô enfrenta no seu ambiente: detecção de linhas de estrada e consequente percepção desta, detecção de obstáculos, semáforos, zona da passadeira e zona de obras. É também descrito um sistema de calibração e aplicação da remoção da perspectiva da imagem, desenvolvido de modo a mapear os elementos percepcionados em distâncias reais. Em consequência do sistema de percepção, é ainda abordado o desenvolvimento de auto-localização integrado numa arquitectura distribuída incluindo navegação com planeamento inteligente. Todo o trabalho desenvolvido no decurso da dissertação é essencialmente centrado no desenvolvimento de percepção robótica no contexto de condução autónoma.
Computer vision and sensor fusion are subjects that are quite recent, however widely adopted in the development of autonomous robots that require adaptability to their surrounding environment. This thesis gives an approach on both in order to achieve perception in the scope of autonomous driving. The use of cameras to achieve this goal is a rather complex subject. Unlike the classic sensorial devices that provide the same type of information with precision and achieve this in a deterministic way, the successive images acquired by a camera are replete with the most varied information, that this ambiguous and extremely dificult to extract. The use of cameras for robotic sensing is the closest we got within the similarities with what is of most importance in the process of human perception, the vision system. Computer vision is a scientific discipline that encompasses areas such as signal processing, artificial intelligence, mathematics, control theory, neurobiology and physics. The support platform in which the study within this thesis was developed, includes ROTA (RObô Triciclo Autónomo) and all elements comprising its environment. In its context, are described approaches that introduced in the platform in order to develop solutions for all the challenges facing the robot in its environment: detection of lane markings and its consequent perception, obstacle detection, trafic lights, crosswalk and road maintenance area. It is also described a calibration system and implementation for the removal of the image perspective, developed in order to map the elements perceived in actual real world distances. As a result of the perception system development, it is also addressed self-localization integrated in a distributed architecture that allows navigation with long term planning. All the work developed in the course of this work is essentially focused on robotic perception in the context of autonomous driving.

In recent years, autonomous driving has become an increasingly practical technology. With state-of-the-art computer and sensor engineering, autonomous vehicles may be produced and widely used for travel and logistics in the near future. They have great potential to reduce traffic accidents, improve transportation efficiency, and release people from driving tasks while commuting. Researchers have built autonomous vehicles that can drive on public roads and handle normal surrounding traffic and obstacles. However, in situations like lane changing and merging, the autonomous vehicle faces the challenge of performing smooth interaction with human-driven vehicles. To do this, autonomous vehicle intelligence still needs to be improved so that it can better understand and react to other human drivers on the road. In this thesis, we argue for the importance of implementing ”socially cooperative driving”, which is an integral part of everyday human driving, in autonomous vehicles. An intention-integrated Prediction- and Cost function-Based algorithm (iPCB) framework is proposed to enable an autonomous vehicles to perform cooperative social behaviors. We also propose a behavioral planning framework to enable the socially cooperative behaviors with the iPCB algorithm. The new architecture is implemented in an autonomous vehicle and can coordinate the existing Adaptive Cruise Control (ACC) and Lane Centering interface to perform socially cooperative behaviors. The algorithm has been tested in over 500 entrance ramp and lane change scenarios on public roads in multiple cities in the US and over 10; 000 in simulated case and statistical testing. Results show that the proposed algorithm and framework for autonomous vehicle improves the performance of autonomous lane change and entrance ramp handling. Compared with rule-based algorithms that were previously developed on an autonomous vehicle for these scenarios, over 95% of potentially unsafe situations are avoided.

Autonomous driving is becoming popular nowadays. In order for autonomouscars to be fully accepted, high demands are placed on the safety side. Onesafety critical issue is the robustness to disturbances. In this work, a robustmodel predictive controller is designed for an autonomous vehicle. More specifically,robust output feedback model predictive control (ROFMPC) is used, androbustness is guaranteed through the use of robust invariant sets. The vehicleis modeled using a discretized, and linearized, version of a simple kinematicbicycle model, expressed in road-aligned coordinates. It is investigated for howlarge uncertainties robustness, and stability, can be guaranteed. Both externaldisturbances and measurement noise are considered. A steady-state Kalmanfilter is used to estimate the state of the vehicle. Two cases have been studiedin simulation; straight line and curved line trajectory following. Results fromsimulations show that robustness can be ensured if the uncertainties in the systemare sufficiently small. Finally, the ROFMPC algorithm is implemented onan F1/10 RC car.
Självkörande bilar blir alltmer populärt. För att självkörande bilar ska bliallmänt accepterade, har höga krav ställts på säkerheten. En viktig sak ursäkerhetssynpunkt är huruvida sytemet kan hantera störningar. I det här arbetetdesignas en robust modelprediktiv regulator för ett självkörande fordon.Mer specifikt används ”Robust Output Feedback Model Predictive Control”(ROFMPC) och robusthet, gentemot både externa störningar och mätbrus,garanteras genom användingen av robust invarianta mängder. Fordonet modellerasmed hjälp av en diskretiserad och linjäriserad enkel cykelliknande modell,uttryckt i naturliga koordinater. Två olika fall studeras genom simulering; delsdå fordonet ska följa en rak bana och dels då det ska följa en krökt bana. Ettstationärt Kalman filter används till att uppskatta fordonets tillstånd. Resultatenfrån simuleringarna visar på att robusthet kan garanteras om störningarnaär tillräckligt små. Slutligen är den robusta MPC-regulatorn implementerad påen F1/10 RC bil.

Cette thèse de doctorat, dans le contexte applicatif de la conduite autonome, se concentre sur l'exploration des mécanismes favorisant la diversité dans les modèles génératifs, qui produisent une distribution probabiliste des trajectoires futures étant donné les trajectoires passées. Les ensembles de données de prévision de trajectoire ne fournissant qu'une trajectoire future pour une trajectoire passée et une disposition spatiale de scène données, de nombreuses méthodes existantes se concentrent sur la précision de la meilleure trajectoire prédite par rapport à la trajectoire future (vérité terrain). Nous visons à étendre ces méthodes en améliorant la diversité intrinsèque de la distribution prédite, à travers la création d'un mécanisme d'échantillonnage conscient de la diversité qui remplace l'échantillonnage séquentiel traditionnel à partir de modèles génératifs tels que les autoencodeurs variationnels (VAE). Nous offrons une manière de générer des échantillons selon la diversité exhibée dans l'ensemble de données d'entraînement, non seulement centrés autour du mode majoritaire. L'amélioration de la diversité, validée sur nuScenes à travers un ensemble complet de métriques, est intéressante en ce qui concerne la sécurité et la fluidité de l'opération de planification, subséquente à la prévision de trajectoire. En approfondissant l'aspect de la diversité dans des scénarios rares mais critiques pour la sécurité, nous nous posons la question de l'expression de la diversité des événements possibles mais encore non représentés dans l'ensemble de données d'entraînement. Cette ligne de questionnement soulève l'exploration d'un aspect beaucoup plus difficile : la découverte. Afin de générer une distribution contenant des modes non présents dans l'ensemble de données d'entraînement, nous devons soigneusement développer la distribution d'entraînement selon une fonction d'admissibilité externe. L'équilibre délicat entre permettre au décodeur d'un modèle génératif de générer à partir de codes latents inconnus et la nécessité de générer des échantillons admissibles est exploré dans la deuxième partie de cette thèse, avec des résultats intéressants sur un ensemble de données jouet
This PhD thesis, in the applicative context of autonomous driving, focuses on the exploration of diversity promoting mechanisms in generative models, which generate a probabilistic distribution of future trajectories given past trajectories. As trajectory forecasting datasets only provide one ground truth trajectory for a given past trajectory and scene spatial layout, many existing methods focus on the accuracy of the best predicted trajectory with respect to the ground truth trajectory. We aim to expand these methods by improving the intrinsic diversity of the predicted distribution, through the creation of a diversity-aware sampling mechanism that replaces traditional sequential sampling from generative models such as variational autoencoders (VAEs). We provide a way to generate samples according to the diversity exhibited in the training dataset, not only centered around the majority mode. The improvement of diversity, validated on nuScenes through a comprehensive set of metrics, is interesting with regard to the safety and smoothness of the planning operation, subsequent to trajectory forecasting. Furthering the diversity aspect in rare but safety-critical scenarios, we ask ourselves the question of expressing the diversity of events that are possible but yet unrepresented in the training dataset. This line of questioning raises the exploration of a much more challenging aspect: discovery. In order to generate a distribution that contains modes not present in the training dataset, we must carefully grow the training distribution according to an external admissibility function. The delicate balance between allowing the decoder of a generative model to generate from unknown latent codes and the necessity of generating admissible samples is explored in the second part of this thesis, with interesting results on a toy dataset

This degree project investigates the suitability of using Behavior Trees (BT) as an architecture for the behavioral layer in autonomous driving. BTs originate from video game development but have received attention in robotics research the past couple of years. This project also includes implementation of a simulated traffic environment using the Unity3D engine, where the use of BTs is evaluated and compared to an implementation using finite-state machines (FSM). After the initial implementation, the simulation along with the control architectures were extended with additional behaviors in four steps. The different versions were evaluated using software maintainability metrics (Cyclomatic complexity and Maintainability index) in order to extrapolate and reason about more complex implementations as would be required in a real autonomous vehicle. It is concluded that as the AI requirements scale and grow more complex, the BTs likely become substantially more maintainable than FSMs and hence may prove a viable alternative for autonomous driving.

Questa tesi rappresenta il risultato di un tirocinio svolto presso il reparto di test di Siemens Industry Software NV Leuven, in Belgio. Il primo obiettivo è stato quello di avere una visione generale sul settore della guida autonoma e delle relative tecnologie: si presenta quindi un'analisi della letteratura. Il campo è stato quindi ristretto ai riconoscitori di oggetti 2D che utilizzano sensori automotive come dispositivi di input. Dopo uno studio dello stato dell'arte di architetture di reti neurali e dei corrispondenti dataset usati per l'allenamento in questo settore, la domanda di ricerca è stata come validare la robustezza del sistema in condizioni reali, in particolare in degradate condizioni del manto stradale. Non essendo stato possibile portare a termine una campagna di test su strada per via dell'emergenza sanitaria, la decisione è stata di proseguire lo studio mediante la sperimentazione e testing virtuale. Questo inoltre ha permesso di estendere gli studi sulla scarsità di dati che affligge l'apprendimento automatico, consentendo di verificare se un ambiente virtuale può essere utile per la generazione di dati per l'allenamento e se può aiutare ad affrontare il problema del'adattamento del dominio. Quest'ultimo è stato affontato sotto due sfumature diverse. Da un lato l'intrinsica differenza tra dati reali e dati sintetici, dall'altro l'abilità di un sistema di adattarsi ad un nuovo dominio reale, che quindi presenta, ad esempio, delle condizioni ambientali differenti.

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2012.
Cataloged from PDF version of thesis.
Includes bibliographical references (p. 75-79).
We introduce CarSpeak, a communication system for autonomous driving. CarSpeak enables a car to query and access sensory information captured by other cars in a manner similar to how it accesses information from its local sensors. CarSpeak adopts a content-centric approach where information objects - i.e., regions along the road - are first class citizens. It names and accesses road regions using a multi-resolution system, which allows it to scale the amount of transmitted data with the available bandwidth. CarSpeak also changes the MAC protocol so that, instead of having nodes contend for the medium, contention is between road regions, and the medium share assigned to any region depends on the number of cars interested in that region. CarSpeak is implemented in a state-of-the-art autonomous driving system and tested on indoor and outdoor hardware testbeds including an autonomous golf car and 10 iRobot Create robots. In comparison with a baseline that directly uses 802.11, CarSpeak reduces the time for navigating around obstacles by 2.4x, and reduces the probability of a collision due to limited visibility by 14 x.
by Swarun Suresh Kumar.
S.M.

Articulated vehicles are widely used in the economically vital cargo industry as they provide a greater maneuverability than their rigid counterparts. Hence, autonomous driving systems for articulated vehicles have become the subject of intense research in the robotic community. This thesis analyzes the reverse motion of an articulated vehicle, namely a tractor-trailer with one on-axle hitched semitrailer, and develops a full autonomous driving system that enables reverse parking in the presence of static obstacles. The motion controller used in the autonomous driving system is based on a two-level feedback control system, with a path stabilization controller in the first level and a hitch angle controller in the second level. The path planner used is a modified RRT planner where the Dubins path has been incorporated in order to enable the planning towards a goal pose rather than merely a goal region. The modifications made have resulted in several improvements, such as more accurate planning and higher computational efficiency. Using a 1:32 scale remote controlled tractor-trailer, and a Qualisys motion capture system for pose estimation, the autonomous driving system was successfully implemented and validated.

Autonomous vehicles have been the subject of intense research, resulting in many of the latest cars being at least partly self driving. Cooperative driving extends this to a group of vehicles called a platoon, relying on com-munication between the vehicles in order to increase safety and improve the ˛ow of tra°c. This thesis is partly done in context of Grand Cooperative Driving Challenge (GCDC) 2016 where KTH has participated with a Scania truck and the Research Concept Vehicle (RCV), an electric prototype car.Trajectory planning is investigated for the longitudinal control of both the truck and the RCV. This planner is to ensure that the vehicles reached a position in a given time and a desired velocity. This is done using Pon-tryagin's minimum principle and interpolation.A more advanced planner based on Model Predictive Control (MPC) is used to avoid collisions in two di˙erent scenarios. One considers obstacle avoidance in the form of an overtake and the other a lane change scenario were the vehicle needs to decide how to position itself relative to the other vehicles.Simulations of the longitudinal control and planning of the truck did show that it could time the position and speed with a position error of less than 2m and speed error less than 0.2 m/s, assuming a distance of 120-200 m, a time interval of 40s and goal speed of 7m/s. The same simulation for the RCV had a distance error of less than 0.3m and a speed error below 0.2m.Simulations of the RCV using MPC planners showed that overtaking and lane changes could be performed. When performing the lane change the RCV managed to maintain a longitudinal distance of at least 1m, even if the other vehicles are slowing down or increasing their speed. The overtaking could also be successfully performed although with small margins, having a lateral distance of 0.5 m to the vehicle being overtaken.
Autonoma fordon har länge varit ett intensivt forskningsområde vilket resulterat i att många av de senaste bilana är åtminstone delvis självkörande. Cooperative driving utvidgar detta till en grupp fordon som kommunicerar med varandra för att öka säkerheten och få trafiken att flyta bättre. Den här uppsatsen baseras på Grand Cooperative Divers Challange (GCDC) 2016 där KTH deltog med en Scania lastbil och en elbil kallad Research Concept Vehicle (RCV). Rörelseplannering har undersökts för longitudinell kontrol av lastbilen och RCV. Den här planeraren ska se till att fordonet når en given position inom en viss tid och med en önskad hastighet. För detta ändamål används vad som kallas "Pontryagin's minimum principle" och interpolation. En mer avancerad planerare baserad på Model Predictive Control (MPC) används för att undvika kollisioner i två olika situationer. Den ena simulerar en omkörning och den andra ett filbyte med flera andra fordon i den intilliggande filen. Simuleringar av den longitudinella kontrollen av lastbilen visade att den kunde nå en position och hastighet med ett fel mindre än 2m respektive 0,2m/s då sträckor mellan 120-200m, ett tidsintervall på 40s och önskad hastighet på 7m/s används. Samma simuleringar med RCV hade ett positionsfel mindre än 0,3m och hastighetsfel under 0,2m/s. Simuleringar med RCV då MPC används visade att omkörningar och filbyten kunde genomföras. Filbyten kunde genomföras med ett longitudinellt avstånd på minst 1m, även då övriga fordon saktar ner eller ökar farten. Omkörningar kunde också genomföras om än med små marginaler. Det laterala avståndet var 0,5m till det omkörda fordonet.

The field of automated vehicle demands outstanding reliability figures to be matched by the artificially driving agents. The software architectures commonly used originate from decades of automation engineering, when robots operated only in confined environments on predefined tasks. On the other hand, autonomous driving represents an “into the wild” application for robotics. The architectures embraced until now may not be sufficiently robust to comply with such an ambitious goal. This research activity proposes a bio-inspired sensorimotor architecture for cognitive robots that addresses the lack of autonomy inherent to the rules-based paradigm. The new architecture finds its realization in an agent for autonomous driving named “Co-driver”. The Agent synthesis was extensively inspired by biological principles that contribute to give the Co-driver some cognitive abilities. Worth to be mentioned are the “simulation hypothesis of cognition” and the “affordance competition hypothesis”. The former is mainly concerned with how the Agent builds its driving skills, whereas the latter yields an interpretable agent notwithstanding the complex behaviors produced. Throughout the essay, the Agent is explained in detail, together with the bottom-up learning framework adopted. Overall, the research effort bore an effectively performing autonomous driving agent whose underlying architecture provides considerable adaptation capability. The thesis also discusses the aspects related to the implementation of the proposed ideas into a versatile software that supports both simulation environments and real vehicle platforms. The step-by-step explanation of the Co-driver is made up of theoretical considerations supported by working simulation examples, some of which are also released open-source to the research community as a driving benchmark. Eventually, guidelines are given for future research activities that may originate from the Agent and the hierarchical training framework devised. First and foremost, the exploitation of the hierarchical training framework to discover optimized longer-term driving policies.

Alloy is a lightweight formal modeling tool that generates instances of a software specification to check properties of the design. This work demonstrates the use of Alloy for the rapid development of autonomous vehicle driving protocols. We contribute two driving protocols: a Normal protocol that represents the unpredictable yet safe driving behavior of typical human drivers, and a Connected protocol that employs connected technology for cooperative autonomous driving. Using five properties that define safe and productive driving actions, we analyze the performance of our protocols in mixed traffic. Lightweight formal modeling is a valuable way to reason about driving protocols early in the development process because it can automate the checking of safety and productivity properties and prevent costly design flaws.

We present an autonomous driving agent on a simulated highway driving scenario with vehicles such as cars and trucks moving with stochastically variable velocity profiles. The focus of the simulated environment is to test tactical decision making in highway driving scenarios. When an agent (vehicle) maintains an optimal range of velocity it is beneficial both in terms of energy efficiency and greener environment. In order to maintain an optimal range of velocity, in this thesis work I proposed two novel reward structures: (a) gaussian reward structure and (b) exponential rise and fall reward structure. I trained respectively two deep reinforcement learning agents to study their differences and evaluate their performance based on a set of parameters that are most relevant in highway driving scenarios. The algorithm implemented in this thesis work is double-dueling deep-Q-network with prioritized experience replay buffer. Experiments were performed by adding noise to the inputs, simulating Partially Observable Markov Decision Process in order to obtain reliability comparison between different reward structures. Velocity occupancy grid was found to be better than binary occupancy grid as input for the algorithm. Furthermore, methodology for generating fuel efficient policies has been discussed and demonstrated with an example.
Vi presenterar ett autonomt körföretag på ett simulerat motorvägsscenario med fordon som bilar och lastbilar som rör sig med stokastiskt variabla hastighetsprofiler. Fokus för den simulerade miljön är att testa taktiskt beslutsfattande i motorvägsscenarier. När en agent (fordon) upprätthåller ett optimalt hastighetsområde är det fördelaktigt både när det gäller energieffektivitet och grönare miljö. För att upprätthålla ett optimalt hastighetsområde föreslog jag i detta avhandlingsarbete två nya belöningsstrukturer: (a) gaussisk belöningsstruktur och (b) exponentiell uppgång och nedgång belöningsstruktur. Jag utbildade respektive två djupförstärkande inlärningsagenter för att studera deras skillnader och utvärdera deras prestanda baserat på en uppsättning parametrar som är mest relevanta i motorvägsscenarier. Algoritmen som implementeras i detta avhandlingsarbete är dubbel-duell djupt Q- nätverk med prioriterad återuppspelningsbuffert. Experiment utfördes genom att lägga till brus i ingångarna, simulera delvis observerbar Markov-beslutsprocess för att erhålla tillförlitlighetsjämförelse mellan olika belöningsstrukturer. Hastighetsbeläggningsgaller visade sig vara bättre än binärt beläggningsgaller som inmatning för algoritmen. Dessutom har metodik för att generera bränsleeffektiv politik diskuterats och demonstrerats med ett exempel.

Robust and persistent localisation is essential for ensuring the safe operation of autonomous vehicles. When operating in vast and diverse urban driving environments, autonomous vehicles are frequently exposed to operating situations that violate the assumptions of algorithms or lead to a loss of localisation. To guarantee driving safety, localisation systems must achieve a high level of accuracy all of the time and anywhere, without the need for human intervention. To satisfy these requirements, we propose a novel localisation framework that can coordinate a multiple sensor switching strategy that ensures sensors and feature types are used in suitable environments. Three sensor modalities are built for global localisation observation: GPS, lidar landmarks, and visual features to provide the redundancy and continuous availability of global localisation updates. We demonstrate the localisation performance of the proposed framework in the University of Sydney Campus dataset acquired over an 18 months period. Accurate localisation for many global sensors relies on a highly consistent long-term map. We developed methodologies to make maps for lidar landmarks and visual features that localisation can repeatedly use during an 18 months period. To enable multi-sensor transition, we developed methods to register maps of different sensors and feature types in the same geographic coordinate system. Map global drift and inter-sensor map biases are also minimized. Localisation systems are seldom evaluated for their robustness, and localisation ground truth such as RTK is hard to obtain in many urban environments. We propose novel metrics to effectively quantify localisation robustness without requiring accurate ground truth. We use these metrics to conduct a comprehensive analysis of the application of these metrics against single and multi-modal localisation strategies developed in this thesis.

Object detection is one of the most important directions in computer vision, where the task is to find out the targets of interest in an image and determine their location and class. With the development of deep learning in recent years, object detection has formed a wide range of applications in various fields, such as intelligent robotics and autonomous driving. And with the expansion of application scenarios, the object detection task has been extended for temporal and 3D spacial. In the temporal dimension, image data in most application scenarios are acquired as video streams, so video-based object detection tasks can perform information fusion in the temporal dimension. In the spatial dimension it is difficult to accurately restore the 3D spatial position of an object because of the lack of depth information in the image. Point cloud data is the data of points with the information of object’s 3D position acquired by LIDAR, which is good to make up the problem of image data without depth information. This has led to the study of object detection based on point cloud data. The strong demand of autonomous driving in the industry has led to vigorous interest in 3D object detection and resulted in many excellent 3D object detection algorithms. However, the vast majority of algorithms only model single-frame data, ignoring the temporal clue in video sequence. In this work, we propose a new transformer, called Temporal-Channel Transformer (TCTR), to model the temporal-channel domain and spatial-wise relationships for video object detecting from Lidar data. As the special design of this transformer, the information encoded in the encoder is different from that in the decoder. The encoder encodes temporal-channel information of multiple frames while the decoder decodes the spatial-wise information for the current frame in a voxel-wise manner. Specifically, the temporal-channel encoder of the transformer is designed to encode the information of different channels and frames by utilizing the correlation among features from different channels and frames. On the other hand, the spatial decoder of the transformer decodes the information for each location of the current frame. Before conducting the object detection with detection head, a gate mechanism is further deployed for re-calibrating the features of current frame, which filters out the object-irrelevant information by repetitively refining the representation of target frame along with the up-sampling process. Experimental results reveal that TCTR achieves the state-of-the-art performance in grid voxel-based 3D object detection on the nuScenes benchmark. Our work wants to advance the accuracy of 3D object detection as well as inspire more research on point cloud time-series data

Machine Learning, and in particular Deep Learning, are extremely capable tools for solving problems which are difficult, or intractable to tackle analytically. Application areas include pattern recognition, computer vision, speech and natural language processing. With the automotive industry aiming for increasing amount of automation in driving, the problems to solve become increasingly complex, which appeals to the use of supervised learning methods from Machine Learning and Deep Learning. With this approach, solutions to the problems are learned implicitly from training data, and inspecting their correctness is not possible directly. This presents concerns when the resulting systems are used to support safety-critical functions, as is the case with autonomous driving of automotive vehicles. This thesis studies the safety concerns related to learning systems within autonomous driving and applies a safety monitoring approach to a collision avoidance scenario. Experiments are performed using a simulated environment, with a deep learning system supporting perception for vehicle control, and a safety monitor for collision avoidance. The related operational situations and safety constraints are studied for an autonomous driving function, with potential faults in the learning system introduced and examined. Also, an example is considered for a measure that indicates trustworthiness of the learning system during operation.
Maskininlärning, och i synnerhet deep learning, är extremt kapabla verktyg för att lösa problem  som är svåra, eller omöjliga att hantera analytiskt. Applikationsområden inkluderar  mönsterigenkänning, datorseende, tal‐ och språkförståelse. När utvecklingen inom bilindustrin  går mot en ökad grad av automatisering, blir problemen som måste lösas alltmer komplexa,  vilket har lett till ett ökat användande av metoder från maskininlärning och deep learning. Med  detta tillvägagångssätt lär sig systemet lösningen till ett problem implicit från träningsdata och  man kan inte direkt utvärdera lösningens korrekthet. Detta innebär problem när systemet i  fråga är del av en säkerhetskritisk funktion, vilket är fallet för självkörande fordon. Detta  examensarbete behandlar säkerhetsaspekter relaterade till maskininlärningssystem i autonoma  fordon och applicerar en safety monitoring‐metodik på en kollisionsundvikningsfunktion.  Simuleringar utförs, med ett deep learning‐system som del av systemet för perception, som ger  underlag för styrningen av fordonet, samt en safety monitor för kollisionsundvikning. De  relaterade operationella situationerna och säkerhetsvillkoren studeras för en autonom  körnings‐funktion, där potentiella fel i det lärande systemet introduceras och utvärderas.  Vidare introduceras ett förslag på ett mått på trovärdighet hos det lärande systemet under  drift.

The field of automated vehicle demands outstanding reliability figures to be matched by the artificially driving agents. The software architectures commonly used originate from decades of automation engineering, when robots operated only in confined environments on predefined tasks. On the other hand, autonomous driving represents an “into the wild” application for robotics. The architectures embraced until now may not be sufficiently robust to comply with such an ambitious goal. This research activity proposes a bio-inspired sensorimotor architecture for cognitive robots that addresses the lack of autonomy inherent to the rules-based paradigm. The new architecture finds its realization in an agent for autonomous driving named “Co-driver”. The Agent synthesis was extensively inspired by biological principles that contribute to give the Co-driver some cognitive abilities. Worth to be mentioned are the “simulation hypothesis of cognition” and the “affordance competition hypothesis”. The former is mainly concerned with how the Agent builds its driving skills, whereas the latter yields an interpretable agent notwithstanding the complex behaviors produced. Throughout the essay, the Agent is explained in detail, together with the bottom-up learning framework adopted. Overall, the research effort bore an effectively performing autonomous driving agent whose underlying architecture provides considerable adaptation capability. The thesis also discusses the aspects related to the implementation of the proposed ideas into a versatile software that supports both simulation environments and real vehicle platforms. The step-by-step explanation of the Co-driver is made up of theoretical considerations supported by working simulation examples, some of which are also released open-source to the research community as a driving benchmark. Eventually, guidelines are given for future research activities that may originate from the Agent and the hierarchical training framework devised. First and foremost, the exploitation of the hierarchical training framework to discover optimized longer-term driving policies.

The field of automated vehicle demands outstanding reliability figures to be matched by the artificially driving agents. The software architectures commonly used originate from decades of automation engineering, when robots operated only in confined environments on predefined tasks. On the other hand, autonomous driving represents an “into the wild” application for robotics. The architectures embraced until now may not be sufficiently robust to comply with such an ambitious goal. This research activity proposes a bio-inspired sensorimotor architecture for cognitive robots that addresses the lack of autonomy inherent to the rules-based paradigm. The new architecture finds its realization in an agent for autonomous driving named “Co-driver”. The Agent synthesis was extensively inspired by biological principles that contribute to give the Co-driver some cognitive abilities. Worth to be mentioned are the “simulation hypothesis of cognition” and the “affordance competition hypothesis”. The former is mainly concerned with how the Agent builds its driving skills, whereas the latter yields an interpretable agent notwithstanding the complex behaviors produced. Throughout the essay, the Agent is explained in detail, together with the bottom-up learning framework adopted. Overall, the research effort bore an effectively performing autonomous driving agent whose underlying architecture provides considerable adaptation capability. The thesis also discusses the aspects related to the implementation of the proposed ideas into a versatile software that supports both simulation environments and real vehicle platforms. The step-by-step explanation of the Co-driver is made up of theoretical considerations supported by working simulation examples, some of which are also released open-source to the research community as a driving benchmark. Eventually, guidelines are given for future research activities that may originate from the Agent and the hierarchical training framework devised. First and foremost, the exploitation of the hierarchical training framework to discover optimized longer-term driving policies.

Dans cette thèse, nous nous intéressons à des problèmes liés à la reconstruction et la complétion des scènes 3D à partir de nuages de points de densité hétérogène. Nous étudions l'utilisation de grilles d'occupation tridimensionnelles pour la reconstruction d'une scène 3D à partir de plusieurs observations. Nous proposons d'exploiter les informations de trajet des rayons pour résoudre des ambiguïtés dans les cellules partiellement occupées. Notre approche permet de réduire les imprécisions dues à la discrétisation et d'effectuer des mises à jour d'occupation des cellules dans des scénarios dynamiques. Puis, dans le cas où le nuage de points correspond à une seule observation de la scène, nous introduisons un algorithme de reconstruction de surface implicite 3D capable de traiter des données de densité hétérogène en utilisant une stratégie de voisinages adaptatifs. Notre méthode permet de compléter de petites zones manquantes de la scène et génère une représentation continue de la scène. Enfin, nous nous intéressons aux approches d'apprentissage profond adaptées à la complétion sémantique d'une scène 3D. Après avoir présenté une étude approfondie des méthodes existantes, nous introduisons une nouvelle méthode de complétion sémantique multi-échelle appropriée aux scenarios en extérieur. Pour ce faire, nous proposons une architecture constituée d'un réseau neuronal convolutif hybride basé sur une branche principale 2D et comportant des têtes de segmentation 3D pour prédire la scène sémantique complète à différentes échelles. Notre approche est plus légère et plus rapide que les approches existantes, tout en ayant une efficacité similaire
In this thesis, we address the challenges of 3D scene reconstruction and completion from sparse and heterogeneous density point clouds. Therefore proposing different techniques to create a 3D model of the surroundings.In the first part, we study the use of 3-dimensional occupancy grids for multi-frame reconstruction, useful for localization and HD-Maps applications. This is done by exploiting ray-path information to resolve ambiguities in partially occupied cells. Our sensor model reduces discretization inaccuracies and enables occupancy updates in dynamic scenarios.We also focus on single-frame environment perception by the introduction of a 3D implicit surface reconstruction algorithm capable to deal with heterogeneous density data by employing an adaptive neighborhood strategy. Our method completes small regions of missing data and outputs a continuous representation useful for physical modeling or terrain traversability assessment.We dive into deep learning applications for the novel task of semantic scene completion, which completes and semantically annotates entire 3D input scans. Given the little consensus found in the literature, we present an in-depth survey of existing methods and introduce our lightweight multiscale semantic completion network for outdoor scenarios. Our method employs a new hybrid pipeline based on a 2D CNN backbone branch to reduce computation overhead and 3D segmentation heads to predict the complete semantic scene at different scales, being significantly lighter and faster than existing approaches

Nowadays, autonomous driving revolution is getting closer to reality. To achieve the Autonomous driving the first step is to develop the Advanced Driver Assistance System (ADAS). Driver-assistance systems are one of the fastest-growing segments in automotive electronics since already there are many forms of ADAS available. To investigate state of art of development of ADAS towards Autonomous Driving, we develop Systematic Mapping Study (SMS). SMS methodology is used to collect, classify, and analyze the relevant publications. A classification is introduced based on the developments carried out in ADAS towards Autonomous driving. According to SMS methodology, we identified 894 relevant publications about ADAS and its developmental journey toward Autonomous Driving completed from 2012 to 2016. We classify the area of our research under three classifications: technical classifications, research types and research contributions. The related publications are classified under thirty-three technical classifications. This thesis sheds light on a better understanding of the achievements and shortcomings in this area. By evaluating collected results, we answer our seven research questions. The result specifies that most of the publications belong to the Models and Solution Proposal from the research type and contribution. The least number of the publications belong to the Automated…Autonomous driving from the technical classification which indicated the lack of publications in this area.

Autonomous vehicles will help society if they can easily support a broad range of driving environments, conditions, and vehicles. Achieving this requires reducing the complexity of the algorithmic system, easing the collection of training data, and verifying operation using real-world experiments. Our work addresses these issues by utilizing a reflexive neural network that translates images into steering and throttle commands. This network is trained using simulation data from Grand Theft Auto V~\cite{gtav}, which we augment to reduce the number of simulation hours driven. We then validate our work using a RC car system through numerous tests. Our system successfully drive 98 of 100 laps of a track with multiple road types and difficult turns; it also successfully avoids collisions with another vehicle in 90\% of the trials.

The urban population is growing at an exponential rate throughout the world leading to the problems related to swift and speedy mobility or issues caused by convectional mobility options. This study illustrates and explores the new ways to transport people specially taking into account the self-driving cars concept and discusses the concept of mobility 4.0 (smart / intelligent mobility) and briefly highlights the technological aspects of autonomous vehicles, adaptation advantages and progress in laws and legislations of autonomous vehicle. The study is primarily qualitative and relies on the work of previous researcher, technical reports and blogs but the part of this study is quantitative where empirical data was collected from the experts in a conference held by BBG Austria. The result of the studies shows adaptation readiness of Austrian professional market and business prospects associated with autonomous vehicles Moreover, different business models are suggested, which could be adopted to incorporate the driverless vehicles in day-to-day life of an individual living in urban environment. The models basically suggest that the adaptation of the technology would help curbing transport externalities especially external cost associated to transportation of each individual; which includes congestion, accident, infrastructure costs and environmental costs which are incurred by least efficient conventional cars and would also help shrinking the diseases like premature mortality, aggravation of respiratory as well as cardiovascular disease and sleep disturbance which are the result of city level congestion and pollution. (author's abstract)
Series: Schriftenreihe des Instituts für Transportwirtschaft und Logistik - Verkehr

Platooning and cooperative driving can decrease the emissions of greenhouse gases and increase the traffic capacity of the roads. The Grand Cooperative Driving Challenge, GCDC, is a competition that will be held in May 2016 focusing on cooperative driving. A cornerstone in the cooperative driving is autonomous driving. The main objective of this thesis is to design a Model Predictive Control for a truck so it autonomously can perform the following tasks: follow a straight road, make a lane change and make a turn. Constraints are added to the vehicle states and the control signals. Additionally, a constraint is added to make the vehicle keep a safe distance to preceding vehicles. A Linear Time-Varying (LTV) MPC for reference tracking is derived. To use the MPC for reference tracking references for all the states and control signals are derived. For the lane change and turn scenario Bezier curves are used to obtain the position references. The controller is implemented in MATLAB and validated through simulations. For the simulations both a bicycle model of the vehicle and a more complicated four wheel model are used. The latter is implemented in Simulink. The bicycle model is on-line linearised around the references in order to be used as the prediction model for the LTV-MPC. The simulations show that the controller can make the vehicle perform the above mentioned tasks. With a horizon of 18 the time in average to perform one iteration of the control loop is 0.02 s. The maximum deviation in the lateral direction is 0.10 m and occurs for the turn scenario when the four wheel model is used for the simulations. Simulations are also done with a preceding vehicle. The controller is able to make the vehicle keep a safe distance to the preceding vehicle. If the preceding vehicle is driving slower than the controlled vehicle the controller is able to decrease the velocity of the controlled vehicle. In addition to the above mentioned, simulations are also done where disturbances and noise, separately, are added. As a disturbance an error in the start position is used. The vehicle can start 1.3 m from the real start position, in the lateral direction, and still find its way back to the trajectory. The noise is added as white noise to the position updates. The controller can deal with noise with a standard deviation up to 0.3 m.
Kolonnkörning och kooperativkörning kan minska utsläppen av växthusgaser och öka trafikkapaciteten på vägarna. The Grand Cooperative Driving Challenge, GCDC, är en tävling med fokus på kooper-ativkörning som kommer att hållas i maj 2016. En hörnsten i kooperativkörning är autonomkörning. Det huvudsakliga målet med det här examensarbetet ¨ar att designa en MPC för en lastbil så att den autonomt kan genomföra följande: följa en rak väg, göra ett filbyte samt göra en sväng. Begränsningar är lagda på fordonets tillstånd och kontrollsignalerna. Utöver det begränsas avståndetet till framförvarande fordon. En linjär tidsvariant (LTV) MPC för referensföljning tas fram. För att använda MPC:n för referensföljning härleds referenser för fordonets tillstånd och kontrollsignaler. För filbytet och svängen används Bezier kurvor för att få fram positionsreferenserna. Regulatorn implementeras i MATLAB och valideras genom simuleringar. För simuleringarna används både en cykelmodell av fordonet och en mer komplicerad fyrhjuls-modell. Den senare implementeras i Simulink. Cykelmodellen linjäriseras online kring referenserna i syfte att användas som prediktionsmodell för LTV-MPC. Simuleringarna visar att regulatorn kan få fordonet att genomföra de ovan nämnda uppgifterna. Med en horisont på 18 ¨ar tiden det tar att genomföra en iteration av regulator-loopen i genomsnitt 0.02 sekunder. Den maximala avvikelsen i den laterala riktningen ¨ar 0.10 meter och uppstår när simuleringar görs för en sväng med fyrhjuls-modellen som fordonsmodell. Simuleringar görs även med ett framförvarande fordon. Regulatorn kan få fordonet att hallå ett säkerhetsavstånd till fordonet framför. Regulatorn kan vidare få fordonet att sänka hastigheten om det framförvarande fordonet kör långsammare. Utöver de ovan nämnda simuleringarna görs simuleringar där störningar och brus, var för sig, introduceras. Som störning används ett fel i startpositionen. Fordonet kan starta 1.3 meter från den korrekta startpositionen, i den laterala riktningen, och hitta tillbaks till referensbanan. Bruset adderas som vitt brus på positionsuppdateringarna. Regulatorn kan hantera vitt brus med en stan-dardavvikelse på upp till 0.3 meter.

L’anotació manual d’imatges per desenvolupar sistemes basats en visió per computador ha estat un dels punts més problemàtics des que s’utilitza aprenentatge automàtic per a això. Aquesta tesi es centra en aprofitar les dades sintètiques per alleujar el cost de les anotacions manuals en tres tasques de percepció relacionades amb l’assistència a la conducció i la conducció autònoma. En tot moment assumim l’ús de xarxes neuronals convolucionals per al desenvolupament dels nostres models profunds de percepció. La primera tasca planteja el reconeixement de senyals de trànsit, un problema de classificació d’imatges. Assumim que el nombre de classes de senyals de trànsit a reconèixer s’ha d’incrementar sense haver pogut anotar noves imatges amb què realitzar el corresponent reentrenament. Demostrem que aprofitant les dades sintètiques de les noves classes i transformant-les amb una xarxa adversària-generativa (GAN, de les seves sigles en anglès) entrenada amb les classes conegudes (sense usar mostres de les noves classes), és possible reentrenar la xarxa neuronal per classificar tots els senyals en una proporció ~1/4 entre classes noves i conegudes. La segona tasca consisteix en la detecció de vehicles i vianants (objectes) en imatges. En aquest cas, assumim la recepció d’un conjunt d’imatges sense anotar. L’objectiu és anotar automàticament aquestes imatges perquè així es puguin utilitzar posteriorment en l’entrenament del detector d’objectes que desitgem. Per assolir aquest objectiu, vam partir de dades sintètiques anotades i proposem un mètode d’aprenentatge semi-supervisat basat en la idea del co-aprenentatge. A més, utilitzem una GAN per reduir la distància entre els dominis sintètic i real abans d’aplicar el co-aprenentatge. Els nostres resultats quantitatius mostren que el procediment desenvolupat permet anotar el conjunt d’imatges d’entrada amb la precisió suficient per entrenar detectors d’objectes de forma efectiva; és a dir, tan precisos com si les imatges s’haguessin anotat manualment. A la tercera tasca deixem enrere l’espai 2D de les imatges, i ens centrem en processar núvols de punts 3D provinents de sensors LiDAR. El nostre objectiu inicial era desenvolupar un detector d’objectes 3D (vehicles, vianants, ciclistes) entrenat en núvols de punts sintètics estil LiDAR. En el cas de les imatges es podia esperar el problema de canvi de domini degut a les diferències visuals entre les imatges sintètiques i reals. Però, a priori, no esperàvem el mateix en treballar amb núvols de punts LiDAR, ja que es tracta d’informació geomètrica provinent del mostreig actiu del món, sense que l’aparença visual influeixi. No obstant això, a la pràctica, hem vist que també apareixen els problemes d’adaptació de domini. Factors com els paràmetres de mostreig del LiDAR, la configuració dels sensors a bord del vehicle autònom, i l’anotació manual dels objectes 3D, indueixen diferències de domini. A la tesi demostrem aquesta observació mitjançant un exhaustiu conjunt d’experiments amb diferents bases de dades públiques i detectors 3D disponibles. Per tant, en relació amb la tercera tasca, el treball s’ha centrat finalment en el disseny d’una GAN capaç de transformar núvols de punts 3D per portar-los d’un domini a un altre, un tema relativament inexplorat.Finalment, cal esmentar que tots els conjunts de dades sintètiques usats en aquestes tres tasques han estat dissenyats i generats en el context d’aquesta tesi doctoral i es faran públics. En general, considerem que aquesta tesi presenta un avanç en el foment de la utilització de dades sintètiques per al desenvolupament de models profunds de percepció, essencials en el camp de la conducció autònoma.
La anotación manual de imágenes para desarrollar sistemas basados en visión por computador ha sido uno de los puntos más problemáticos desde que se utiliza aprendizaje automático para ello. Esta tesis se centra en aprovechar los datos sintéticos para aliviar el coste de las anotaciones manuales en tres tareas de percepción relacionadas con la asistencia a la conducción y la conducción autónoma. En todo momento asumimos el uso de redes neuronales convolucionales para el desarrollo de nuestros modelos profundos de percepción. La primera tarea plantea el reconocimiento de señales de tráfico, un problema de clasificación de imágenes. Asumimos que el número de clases de señales de tráfico a reconocer se debe incrementar sin haber podido anotar nuevas imágenes con las que realizar el correspondiente reentrenamiento. Demostramos que aprovechando los datos sintéticos de las nuevas clases y transformándolas con una red adversaria-generativa (GAN, de sus siglas en inglés) entrenada con las clases conocidas (sin usar muestras de las nuevas clases), es posible reentrenar la red neuronal para clasificar todas las señales en una proporción de ~1/4 entre clases nuevas y conocidas. La segunda tarea consiste en la detección de vehículos y peatones (objetos) en imágenes. En este caso, asumimos la recepción de un conjunto de imágenes sin anotar. El objetivo es anotar automáticamente esas imágenes para que así se puedan utilizar posteriormente en el entrenamiento del detector de objetos que deseemos. Para alcanzar este objetivo, partimos de datos sintéticos anotados y proponemos un método de aprendizaje semi-supervisado basado en la idea del co-aprendizaje. Además, utilizamos una GAN para reducir la distancia entre los dominios sintético y real antes de aplicar el co-aprendizaje. Nuestros resultados cuantitativos muestran que el procedimiento desarrollado permite anotar el conjunto de imágenes de entrada con la precisión suficiente para entrenar detectores de objetos de forma efectiva; es decir, tan precisos como si las imágenes se hubiesen anotado manualmente. En la tercera tarea dejamos atrás el espacio 2D de las imágenes, y nos centramos en procesar nubes de puntos 3D provenientes de sensores LiDAR. Nuestro objetivo inicial era desarrollar un detector de objetos 3D (vehículos, peatones, ciclistas) entrenado en nubes de puntos sintéticos estilo LiDAR. En el caso de las imágenes cabía esperar el problema de cambio de dominio debido a las diferencias visuales entre las imágenes sintéticas y reales. Pero, a priori, no esperábamos lo mismo al trabajar con nubes de puntos LiDAR, ya que se trata de información geométrica proveniente del muestreo activo del mundo, sin que la apariencia visual influya. Sin embargo, en la práctica, hemos visto que también aparecen los problemas de adaptación de dominio. Factores como los parámetros de muestreo del LiDAR, la configuración de los sensores a bordo del vehículo autónomo, y la anotación manual de los objetos 3D, inducen diferencias de dominio. En la tesis demostramos esta observación mediante un exhaustivo conjunto de experimentos con diferentes bases de datos públicas y detectores 3D disponibles. Por tanto, en relación a la tercera tarea, el trabajo se ha centrado finalmente en el diseño de una GAN capaz de transformar nubes de puntos 3D para llevarlas de un dominio a otro, un tema relativamente inexplorado. Finalmente, cabe mencionar que todos los conjuntos de datos sintéticos usados en estas tres tareas han sido diseñados y generados en el contexto de esta tesis doctoral y se harán públicos. En general, consideramos que esta tesis presenta un avance en el fomento de la utilización de datos sintéticos para el desarrollo de modelos profundos de percepción, esenciales en el campo de la conducción autónoma.
Manually annotating images to develop vision models has been a major bottleneck since computer vision and machine learning started to walk together. This thesis focuses on leveraging synthetic data to alleviate manual annotation for three perception tasks related to driving assistance and autonomous driving. In all cases, we assume the use of deep convolutional neural networks (CNNs) to develop our perception models. The first task addresses traffic sign recognition (TSR), a kind of multi-class classification problem. We assume that the number of sign classes to be recognized must be suddenly increased without having annotated samples to perform the corresponding TSR CNN re-training. We show that leveraging synthetic samples of such new classes and transforming them by a generative adversarial network (GAN) trained on the known classes (i.e., without using samples from the new classes), it is possible to re-train the TSR CNN to properly classify all the signs for a ~1/4 ratio of new/known sign classes. The second task addresses on-board 2D object detection, focusing on vehicles and pedestrians. In this case, we assume that we receive a set of images without the annotations required to train an object detector, i.e., without object bounding boxes. Therefore, our goal is to self-annotate these images so that they can later be used to train the desired object detector. In order to reach this goal, we leverage from synthetic data and propose a semi-supervised learning approach based on the co-training idea. In fact, we use a GAN to reduce the synth-to-real domain shift before applying co-training. Our quantitative results show that co-training and GAN-based image-to-image translation complement each other up to allow the training of object detectors without manual annotation, and still almost reaching the upper-bound performances of the detectors trained from human annotations. While in previous tasks we focus on vision-based perception, the third task we address focuses on LiDAR pointclouds. Our initial goal was to develop a 3D object detector trained on synthetic LiDAR-style pointclouds. While for images we may expect synth/real-to-real domain shift due to differences in their appearance (e.g. when source and target images come from different camera sensors), we did not expect so for LiDAR pointclouds since these active sensors factor out appearance and provide sampled shapes. However, in practice, we have seen that it can be domain shift even among real-world LiDAR pointclouds. Factors such as the sampling parameters of the LiDARs, the sensor suite configuration on-board the ego-vehicle, and the human annotation of 3D bounding boxes, do induce a domain shift. We show it through comprehensive experiments with different publicly available datasets and 3D detectors. This redirected our goal towards the design of a GAN for pointcloud-to-pointcloud translation, a relatively unexplored topic. Finally, it is worth to mention that all the synthetic datasets used for these three tasks, have been designed and generated in the context of this PhD work and will be publicly released. Overall, we think this PhD presents several steps forward to encourage leveraging synthetic data for developing deep perception models in the field of driving assistance and autonomous driving.
Universitat Autònoma de Barcelona. Programa de Doctorat en Informàtica

For safe mobility, an autonomous vehicle must perceive the surroundings accurately. There are many perception tasks associated with understanding the local environment such as object detection, localization, and lane analysis. Object detection, in particular, plays a vital role in determining an object’s location and classifying it correctly and is one of the challenging tasks in the self-driving research area. Before employing an object detection module in autonomous vehicle testing, an organization needs to have a precise analysis of the module. Hence, it becomes crucial for a company to have an evaluation framework to evaluate an object detection algorithm’s performance. This thesis develops a comprehensive framework for evaluating and analyzing object detection algorithms, both 2D (camera images based) and 3D (LiDAR point cloud-based). The pipeline developed in this thesis provides the ability to evaluate multiple models with ease, signified by the key performance metrics, Average Precision, F-score, and Mean Average Precision. 40-point interpolation method is used to calculate the Average Precision.
För säker rörlighet måste ett autonomt fordon uppfatta omgivningen exakt. Det finns många uppfattningsuppgifter associerade med att förstå den lokala miljön, såsom objektdetektering, lokalisering och filanalys. I synnerhet objektdetektering spelar en viktig roll för att bestämma ett objekts plats och klassificera det korrekt och är en av de utmanande uppgifterna inom det självdrivande forskningsområdet. Innan en anställd detekteringsmodul används i autonoma fordonsprovningar måste en organisation ha en exakt analys av modulen. Därför blir det avgörande för ett företag att ha en utvärderingsram för att utvärdera en objektdetekteringsalgoritms prestanda. Denna avhandling utvecklar ett omfattande ramverk för utvärdering och analys av objektdetekteringsalgoritmer, både 2 D (kamerabilder baserade) och 3 D (LiDAR-punktmolnbaserade). Rörledningen som utvecklats i denna avhandling ger möjlighet att enkelt utvärdera flera modeller, betecknad med nyckelprestandamätvärdena, Genomsnittlig precision, F-poäng och genomsnittlig genomsnittlig precision. 40-punkts interpoleringsmetod används för att beräkna medelprecisionen.

Syftet med denna studie var att bygga, träna och testa two olika självkörande agenter med hjälp av maskininlärningstekniker, specifikt neurala nätverk. För att träna agenterna användes en teknik kallad Imitationsinlärning. Imitationsinlärning är en teknik för att lära agenter sekventiell beslutsfattning genom demostrering från en expert (vanligtvis en människa). Två något olika nätverksarkitekturer jämfördes. Skillnaden mellan dessa var att den kontrollmodul för en specifik intention (vilket här betecknar t.ex. åt vilket håll man ska köra i en korsning) var placerad antingen tidigt eller sent i det neurala nätverket. Testningen av de tränade modellerna observerade körningsförmåga såsom antalet lyckade intentioner och hastighet. Den tidiga modellen hade en signifikant högre hastighet än den sena modellen och verkade också köra generellt “bättre” än den sena modellen dock utan signifikans från den statistiska utvärderingen. Detta kan vara en effekt av en för liten stickprovstorlek, vilket kunde åtgärdats med användning av andra verktyg vid träning och testning. Det upptäcktes också att den tidiga modellens gas-värden som ackumulerades under testningen, var närmare expertens gas-distribution.
This study aimed to build, train, and test two different autonomous vehicle (AV) agents by using machine learning techniques, specifically neural network architectures. To be able to train the agents a technique called Imitation Learning was used. Imitation Learning is an approach for learning sequential decision-making from demonstrations provided by an expert. Two slightly different neural network architectures were compared. The difference was that the intentional command module (which denotes what direction to take in an intersection for example) was located either in the beginning or the end of the respective networks. The testing of the trained models was looking at their driving capabilities such as intentions completed and speed. The early model was significantly faster than the late model and seemed to be “better” at driving in general but with no significant difference from the late network. This could be an effect of the sample size being too small, which could have been rectified with different tools used in the training and testing. Additionally, it was found that the early model’s gas values, acquired at testing time, were closer to the expert gas distribution.

Autonomous driving technology is now evolving at an unprecedented speed. HD maps, which are embedded with highly precise and detailed road spatial and object information, play an important role in supporting autonomous vehicles. This thesis presents the development of a semi-automated HD map creation and updating method that is capable of extracting basic road feature information to HD maps by employing raw MLS point cloud data. The proposed HD map creation method consists of four steps: Road edge extraction, road surface extraction, road marking extraction and driving line generation. First, an existing curb-based road edge detection method is applied to extract road edge candidate points according to the elevation difference and slope between points. This thesis develops an edge vectorization algorithm based on the point's distance-to-trajectory. Then, the road surface is extracted by filtering the points inside fitted edges on the XY plane within a range of the ground elevation. In the next step, instead of using intensity to detect road markings used by most studies, this thesis fuses point clouds and images to assign each point with an RGB value to segment marking points. Marking objects are extracted by conditional Euclidean clustering and classified according to a manually defined decision tree. Finally, driving lines are generated based on the vectorized road edge and lane markings. The HD map update method varies depending on which data source is updated for the road segments, including updating images only, updating point clouds only and updating both images and point clouds. The method is evaluated by six point clouds and image datasets collected from different types of roads. The extracted road edges are assessed by both length- and buffer-based assessment methods. The results indicate that the road edge extraction algorithm performs well in all three dimensions. The road surface extraction results confirm the high accuracy of extracted edges. In addition, the quantitative evaluations of road markings demonstrate that the proposed road marking extraction method achieves an average recall, precision, and F1-score of 94.50%, 81.65% and 87.09%.

Mestrado em Engenharia de Computadores e Telemática
Os robôs autónomos e os veículos não tripulados são vertentes da robótica de forte investigação durante os últimos anos, especialmente para o desenvolvimento de veículos autónomos destinados à exploração de lugares inóspitos. Um robô autónomo é uma máquina que consegue ser independente e regida pelas suas próprias leis, um sistema que consegue sobreviver num ambiente natural sem intervenção humana. Hoje em dia são muitos os sistemas disponíveis (como por exemplo GPS e visão computorizada) que ajudam os robôs a sobreviver, sendo o grande desafio integrá-los de forma a produzir um ser mais inteligente, sólido e confiável. O Departamento de Electrónica, Telecomunicações e Informática da Universidade de Aveiro tem realizado, nos últimos anos, trabalho na área da condução autónoma. Um dos objectivos desse trabalho tem sido o desenvolvimento de veículos autónomos para participação na prova de condução autónoma do Festival Nacional de Robótica, na qual participa desde 2001. O software de controlo de alto nível do veículo actual assentava numa estrutura sequencial o que torna complexa a tarefa de manutenção e integração de novas funcionalidades. Actualmente existem módulos para a descrição do mundo bem como para o planeamento de trajectórias sobre esses modelos e um módulo para percepção a partir de imagem. O objectivo deste trabalho foi o de reestruturar todo o software de alto nível, tornando-o modular e concorrente permitindo dessa forma uma melhor manutenção, actualização e evolução. Fica assim mais fácil a substituição de módulos, a incorporação de novas funcionalidades e o trabalho de equipa. A nova estrutura assenta na existência de uma memória central e partilhada, tipo blackboard, onde os vários módulos recolhem os dados de que necessitam e depositam os dados produzidos. Este novo modelo arquitectural foi implementado no veículo e testado durante a edição de 2010 do Festival Nacional de Robótica, tendo alcançado o terceiro lugar. A arquitectura apresentada incorporou vários sistemas já existentes, tendo como principais vantagens a modularidade e extensibilidade dentro de um ambiente concorrente e com informação distribuída.
The autonomous robots and unmanned vehicles have been an intensive fi eld of research in the last years for driverless cars and harsh environments exploration. An autonomous robot is a machine that can work in an independent way and subject to its own laws only, a system that can survive in the real-world environment without human intervention. Today many systems are available (e.g. GPS, Computer Vision) that aid the robots to survive, being the biggest challenge put all together and create an agent more intelligent, robust and reliable. The Department of Electronics, Telecommunications and Informatics of University of Aveiro in the last years has worked in the autonomous driving area. One point of this work has been the developing of autonomous vehicles for participation in the autonomous driving inside competition of the Portuguese Robotics Open, where it has participated since 2001. The high level software that controls the actual vehicle is based on a sequential structure that turns maintenance and integration of new modules in a complex task. Currently, there are modules to carry out several basic tasks, namely, image perception and integration, \world" representation, and creation of trajectory plans The aim of this work is the reorganization of the existent high-level software, following a modular and concurrent paradigm. The new software organization makes it easier to replace software modules and to add new functionalities, enhancing team work development and maintenance. The new structure is based on the existence of a central shared memory, like a blackboard, where the modules collect data that they need as well as publish produced data. This new architectural approach has been implemented in the ROTA robot and it was tested in the 2010 Portuguese Robotics Open (where it ranked 3rd). The proposed architecture links several existing systems and has as strongest points modularity and extensibility in concurrent environment with distributed information.

Les véhicules autonomes, qualifiés aussi de véhicules sans conducteur, deviennent dans certains contextes une réalité tangible et partageront très bientôt nos routes avec d’autres véhicules classiques. Pour qu’un véhicule autonome se déplace de manière sécurisée, il doit savoir où il se trouve et ce qui l’entoure dans l’environnement. Pour la première tâche, pour déterminer sa position dans l’environnement, il doit se localiser selon six degrés de liberté (position et angles de rotation). Alors que pour la deuxième tâche, une bonne connaissance de cet environnement « proche » est nécessaire, ce qui donne lieu à une solution sous forme de cartographie. Par conséquent, pour atteindre le niveau de sécurité souhaité des véhicules autonomes, une localisation précise est primordiale. Cette localisation précise permet au véhicule non seulement de se positionner avec précision, mais également de trouver sa trajectoire optimale et d’éviter efficacement les collisions avec des objets statiques et dynamiques sur son trajet. Actuellement, la solution la plus répandue est le système de positionnement (GPS). Ce système ne permet qu’une précision limitée (de l’ordre de plusieurs mètres) et bien que les systèmes RTK (RealTime Kinematic) et DGPS (Differential GPS) aient atteint une précision bien plus satisfaisante, ces systèmes restent sensibles au masquage des signaux, et aux réflexions multiples, en particulier dans les zones urbaines denses. Toutes ces déficiences rendent ces systèmes inadaptés pour traiter des tâches critiques telles que l’évitement des collisions. Une alternative qui a récemment attiré l’attention des experts (chercheurs et industriels), consiste à utiliser une carte à priori pour localiser la voiture de l’intérieur de celui-ci. En effet, les cartes facilitent le processus de navigation et ajoutent une couche supplémentaire de sécurité et de compréhension. Le véhicule utilise ses capteurs embarqués pour comparer ce qu’il perçoit à un moment donné avec ce qui est stocké dans sa mémoire. Les cartes à priori permettent donc au véhicule de mieux se localiser dans son environnement en lui permettant de focaliser ses capteurs et la puissance de calcul uniquement sur les objets en mouvement. De cette façon, le véhicule peut prédire ce qui devrait arriver et voir ensuite ce qui se passe réellement en temps réel, et donc peut prendre une décision sur ce qu’il faut faire.Cette thèse vise donc à développer des outils permettant une localisation précise d’un véhicule autonome dans un environnement connu à priori. Cette localisation est déterminée par appariement (Map-matching) entre une carte de l’environnement disponible a priori et les données collectées au fur et à mesure que le véhicule se déplace. Pour ce faire, deux phases distinctes sont déployées. La première permet la construction de la carte, avec une précision centimétrique en utilisant des techniques de construction de cartes statiques ou dynamiques. La seconde correspond à la capacité de localiser le véhicule dans cette carte 3D en l’absence d’infrastructures dédiées comprenant le système GPS, les mesures inertielles (IMU) ou des balises.Au cours de ce travail, différentes techniques sont développées pour permettre la réalisation des deux phases mentionnées ci-dessus. Ainsi, la phase de construction de cartes, qui consiste à recaler des nuages de points capturés pour construire une représentation unique et unifiée de l’environnement, correspond au problème de la localisation et de la cartographie simultanée (SLAM). Afin de faire face à ce problème, nous avons testé et comparé différentes méthodes de recalage. Cependant, l’obtention de cartes précises nécessite des nuages de points très denses, ce qui les rend inefficaces pour une utilisation en temps réel. Dans ce contexte, une nouvelle méthode de réduction des points est proposée. (...)
The concept of self-driving vehicles is becoming a happening reality and will soon share our roads with other vehicles –autonomous or not-. For a self-driving car to move around in its environment in a securely, it needs to sense to its immediate environment and most importantly localize itself to be able to plan a safe trajectory to follow. Therefore, to perform tasks suchas trajectory planning and navigation, a precise localization is of upmost importance. This would further allow the vehicle toconstantly plan and predict an optimal path in order to weave through cluttered spaces by avoiding collisions with other agentssharing the same space as the latter. For years, the Global Positioning System (GPS) has been a widespread complementary solution for navigation. The latter allows only a limited precision (range of several meters). Although the Differential GPSand the Real Time Kinematic (RTK) systems have reached considerable accuracy, these systems remain sensitive to signal masking and multiple reflections, offering poor reliability in dense urban areas. All these deficiencies make these systems simply unsuitable to handle hard real time constraints such as collision avoidance. A prevailing alternative that has attracted interest recently, is to use upload a prior map in the system so that the agent can have a reliable support to lean on. Indeed,maps facilitate the navigation process and add an extra layer of security and other dimensions of semantic understanding. The vehicle uses its onboard sensors to compare what it perceives at a given instant to what is stored in the backend memory ofthe system. In this way, the autonomous vehicle can actually anticipate and predict its actions accordingly.The purpose of this thesis is to develop tools allowing an accurate localization task in order to deal with some complex navigation tasks outlined above. Localization is mainly performed by matching a 3D prior map with incoming point cloudstructures as the vehicle moves. Three main objectives are set out leading with two distinct phases deployed (the map building and the localization). The first allows the construction of the map, with centimeter accuracy using static or dynamic laser surveying technique. Explicit details about the experimental setup and data acquisition campaigns thoroughly carried outduring the course of this work are given. The idea is to construct efficient maps liable to be updated in the long run so thatthe environment representation contained in the 3D models are compact and robust. Moreover, map-building invariant on any dedicated infrastructure is of the paramount importance of this work in order to rhyme with the concept of flexible mapping and localization. In order to build maps incrementally, we rely on a self-implementation of state of the art iterative closest point (ICP) algorithm, which is then upgraded with new variants and compared to other implemented versions available inthe literature. However, obtaining accurate maps requires very dense point clouds, which make them inefficient for real-time use. Inthis context, the second objective deals with points cloud reduction. The proposed approach is based on the use of both colorinformation and the geometry of the scene. It aims to find sets of 3D points with the same color in a very small region and replacing each set with one point. As a result, the volume of the map will be significantly reduced, while the proprieties of this map such as the shape and color of scanned objects remain preserved.The third objective resort to efficient, precise and reliable localization once the maps are built and treated. For this purpose, the online data should be accurate, fast with low computational effort whilst maintaining a coherent model of the explored space. To this end, the Velodyne HDL-32 comes into play. (...)

1. GEAR 2030 and Strategy 2018-2020 – Comparative analysis of the competitive position of the EU automotive industry and the impact of the introduction of autonomous vehicles [Electronic resource] // Publications Office of the EU. – 2020. – Access mode: https://cutt.ly/QcoLTmU. 2. Unmanned multi-purpose vehicles: modern technologies / O. Ya. Nikonov, L. E. Kulakova, T. O. Polosukhina, V. O. Chernyshov. // Automotive and Electronics. Modern technology.. – 2017. – №11. – С. 46–49. Scientific adviser - doctor of Economics, professor Yanchuk M.B.
The idea of autonomous car control has existed for almost a century. However, only now advances in sensors, efficient drives, new materials, and increased computing power led to the realization of this idea
Ідея автономного управління автомобілем існує майже століття. Однак лише зараз досягнення в сенсорах, ефективних приводах, нових матеріалах та збільшеній обчислювальній потужності призвели до її реалізації.

Highly automated driving systems promise increased road traffic safety, as well as positive impacts on sustainable transportation by means of increased traffic efficiency and environmental friendliness. The design and development of such systems require scientific advances in a number of areas. One area is the vehicle's electrical/electronic (E/E) architecture. The E/E architecture can be presented using a number of views, of which an important one is the functional view. The functional view describes the decomposition of the system into its main logical components, along with the hierarchical structure, the component inter-connections, and requirements. When this view captures the principal ideas and patterns that constitute the foundation of a variety of specific architectures, it may be termed as a reference architecture. Two reference architectures for highly automated driving form the principal contribution of this thesis. The first reference architecture is for cooperative driving. In a cooperative driving situation, vehicles and road infrastructure in the vicinity of a vehicle continuously exchange wireless information and this information is then used to control the motion of the vehicle. The second reference architecture is for autonomous driving, wherein the vehicle is capable of driver-less operation even without direct communication with external entities. The description of both reference architectures includes their main components and the rationale for how these components should be distributed across the architecture and its layers. These architectures have been validated via multiple real-world instantiations, and the guidelines for instantiation also form part of the architecture description. A comparison with similar architectures is also provided, in order to highlight the similarities and differences. The comparisons show that in the context of automated driving, the explicit recognition of components for semantic understanding, world modeling, and vehicle platform abstraction are unique to the proposed architecture. These components are not unusual in architectures within the Artificial Intelligence/robotics domains; the proposed architecture shows how they can be applied within the automotive domain. A secondary contribution of this thesis is a description of a lightweight, four step approach for model based systems engineering of highly automated driving systems, along with supporting model classes. The model classes cover the concept of operations, logical architecture, application software components, and the implementation platforms. The thesis also provides an overview of current implementation technologies for cognitive driving intelligence and vehicle platform control, and recommends a specific setup for development and accelerated testing of highly automated driving systems, that includes model- and hardware-in-the-loop techniques in conjunction with a publish/subscribe bus. Beyond the more "traditional" engineering concepts, the thesis also investigates the domain of machine consciousness and computational self-awareness. The exploration indicates that current engineering methods are likely to hit a complexity ceiling, breaking through which may require advances in how safety-critical systems can self-organize, construct, and evaluate internal models to reflect their perception of the world. Finally, the thesis also presents a functional architecture for the brake system of an autonomous truck. This architecture proposes a reconfiguration of the existing brake systems of the truck in a way that provides dynamic, diversified redundancy, and an increase in the system reliability and availability, while meeting safety requirements.

QC 20151216

With over 1.35 million fatalities related to traffic accidents worldwide, autonomous driving was foreseen at the beginning of this century as a feasible solution to improve security in our roads. Nevertheless, it is meant to disrupt our transportation paradigm, allowing to reduce congestion, pollution, and costs, while increasing the accessibility, efficiency, and reliability of the transportation for both people and goods. Although some advances have gradually been transferred into commercial vehicles in the way of Advanced Driving Assistance Systems (ADAS) such as adaptive cruise control, blind spot detection or automatic parking, however, the technology is far from mature. A full understanding of the scene is actually needed so that allowing the vehicles to be aware of the surroundings, knowing the existing elements of the scene, as well as their motion, intentions and interactions. In this PhD dissertation, we explore new approaches for understanding driving scenes from 3D LiDAR point clouds by using Deep Learning methods. To this end, in Part I we analyze the scene from a static perspective using independent frames to detect the neighboring vehicles. Next, in Part II we develop new ways for understanding the dynamics of the scene. Finally, in Part III we apply all the developed methods to accomplish higher level challenges such as segmenting moving obstacles while obtaining their rigid motion vector over the ground. More specifically, in Chapter 2 we develop a 3D vehicle detection pipeline based on a multi-branch deep-learning architecture and propose a Front (FR-V) and a Bird’s Eye view (BE-V) as 2D representations of the 3D point cloud to serve as input for training our models. Later on, in Chapter 3 we apply and further test this method on two real uses-cases, for pre-filtering moving obstacles while creating maps to better localize ourselves on subsequent days, as well as for vehicle tracking. From the dynamic perspective, in Chapter 4 we learn from the 3D point cloud a novel dynamic feature that resembles optical flow from RGB images. For that, we develop a new approach to leverage RGB optical flow as pseudo ground truth for training purposes but allowing the use of only 3D LiDAR data at inference time. Additionally, in Chapter 5 we explore the benefits of combining classification and regression learning problems to face the optical flow estimation task in a joint coarse-and-fine manner. Lastly, in Chapter 6 we gather the previous methods and demonstrate that with these independent tasks we can guide the learning of higher challenging problems such as segmentation and motion estimation of moving vehicles from our own moving perspective.
Con más de 1,35 millones de muertes por accidentes de tráfico en el mundo, a principios de siglo se predijo que la conducción autónoma sería una solución viable para mejorar la seguridad en nuestras carreteras. Además la conducción autónoma está destinada a cambiar nuestros paradigmas de transporte, permitiendo reducir la congestión del tráfico, la contaminación y el coste, a la vez que aumentando la accesibilidad, la eficiencia y confiabilidad del transporte tanto de personas como de mercancías. Aunque algunos avances, como el control de crucero adaptativo, la detección de puntos ciegos o el estacionamiento automático, se han transferido gradualmente a vehículos comerciales en la forma de los Sistemas Avanzados de Asistencia a la Conducción (ADAS), la tecnología aún no ha alcanzado el suficiente grado de madurez. Se necesita una comprensión completa de la escena para que los vehículos puedan entender el entorno, detectando los elementos presentes, así como su movimiento, intenciones e interacciones. En la presente tesis doctoral, exploramos nuevos enfoques para comprender escenarios de conducción utilizando nubes de puntos en 3D capturadas con sensores LiDAR, para lo cual empleamos métodos de aprendizaje profundo. Con este fin, en la Parte I analizamos la escena desde una perspectiva estática para detectar vehículos. A continuación, en la Parte II, desarrollamos nuevas formas de entender las dinámicas del entorno. Finalmente, en la Parte III aplicamos los métodos previamente desarrollados para lograr desafíos de nivel superior, como segmentar obstáculos dinámicos a la vez que estimamos su vector de movimiento sobre el suelo. Específicamente, en el Capítulo 2 detectamos vehículos en 3D creando una arquitectura de aprendizaje profundo de dos ramas y proponemos una vista frontal (FR-V) y una vista de pájaro (BE-V) como representaciones 2D de la nube de puntos 3D que sirven como entrada para entrenar nuestros modelos. Más adelante, en el Capítulo 3 aplicamos y probamos aún más este método en dos casos de uso reales, tanto para filtrar obstáculos en movimiento previamente a la creación de mapas sobre los que poder localizarnos mejor en los días posteriores, como para el seguimiento de vehículos. Desde la perspectiva dinámica, en el Capítulo 4 aprendemos de la nube de puntos en 3D una característica dinámica novedosa que se asemeja al flujo óptico sobre imágenes RGB. Para ello, desarrollamos un nuevo enfoque que aprovecha el flujo óptico RGB como pseudo muestras reales para entrenamiento, usando solo information 3D durante la inferencia. Además, en el Capítulo 5 exploramos los beneficios de combinar los aprendizajes de problemas de clasificación y regresión para la tarea de estimación de flujo óptico de manera conjunta. Por último, en el Capítulo 6 reunimos los métodos anteriores y demostramos que con estas tareas independientes podemos guiar el aprendizaje de problemas de más alto nivel, como la segmentación y estimación del movimiento de vehículos desde nuestra propia perspectiva
Amb més d’1,35 milions de morts per accidents de trànsit al món, a principis de segle es va predir que la conducció autònoma es convertiria en una solució viable per millorar la seguretat a les nostres carreteres. D’altra banda, la conducció autònoma està destinada a canviar els paradigmes del transport, fent possible així reduir la densitat del trànsit, la contaminació i el cost, alhora que augmentant l’accessibilitat, l’eficiència i la confiança del transport tant de persones com de mercaderies. Encara que alguns avenços, com el control de creuer adaptatiu, la detecció de punts cecs o l’estacionament automàtic, s’han transferit gradualment a vehicles comercials en forma de Sistemes Avançats d’Assistència a la Conducció (ADAS), la tecnologia encara no ha arribat a aconseguir el grau suficient de maduresa. És necessària, doncs, una total comprensió de l’escena de manera que els vehicles puguin entendre l’entorn, detectant els elements presents, així com el seu moviment, intencions i interaccions. A la present tesi doctoral, explorem nous enfocaments per tal de comprendre les diferents escenes de conducció utilitzant núvols de punts en 3D capturats amb sensors LiDAR, mitjançant l’ús de mètodes d’aprenentatge profund. Amb aquest objectiu, a la Part I analitzem l’escena des d’una perspectiva estàtica per a detectar vehicles. A continuació, a la Part II, desenvolupem noves formes d’entendre les dinàmiques de l’entorn. Finalment, a la Part III apliquem els mètodes prèviament desenvolupats per a aconseguir desafiaments d’un nivell superior, com, per exemple, segmentar obstacles dinàmics al mateix temps que estimem el seu vector de moviment respecte al terra. Concretament, al Capítol 2 detectem vehicles en 3D creant una arquitectura d’aprenentatge profund amb dues branques, i proposem una vista frontal (FR-V) i una vista d’ocell (BE-V) com a representacions 2D del núvol de punts 3D que serveixen com a punt de partida per entrenar els nostres models. Més endavant, al Capítol 3 apliquem i provem de nou aquest mètode en dos casos d’ús reals, tant per filtrar obstacles en moviment prèviament a la creació de mapes en els quals poder localitzar-nos millor en dies posteriors, com per dur a terme el seguiment de vehicles. Des de la perspectiva dinàmica, al Capítol 4 aprenem una nova característica dinàmica del núvol de punts en 3D que s’assembla al flux òptic sobre imatges RGB. Per a fer-ho, desenvolupem un nou enfocament que aprofita el flux òptic RGB com pseudo mostres reals per a entrenament, utilitzant només informació 3D durant la inferència. Després, al Capítol 5 explorem els beneficis que s’obtenen de combinar els aprenentatges de problemes de classificació i regressió per la tasca d’estimació de flux òptic de manera conjunta. Finalment, al Capítol 6 posem en comú els mètodes anteriors i demostrem que mitjançant aquests processos independents podem abordar l’aprenentatge de problemes més complexos, com la segmentació i estimació del moviment de vehicles des de la nostra pròpia perspectiva.

The increasing usage of electronics and software in a modern automobile enables realization of many advanced features. One such feature is autonomous driving. Autonomous driving means that a human driver’s intervention is not required to drive the automobile; rather, theautomobile is capable of driving itself. Achieving automobile autonomyrequires research in several areas, one of which is the area of automotive electrical/electronics (E/E) architectures. These architectures deal with the design of the computer hardware and software present inside various subsystems of the vehicle, with particular attention to their interaction and modularization. The aim of this thesis is to investigate how automotive E/E architectures should be designed so that 1) it ispossible to realize autonomous features and 2) a smooth transition canbe made from existing E/E architectures, which have no explicit support for autonomy, to future E/E architectures that are explicitly designed for autonomy.The thesis begins its investigation by considering the specific problem of creating autonomous behavior under cooperative driving condi-tions. Cooperative driving conditions are those where continuous wireless communication exists between a vehicle and its surroundings, which consist of the local road infrastructure as well as the other vehicles in the vicinity. In this work, we define an original reference architecture for cooperative driving. The reference architecture demonstrates how a subsystem with specific autonomy features can be plugged into an existing E/E architecture, in order to realize autonomous driving capabilities. Two salient features of the reference architecture are that it isminimally invasive and that it does not dictate specific implementation technologies. The reference architecture has been instantiated on two separate occasions and is the main contribution of this thesis. Another contribution of this thesis is a novel approach to the design of general, autonomous, embedded systems architectures. The approach introduces an artificial consciousness within the architecture, that understands the overall purpose of the system and also how the different existing subsystems should work together in order to meet that purpose.This approach can enable progressive autonomy in existing embedded systems architectures, over successive design iterations.

QC 20130412

Thesis: M.C.P., Massachusetts Institute of Technology, Department of Urban Studies and Planning, 2018.
Cataloged from PDF version of thesis.
Includes bibliographical references (pages 95-101).
Autonomous vehicles (AV) have captured the collective imagination of everyone from traditional auto manufacturers to computer software startups, from government administrators to urban planners. This thesis articulates a likely future for the deployment of AVs. Through stakeholder interviews and industry case studies, I show that there is general optimism about the progress of AV technology and its power to positively impact society. Stakeholders across sectors are expecting a future of autonomous electric fleets, but have divergent attitudes toward the regulation needed to facilitate its implementation. I demonstrate that, given the immense upfront capital investments and the nature of network effects intrinsic to data-intensive platforms, the autonomous mobility-as-a-service system is likely to tend toward a natural monopoly. This view is corroborated by key informants as well as recent industry trends. In order to better anticipate the characteristics of this emerging platform, I look back at the developmental trajectories of two classic public utilities - telecommunications and the electricity industry. I argue that the aspiring monopolists in autonomous mobility, like icons in these traditional industries, will succeed in supplanting a legacy technology with a new, transformative one, and use pricing and market consolidation tactics to gain regional dominance. The discussion on monopoly power is then adapted to the new business models of internet-enabled technology giants, and I examine two additional industry case studies in Google and Amazon. I argue that the autonomous mobility platform will first be designed to prioritize scale over everything else, including profits, and that firms are likely to pursue both horizontal and vertical integration strategies to achieve sustained market leadership. I conclude by recommending next steps for reining in platforms that may harm the public interest, and encourage planners to traverse disciplinary boundaries to better facilitate discussions between innovators and regulators.
by Joanne Wong.
M.C.P.

Thesis: S.M., Massachusetts Institute of Technology, Department of Aeronautics and Astronautics, 2018.
Cataloged from PDF version of thesis.
Includes bibliographical references (pages 89-91).
Driving is often stressful and dangerous due to uncertainty in the actions of nearby vehicles. Having the ability to model driving maneuvers qualitatively and guarantee safety bounds in uncertain traffic scenarios are two steps towards building trust in vehicle autonomy. In this thesis, we present an approach to the problem of Qualitative Autonomous Driving (QAD) using risk-bounded conditional planning. First, we present Incremental Risk-aware AO* (iRAO*), an online conditional planning algorithm that builds off of RAO* for use in larger dynamic systems like driving. An illustrative example is included to better explain the behavior and performance of the algorithm. Second, we present a Chance-Constrained Hybrid Multi-Agent MDP as a framework for modeling our autonomous vehicle in traffic scenarios using qualitative driving maneuvers. Third, we extend our driving model by adding variable duration to maneuvers and develop two approaches to the resulting complexity. We present planning results from various driving scenarios, as well as from scaled instances of the illustrative example, that show the potential for further applications. Finally, we propose a QAD system, using the different tools developed in the context of this thesis, and show how it would fit within an autonomous driving architecture.
by Matthew Quinn Deyo.
S.M.

This dissertation intends to provide theoretical and practical contributions on the development of deep learning algorithms for autonomous driving applications. The research is motivated by the need of deep neural networks (DNNs) to get a full understanding of the surrounding area and to be executed on real driving scenarios with real vehicles equipped with specific hardware, such as memory constrained (DSP or GPU platforms) or multiple optical sensors, which constraints the algorithm's development forcing the designed deep networks to be accurate, with minimum number of operations and low memory consumption. The main objective of this thesis is, on one hand, the research in the actual limitations of DL-based algorithms that prevent them of being integrated in nowadays' ADAS (Autonomous Driving System) functionalities, and on the other hand, the design and implementation of deep learning algorithms able to overcome such constraints to be applied on real autonomous driving scenarios, enabling their integration in low memory hardware platforms and avoiding sensor redundancy. Deep learning (DL) applications have been widely exploited over the last years but have some weak points that need to be faced and overcame in order to fully integrate DL into the development process of big manufacturers or automotive companies, like the time needed to design, train and validate and optimal network for a specific application or the vast knowledge from the required experts to tune hyperparameters of predefined networks in order to make them executable in the target platform and to obtain the biggest advantage of the hardware resources. During this thesis, we have addressed these topics and focused on the implementations of breakthroughs that would help in the industrial integration of DL-based applications in the automobile industry. This work has been done as part of the "Doctorat Industrial" program, at the company FICOSA ADAS, and it is because of the possibilities that developing this research at the company's facilities have brought to the author, that a direct impact of the achieved algorithms could be tested on real scenarios to proof their validity. Moreover, in this work, the author investigates deep in the automatic design of deep neural networks (DNN) based on state-of-the-art deep learning frameworks like NAS (neural architecture search). As stated in this work, one of the identified barriers of deep learning technology in nowadays automobile companies is the difficulty of developing light and accurate networks that could be integrated in small system on chips (SoC) or DSP. To overcome this constraint, the author proposes a framework named E-DNAS for the automatic design, training and validation of deep neural networks to perform image classification tasks and run on resource-limited hardware platforms. This apporach have been validated on a real system on chip by the company Texas Instrumets (tda2x) provided by the company, whose results are published within this thesis. As an extension of the mentioned E-DNAS, in the last chapter of this work the author presents a framework based on NAS valid for detecting objects whose main contribution is a learnable and fast way of finding object proposals on images that, on a second step, will be classified into one of the labeled classes.
Esta disertación tiene como objetivo principal proporcionar contribuciones teóricas y prácticas sobre el desarrollo de algoritmos de aprendizaje profundo para aplicaciones de conducción autónoma. La investigación está motivada por la necesidad de redes neuronales profundas (DNN) para obtener una comprensión completa del entorno y para ejecutarse en escenarios de conducción reales con vehículos reales equipados con hardware específico, los cuales tienen memoria limitada (plataformas DSP o GPU) o utilizan múltiples sensores ópticos Esto limita el desarrollo del algoritmo obligando a las redes profundas diseñadas a ser precisas, con un número mínimo de operaciones y bajo consumo de memoria y energía. El objetivo principal de esta tesis es, por un lado, investigar las limitaciones reales de los algoritmos basados en DL que impiden que se integren en las funcionalidades ADAS (Autonomous Driving System) actuales, y por otro, el diseño e implementación de algoritmos de aprendizaje profundo capaces de superar tales limitaciones para ser aplicados en escenarios reales de conducción autónoma, permitiendo su integración en plataformas de hardware de baja memoria y evitando la redundancia de sensores. Las aplicaciones de aprendizaje profundo (DL) se han explotado ampliamente en los últimos años, pero tienen algunos puntos débiles que deben enfrentarse y superarse para integrar completamente la DL en el proceso de desarrollo de los grandes fabricantes o empresas automobilísticas, como el tiempo necesario para diseñar, entrenar y validar una red óptima para una aplicación específica o el vasto conocimiento de los expertos requeridos para tunear hiperparámetros de redes predefinidas con el fin de hacerlas ejecutables en una plataforma concreta y obtener la mayor ventaja de los recursos de hardware. Durante esta tesis, hemos abordado estos temas y nos hemos centrado en las implementaciones de avances que ayudarían en la integración industrial de aplicaciones basadas en DL en la industria del automóvil. Este trabajo se ha realizado en el marco del programa "Doctorat Industrial", en la empresa FICOSA ADAS, y es por las posibilidades que la empresa ha ofrecido que se ha podido demostrar un impacto rápido y directo de los algoritmos conseguidos en escenarios de test reales para probar su validez. Además, en este trabajo, se investiga en profundidad el diseño automático de redes neuronales profundas (DNN) basadas en frameworks de deep learning de última generación como NAS (neural architecture search). Como se afirma en esta tesis, una de las barreras identificadas de la tecnología de aprendizaje profundo en las empresas automotrices de hoy en día es la dificultad de desarrollar redes ligeras y precisas que puedan integrarse en pequeños systems on chip(SoC) o DSP. Para superar esta restricción, se propone un framework llamado E-DNAS para el diseño automático, entrenamiento y validación de redes neuronales profundas para realizar tareas de clasificación de imágenes y ejecutarse en plataformas de hardware con recursos limitados. Este apporach ha sido validado en un system on chip real de la empresa Texas Instrumets (tda2x) facilitado por FICOSA ADAS, cuyos resultados se publican dentro de esta tesis. Como extensión del mencionado E-DNAS, en el último capítulo de este trabajo se presenta un framework basado en NAS válido para la detección de objetos cuya principal contribución es una forma fácil y rápida de encontrar propuestas de objetos en imágenes que, en un segundo paso, se clasificará en una de las clases etiquetadas.
Automàtica, robòtica i visió