Synthetic Data Generator and Vehicle Semantic Segmentation

The deep learning case study project
The test assignment for DL job offer
From the development of Synthetic Data Generator to DeepLab v3 fine tuning
Synthetic Data Generator developed in Unity3D
DeepLab v3 fine tuned in PyTorch environment
Few scripts for data formatting and data conversion written

This page is under construction.

[CLICK TO SHOW VIDEO]

Scene with orbiter camera

Unity3D Engine used to build synthetic image data generator
Camera orbits over 3D object in the center of scene and takes photos
Camera pitch and roll angles can vary
Camera yaw angle automatically adjusted to be oriented to object in the scene
Information about camera position and orientation are available in real-time
Photos can be taken in any moment
Generator predefined to take a photo each 10 degrees while orbiting the object

Real Scene

Generated Binary Mask

Generated Multiclass Mask

Generated Dataset

Generated Dataset available as follow

Original Image (1st Image)
Ground Truth (GT) Mask for Binary Semantic Segmentation (2nd Image)
GT Mask for Multiclass Semantic Segmentation (3rd Image)

DOWNLOAD DATASET

Original Image - Ground Truth Mask Pairs

Binary Semantic Segmentation - Two Classes (Background and Car)

Multiclass Semantic Segmentation - Several Classes (Background and Car parts)

DeepLab v3 Model for Semantic Segmentation

DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

In this work we address the task of semantic image segmentation with Deep Learning and make three main contributions that are experimentally shown to have substantial practical merit. First, we highlight convolution with upsampled filters, or ‘atrous convolution’, as a powerful tool in dense prediction tasks. Atrous convolution allows us to explicitly control the resolution at which feature responses are computed within Deep Convolutional Neural Networks. It also allows us to effectively enlarge the field of view of filters to incorporate larger context without increasing the number of parameters or the amount of computation. Second, we propose atrous spatial pyramid pooling (ASPP) to robustly segment objects at multiple scales. ASPP probes an incoming convolutional feature layer with filters at multiple sampling rates and effective fields-of-views, thus capturing objects as well as image context at multiple scales. Third, we improve the localization of object boundaries by combining methods from DCNNs and probabilistic graphical models. The commonly deployed combination of max-pooling and downsampling in DCNNs achieves invariance but has a toll on localization accuracy. We overcome this by combining the responses at the final DCNN layer with a fully connected Conditional Random Field (CRF), which is shown both qualitatively and quantitatively to improve localization performance. Our proposed “DeepLab” system sets the new state-of-art at the PASCAL VOC-2012 semantic image segmentation task, reaching 79.7% mIOU in the test set, and advances the results on three other datasets: PASCAL-Context, PASCAL-Person-Part, and Cityscapes. All of our code is made publicly available online.

Implementation

PyTorch Implementation
Transfer Learning Project from GitHub by Manpreet Singh
Fine Tuned Model for Car Segmentation Problem

DeepLab v3 Source Model

Whole project realized by Manpreet Singh posted at GitHub..

Car Segmentation Model

Fine tuned DeepLab v3 model. A common PyTorch convention is to save models using either a .pt or .pth file extension.

Page updated

Google Sites

Report abuse