Abstract:

This project strives to build an Android App that is capable of collecting the road scenery and car driving data and uses such data to train a neural network model to provide the correct instructions during autonomous driving. It is a fact that every road has already been driven on by someone at some point and will likely be driven on by someone again in the future. It is also a fact that mobile phones are ubiquitous, so it seems promising to collect car driving data using a mobile application and then use that data to train autonomous vehicle AI systems. We will in this project develop applications to actually do this. We will use the Convolutional Neural Network (CNN) model and the training will be done through Python sklearn library.

CS297 Results

I have developed and tested three separate Android Apps to detect a user's real time GPS location, to capture a vehicle's movement to correlate with steering wheel's movement direction and degrees, and to record the traffic scenery and save to an external storage. All serves as the preparation for the coming integrated autonomous driving App.

Proposed Schedule

Key Deliverables:

• Software
  • Fully integrated Android App to record roadside scenery video, real time car location, and car motion status, and send the collected data to the database
  • TensorFlow model to process the image and video data collected by the App during car's driving with reasonable accuracy
  • Explore possible boost of processing power for image and video training
• Report
  • How to select the correct features to train TensorFlow model

Innovations and Challenges

• Ability to process different sources of data collected during car's motion by the integrated Android App is one challenge and innovation of this project. The main gain of this App is to crowd source data collection so the data generated by different users are going to be processed.
• Another innovation is the different types of data that the App can collect:real time car location, car's motion status, and roadside scenery recording.
• The third challenge and innovation is to efficiently process the streaming videos as TensorFlow model. This requires very solid processing power to generate the recommendation for steering wheels in a timely manner.

References:

[Long2007] A Review of Intelligent Systems Software for Autonomous Vehicles. Long,L., Hanford,S., Janrathitikarn, O., Sinsley, G., Miller, J. Proceedings of the 2007 IEEE Symposium on Computational Intelligence in Security and Defense Applications. 2007.

[Menze2015] Object Scene Flow for Autonomous Vehicles. Menze,M.,Geiger,A. Computer Vision Foundation. 2015.

[Pomerleau1991] Efficient Training of Artificial Neural Networks for Autonomous. Navigation. Pomerleau, D.A. In Neural Computation 3:1 pp. 88-97. 1991.