CS298 Proposal
Crowd-sourcing Data for Autonomous Driving and Applying It
Yaoyan Xi (Yaoyan.Xi@sjsu.edu)
Advisor: Dr. Chris Pollett (Chris@Pollett.org)
Committee Members: Dr. Robert Chun (Robert.Chun@sjsu.edu)
Dr. Mingkun Li (Mingkun_Li@apple.com)
Abstract:
This project strives to build an Android App that is capable of collecting the road scenery and car driving data and uses such data to train a neural network model to provide the correct instructions during autonomous driving. It is a fact that every road has already been driven on by someone at some point and will likely be driven on by someone again in the future. It is also a fact that mobile phones are ubiquitous, so it seems promising to collect car driving data using a mobile application and then use that data to train autonomous vehicle AI systems. We will in this project develop applications to actually do this. We will use the Convolutional Neural Network (CNN) model and the training will be done through Python sklearn library.
CS297 Results
I have developed and tested three separate Android Apps to detect a user's real time GPS location, to capture a vehicle's movement to correlate with steering wheel's movement direction and degrees, and to record the traffic scenery and save to an external storage. All serves as the preparation for the coming integrated autonomous driving App.
Proposed Schedule
Week 1:
Aug 25 - Aug 31 | Build fully integrated functional Android App |
Week 2:
Sep 1 - Sep 7 | Build fully integrated functional Android App |
Week 3:
Sep 8 - Sep 14 | Deliverable 1: Build fully integrated functional Android App that can collect real time location and car motion data. |
Week 4:
Sep 15 - Sep 21 | Build database and backend connection with integrated App |
Week 5:
Sep 22 - Sep 28 | Build database and backend connection with integrated App |
Week 6:
Sep 29 - Oct 5 | Deliverable 2: Build database and backend connection infrastructure capable of saving and retrieving the data |
Week 7:
Oct 6 - Oct 12 | Testing to collect static images and video with App in real cars |
Week 8:
Oct 13 - Oct 19 | Testing to collect static images and video with App in real cars |
Week 9:
Oct 20 - Oct 26 | Deliverable 3: build dataset from cars with App |
Week 10:
Oct 27 - Nov 2 | Build and train TensorFlow model |
Week 11:
Nov 3 - Nov 9 | Build and train TensorFlow model |
Week 12:
Nov 10 - Nov 16 | Build and train TensorFlow model |
Week 13:
Nov 17 - Nov 23 |
Deliverable 4: Build and train TensorFlow model with the dataset built in Deliverable 4 to recommend driving instructions for cars
|
Week 14:
Nov 24 - Nov 30 | Composing report |
Week 15:
Dec 1 - Dec 7 | Deliverable 5: Composing report |
Week 16:
Dec 8 - Dec 14 |
Deliverable 6 - Presentation
|
Key Deliverables:
- Software
- Fully integrated Android App to record roadside scenery video, real time car location, and car motion status, and send the collected data to the database
- TensorFlow model to process the image and video data collected by the App during car's driving with reasonable accuracy
- Explore possible boost of processing power for image and video training
- Report
- How to select the correct features to train TensorFlow model
Innovations and Challenges
- Ability to process different sources of data collected during car's motion by the integrated Android App is one challenge and innovation of this project. The main gain of this App is to crowd source data collection so the data generated by different users are going to be processed.
- Another innovation is the different types of data that the App can collect:real time car location, car's motion status, and roadside scenery recording.
- The third challenge and innovation is to efficiently process the streaming videos as TensorFlow model. This requires very solid processing power to generate the recommendation for steering wheels in a timely manner.
References:
[Long2007] A Review of Intelligent Systems Software for Autonomous Vehicles. Long,L., Hanford,S., Janrathitikarn, O., Sinsley, G., Miller, J. Proceedings of the 2007 IEEE Symposium on Computational Intelligence in Security and Defense Applications. 2007.
[Menze2015] Object Scene Flow for Autonomous Vehicles. Menze,M.,Geiger,A. Computer Vision Foundation. 2015.
[Pomerleau1991] Efficient Training of Artificial Neural Networks for Autonomous. Navigation. Pomerleau, D.A. In Neural Computation 3:1 pp. 88-97. 1991. |