Chris Pollett > Students >

    ( Print View)




    [Deliverable 1: GPS App]

    [Deliverable 2: Image Feature Selection]

    [Deliverable 3: Accelerometer and Video Recording Apps]

    [Deliverable 4: Model Survey]

    [Deliverable 5: CS297 Report]

    [Menze Paper Review Slides]

    [Pomerleau Paper Review Slides]



CS298 Proposal

Crowd-sourcing Data for Autonomous Driving and Applying It

Yaoyan Xi (

Advisor: Dr. Chris Pollett (

Committee Members: Dr. Robert Chun (
Dr. Mingkun Li (


This project strives to build an Android App that is capable of collecting the road scenery and car driving data and uses such data to train a neural network model to provide the correct instructions during autonomous driving. It is a fact that every road has already been driven on by someone at some point and will likely be driven on by someone again in the future. It is also a fact that mobile phones are ubiquitous, so it seems promising to collect car driving data using a mobile application and then use that data to train autonomous vehicle AI systems. We will in this project develop applications to actually do this. We will use the Convolutional Neural Network (CNN) model and the training will be done through Python sklearn library.

CS297 Results

I have developed and tested three separate Android Apps to detect a user's real time GPS location, to capture a vehicle's movement to correlate with steering wheel's movement direction and degrees, and to record the traffic scenery and save to an external storage. All serves as the preparation for the coming integrated autonomous driving App.

Proposed Schedule

Week 1: Aug 25 - Aug 31Build fully integrated functional Android App
Week 2: Sep 1 - Sep 7Build fully integrated functional Android App
Week 3: Sep 8 - Sep 14Deliverable 1: Build fully integrated functional Android App that can collect real time location and car motion data.
Week 4: Sep 15 - Sep 21Build database and backend connection with integrated App
Week 5: Sep 22 - Sep 28Build database and backend connection with integrated App
Week 6: Sep 29 - Oct 5Deliverable 2: Build database and backend connection infrastructure capable of saving and retrieving the data
Week 7: Oct 6 - Oct 12Testing to collect static images and video with App in real cars
Week 8: Oct 13 - Oct 19Testing to collect static images and video with App in real cars
Week 9: Oct 20 - Oct 26Deliverable 3: build dataset from cars with App
Week 10: Oct 27 - Nov 2Build and train TensorFlow model
Week 11: Nov 3 - Nov 9Build and train TensorFlow model
Week 12: Nov 10 - Nov 16Build and train TensorFlow model
Week 13: Nov 17 - Nov 23 Deliverable 4: Build and train TensorFlow model with the dataset built in Deliverable 4 to recommend driving instructions for cars
Week 14: Nov 24 - Nov 30Composing report
Week 15: Dec 1 - Dec 7Deliverable 5: Composing report
Week 16: Dec 8 - Dec 14 Deliverable 6 - Presentation

Key Deliverables:

  • Software
    • Fully integrated Android App to record roadside scenery video, real time car location, and car motion status, and send the collected data to the database
    • TensorFlow model to process the image and video data collected by the App during car's driving with reasonable accuracy
    • Explore possible boost of processing power for image and video training
  • Report
    • How to select the correct features to train TensorFlow model

Innovations and Challenges

  • Ability to process different sources of data collected during car's motion by the integrated Android App is one challenge and innovation of this project. The main gain of this App is to crowd source data collection so the data generated by different users are going to be processed.
  • Another innovation is the different types of data that the App can collect:real time car location, car's motion status, and roadside scenery recording.
  • The third challenge and innovation is to efficiently process the streaming videos as TensorFlow model. This requires very solid processing power to generate the recommendation for steering wheels in a timely manner.


[Long2007] A Review of Intelligent Systems Software for Autonomous Vehicles. Long,L., Hanford,S., Janrathitikarn, O., Sinsley, G., Miller, J. Proceedings of the 2007 IEEE Symposium on Computational Intelligence in Security and Defense Applications. 2007.

[Menze2015] Object Scene Flow for Autonomous Vehicles. Menze,M.,Geiger,A. Computer Vision Foundation. 2015.

[Pomerleau1991] Efficient Training of Artificial Neural Networks for Autonomous. Navigation. Pomerleau, D.A. In Neural Computation 3:1 pp. 88-97. 1991.