Chris Pollett > Students >

    ( Print View)



    [CS 297 Proposal]

    [Deliverable 1]

    [Rapid Object Detection using a Boosted Cascade of Simple Features-PDF]

    [Deliverable 2]

    [Deliverable 3-PDF]

    [Deliverable 4]

    [CS297 Report-PDF]

    [CS298 Proposal]

    [CS298 Report-PDF]

    [CS298 Defense Slides-PDF]

CS297 Proposal

Detecting and Predicting Visual Affordance of objects in a given environment.

Bhumika Kaur Matharu (

Advisor: Dr. Chris Pollett


As stated by psychologist Gibson, 'The affordances of the environment are what it offers the animal, what it provides or furnishes, either for good or ill. The word affordance implies the complementarity of the animal and the environment'. The property of an object and how that object can be used by the user in the environment defines the affordances of the environment. In this project, we would predict the affordances of the environment from the learned affordances. We will create our dataset using Unity consisting of thousands of videos of grocery shopping store. On those videos, we will train our model and using these learned affordances we will try to predict the affordance of an unseen environment.


Week 1: Aug 25 - Aug 31Kickoff meeting, discuss topics and draft proposal
Week 2: Aug 31 - Sept 1Finalize topic, discuss deliverables and possible scope of project
Week 3: Sept 1 - Sept 8Keras Bootcamp
Week 4: Sept 8 - Sept 15Read about Image detection in Open CV and Keras. Continue working on Deliverable 1
Week 5: Sept 15 - Sept 22Perform object detection using OpenCV
Week 6: Sept 22 - Sept 29Look at possible approaches and resolve issues related to Deliverable 1
Week 7: Sept 29 - Oct 06Progress on Deliverable 1
Week 8: Oct 06 - Oct 13Show progress and complete Deliverable 1. Deliverable 1 Due
Week 9: Oct 13 - Oct 19Learn Unity to create video dataset.
Week 10: Oct 20 - Oct 26Create sample videos for the dataset
Week 11: Oct 27 - Nov 02Deliverable 2 Due.
Week 12: Nov 03 - Nov 09Research AI techniques for Affordance Prediction.
Week 13: Nov 10 - Nov 16Measure efficiency of chosen AI techniques. Decide AI approach and architecture.
Week 14: Nov 17 - Nov 23Deliverable 3 Due.
Week 15: Dec 01 - Dec 07Start CS297 report. Revise draft of report.
Week 16: Dec 08 - Dec 14Deliverable 4 due - CS 297 report.


The full project will be done when CS298 is completed. The following will be done by the end of CS297:

1. Perform object detection and image classification using OpenCV and Keras

2. Generate synthetic video dataset of Humanoid picking objects using Unity

3. Research existing techniques(or methodologies) to predict the visual affordance of unseen environment.

4. Try to implement model architecture of one of the selected research techniques

5. CS 297 report due.


[1] Mohammed Hassanin, Salman Khan and Murat Tahtali, 2018, Visual Affordance and Function Understanding: A Survey (

[2] Tucker Hermans, James M. Rehg and Aaron Bobick. Affordance Prediction via Learned Object Attributes (

[3] Deep Learning with PyTorch: A 60 minute blitz (

[4] Paul Viola, Michael Jones, 2001, Rapid Object Detection using a Boosted Cascade of Simple Features
( Paper )

[5] Haoxiang Li, Zhe Lin, Xiaohui Shen, Jonathan Brandt, Gang Hua, 2015, A Convolutional Neural Network Cascade for Face Detection
( Paper )