Chris Pollett > Students >

    ( Print View)



    [CS 297 Proposal]

    [Deliverable 1]

    [Comparison between ARKit and ARCore - PDF]

    [2D to 3D Cartoon Modeling - PDF]

    [YOLO Object Detection iOS - PDF]

    [Deliverable 2]

    [Deliverable 3]

    [ARCritique - PDF]

    [Deliverable 4]

    [CS 297 Report - PDF]

    [CS 298 Proposal]

    [CS 298 Report - 3D AR Reconstruction - PDF]

    [3D AR Reconstruction Slides - PDF]

CS298 Proposal

Shoe AR Reconstruction

Sneh Kothari (

Advisor: Dr. Chris Pollett

Committee Members: Dr. William Andreopoulos, Dr. Robert Chun


People generally want opinions from others when buying shoes offline. Clicking and sending images of the shoe is not an ideal solution as it does not give the complete feel of the shoe. This problem can be solved using Shoe AR Reconstruction an app intended to make shoe shopping easier. The user clicks a photo/video of a shoe. This image data is converted to form a mesh that can be shared. On receiving a model the user can open it in the app and interact with it to get a better look and feel of the shoe. This can be particularly useful in something like Amazon where sellers can click images to build a mesh and customers can render this mesh for a better buying experience. The solution to this is building an Augmented Reality app that build 3D mesh of shoes and allows users to make an informed decision about the product they are buying.

CS297 Results

  • Built a demo Augmented Reality App using frameworks provided by Apple. Compared different ways to build an AR app.
  • Explored different ways to build a 3D mesh from 2D image data. Concluded that the mesh building will happen on server side.
  • Implemented deep linking to make sharing easier and pass data along with the link
  • Built an app to view and interact with shoes in Augmented Reality. Tried different versions and decided on one.

Proposed Schedule

Week 1: Jan 31 - Feb 7Organizational meeting with advisor
Week 2: Feb 8 - Feb 14Read blogs and research papers and come up with a plan for the model building
Week 3: Feb 15 - Feb 21Implement 3D model building on server
Week 4: Feb 22 - Feb 28Implement 3D model building on server
Week 5: Mar 1 - Mar 7Implement 3D model building on server
Week 6: Mar 8 - Mar 14Implement 3D model building on server
Week 7: Mar 15 - Mar 21Build app to capture images for building 3D model
Week 8: Mar 22 - Mar 28Build app to display 3D model
Week 9: Mar 29 - Apr 4Build app to display 3D model
Week 10: Apr 5 - Apr 11Integrate server image with app and connect using deep links
Week 11: Apr 12 - Apr 18Integrate server image with app and connect using deep links
Week 12: Apr 19 - Apr 25Start CS 298 Report and Presentation
Week 13: Apr 26 - May 2Work on CS 298 Report and Presentation
Week 14: May 3 - May 9Finish CS 298 Report and Presentation

Key Deliverables:

  • Software
    • Code which runs on server to build a 3D mesh from 2D image data
    • Build an app that allows users to capture images that will be used for building the 3D model
    • Implement a viewing experience for the shoe model in Augmented Reality
  • Report
    • CS 298 Report
    • CS 298 Presentation

Innovations and Challenges

  • Figuring out how to build a 3D mesh from 2D data
  • Building an app that lets users interact with 3D objects and have a good experience


[2020] ARKit and ARCore in serve to augmented reality. Z. Oufqir, A. El Abderrahmani and K. Satori. International Conference on Intelligent Systems and Computer Vision (ISCV). 2020.

[2017] MagicToon: A 2D-to-3D creative cartoon modeling system with mobile AR. L. Feng, X. Yang and S. Xiao. IEEE Virtual Reality (VR). 2017.

[2018] Integrating YOLO Object Detection with Augmented Reality for iOS Apps. S. Mahurkar. IEEE Annual Ubiquitous Computing, Electronics and Mobile Communication Conference. 2018.

[2021] ARCritique: Supporting Remote Design Critique of Physical Artifacts through Collaborative Augmented Reality. Y. Li, D. Hicks, W. S. Lages, S. Won Lee, A. Sharma and D. A. Bowman. 2021 IEEE Conference on Virtual Reality and 3D User Interfaces