Chris Pollett > Students >
Sneh
    ( Print View)

    [Bio]

    [Blog]

    [CS 297 Proposal]

    [Deliverable 1]

    [Comparison between ARKit and ARCore - PDF]

    [2D to 3D Cartoon Modeling - PDF]

    [YOLO Object Detection iOS - PDF]

    [Deliverable 2]

    [Deliverable 3]

    [ARCritique - PDF]

    [Deliverable 4]

    [CS 297 Report - PDF]

    [CS 298 Proposal]

    [CS 298 Report - 3D AR Reconstruction - PDF]

    [3D AR Reconstruction Slides - PDF]

CS297 Proposal

Shoe AR Reconstruction

Sneh Kothari (sneharvind.kothari@sjsu.edu)

Advisor: Dr. Chris Pollett

Description:

People generally want opinions from others when buying shoes offline. Clicking and sending images of the shoe is not an ideal solution as it does not give the complete feel of the shoe. This problem can be solved using Shoe AR Reconstruction an app intended to make shoe shopping easier. The user clicks a photo/video of a shoe. This image data is converted to form a mesh that can be shared. On receiving a model the user can open it in the app and interact with it to get a better look and feel of the shoe. This can be particularly useful in something like Amazon where sellers can click images to build a mesh and customers can render this mesh for a better buying experience.

Schedule:

Week 1: Aug 24-Aug 30Introductory
Week 2: Aug 31-Sep 6Finalize Proposal, Read up on ARKit, RealityKit.
Week 3: Sep 7-Sep 13Work on paper [1]
Week 4: Sep 14-Sep 20Work on deliverable [1]
Week 5: Sep 21-Sep 27Work on paper [2]
Week 6: Sep 28-Oct 4Work on paper [3]
Week 7: Oct 5-Oct 11Work on deliverable [2]
Week 8: Oct 12-Oct 18Work on deliverable [2]
Week 9: Oct 19-Oct 25Work on deliverable [3]
Week 10: Oct 26-Nov 1Work on deliverable [3]
Week 11: Nov 2-Nov 8Work on paper [4]
Week 12: Nov 9-Nov 15Work on deliverable [4]
Week 13: Nov 16-Nov 22Work on deliverable [4]
Week 14: Nov 23-Nov 29Work on deliverable [4]
Week 15: Nov 30-Dec 6Work on deliverable [5]

Deliverables:

The full project will be done when CS298 is completed. The following will be done by the end of CS297:

1. Implement a basic AR app for displaying teapots in different settings and allowing resizing interaction.
Also saving the rooms with the updated pot sizes.

2. Build an AR app implementing object capture to capture 2D image data for 3D modeling.
The object capture will be tested for increasingly complex objects starting from a tennis ball. Work on making two different types of meshes.

3. Try out different methods of sharing/compression for AR models.
Examine mesh sizes and explore sharing options. These can be peer-to-peer(iMessage) or server-to-device(S3 buckets).
Implement deep link for the shared file.

4. Render a 3D model on the user's app and try multiple versions of interaction with the object.

  • Fix the model in space and the user moves the phone to examine the model
  • The phone does not move and the user can spin the model and interact with gestures

5. Finish report.

References:

[2020] ARKit and ARCore in serve to augmented reality. Z. Oufqir, A. El Abderrahmani and K. Satori. International Conference on Intelligent Systems and Computer Vision (ISCV). 2020.

[2017] MagicToon: A 2D-to-3D creative cartoon modeling system with mobile AR. L. Feng, X. Yang and S. Xiao. IEEE Virtual Reality (VR). 2017.

[2018] Integrating YOLO Object Detection with Augmented Reality for iOS Apps. S. Mahurkar. IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conference. 2018.

[2021] ARCritique: Supporting Remote Design Critique of Physical Artifacts through Collaborative Augmented Reality. Y. Li, D. Hicks, W. S. Lages, S. Won Lee, A. Sharma and D. A. Bowman. 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). 2021