CS297 Proposal
Shoe AR Reconstruction
Sneh Kothari (sneharvind.kothari@sjsu.edu)
Advisor: Dr. Chris Pollett
Description:
People generally want opinions from others when buying shoes offline.
Clicking and sending images of the shoe is not an ideal solution as it does not give the complete feel of the shoe.
This problem can be solved using Shoe AR Reconstruction an app intended to make shoe shopping easier.
The user clicks a photo/video of a shoe. This image data is converted to form a mesh that can be shared.
On receiving a model the user can open it in the app and interact with it to get a better look and feel of the shoe.
This can be particularly useful in something like Amazon where sellers can click images to build a mesh and customers can render this mesh for a better buying experience.
Schedule:
Week 1:
Aug 24-Aug 30 | Introductory |
Week 2:
Aug 31-Sep 6 | Finalize Proposal, Read up on ARKit, RealityKit. |
Week 3:
Sep 7-Sep 13 | Work on paper [1] |
Week 4:
Sep 14-Sep 20 | Work on deliverable [1] |
Week 5:
Sep 21-Sep 27 | Work on paper [2] |
Week 6:
Sep 28-Oct 4 | Work on paper [3] |
Week 7:
Oct 5-Oct 11 | Work on deliverable [2] |
Week 8:
Oct 12-Oct 18 | Work on deliverable [2] |
Week 9:
Oct 19-Oct 25 | Work on deliverable [3] |
Week 10:
Oct 26-Nov 1 | Work on deliverable [3] |
Week 11:
Nov 2-Nov 8 | Work on paper [4] |
Week 12:
Nov 9-Nov 15 | Work on deliverable [4] |
Week 13:
Nov 16-Nov 22 | Work on deliverable [4] |
Week 14:
Nov 23-Nov 29 | Work on deliverable [4] |
Week 15:
Nov 30-Dec 6 | Work on deliverable [5] |
Deliverables:
The full project will be done when CS298 is completed. The following will
be done by the end of CS297:
1. Implement a basic AR app for displaying teapots in different settings and allowing resizing interaction.
Also saving the rooms with the updated pot sizes.
2. Build an AR app implementing object capture to capture 2D image data for 3D modeling.
The object capture will be tested for increasingly complex objects starting from a tennis ball. Work on making two different types of meshes.
3. Try out different methods of sharing/compression for AR models.
Examine mesh sizes and explore sharing options. These can be peer-to-peer(iMessage) or server-to-device(S3 buckets).
Implement deep link for the shared file.
4. Render a 3D model on the user's app and try multiple versions of interaction with the object.
- Fix the model in space and the user moves the phone to examine the model
- The phone does not move and the user can spin the model and interact with gestures
5. Finish report.
References:
[2020] ARKit and ARCore in serve to augmented reality. Z. Oufqir, A. El Abderrahmani and K. Satori. International Conference on Intelligent Systems and Computer Vision (ISCV). 2020.
[2017] MagicToon: A 2D-to-3D creative cartoon modeling system with mobile AR. L. Feng, X. Yang and S. Xiao. IEEE Virtual Reality (VR). 2017.
[2018] Integrating YOLO Object Detection with Augmented Reality for iOS Apps. S. Mahurkar. IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conference. 2018.
[2021] ARCritique: Supporting Remote Design Critique of Physical Artifacts through Collaborative Augmented Reality. Y. Li, D. Hicks, W. S. Lages, S. Won Lee, A. Sharma and D. A. Bowman. 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). 2021 |