CS297 Proposal
Quantifying Deep Fake Detection Accuracy for a Variety of Natural Settings.
Pratikkumar Prajapati (pratikkumar.prajapati@sjsu.edu)
Advisor: Dr. Chris Pollett
Description:
In this era of fake news, fake videos are on the rise and it can lead to catastrophic consequences in society; imagine a fake video
of a political leader announcing something horrific! Four types of facial manipulations are popular to generate fake videos a.k.a.
Deep Fakes. The techniques used to generate Deep Fakes are (i) entire face synthesis, (ii) face identify swap, (iii) facial attributes
manipulation, and (iv) facial expression manipulation. Machine learning techniques, typically autoencoders and generative adversarial
networks (GANs) are used to generate such fake videos. With the advancement of artificial intelligence, many researchers have proposed
various models to detect Deep Fakes, but fake video generation techniques are becoming smarter and more realistic fake videos are getting
generated easily. In this project, we would quantify Deep Fakes generation and detection accuracy for a variety of natural settings.
We would propose a GAN model to generate and detect deep fakes and analyze its accuracy.
Schedule:
Week 1:
Jan 28 - Feb 03 | Kickoff meeting, and draft schedule. |
Week 2:
Feb 04 - Feb 10 | PyTorch bootcamp. |
Week 3:
Feb 11 - Feb 17 | Work on Deliverable 1. |
Week 4:
Feb 18 - Feb 24 | Deliverable 1 due. Understand Autoencoder - chapter [1]. |
Week 5:
Feb 25 - Mar 02 | Work on Deliverable 2. |
Week 6:
Mar 03 - Mar 09 | Deliverable 2 due. Understand GAN - paper [2]. |
Week 7:
Mar 10 - Mar 16 | Work on Deliverable 3. |
Week 8:
Mar 17 - Mar 23 | Deliverable 3 due. Read paper [3], [4]. |
Week 9:
Mar 24 - Mar 30 | Start working on deliverable 4 to swap face in image. |
Week 10:
Mar 31 - Apr 06 | Spring break (May 30- Apr 03) |
Week 11:
Apr 08 - Apr 13 | Continue, |
Week 12:
Apr 14 - Apr 20 | Deliverable 4 due. Read paper [5]. |
Week 13:
Apr 21 - Apr 27 | Start working on deliverable 5 to extend face-swapping for video. |
Week 14:
Apr 28 - May 04 | Continue, |
Week 15:
May 05 - May 11 | Deliverable 5 due. Work on CS 297 report. |
Week 16:
May 12th | Deliverable 6 due. |
Deliverables:
The full project will be done when CS298 is completed. The following will
be done by the end of CS297:
1. Implement CNN in PyTorch to classify Devanagari characters.
2. Implement Autoencoder in PyTorch to generate Devanagari characters.
3. Implement GAN in PyTorch to generate Devanagari characters.
4. Face-swap GAN implementation for Image.
5. Develop a framework of face-swapping GAN for Video in a newsroom setting. This framework would be extended and improved as part of CS 298.
6. CS 297 report due.
References:
[1] Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. The MIT Press, ch. 14
[2] Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio. 2014.
Generative adversarial networks. CoRR abs/1406.2661
[3] Tolosana, Ruben, et al. "DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection." arXiv preprint arXiv:2001.00179 (2020).
[4] Hukkelas, H.; Mester, R.; and Lindseth, F. 2019. Deepprivacy: A generative adversarial network for face anonymization. In Bebis, G.; Boyle, R.;
Parvin, B.; Koracin, D.; Ushizima, D.; Chai, S.; Sueda, S.; Lin, X.; Lu, A.; Thalmann, D.; Wang, C.; and Xu, P., eds., Advances in Visual Computing,
565-578. Cham: Springer International Publishing.
[5] E. Sabir, J. Cheng, A. Jaiswal, W. AbdAlmageed, I. Masi, and P. Natarajan, "Recurrent Convolutional Strategies for Face Manipulation Detection in
Videos," in Proc. Conference on Computer Vision and Pattern Recognition Workshops, 2019.
|