CS297 Proposal

Quantifying Deep Fake Detection Accuracy for a Variety of Natural Settings.

Pratikkumar Prajapati (pratikkumar.prajapati@sjsu.edu)

Advisor: Dr. Chris Pollett

Description:

In this era of fake news, fake videos are on the rise and it can lead to catastrophic consequences in society; imagine a fake video
of a political leader announcing something horrific! Four types of facial manipulations are popular to generate fake videos a.k.a.
Deep Fakes. The techniques used to generate Deep Fakes are (i) entire face synthesis, (ii) face identify swap, (iii) facial attributes
manipulation, and (iv) facial expression manipulation. Machine learning techniques, typically autoencoders and generative adversarial
networks (GANs) are used to generate such fake videos. With the advancement of artificial intelligence, many researchers have proposed
various models to detect Deep Fakes, but fake video generation techniques are becoming smarter and more realistic fake videos are getting
generated easily. In this project, we would quantify Deep Fakes generation and detection accuracy for a variety of natural settings.
We would propose a GAN model to generate and detect deep fakes and analyze its accuracy.

Schedule:

Week 1: Jan 28 - Feb 03Kickoff meeting, and draft schedule.
Week 2: Feb 04 - Feb 10PyTorch bootcamp.
Week 3: Feb 11 - Feb 17Work on Deliverable 1.
Week 4: Feb 18 - Feb 24Deliverable 1 due. Understand Autoencoder - chapter [1].
Week 5: Feb 25 - Mar 02Work on Deliverable 2.
Week 6: Mar 03 - Mar 09Deliverable 2 due. Understand GAN - paper [2].
Week 7: Mar 10 - Mar 16Work on Deliverable 3.
Week 8: Mar 17 - Mar 23Deliverable 3 due. Read paper [3], [4].
Week 9: Mar 24 - Mar 30Start working on deliverable 4 to swap face in image.
Week 10: Mar 31 - Apr 06Spring break (May 30- Apr 03)
Week 11: Apr 08 - Apr 13Continue,
Week 12: Apr 14 - Apr 20Deliverable 4 due. Read paper [5].
Week 13: Apr 21 - Apr 27Start working on deliverable 5 to extend face-swapping for video.
Week 14: Apr 28 - May 04Continue,
Week 15: May 05 - May 11Deliverable 5 due. Work on CS 297 report.
Week 16: May 12thDeliverable 6 due.

Deliverables:

The full project will be done when CS298 is completed. The following will be done by the end of CS297:

1. Implement CNN in PyTorch to classify Devanagari characters.

2. Implement Autoencoder in PyTorch to generate Devanagari characters.

3. Implement GAN in PyTorch to generate Devanagari characters.

4. Face-swap GAN implementation for Image.

5. Develop a framework of face-swapping GAN for Video in a newsroom setting. This framework would be extended and improved as part of CS 298.

6. CS 297 report due.

References:

[1] Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. The MIT Press, ch. 14

[2] Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio. 2014.
Generative adversarial networks. CoRR abs/1406.2661

[3] Tolosana, Ruben, et al. "DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection." arXiv preprint arXiv:2001.00179 (2020).

[4] Hukkelas, H.; Mester, R.; and Lindseth, F. 2019. Deepprivacy: A generative adversarial network for face anonymization. In Bebis, G.; Boyle, R.;
Parvin, B.; Koracin, D.; Ushizima, D.; Chai, S.; Sueda, S.; Lin, X.; Lu, A.; Thalmann, D.; Wang, C.; and Xu, P., eds., Advances in Visual Computing,
565-578. Cham: Springer International Publishing.

[5] E. Sabir, J. Cheng, A. Jaiswal, W. AbdAlmageed, I. Masi, and P. Natarajan, "Recurrent Convolutional Strategies for Face Manipulation Detection in
Videos," in Proc. Conference on Computer Vision and Pattern Recognition Workshops, 2019.