Abstract:

Photos taken by digital cameras leave identifiable traces imprinted on the resulting image. Digital camera models have unique image processing pipelines that generate noise in a deterministic manner that can be used to identify the camera source. These fingerprints that are left behind can be estimated from a series of images of a camera and then be used to identify the make and model of the camera that took the image. After denoising an image, a noise pattern that is taken from a non original camera can be added to the denoised image to fool a discriminator model that classifies images based on camera origin. Various models and techniques such as principle component analysis and conditional generative adversarial networks will be used to achieve a generative model. Face swapping techniques will be applied to verify the robustness of the model against image modifications.

Keywords – Machine learning, computer vision, image forensics, Generative Adversarial Network (GAN), wavelet transform,

CS297 Results

• Found sensor noise pattern for my own phone based on averages
• Calculated noise print with subtraction of average pixel value on a photo taken from a personal phone
• Implemented, analyzed, and compared a variety of methods for denoising using different types of wavelet transforms and noise generation methods
• Ran benchmarking models on a local machine to obtain an understanding on modern benchmarks for this type of problem
• Implemented a GAN network to generate and identify digits on the MNIST dataset

Proposed Schedule

Key Deliverables:

• A software application written in Python capable of:
  • Removing camera sensor noise data from images while retaining most details of the image
  • Identifying camera make and model from an image taken by cameras from a fixed list of cameras using sensor noise data
  • Modifying sensor noise data in an image to make it look like the image was taken from a particular camera from a fixed list
  • Face swapping capabilities for images that contain faces
• Report
  • CS298 Report
  • CS298 Presentation

Innovations and Challenges

• This will be the only currently known system to attempt to denoise and overlay a different noise pattern from a target phone model onto an image.
• The use of embedded layers in a conditional generative adversarial network will be difficult.
• High levels of accuracy on discriminatory models are necessary to prove reliable and permissible in the court of law. This program would be able to prove that noise spoofing render noise discrimation models unfit to be used in court.
• PCA would be able to vastly speed up training time with dimensionality reductions. Most noise discrimination models are complex and have long training times.

References:

[1] J. Lukas, J. Fridrich and M. Goljan, "Digital camera identification from sensor pattern noise," in IEEE Transactions on Information Forensics and Security, vol. 1, no. 2, pp. 205-214, June 2006, doi: 10.1109/TIFS.2006.873602.

[2] D. Cozzolino and L. Verdoliva, "Noiseprint: A CNN-Based Camera Model Fingerprint," in IEEE Transactions on Information Forensics and Security, vol. 15, pp. 144-159, 2020, doi: 10.1109/TIFS.2019.2916364.

[3] S. Samaras, V. Mygdalis and I. Pitas, "Robustness in blind camera identification," 2016 23rd International Conference on Pattern Recognition (ICPR), 2016, pp. 3874-3879, doi: 10.1109/ICPR.2016.7900239.

[4] L. Bondi, L. Baroffio, D. Güera, P. Bestagini, E. J. Delp and S. Tubaro, "First Steps Toward Camera Model Identification With Convolutional Neural Networks," in IEEE Signal Processing Letters, vol. 24, no. 3, pp. 259-263, March 2017, doi: 10.1109/LSP.2016.2641006.

[5] R. Li, Y. Guan and C. Li, "PCA-based denoising of Sensor Pattern Noise for source camera identification," 2014 IEEE China Summit & International Conference on Signal and Information Processing (ChinaSIP), 2014, pp. 436-440, doi: 10.1109/ChinaSIP.2014.6889280.