Purpose: The goal for this deliverable is to first discover what a sensor noise pattern would look like from pictures with my own camera. The first step was to take a series of pictures using my cellphone's camera. I set up my camera and took a series of images of a black piece of carboard with the camera in a fixed position along with fixed position light source. The reasoning behind this setup was to capture the noise generated by the camera's internal components and minimize other sources of noise due to environmental factors. I hoped to generate some sort of fingerprint based off of the noise from these images. In order to generate noise in this fashion I used the Python Image library (PIL) to process the images. I looped through all of the images and stored all rgbs values of the image to a 3d array. I then created a new 3d array that had entries that represented the average (R,G,B) value of each pixel for each pixel location. I then subtracted that value from each picture array to get a new array that represented the sensor noise pattern for each corresponding picture. This was the quickest rudimentary test that we came up with to get a sensor pattern.

Conclusion: Assumed that there would be a majority of black pixels due to the the average of pictures almost identical would deviate very little from each other, thus producing a delta from the average. This is due to the fact that low values in (R,G,B) arrays correspond to dark colors. However, the resulting pattern had more white pixels than expected. We assume this is because when we converted signed ints to unsigned ints, the two-compliment representation causes the conversion to turn small negatives into large numbers. This turns some spots white. On average some spots will be small numbers and other spots will be large numbers (represented by white and black pixels). Also noticed larger blocks of pixels near the edges of the picture and finer grains of noise near the center. This indicates that the camera is more sensitive towards the middle of the lens. It is also possible that whiter images have less noise (larger blocks in the noise map). Upon comparing the my experiment with that of the paper 1's, I noticed similar and different results/methods. Paper 1 used pictures of a light box and used digital cameras with infinite exposure settings. The white balance was also set to gray to avoid saturation from overwriting noise. This method was to avoid dark current that is present in dark pixels from over influencing the noise gradient since dark current is more heavily influenced by environmental factors. They wanted the fingerprint of the camera to be base on component manufacturing deviations that is consistent among any images taken by the camera. One similar conclusion that we both noticed was the vignette of the image. The sensor seemed to be less sensitive on the edges of the screen than the middle. The paper did a lot more tests than I did with different transformation functions that just subtraction from the average. I will attempt to use different analyzing functions on the source images to produce sensor patterns in Deliverable 2.

Code:

from PIL import Image
import os, sys
import numpy as np

os.chdir('C:/Users/Justin/Documents/SJSU/masters_project/blk')
print("Current working directory: {0}".format(os.getcwd()))

pixel_value_dict = {}

height = 0
width = 0
channels = 0



for infile in os.listdir('.'):
    try:
        with Image.open(infile) as im:
            print(f'{infile}, {im.format}, {im.size}x{im.mode}')
            height, width = im.size
            pixel_values = list(im.getdata())
            if im.mode == "RGB":
                channels = 3
            elif im.mode == "L":
                channels = 1
            else:
                print("Unknown mode: %s" % im.mode)
            pixel_values = np.array(pixel_values).reshape((width, height, channels))
            pixel_value_dict[infile] = pixel_values
    except OSError:
        pass

count = 0

temp = np.zeros((width, height, channels))

for fname in pixel_value_dict:
    count += 1
    print('shape of temp :', temp.shape)
    print('shape of pixel_value_dict[fname] :', pixel_value_dict[fname].shape)
    temp = np.add(temp, pixel_value_dict[fname])
avg = temp/count

avg_filter = Image.fromarray(avg.astype(np.uint8))
avg_filter.save('zavg_filter.png')


for infile in os.listdir('.'):
    try:
        with Image.open(infile) as im:
            print(f'{infile}, {im.format}, {im.size}x{im.mode}')
            pixel_values = list(im.getdata())
            if im.mode == "RGB":
                channels = 3
            elif im.mode == "L":
                channels = 1
            else:
                print("Unknown mode: %s" % im.mode)
            pixel_values = np.array(pixel_values).reshape((width, height, channels))
            temp_noise = Image.fromarray((pixel_values - avg_filter).astype(np.uint8))
            temp_noise.save(f'noise_{infile}.png')
            
    except OSError:
        pass

print('end')

Example image:

Example sensor noise pattern image: