Chris Pollett >
Students >
Pratikkumar
( Print View)
[Bio]
[Blog]
[CS 297 Proposal]
[PyTorch Bootcamp-IPYNB]
[Basics of Sanskrit-PDF]
[Autoencoders-PDF]
[Overview of GANs-PDF]
[DeepFakes and Beyond_A Survey-PDF]
[DeepPrivacy-PDF]
[Fake_Video_Detection-PDF]
[Deliverable 1]
[Deliverable 2]
[Deliverable 3]
[Deliverable 4]
[CS297_Final_Report-PDF]
[CS298 Proposal]
[CS298_Final_Report-PDF]
[CS298_slides-PDF]
[CS298_Code-ZIP]
|
Deliverable 4: Face-swap GAN in PyTorch
Data Collection:
GAN model architecture
- DeepPrivacy (https://arxiv.org/abs/1909.04538) paper's reference implementation is open-sourced by the authors at https://github.com/hukkelas/DeepPrivacy.
This code is taken as reference and further modification are made.
- The generator of the original DeepPivacy GAN is shown in Figure 1. The architecture includes all the layers shown in the picture, in
an encoder-decoder pair. We have simplified the generator architecture by removing layers marked by orange crosses. The model was trained
on a high-end machine for approx 50 hours. The discriminator is not changed.
- The specification of the high-end machine: Intel 28 cores CPU, 64GB dram, Nvidia Titan RTX GPU (24GB vram)
Figure 1. The architecture of the Generator of the DeepPrivacy GAN.
Architecutes of the models can be seen in below table.
Model | Original Generator architecture
| New Generator architecture |
Generator |
NetworkWrapper(
(network): Generator(
(to_rgb_new): WSConv2dConv2d-[3, 256, 1, 1]
(to_rgb_old): WSConv2dConv2d-[3, 256, 1, 1]
(core_blocks_down): ModuleList(
(0): UnetDownSamplingBlock(
(model): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
(1): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
)
)
)
(core_blocks_up): ModuleList(
(0): Sequential(
(0): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 295, 1, 1]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
(1): UnetUpsamplingBlock(
(model): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
(1): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
)
)
)
(1): UpSamplingBlock()
)
)
(new_up): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 519, 1, 1]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
(1): UnetUpsamplingBlock(
(model): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
(1): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
)
)
)
(old_up): Sequential()
(new_down): Sequential(
(0): UnetDownSamplingBlock(
(model): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
(1): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
)
)
)
(from_rgb_new): Sequential(
(0): WSConv2dConv2d-[256, 3, 1, 1]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
(from_rgb_old): Sequential(
(0): AvgPool2d(kernel_size=[2, 2], stride=[2, 2], padding=0)
(1): Sequential(
(0): WSConv2dConv2d-[256, 3, 1, 1]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
)
(upsampling): UpSamplingBlock()
(downsampling): AvgPool2d(kernel_size=2, stride=2, padding=0)
)
(forward_block): Generator(
(to_rgb_new): WSConv2dConv2d-[3, 256, 1, 1]
(to_rgb_old): WSConv2dConv2d-[3, 256, 1, 1]
(core_blocks_down): ModuleList(
(0): UnetDownSamplingBlock(
(model): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
(1): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
)
)
)
(core_blocks_up): ModuleList(
(0): Sequential(
(0): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 295, 1, 1]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
(1): UnetUpsamplingBlock(
(model): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
(1): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
)
)
)
(1): UpSamplingBlock()
)
)
(new_up): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 519, 1, 1]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
(1): UnetUpsamplingBlock(
(model): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
(1): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
)
)
)
(old_up): Sequential()
(new_down): Sequential(
(0): UnetDownSamplingBlock(
(model): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
(1): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
)
)
)
(from_rgb_new): Sequential(
(0): WSConv2dConv2d-[256, 3, 1, 1]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
(from_rgb_old): Sequential(
(0): AvgPool2d(kernel_size=[2, 2], stride=[2, 2], padding=0)
(1): Sequential(
(0): WSConv2dConv2d-[256, 3, 1, 1]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
)
(upsampling): UpSamplingBlock()
(downsampling): AvgPool2d(kernel_size=2, stride=2, padding=0)
)
)
|
NetworkWrapper(
(network): Generator(
(to_rgb_new): WSConv2dConv2d-[3, 256, 1, 1]
(to_rgb_old): WSConv2dConv2d-[3, 256, 1, 1]
(core_blocks_down): ModuleList(
(0): UnetDownSamplingBlock(
(model): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
)
)
)
(core_blocks_up): ModuleList(
(0): Sequential(
(0): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 295, 1, 1]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
(1): UnetUpsamplingBlock(
(model): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
)
)
)
(1): UpSamplingBlock()
)
)
(new_up): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 519, 1, 1]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
(1): UnetUpsamplingBlock(
(model): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
)
)
)
(old_up): Sequential()
(new_down): Sequential(
(0): UnetDownSamplingBlock(
(model): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
)
)
)
(from_rgb_new): Sequential(
(0): WSConv2dConv2d-[256, 3, 1, 1]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
(from_rgb_old): Sequential(
(0): AvgPool2d(kernel_size=[2, 2], stride=[2, 2], padding=0)
(1): Sequential(
(0): WSConv2dConv2d-[256, 3, 1, 1]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
)
(upsampling): UpSamplingBlock()
(downsampling): AvgPool2d(kernel_size=2, stride=2, padding=0)
)
(forward_block): Generator(
(to_rgb_new): WSConv2dConv2d-[3, 256, 1, 1]
(to_rgb_old): WSConv2dConv2d-[3, 256, 1, 1]
(core_blocks_down): ModuleList(
(0): UnetDownSamplingBlock(
(model): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
)
)
)
(core_blocks_up): ModuleList(
(0): Sequential(
(0): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 295, 1, 1]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
(1): UnetUpsamplingBlock(
(model): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
)
)
)
(1): UpSamplingBlock()
)
)
(new_up): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 519, 1, 1]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
(1): UnetUpsamplingBlock(
(model): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
)
)
)
(old_up): Sequential()
(new_down): Sequential(
(0): UnetDownSamplingBlock(
(model): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
)
)
)
(from_rgb_new): Sequential(
(0): WSConv2dConv2d-[256, 3, 1, 1]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
(from_rgb_old): Sequential(
(0): AvgPool2d(kernel_size=[2, 2], stride=[2, 2], padding=0)
(1): Sequential(
(0): WSConv2dConv2d-[256, 3, 1, 1]
(1): LeakyReLU(negative_slope=0.2)
(2): PixelwiseNormalization()
)
)
(upsampling): UpSamplingBlock()
(downsampling): AvgPool2d(kernel_size=2, stride=2, padding=0)
)
)
|
Discriminator |
NetworkWrapper(
(network): DeepDiscriminator(
(from_rgb_new): Sequential(
(0): WSConv2dConv2d-[256, 6, 1, 1]
(1): LeakyReLU(negative_slope=0.2)
)
(from_rgb_old): Sequential(
(0): AvgPool2d(kernel_size=[2, 2], stride=[2, 2], padding=0)
(1): Sequential(
(0): WSConv2dConv2d-[256, 6, 1, 1]
(1): LeakyReLU(negative_slope=0.2)
)
)
(new_block): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 263, 1, 1]
(1): LeakyReLU(negative_slope=0.2)
)
(1): ResNetBlock(
(conv): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
)
(1): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
)
(2): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
)
(3): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
)
)
)
(2): Sequential(
(0): WSConv2dConv2d-[256, 256, 1, 1]
(1): LeakyReLU(negative_slope=0.2)
)
(3): AvgPool2d(kernel_size=[2, 2], stride=[2, 2], padding=0)
)
(core_model): Sequential(
(0): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 263, 1, 1]
(1): LeakyReLU(negative_slope=0.2)
)
(1): ResNetBlock(
(conv): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
)
(1): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
)
(2): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
)
)
)
(2): Sequential(
(0): WSConv2dConv2d-[256, 256, 4, 4]
(1): LeakyReLU(negative_slope=0.2)
)
)
)
(output_layer): WSLinear(
(linear): Linear(in_features=256, out_features=1, bias=False)
)
)
(forward_block): DeepDiscriminator(
(from_rgb_new): Sequential(
(0): WSConv2dConv2d-[256, 6, 1, 1]
(1): LeakyReLU(negative_slope=0.2)
)
(from_rgb_old): Sequential(
(0): AvgPool2d(kernel_size=[2, 2], stride=[2, 2], padding=0)
(1): Sequential(
(0): WSConv2dConv2d-[256, 6, 1, 1]
(1): LeakyReLU(negative_slope=0.2)
)
)
(new_block): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 263, 1, 1]
(1): LeakyReLU(negative_slope=0.2)
)
(1): ResNetBlock(
(conv): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
)
(1): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
)
(2): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
)
(3): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
)
)
)
(2): Sequential(
(0): WSConv2dConv2d-[256, 256, 1, 1]
(1): LeakyReLU(negative_slope=0.2)
)
(3): AvgPool2d(kernel_size=[2, 2], stride=[2, 2], padding=0)
)
(core_model): Sequential(
(0): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 263, 1, 1]
(1): LeakyReLU(negative_slope=0.2)
)
(1): ResNetBlock(
(conv): Sequential(
(0): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
)
(1): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
)
(2): Sequential(
(0): WSConv2dConv2d-[256, 256, 3, 3]
(1): LeakyReLU(negative_slope=0.2)
)
)
)
(2): Sequential(
(0): WSConv2dConv2d-[256, 256, 4, 4]
(1): LeakyReLU(negative_slope=0.2)
)
)
)
(output_layer): WSLinear(
(linear): Linear(in_features=256, out_features=1, bias=False)
)
)
)
|
No Change. Used original discriminator only.
|
Code
PyTorch implementation can be found at: deep_privacy.tgz. It contains the changes made in deep_privacy python module from the original paper.
Sample output
Conclusion
The modified Deep Privacy model also seems to work up to a certain level. Some of the images in sample output seem to generate more human-like faces with different features.
Training longer might generate even more realistic faces.
|