Chris Pollett > Students >

    ( Print View)



    [CS 297 Proposal]


    [Gen Videos Scene Dynamics-PDF]

    [3D CNNs Human Action RECOG-PDF]


    [Deliverable 1]

    [Deliverable 2]

    [Deliverable 3]

    [Deliverable 4]

    [CS297 Report - PDF]

    [CS 298 Proposal]

    [CS 298 Report - PDF]

    [CS 298 Slides - PDF]

Use pix2pix to generate videos


This is the first atempt to generate a video with 3D GAN.


UCF101-Action Recognition dataset and MPII Cooking Activities dataset


1) numpy

2) keras

3) matplotlib

4) skimage

5) skvideo

Steps to Generate videos:

1) Use an open source tool inside magenta to generate paired video frames.

Paired frame:

paired image

2) Copied paired frames to a folder, and run to create a npz file as the training data set

3) Run to train the pix2pix model

4) Copy at least one frame as the input, and run to predict a sequence of frames

5) Run to combine the predicted frames to a mp4 video


128 X 128 frames random length video

Download predicted video with all original frames

Download predicted video with one original frame


1) Pix2pix is very good to learn picture to picture mappings

2) It can generate very good videos if training with all original frames

3) It only learn limited length of frame to frame mappings. In this project, it only learns transition from two continuous frames

4) It cannot predict longer videos if just use one origin frame as input.

5) Sometimes, it cannot predict video well if no much difference between the continuous two frames. In this case, this model only can predict the same frames over and over again.