Affiliation

3rd Year, PhD
Computer Science
New York University
New York, NY, USA





Email:
zeiler@cs.nyu.edu















Facial Expression Transfer with Input-Output Temporal Restricted Boltzmann Machines

Matthew D. Zeiler, Graham W. Taylor, Leonid Sigal, Iain Matthews, and Rob Fergus
Neural Information Processing Systems(December 12-17, 2011)

Abstract

We present a type of Temporal Restricted Boltzmann Machine that defines a prob- ability distribution over an output sequence conditional on an input sequence. It shares the desirable properties of RBMs: efficient exact inference, an exponen- tially more expressive latent state than HMMs, and the ability to model nonlinear structure and dynamics. We apply our model to a challenging real-world graphics problem: facial expression transfer. Our results demonstrate improved perfor- mance over several baselines modeling high-dimensional 2D and 3D data.

Paper(.pdf)  Supplementary(.pdf)  Code  Videos

Code

 Code for training an Input-Output Temporal Restricted Boltzman Machine (IOTRBM) and a Factored 3-way IOTRBM in MATLAB.

Facial Expression Transfer with Input-Output Temporal Restricted Boltzmann Machines

Author:
Matthew Zeiler (zeiler@cs.nyu.edu)
Date:
Sep 30, 2011
Version:
1.0.0 - This is the initial release of this toolbox and may contain unkown bugs.
Instructions for Using this Toolbox:
Download and unzip this toolbox in the location of choice. To setup this toolbox, simply open matlab, cd to this directory, and type "setupRetargetToolbox" (without quotes) in the command window. This sets up your path to include the required directories (you may want to add these paths into your startup.m file as well however). You can also read the setupRetargetToolbox.m file for some basic documentation about the Toolbox. You will likely have to add Piotr Dollar's toolbox to your path for pca function. It can be found at: http://vision.ucsd.edu/~pdollar/toolbox/doc/
This toolbox provides MATLAB implementations of ioTRBMs and FIOTRBM models for use in facial retargeting expeirments.
Download (.zip)  Documentation (html)

Videos
The player will show in this paragraph unless you do not have flash player installed.

[2D Comparison Retarget.]

[2D Retarget Under Noise.]

[3D Comparison Retarget.]

[3D Retarget Compared to Autoregressive Model.]

These videos are encoded using H.264 and so to view them with this integrated player you will need a relatively new version of Adobe Flash Player (called version 9 Update 3 or v9.0.115.0 which was released on December 3, 2007).