Project 3: Face Morphing

Part 1: Defining Correspondences


I selected 45 points of correspondence for both a picture of my brother and a picture of myself, making sure to include the corners. These defined important parts of the face, including the locations of the face, eyes, nose, mouth, and eyebrows. Then, I found the average of these points of correspondence and use these average points to generate a Delaunay triangulation.

Daniel Triangulation
Average Triangulation
Kelly Triangulation

Part 2: Mid-Way Face


To morph together two images of the same dimensions, I computed the average shape from the average Delaunay triangulation from part 1 using a warp_frac to determine what ratio of image1 would be incorporated with image2. Then, I had to define an affine transformation that warped both faces into this average mold. I iterated over every triangle in my triangulation and founds its affine matrix by doing matrix multiplication. I populated the triangle vertices of my original image into a matrix that I multiply to an affine matrix with placeholder variable values, and set this equal to a matrix of triangle vertices of my average image. I also padded my vertex points with 1s to ensure the multiplication dimensions made sense. Once I found my affine matrix, I took the inverse such that I could perform inverse warping so that when I multiply this inverse matrix with points from the average image, it will essentially map me back to the coordinates of where that point would lie on my original image. Using these inverse wrap affine matrices, I was able to warp each image into the average triangulation.

Lastly, I needed to average the colors together. I separated my image into its rgb channels so that I could work with them individually. I then populated colors at coordinates in my original image using inverse wrapping by iterating over the triangles. I used skimage.draw.polygon to retrieve all the pixels within a triangle, and used 2D binary interpolation to interpolate any in-between colors. I also cross-dissolved using a dissolve_frac to determine how much of image1's colors would be incorpoated with image2's colors.

Daniel
Morphed
Kelly

Part 3: Morph Sequence


I wrote a function morph() which takes in both images and their points, the triangulation, as well as a warp_frac and dissolve_frac to determine the ratios of shaping and coloring from images 1 and 2. A warp_frac of 1 would mean the shape is completely determined by image 1 and 0 would mean the shape is completely determined by image 2. A dissolve_frac of 1 would mean the color is completely determined by image 1 and 0 would mean the color is completely determined by image 2. To create a morph sequence, I created images with the same warp_frac and dissolve_frac over a range of [0, 1] to generate 44 frames that morph from Daniel to Kelly.

Part 4: Mean Face of Population


I used the Danes population and found the average face of the entire population. This was done by using my previously defined morph function and using a warp_frac of 0 and a dissolve_frac of 1 to use the average Danes shape but the colors of each individual.

Triangulation
Triangulation Over Mean Face
Danes Mean Face

Here's a couple of individual examples of people's faces being morphed into the Danes average shape.

01-1m Original
01-1m Warped
01-5m Original
01-5m Warped

Then, this is what it looks like when I morph my face onto the Dane's average shape, and Dane's face onto my own face's shape. The output is a little jankier due to the less consistent nature of my own picture with Dane's as well as a bigger aspect ratio, hence why Dane's face is being slightly stretched while mine is shrinked.

Kelly
My Face Morphed to Danes Shape
Danes
Danes Face Morphed to My Shape

Part 5: Caricatures


To produce caricatures, I extrapolated from the population mean by modifying my own defined alpha value. This value is multiplied by the shape differences between the average face with my face, and then added to my own face geometry. Hence, the greater the magnitude of alpha and as it strays further away from 0 (which would be my original face), the greater these differences are and the more exaggerated the feature became.

Alpha = -0.5
Alpha = -0.25
Alpha = 0 (Original)
Alpha = 0.25
Alpha = 0.5
Alpha = 0.75
Alpha = 1
Alpha = 1.25

Bells & Whistles


My friends and I created a face-morphing music video together (featuring Jaewon Lee, Rohan Gulati, Victor Zhou, Natalie Wei, and Jennifer Yin). We made sure all our images were 500 x 500 pixels and performed the same face morphing. However, we only labeled corresponding points just around the face and not the corners, hence leading to a black background.
Music Video

×

Website Template: w3.css