3D Morphorble Model

[ 3d-face  3dmm  flame  FaceScape  FaceWareHouse  ]

This notebooks describes the models which could represent animatable 3D face mesh, which is usually referred 3D morphorble model (3DMM).

Screen Shot 2021-04-24 at 8.24.05 PM

FaceWareHouse

FaceWareHouse is a database of 3D facial expressions for visual computing applications. Some highlights:

  • learned from data captured from Kinect includes 159 subjects and 20 expressins each. The quality and resolution of Kinect depth camea is at millimeter level;
  • bilinear model with identity and expression on different axis \(V=C_r\times W_{id}\times W_{exp}\), with \(W_{id}\in\mathbb{R}^{150}\) and \(W_{exp}\in\mathbb{R}^{47}\);

Screen Shot 2021-04-24 at 8.32.42 PM

FLAME

FLAME parametrizes the 3D face mesh by identity \(\beta\in\mathbb{R}^{100}\), expression \(\psi\in\mathbb{R}^{50}\) and head pose \(\theta\in\mathbb{R}^{15}\)). Some highlights of FLAME:

  • learned from 33000 3D scans of several hertergenous dataset including about 3800 identities, 8000 scans for poses and 69000 frames for expressions;
  • strictly linear model for identity, expression and pose;
  • the idenity and expression basis are orthornomal and computed from data. The number of basis are configurable;
  • pose contains global pose, neck pose, jaw pose and two pupil poses;
  • FLAME also supports different rendering to output the 2D image.

FLAME could be written as:

\[M(\beta,\theta,\psi)=W(T+B_S(\beta,S)+B_p(\theta,P)+B_E(\psi,E),J(\beta),\theta)\]

Screen Shot 2021-04-23 at 12.21.05 AM

This figure shows the coverage of variability given different number of basis.

Screen Shot 2021-04-23 at 12.21.29 AM

FaceScape

FaceScape provides large-scale high-quality 3D face datasets, parametric models, docs and toolkits about 3D face related technology. It is published in CVPR 2020 and code is available in github. FaceScape utilizes a bilinear model instead of linear model in FLAME.

Some highlights of FaceScape:

  • learned from 18760 scans from 938 subjects and 20 expressions each;
  • bilinear model with identity and expression on different axis \(V=C_r\times W_{id}\times W_{exp}\), with \(W_{id}\in\mathbb{R}^{938}\) and \(W_{exp}\in\mathbb{R}^{52}\);
  • captured with high quality multi-view systemconsists of 68 DSLR cameras, 30 of which capture 8K images focusing on front side, and the other cameras capture 4K level images for the side part.
Written on April 24, 2021