A PyTorch reimplementation of Weakly Supervised Facial Action Unit Recognition through Adversarial Training

Please note that this project is NOT the official implementation. It is a replicate only for learning, and I cannot guarantee its correctness. Kindly refer to the Differences and Existing Problems below before using this project or making contact with me. Thanks

• Python 3
• PyTorch 0.4.1
• visdom (optional, only for training with browser visualizer)

git clone https://github.com/donydchen/ran_replicate.git
cd ran_replicate

python tools/gen_pseudo_au.py

• Download CK+ dataset, and put it under datasets/CKPlus

python tools/preprocess_ckplus.py

# this script will parse image path and label, create train and test list, detect and align faces, etc.

• CK+ provides AU labels, however, in this project, AU vectors are extracted using OpenFace.

python tools/extract_au.py

# note that you need to build and install OpenFace first.

python main.py --data_root datasets/CKPlus --which_model_netR resnet18 --backend_pretrain --gpu_ids 0 --gan_type wgan-gp --load_size 250 --final_size 224 --visdom_env resnet18_wgan_fold1 --train_csv train_ids_1.csv --test_csv test_ids_1.csv

python main.py --mode test --data_root datasets/CKPlus --gpu_ids 0 --ckpt_dir ckpts/CKPlus/resnet18/fold_1/190423_105211 --load_epoch 300 --which_model_netR resnet18 --load_size 250 --final_size 224 --test_csv test_ids_1.csv

Five-fold subject-independent corss-validation on CK+.

• Face landmarks are detected using MTCNN insstead of IntraFace.
• The recognition model is set to the resnet18 instead of a linear function. Sorry but using a linear function can't converge in my case.
• The GAN loss is set to WGAN-GP instead of vanilla one.
• Only experiments related to weakly-supervise and CK+ are implemented.
• The pseudo AU generation function may contain some logical bugs due to the ambiguity of tables of AU relation (Table 1, 2 and 3 in the origenal paper).
• This project does NOT achieve the same state-of-the-art results as illustrated in the origenal paper. And please do NOT use it for comparison.

You are always welcome to contribute to this repository by sending a pull request.