Download UPSTREAM_README.md from mcflurryshuoz/AffectGPT-RL: direct link, hf CLI and curl.
- Browser
- Download file 18.6 kB
-
https://huggingface.co/mcflurryshuoz/AffectGPT-RL/resolve/main/UPSTREAM_README.md
- Command line
-
hf download hf://mcflurryshuoz/AffectGPT-RL/UPSTREAM_README.md
-
curl -L -o UPSTREAM_README.md https://huggingface.co/mcflurryshuoz/AffectGPT-RL/resolve/main/UPSTREAM_README.md
AffectGPT-RL: Revealing Roles of Reinforcement Learning in Open-Vocabulary Emotion Recognition
Zheng Lian, Fan Zhang, Lan Chen, Yazhou Zhang, Rui Liu, Jinyang Wu, Haoyu Chen, Xiaobai Li, Xiaojiang Peng, Bin He, Jianhua Tao
π Paper: arXiv Β· paper/NeurIPS_2026__AffectGPT_RL.pdf
If our project helps you, please give us a star β on GitHub to support us. ππ

π Overview
|
Open-Vocabulary Multimodal Emotion Recognition (OV-MER) removes the restriction of a predefined label space, enabling fine-grained emotion understanding. Instead of relying on discriminative classifiers, OV-MER uses generative models to capture the full spectrum of emotions, and evaluates them with emotion wheels (EWs). Existing approaches are trained with a token-level loss, which is misaligned with the metrics actually used in OV-MER: joyful and awful share high token-level similarity yet are semantically far apart. Worse, EW-based metrics are non-differentiable β they involve stochastic sampling, discrete word-to-word clustering, and set-cardinality operations β so they cannot be optimized by gradient backpropagation at all. AffectGPT-RL addresses this with reinforcement learning, which can optimize non-differentiable objectives. EW-based metrics are used as reward functions, so that the training objective is directly aligned with the evaluation metric. Extensive experiments reveal the role of reinforcement learning in this task. |
π₯ Highlights
|
(Framework) This is the first work to reveal the role of reinforcement learning in OV-MER. (Experiment) We conduct systematic experiments, revealing the role of the reasoning process, the impact of different rewards, and the generalization of reinforcement learning to other tasks. (Performance) Our method achieves state-of-the-art performance on fine-grained and basic emotion recognition, highlighting the potential of reinforcement learning in affective computing. |

|
Pipeline. Training has two phases. In the cold-start phase we use a large-scale, coarse-grained dataset to build emotion understanding and output-format alignment. In the RL phase we use a high-quality, fine-grained dataset for reward calculation and policy updates. run-release.sh collects the whole release as five commands in four stages: Stage 0 sets up the environment and prepares the data, Stage 1 (cold start) and Stage 2 (RL) each run a training instance followed by an inference instance, and Stage 3 computes the emotion-wheel score. The Reproduction section below documents exactly these instances. |
π οΈ Requirements and Installation
|
Validated environment β every command in run-release.sh was run in the conda env
The RL models import |
conda create -n vllm3 python=3.10.15
conda activate vllm3
# 1) torch first: vLLM 0.11.0 pins torch==2.8.0 / torchvision==0.23.0 / torchaudio==2.8.0,
# so the CUDA build has to be chosen explicitly here
pip install torch==2.8.0 torchvision==0.23.0 torchaudio==2.8.0 \
--index-url https://download.pytorch.org/whl/cu129
# 2) vLLM (this also pulls xformers==0.0.32.post1)
pip install vllm==0.11.0
# 3) transformers LAST: vllm 0.11.0 declares transformers>=4.55.2 while the validated env pins
# 4.53.2. Installing it after vllm stops pip from upgrading it back; `pip check` will still
# report this single conflict, which is expected.
pip install transformers==4.53.2 tokenizers==0.21.4 trl==0.19.1 peft==0.15.2
# 4) the remaining AffectGPT dependencies
pip install decord einops omegaconf webdataset pytorchvideo timm opencv-python \
pandas scikit-learn scipy openpyxl tqdm matplotlib seaborn \
iopath ftfy regex PyYAML sentencepiece protobuf
Sanity check β the versions that matter, plus a visible GPU:
python -c "import torch, vllm, transformers; \
print(torch.__version__, torch.version.cuda, torch.cuda.is_available(), \
vllm.__version__, transformers.__version__)"
# 2.8.0+cu129 12.9 True 0.11.0 4.53.2
On the CUDA build: the validated machine's driver is 570.195.03, which reports CUDA 12.8, and it runs the cu129 wheels as they are β CUDA 12.x minor-version compatibility means any driver that supports 12.0 or later runs the whole 12.x line. If your driver predates 525, pick the matching --index-url from the PyTorch wheel index instead.
|
[Environment Preparation] All paths are hard-coded in config.py.
|
π Dataset&Model Preparation
Dataset Preparation
| Dataset | # Samples | Role | Emotion Task |
|---|---|---|---|
| MER-Caption+ | 31,327 | Stage 1: cold-start training | Open-vocabulary + Descriptive |
| MER2025-OV | 1,000 | Stage 2: reinforcement learning | Open-vocabulary |
| OV-MERD+ | 532 | Testing | Open-vocabulary |
| MER-UniBench | 12,799 | Testing | Sentiment + Basic + Open-vocabulary |
|
The label files ship with the repository, so only the raw media has to be downloaded. Two options, as in run-release.sh:
Either way, keep everything under |
Model Preparation
| Model Name | Model Type |
|---|---|
| clip-vit-large-patch14 | Visual Encoder |
| chinese-hubert-large | Audio Encoder |
| Qwen2.5-7B-Instruct | LLM |
| bert-base-uncased | Text Encoder / Tokenizer of the Q-Formers |
| mercaptionplus_βoutputhybird_βbestsetup_βbestfusion_βface_βr1stage1 | Stage 1 (cold start): trained on MER-Caption+ with pre-extracted face as input |
| mercaptionplus_βoutputhybird_βbestsetup_βbestfusion_βface_βr1stage2_βmer25ov1000_βcontinue60_βg4_βformatbeta00_βovlabel_βnumrestrict3 | Stage 2 (RL): GRPO optimization on MER2025-OV (P3 length penalty), continued from the Stage-1 epoch-60 checkpoint |
Data and Pre-trained Checkpoints Structure
dataset # read through config.DATASET_ROOT, which resolves to dataset/ inside this repository
βββ mercaptionplus-emotion.csv # MER-Caption+ open-vocabulary labels, 31,327 samples
βββ mercaptionplus-description.csv # MER-Caption+ descriptions / reasoning, 31,327 samples
βββ mer2025ov1000-emotion.csv # MER2025-OV: the open-vocabulary labels used for RL, 1,000 samples
βββ ovmerdplus-emotion.csv # OV-MERD+ ground-truth labels, 532 samples
βββ subtitle_chieng.csv # all subtitles (Chinese + English)
βββ audio/ # β download: pre-extracted audio
βββ openface_face/ # β download: pre-extracted face files
βββ video/ # β download: raw videos
AffectGPT-RL
βββ emotion_wheel # already included in this repository
βββ models # Available at: https://pan.baidu.com/s/1IvC4H7Xt1AzMFocGMBBbHQ?pwd=hzf9
β βββ chinese-hubert-large # audio encoders
β βββ clip-vit-large-patch14 # video encoders
β βββ Qwen2.5-7B-Instruct # LLM
β βββ bert-base-uncased # text encoder / tokenizer of the Q-Formers
βββ output # created by train.py; the two pre-trained checkpoints are placed here before Stage 2 / inference
βββ mercaptionplus_outputhybird_bestsetup_bestfusion_face_r1stage1
β βββ mercaptionplus_outputhybird_bestsetup_bestfusion_face_r1stage1_20250715180
βββ mercaptionplus_outputhybird_bestsetup_bestfusion_face_r1stage2_mer25ov1000_continue60_g4_formatbeta00_ovlabel_numrestrict3
βββ mercaptionplus_outputhybird_bestsetup_bestfusion_face_r1stage2_mer25ov1000_continue60_g4_formatbeta00_ovlabel_numrestrict3_20251009171
π Reproduction
| Stage | Commands | What it does |
|---|---|---|
| Stage 0 | β | Dataset and model preparation β see Dataset&Model Preparation above |
| Stage 1 | [1]β[2] |
Cold start β train the model to produce output in <think></think><answer></answer> format, then run inference |
| Stage 2 | [1]β[2] |
Reinforcement learning β align the token-level training loss with the semantic-level evaluation metric, then run inference |
| Stage 3 | statistic.py |
Score calculation β the emotion-wheel metric over the inference results |
Stage 1: Cold Start
"train the model to produce output in <think></think><answer></answer> format"
|
We use MER-Caption+, a large-scale dataset, to equip the model with emotion understanding and format alignment. Because descriptive emotion datasets already follow a reasoning-like format, we treat the description as the thinking $o^t$ and the open-vocabulary labels as the answer $o^a$, unifying descriptive and open-vocabulary emotions into a single task. The released configuration uses |
[1] Training
CUDA_VISIBLE_DEVICES=0 python train.py \
--cfg-path=train_configs/mercaptionplus_outputhybird_bestsetup_bestfusion_face_r1stage1.yaml
[2] Inference
Evaluates the Stage-1 checkpoints on OV-MERD+ (OVMERDPlus), every 5 epochs from 10 to 60:
CUDA_VISIBLE_DEVICES=0 python inference_hybird.py --zeroshot --dataset='OVMERDPlus' \
--outside_user_message="Please recognize all possible emotional states of the character." \
--cfg-path=train_configs/mercaptionplus_outputhybird_bestsetup_bestfusion_face_r1stage1.yaml \
--options "inference.test_epochs=10-60" "inference.skip_epoch=5"
Stage 2: Reinforcement Learning
"align the token-level training loss with the semantic-level evaluation metric"
|
We use MER2025-OV, a high-quality dataset, for reward calculation and policy updates. Policy optimization follows GRPO: $G$ outputs are sampled for each prompt, and rewards are normalized within the group to produce relative advantages. The reward is the emotion-wheel metric itself, so this stage directly optimizes what is measured at test time. |
[1] Training
CUDA_VISIBLE_DEVICES=0 python train.py \
--cfg-path=train_configs/mercaptionplus_outputhybird_bestsetup_bestfusion_face_r1stage2_mer25ov1000_continue60_g4_formatbeta00_ovlabel_numrestrict3.yaml
Key options in this config:
| Option | Value | Meaning |
|---|---|---|
model.arch |
length-penalized GRPO policy | Policy without a reference model, with the $P_3$ (log) length penalty |
model.continue_epoch |
60 |
Continue from the Stage-1 epoch-60 checkpoint (located automatically under output/) |
model.num_generations |
4 |
Number of rollouts sampled per prompt (group size G) |
model.beta |
0 |
Weight of the KL term. The released arch is reference-free, so no KL term is computed at all |
model.formatbeta |
0.0 |
Weight of the format reward. In the released arch the reward reduces to the accuracy reward alone |
datasets.mer25ov1000.label_type |
ovlabel |
Train on the open-vocabulary labels of the 1,000 RL samples |
[2] Inference
Evaluates the RL checkpoints on OV-MERD+ every 5 epochs, from epoch 5 to epoch 60 β the range validated by run-release.sh:
CUDA_VISIBLE_DEVICES=0 python inference_hybird.py --zeroshot --dataset='OVMERDPlus' \
--outside_user_message="Please recognize all possible emotional states of the character." \
--cfg-path=train_configs/mercaptionplus_outputhybird_bestsetup_bestfusion_face_r1stage2_mer25ov1000_continue60_g4_formatbeta00_ovlabel_numrestrict3.yaml \
--options "inference.test_epochs=5-60" "inference.skip_epoch=5"
|
Variants. Three length penalties $P_1$, $P_2$, $P_3$ were explored as separate architectures; this release ships only the loosest one, $P_3$, as my_affectgpt/models/affectgpt_r1_noref_ovlabel4.py, which is what the released Stage-2 config |
Stage 3: Score Calculation
Runs both stages of scoring at once, over the results of every stage above:
CUDA_VISIBLE_DEVICES=0 python statistic.py
|
Inference results land in
|
π Main Results
| Model | OV-MERD+ |
|---|---|
| AffectGPT | 62.52 |
| AffectGPT-RL (Ours) | 68.05 |
Note: this release takes OV-MERD+ as the example; results on the other datasets are reported in the paper.
π Citation
If you find this project useful for your research and applications, please cite using this BibTeX:
@inproceedings{lian2026affectgptrl,
title={AffectGPT-RL: Revealing Roles of Reinforcement Learning in Open-Vocabulary Emotion Recognition},
author={Lian, Zheng and Zhang, Fan and Chen, Lan and Zhang, Yazhou and Liu, Rui and Wu, Jinyang and Chen, Haoyu and Li, Xiaobai and Peng, Xiaojiang and He, Bin and Tao, Jianhua},
booktitle={Advances in Neural Information Processing Systems},
year={2026}
}
@inproceedings{lian2025affectgpt,
title={AffectGPT: A New Dataset, Model, and Benchmark for Emotion Understanding with Multimodal Large Language Models},
author={Lian, Zheng and Chen, Haoyu and Chen, Lan and Sun, Haiyang and Sun, Licai and Ren, Yong and Cheng, Zebang and Liu, Bin and Liu, Rui and Peng, Xiaojiang and others},
booktitle={Proceedings of the 42nd International Conference on Machine Learning},
year={2025}
}
π License
|
This project is released under the Apache 2.0 license as found in the LICENSE file. The service is a research preview intended for non-commercial use ONLY. Please get in touch with us if you find any potential violations. |