ICLR 2025

National University of Singapore

This repository provides the code and data for CHiP, a direct preference optimization algorithm aimed at mitigating hallucinations in multimodal large language models (MLLMs) and enhancing cross-modal semantic alignment.

• The capabilities of ideally well-aligned MLLMs. Ideally, for well-aligned MLLMs, the representations of an image and its ground-truth description should be as close as possible, while the representations of ground-truth and hallucinated descriptions should be more distant.

• The gap between existing MLLMs, MLLMs with DPO, and the ideal state. (1) LLaVA and LLaVA+DPO struggles to align image and description representations and to effectively distinguish between hallucinated and non-hallucinated descriptions. (2) The proposed CHiP method, which incorporates both image and fine-grained text preferences, achieves better alignment between images and ground-truth descrip- tions while increasing the distance between ground-truth and hallucinated descriptions.

• ** The framework of CHiP. ** CHiP comprises a hierarchical text preference optimization module that captures fine-grained preferences at the response, segment, and token levels, along with a visual preference optimization module that extracts cross-modal preferences.

For more technical details, kindly refer to the paper.

• 1. Preparing Dataset
• 2. Environment Preparation
• 3. Training
• 4. Evaluation

There are several publicly available training datasets that include preference pairs for multimodal hallucinations. Here, we choose to use the RLHF-V Dataset. You can download from HuggingFace.

The evaluation datasets are publicly available and widely used. They can be downloaded from their official websites and placed in the playground/data directory.

The evaluation datasets used in this study and their download links are listed below:

• Object HalBench (ObjHal) (Download from here!)
• MMHal-Bench (MMHal) (Download from here!)
• HallusionBench (Download from here!)
• AMBER (Download from here!)

1. Clone this repository and navigate to source folder

cd CHiP

2. Build Environment

echo "Creating conda environment"
conda create -n CHiP python=3.10
conda activate CHiP

echo "Installing dependencies"
pip install -e .

• LLaVA CHiP/DPO Training

bash scripts/chip.sh
bash scripts/cmdpo.sh
bash scripts/dpo.sh

• MUFFIN CHiP/DPO Training

bash muffin/script/train/chip.sh
bash muffin/script/train/dpo.sh

Run inference to generate responses

python llava_inference.py --model_name {ckpt_name} --test_datasets {test_datasets} --eval_output {eval_output} 
python muffin/muffin/eval/inference.py --model_name {ckpt_name} --test_datasets {test_datasets} --eval_output {eval_output} 

1. Evaluate on the Object HalBench

• Prepare COCO2014 annotations. The evaluation of Object HalBench relies on the caption and segmentation annotations from the COCO2014 dataset. Please first download the COCO2014 dataset from the COCO dataset's official website.

mkdir coco2014
cd coco2014

wget http://images.cocodataset.org/annotations/annotations_trainval2014.zip

unzip annotations_trainval2014.zip

# eval with gpt-4
python ./eval/eval_gpt_obj_halbench.py --cap_file {cap_file} --openai_key {api_key}

python ./eval/summarize_gpt_obj_halbench_review.py --cap_file {cap_file} 

2. Evaluate on the MMHal Bench

# eval with gpt-4
python ./eval/eval_gpt_mmhal.py --response {response} --openai_key {api_key}

3. Evaluate on the HallusionBench

# Firstly, prepare the response format 
python ./eval/eval_public.py # use eval_hallusionbench function

python HallusionBench/evaluation.py

4. Evaluate on the AMBER

# Firstly, prepare the response format
python ./eval/eval_public.py # use eval_amber function

python AMBER/inference.py

If you find CHiP useful in your research or applications, please kindly cite:

@inproceedings{fu2025chip,
  title={CHiP: Cross-modal Hierarchical Direct Preference Optimization for Multimodal LLMs},
  author={Fu, Jinlan and Huangfu, Shenzhen and Fei, Hao and Shen, Xiaoyu and Hooi, Bryan and Qiu, Xipeng and Ng, See-Kiong},
  journal={Proceedings of the International Conference on Learning Representations},
  year={2025}
}

Our CHiP is developed based on the codebases of LLaVA and Muffin, and we would like to thank the developers of both.