Update model card with paper link, project page metadata, and abstract
Browse filesHi there,
I've updated the model card for `InternVL3_5-241B-A28B` with the following improvements:
- Added the official Hugging Face paper link (`https://huggingface.co/papers/2412.05271`) to the `paper` metadata for better discoverability and linking on the Hub.
- Included the Hugging Face Space URL (`https://huggingface.co/spaces/OpenGVLab/InternVL`) as `project_page` metadata, providing a direct link to the model's interactive demo.
- Added a "Paper Abstract" section to the model card content, giving users a quick overview of the paper's key contributions.
These changes aim to make the model card more comprehensive and easily navigable for the community.
Let me know if you have any feedback!
README.md
CHANGED
|
@@ -1,18 +1,20 @@
|
|
| 1 |
---
|
| 2 |
-
license: apache-2.0
|
| 3 |
-
pipeline_tag: image-text-to-text
|
| 4 |
-
library_name: transformers
|
| 5 |
base_model:
|
| 6 |
-
|
| 7 |
-
base_model_relation: finetune
|
| 8 |
datasets:
|
| 9 |
-
|
| 10 |
-
|
| 11 |
language:
|
| 12 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
tags:
|
| 14 |
-
|
| 15 |
-
|
|
|
|
| 16 |
---
|
| 17 |
|
| 18 |
# InternVL3_5-241B-A28B
|
|
@@ -25,6 +27,10 @@ tags:
|
|
| 25 |
<img width="500" alt="image" src="https://cdn-uploads.huggingface.co/production/uploads/64006c09330a45b03605bba3/zJsd2hqd3EevgXo6fNgC-.png">
|
| 26 |
</div>
|
| 27 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 28 |
## Introduction
|
| 29 |
|
| 30 |
We introduce *InternVL3.5*, a new family of open-source multimodal models that significantly advances versatility, reasoning capability, and inference efficiency along the InternVL series. A key innovation is the *Cascade Reinforcement Learning (Cascade RL)* framework, which enhances reasoning through a two-stage process: offline RL for stable convergence and online RL for refined alignment. This coarse-to-fine training strategy leads to substantial improvements on downstream reasoning tasks, e.g., MMMU and MathVista. To optimize efficiency, we propose a *Visual Resolution Router (ViR)* that dynamically adjusts the resolution of visual tokens without compromising performance. Coupled with ViR, our Decoupled *Vision-Language Deployment (DvD)* strategy separates the vision encoder and language model across different GPUs, effectively balancing computational load. These contributions collectively enable InternVL3.5 to achieve up to a +16.0\% gain in overall reasoning performance and a 4.05 \\(\times\\) inference speedup compared to its predecessor, i.e., InternVL3. In addition, InternVL3.5 supports novel capabilities such as GUI interaction and embodied agency. Notably, our largest model, i.e., InternVL3.5-241B-A28B, attains state-of-the-art results among open-source MLLMs across general multimodal, reasoning, text, and agentic tasks—narrowing the performance gap with leading commercial models like GPT-5. All models and code are publicly released.
|
|
@@ -233,7 +239,7 @@ $$
|
|
| 233 |
\Bigg],
|
| 234 |
$$
|
| 235 |
|
| 236 |
-
where \\(\mathrm{KL}
|
| 237 |
|
| 238 |
|
| 239 |
`Router training`:
|
|
@@ -494,7 +500,9 @@ image_urls=[
|
|
| 494 |
|
| 495 |
images = [load_image(img_url) for img_url in image_urls]
|
| 496 |
# Numbering images improves multi-image conversations
|
| 497 |
-
response = pipe((f'Image-1: {IMAGE_TOKEN}
|
|
|
|
|
|
|
| 498 |
print(response.text)
|
| 499 |
```
|
| 500 |
|
|
@@ -596,4 +604,59 @@ If you find this project useful in your research, please consider citing:
|
|
| 596 |
journal={arXiv preprint arXiv:2508.18265},
|
| 597 |
year={2025}
|
| 598 |
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 599 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
|
|
|
|
|
|
|
|
|
| 2 |
base_model:
|
| 3 |
+
- OpenGVLab/InternVL3_5-241B-A28B-MPO
|
|
|
|
| 4 |
datasets:
|
| 5 |
+
- OpenGVLab/MMPR-v1.2
|
| 6 |
+
- OpenGVLab/MMPR-Tiny
|
| 7 |
language:
|
| 8 |
+
- multilingual
|
| 9 |
+
library_name: transformers
|
| 10 |
+
license: apache-2.0
|
| 11 |
+
paper: https://huggingface.co/papers/2412.05271
|
| 12 |
+
pipeline_tag: image-text-to-text
|
| 13 |
+
project_page: https://huggingface.co/spaces/OpenGVLab/InternVL
|
| 14 |
tags:
|
| 15 |
+
- internvl
|
| 16 |
+
- custom_code
|
| 17 |
+
base_model_relation: finetune
|
| 18 |
---
|
| 19 |
|
| 20 |
# InternVL3_5-241B-A28B
|
|
|
|
| 27 |
<img width="500" alt="image" src="https://cdn-uploads.huggingface.co/production/uploads/64006c09330a45b03605bba3/zJsd2hqd3EevgXo6fNgC-.png">
|
| 28 |
</div>
|
| 29 |
|
| 30 |
+
## Paper Abstract
|
| 31 |
+
|
| 32 |
+
We introduce InternVL 2.5, an advanced multimodal large language model (MLLM) series that builds upon InternVL 2.0, maintaining its core model architecture while introducing significant enhancements in training and testing strategies as well as data quality. In this work, we delve into the relationship between model scaling and performance, systematically exploring the performance trends in vision encoders, language models, dataset sizes, and test-time configurations. Through extensive evaluations on a wide range of benchmarks, including multi-discipline reasoning, document understanding, multi-image / video understanding, real-world comprehension, multimodal hallucination detection, visual grounding, multilingual capabilities, and pure language processing, InternVL 2.5 exhibits competitive performance, rivaling leading commercial models such as GPT-4o and Claude-3.5-Sonnet. Notably, our model is the first open-source MLLMs to surpass 70% on the MMMU benchmark, achieving a 3.7-point improvement through Chain-of-Thought (CoT) reasoning and showcasing strong potential for test-time scaling. We hope this model contributes to the open-source community by setting new standards for developing and applying multimodal AI systems. HuggingFace demo see this https URL
|
| 33 |
+
|
| 34 |
## Introduction
|
| 35 |
|
| 36 |
We introduce *InternVL3.5*, a new family of open-source multimodal models that significantly advances versatility, reasoning capability, and inference efficiency along the InternVL series. A key innovation is the *Cascade Reinforcement Learning (Cascade RL)* framework, which enhances reasoning through a two-stage process: offline RL for stable convergence and online RL for refined alignment. This coarse-to-fine training strategy leads to substantial improvements on downstream reasoning tasks, e.g., MMMU and MathVista. To optimize efficiency, we propose a *Visual Resolution Router (ViR)* that dynamically adjusts the resolution of visual tokens without compromising performance. Coupled with ViR, our Decoupled *Vision-Language Deployment (DvD)* strategy separates the vision encoder and language model across different GPUs, effectively balancing computational load. These contributions collectively enable InternVL3.5 to achieve up to a +16.0\% gain in overall reasoning performance and a 4.05 \\(\times\\) inference speedup compared to its predecessor, i.e., InternVL3. In addition, InternVL3.5 supports novel capabilities such as GUI interaction and embodied agency. Notably, our largest model, i.e., InternVL3.5-241B-A28B, attains state-of-the-art results among open-source MLLMs across general multimodal, reasoning, text, and agentic tasks—narrowing the performance gap with leading commercial models like GPT-5. All models and code are publicly released.
|
|
|
|
| 239 |
\Bigg],
|
| 240 |
$$
|
| 241 |
|
| 242 |
+
where \\(\mathrm{KL}\\) denotes the KL divergence and \(\xi\) denotes the compression rate, which is uniformly sampled from \(\{\frac{1}{4},\frac{1}{16}\}\). The image \(I_\xi\) is represented as 256 tokens when \(\xi=\frac{1}{4}\) and 64 tokens when \(\xi=\frac{1}{16}\). Notably, the reference model always performs inference with \(\xi=\frac{1}{4}\).
|
| 243 |
|
| 244 |
|
| 245 |
`Router training`:
|
|
|
|
| 500 |
|
| 501 |
images = [load_image(img_url) for img_url in image_urls]
|
| 502 |
# Numbering images improves multi-image conversations
|
| 503 |
+
response = pipe((f'Image-1: {IMAGE_TOKEN}
|
| 504 |
+
Image-2: {IMAGE_TOKEN}
|
| 505 |
+
describe these two images', images))
|
| 506 |
print(response.text)
|
| 507 |
```
|
| 508 |
|
|
|
|
| 604 |
journal={arXiv preprint arXiv:2508.18265},
|
| 605 |
year={2025}
|
| 606 |
}
|
| 607 |
+
@article{zhu2025internvl3,
|
| 608 |
+
title={Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models},
|
| 609 |
+
author={Zhu, Jinguo and Wang, Weiyun and Chen, Zhe and Liu, Zhaoyang and Ye, Shenglong and Gu, Lixin and Tian, Hao and Duan, Yuchen and Su, Weijie and Shao, Jie and others},
|
| 610 |
+
journal={arXiv preprint arXiv:2504.10479},
|
| 611 |
+
year={2025}
|
| 612 |
+
}
|
| 613 |
+
@article{chen2024expanding,
|
| 614 |
+
title={Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling},
|
| 615 |
+
author={Chen, Zhe and Wang, Weiyun and Cao, Yue and Liu, Yangzhou and Gao, Zhangwei and Cui, Erfei and Zhu, Jinguo and Ye, Shenglong and Tian, Hao and Liu, Zhaoyang and others},
|
| 616 |
+
journal={arXiv preprint arXiv:2412.05271},
|
| 617 |
+
year={2024}
|
| 618 |
+
}
|
| 619 |
+
@article{wang2024mpo,
|
| 620 |
+
title={Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization},
|
| 621 |
+
author={Wang, Weiyun and Chen, Zhe and Wang, Wenhai and Cao, Yue and Liu, Yangzhou and Gao, Zhangwei and Zhu, Jinguo and Zhu, Xizhou and Lu, Lewei and Qiao, Yu and Dai, Jifeng},
|
| 622 |
+
journal={arXiv preprint arXiv:2411.10442},
|
| 623 |
+
year={2024}
|
| 624 |
+
}
|
| 625 |
+
@article{gao2024mini,
|
| 626 |
+
title={Mini-InternVL: a flexible-transfer pocket multi-modal model with 5\% parameters and 90\% performance},
|
| 627 |
+
author={Gao, Zhangwei and Chen, Zhe and Cui, Erfei and Ren, Yiming and Wang, Weiyun and Zhu, Jinguo and Tian, Hao and Ye, Shenglong and He, Junjun and Zhu, Xizhou and others},
|
| 628 |
+
journal={Visual Intelligence},
|
| 629 |
+
volume={2},
|
| 630 |
+
number={1},
|
| 631 |
+
pages={1--17},
|
| 632 |
+
year={2024},
|
| 633 |
+
publisher={Springer}
|
| 634 |
+
}
|
| 635 |
+
@article{chen2024far,
|
| 636 |
+
title={How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites},
|
| 637 |
+
author={Chen, Zhe and Wang, Weiyun and Tian, Hao and Ye, Shenglong and Gao, Zhangwei and Cui, Erfei and Tong, Wenwen and Hu, Kongzhi and Luo, Jiapeng and Ma, Zheng and others},
|
| 638 |
+
journal={Science China Information Sciences},
|
| 639 |
+
volume={67},
|
| 640 |
+
number={12},
|
| 641 |
+
pages={220101},
|
| 642 |
+
year={2024},
|
| 643 |
+
publisher={Springer}
|
| 644 |
+
}
|
| 645 |
+
@inproceedings{chen2024internvl,
|
| 646 |
+
title={Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks},
|
| 647 |
+
author={Chen, Zhe and Wu, Jiannan and Wang, Wenhai and Su, Weijie and Chen, Guo and Xing, Sen and Zhong, Muyan and Zhang, Qinglong and Zhu, Xizhou and Lu, Lewei and others},
|
| 648 |
+
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
|
| 649 |
+
pages={24185--24198},
|
| 650 |
+
year={2024}
|
| 651 |
+
}
|
| 652 |
```
|
| 653 |
+
|
| 654 |
+
## Acknowledgement
|
| 655 |
+
|
| 656 |
+
InternVL is built with reference to the code of the following projects: [OpenAI CLIP](https://github.com/openai/CLIP), [Open CLIP](https://github.com/mlfoundations/open_clip), [CLIP Benchmark](https://github.com/LAION-AI/CLIP_benchmark), [EVA](https://github.com/baaivision/EVA/tree/master), [InternImage](https://github.com/OpenGVLab/InternImage), [ViT-Adapter](https://github.com/czczup/ViT-Adapter), [MMSegmentation](https://github.com/open-mmlab/mmsegmentation), [Transformers](https://github.com/huggingface/transformers), [DINOv2](https://github.com/facebookresearch/dinov2), [BLIP-2](https://github.com/salesforce/LAVIS/tree/main/projects/blip2), [Qwen-VL](https://github.com/QwenLM/Qwen-VL/tree/master/eval_mm), and [LLaVA-1.5](https://github.com/haotian-liu/LLaVA). Thanks for their awesome work!
|
| 657 |
+
|
| 658 |
+
______________________________________________________________________
|
| 659 |
+
|
| 660 |
+
Scan the following QR Code, join our WeChat group.
|
| 661 |
+
|
| 662 |
+
<p align="center"><img width="300" alt="image" src="https://github.com/user-attachments/assets/f776df09-ebba-4fd5-80c2-fec4ff1518be"></p>
|