Upload Phi3VForCausalLMMoE

README.md

@@ -1,151 +1,199 @@
 ---
-language:
-- multilingual
-license: apache-2.0
 library_name: transformers
-datasets:
-- lamm-mit/Cephalo-Bioinspired-Mechanics-Materials
-- lamm-mit/Cephalo-Wikipedia-Materials
-tags:
-- nlp
-- code
-- vision
-- chemistry
-- engineering
-- biology
-- bio-inspired
-- text-generation-inference
-- materials science
-pipeline_tag: image-text-to-text
-inference:
-  parameters:
-    temperature: 0.3
-widget:
-- messages:
-  - role: user
-    content: <|image_1|>Can you describe what you see in the image?
+tags: []
 ---
-## Model Summary
 
-Cephalo is a series of multimodal materials science focused vision large language models (V-LLMs) designed to integrate visual and linguistic data for advanced understanding and interaction in human-AI or multi-agent AI frameworks. 
+# Model Card for Model ID
 
-A novel aspect of Cephalo's development is the innovative dataset generation method. The extraction process employs advanced algorithms to accurately detect and separate images and their corresponding textual descriptions from complex PDF documents. It involves extracting images and captions from PDFs to create well-reasoned image-text pairs, utilizing large language models (LLMs) for natural language processing. These image-text pairs are then refined and validated through LLM-based NLP processing, ensuring high-quality and contextually relevant data for training. 
+<!-- Provide a quick summary of what the model is/does. -->
 
-Cephalo can interpret complex visual scenes and generating contextually accurate language descriptions and answer queries. 
 
-The model is developed to process diverse inputs, including images and text, facilitating a broad range of applications such as image captioning, visual question answering, and multimodal content generation. The architecture combines a vision encoder model and an autoregressive transformer to process complex natural language understanding. 
 
-![image/png](https://cdn-uploads.huggingface.co/production/uploads/623ce1c6b66fedf374859fe7/kl5GWBP9WS0D4uwd1t3S7.png)
+## Model Details
 
-Cephalo provides a robust framework for multimodal interaction and understanding, including the development of complex generative pipelines to create 2D and 3D renderings of material microstructures as input for additive manufacturing methods.
+### Model Description
 
-This version of Cephalo, lamm-mit/Cephalo-Phi-3-vision-128k-4b-beta, is based on the Phi-3-Vision-128K-Instruct model. The model was trained on a combination of scientific text-image and text-only data. The model has a context length of 128,000 tokens. Further details, see: https://huggingface.co/microsoft/Phi-3-vision-128k-instruct. 
+<!-- Provide a longer summary of what this model is. -->
 
-### Chat Format
+This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
 
-Given the nature of the training data, the Cephalo-Phi-3-vision-128k-4b-beta model is best suited for a single image input wih prompts using the chat format as follows. 
-You can provide the prompt as a single image with a generic template as follow:
-```markdown
-<|user|>\n<|image_1|>\n{prompt}<|end|>\n<|assistant|>\n 
-```
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
 
-where the model generates the text after `<|assistant|>` . For multi-turn conversations, the prompt should be formatted as follows:
+### Model Sources [optional]
 
-```markdown
-<|user|>\n<|image_1|>\n{prompt_1}<|end|>\n<|assistant|>\n{response_1}<|end|>\n<|user|>\n{prompt_2}<|end|>\n<|assistant|>\n 
-```
+<!-- Provide the basic links for the model. -->
 
-### Sample inference code
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
 
-This code snippets show how to get quickly started on a GPU:
+## Uses
 
-```python
-from PIL import Image 
-import requests 
-from transformers import AutoModelForCausalLM 
-from transformers import AutoProcessor 
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
 
-model_id = "lamm-mit/Cephalo-Phi-3-vision-128k-4b-beta" 
+### Direct Use
 
-model = AutoModelForCausalLM.from_pretrained(model_id, device_map="cuda", trust_remote_code=True, torch_dtype="auto")
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
 
-processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True) 
+[More Information Needed]
 
-question = "What is shown in this image, and what is the relevance for materials design? Include a discussion of multi-agent AI."
+### Downstream Use [optional]
 
-messages = [ 
-    {"role": "user", "content": f"<|image_1|>\n{question}"}, 
-    ] 
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
 
-url = "https://d2r55xnwy6nx47.cloudfront.net/uploads/2018/02/Ants_Lede1300.jpg" 
+[More Information Needed]
 
-image = Image.open(requests.get(url, stream=True).raw) 
+### Out-of-Scope Use
 
-prompt = processor.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
 
-inputs = processor(prompt, [image], return_tensors="pt").to("cuda:0") 
+[More Information Needed]
 
-generation_args = { 
-                    "max_new_tokens": 512, 
-                    "temperature": 0.1, 
-                    "do_sample": True, 
-                    "stop_strings": ['<|end|>',
-                                     '<|endoftext|>'],
-                    "tokenizer": processor.tokenizer,
-                  } 
+## Bias, Risks, and Limitations
 
-generate_ids = model.generate(**inputs, eos_token_id=processor.tokenizer.eos_token_id, **generation_args) 
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
 
-# remove input tokens 
-generate_ids = generate_ids[:, inputs['input_ids'].shape[1]:]
-response = processor.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0] 
+[More Information Needed]
 
-print(response) 
-```
-Sample output:
+### Recommendations
 
-![image/png](https://cdn-uploads.huggingface.co/production/uploads/623ce1c6b66fedf374859fe7/5n6oRNHrfwHkBX0QertZp.png)
-<small>Image by [Vaishakh Manohar](https://www.quantamagazine.org/the-simple-algorithm-that-ants-use-to-build-bridges-20180226/)</small>
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
 
-<pre style="white-space: pre-wrap;">
-The image shows a group of red ants (Solenopsis invicta) climbing over a vertical wooden post. The ants are using their long legs and antennae to navigate the rough surface of the wood, demonstrating their ability to adapt to different materials and environments. This behavior is relevant for materials design because it highlights the importance of considering the interactions between materials and living organisms, such as ants, when designing new materials.
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
 
-Multi-agent AI (Artificial Intelligence) is a field of study that focuses on the development of AI systems that can work together with other AI systems to achieve a common goal. In the context of this image, multi-agent AI could be used to design materials that are more compatible with the natural behaviors of living organisms, such as ants, and that can adapt to different environments and conditions.
+## How to Get Started with the Model
 
-By studying the behavior of ants and other living organisms, researchers can gain insights into how materials can be designed to better interact with these organisms and to better mimic their natural behaviors. This can lead to the development of new materials that are more sustainable, efficient, and effective in a variety of applications.
+Use the code below to get started with the model.
 
-In summary, the image of red ants climbing over a wooden post highlights the importance of considering the interactions between materials and living organisms when designing new materials, and the potential of multi-agent AI to help achieve this goal.
-</pre>
+[More Information Needed]
 
+## Training Details
 
-## Dataset generation
+### Training Data
 
-The schematic below shows a visualization of the approach to generate datasets for training the vision model. The extraction process employs advanced algorithms to accurately detect and separate images and their corresponding textual descriptions from complex PDF documents. It involves extracting images and captions from PDFs to create well-reasoned image-text pairs, utilizing large language models (LLMs) for natural language processing. These image-text pairs are then refined and validated through LLM-based NLP processing, ensuring high-quality and contextually relevant data for training.
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
 
-The image below shows reproductions of two representative pages of the scientific article (here, Spivak, Buehler, et al., 2011), and how they are used to extract visual scientific data for training the Cephalo model.
+[More Information Needed]
 
-![image/png](https://cdn-uploads.huggingface.co/production/uploads/623ce1c6b66fedf374859fe7/qHURSBRWEDgHy4o56escN.png)
+### Training Procedure
 
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
 
-## Example applications
+#### Preprocessing [optional]
 
-The paper provides detailed examples and use cases. Here is a visual of a pipeline that consists of 1) analysis of an image provided to Cephalo-Phi-3-vision-128k-4b-beta, 2) generation of an image generation fromt, and 3) generation of a new image using Stable Diffusion XL Turbo.
+[More Information Needed]
 
-![image/png](https://cdn-uploads.huggingface.co/production/uploads/623ce1c6b66fedf374859fe7/3VvHK_c9eJolQvfOrhiBw.png)
 
-A similar mechanism can be employed to generate 3D models:
+#### Training Hyperparameters
 
-![image/png](https://cdn-uploads.huggingface.co/production/uploads/623ce1c6b66fedf374859fe7/6ZsvCZ3x3TGvugly44MMI.png)
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
 
-## Citation
+#### Speeds, Sizes, Times [optional]
 
-Please cite as:
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
 
-```bibtex
-@article{Buehler_Cephalo_2024,
-  title={Cephalo: Multi-Modal Vision-Language Models for Bio-Inspired Materials Analysis and Design},
-  author={Markus J. Buehler},
-  journal={arXiv preprint arXiv:2405.19076},
-  year={2024}
-}
-```
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]

config.json

@@ -0,0 +1,152 @@
+{
+  "_name_or_path": "microsoft/Phi-3-vision-128k-instruct",
+  "architectures": [
+    "Phi3VForCausalLMMoE"
+  ],
+  "attention_dropout": 0.0,
+  "auto_map": {
+    "AutoConfig": "moe_phi3_v.Phi3VForCausalLMMoEConfig",
+    "AutoModelForCausalLM": "moe_phi3_v.Phi3VForCausalLMMoE"
+  },
+  "bos_token_id": 1,
+  "embd_layer": {
+    "embedding_cls": "image",
+    "hd_transform_order": "sub_glb",
+    "projection_cls": "mlp",
+    "use_hd_transform": true,
+    "with_learnable_separator": true
+  },
+  "embd_pdrop": 0.0,
+  "eos_token_id": 2,
+  "hidden_act": "silu",
+  "hidden_size": 3072,
+  "img_processor": {
+    "image_dim_out": 1024,
+    "model_name": "openai/clip-vit-large-patch14-336",
+    "name": "clip_vision_model",
+    "num_img_tokens": 144
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 8192,
+  "k": 1,
+  "max_position_embeddings": 131072,
+  "model_type": "phi3_v_moe",
+  "num_attention_heads": 32,
+  "num_expert_models": 3,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 32,
+  "original_max_position_embeddings": 4096,
+  "pad_token_id": 32000,
+  "resid_pdrop": 0.0,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": {
+    "long_factor": [
+      1.0299999713897705,
+      1.0499999523162842,
+      1.0499999523162842,
+      1.0799999237060547,
+      1.2299998998641968,
+      1.2299998998641968,
+      1.2999999523162842,
+      1.4499999284744263,
+      1.5999999046325684,
+      1.6499998569488525,
+      1.8999998569488525,
+      2.859999895095825,
+      3.68999981880188,
+      5.419999599456787,
+      5.489999771118164,
+      5.489999771118164,
+      9.09000015258789,
+      11.579999923706055,
+      15.65999984741211,
+      15.769999504089355,
+      15.789999961853027,
+      18.360000610351562,
+      21.989999771118164,
+      23.079999923706055,
+      30.009998321533203,
+      32.35000228881836,
+      32.590003967285156,
+      35.56000518798828,
+      39.95000457763672,
+      53.840003967285156,
+      56.20000457763672,
+      57.95000457763672,
+      59.29000473022461,
+      59.77000427246094,
+      59.920005798339844,
+      61.190006256103516,
+      61.96000671386719,
+      62.50000762939453,
+      63.3700065612793,
+      63.48000717163086,
+      63.48000717163086,
+      63.66000747680664,
+      63.850006103515625,
+      64.08000946044922,
+      64.760009765625,
+      64.80001068115234,
+      64.81001281738281,
+      64.81001281738281
+    ],
+    "short_factor": [
+      1.05,
+      1.05,
+      1.05,
+      1.1,
+      1.1,
+      1.1,
+      1.2500000000000002,
+      1.2500000000000002,
+      1.4000000000000004,
+      1.4500000000000004,
+      1.5500000000000005,
+      1.8500000000000008,
+      1.9000000000000008,
+      2.000000000000001,
+      2.000000000000001,
+      2.000000000000001,
+      2.000000000000001,
+      2.000000000000001,
+      2.000000000000001,
+      2.000000000000001,
+      2.000000000000001,
+      2.000000000000001,
+      2.000000000000001,
+      2.000000000000001,
+      2.000000000000001,
+      2.000000000000001,
+      2.000000000000001,
+      2.000000000000001,
+      2.000000000000001,
+      2.000000000000001,
+      2.000000000000001,
+      2.000000000000001,
+      2.1000000000000005,
+      2.1000000000000005,
+      2.2,
+      2.3499999999999996,
+      2.3499999999999996,
+      2.3499999999999996,
+      2.3499999999999996,
+      2.3999999999999995,
+      2.3999999999999995,
+      2.6499999999999986,
+      2.6999999999999984,
+      2.8999999999999977,
+      2.9499999999999975,
+      3.049999999999997,
+      3.049999999999997,
+      3.049999999999997
+    ],
+    "type": "su"
+  },
+  "rope_theta": 10000.0,
+  "sliding_window": 131072,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.41.1",
+  "use_cache": true,
+  "vocab_size": 32064
+}

generation_config.json

@@ -0,0 +1,7 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "pad_token_id": 32000,
+  "transformers_version": "4.41.1"
+}

pytorch_model-00001-of-00004.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:13395a89696b690cb4cdffd5e8bb225e64181a33535417761e830a5ce4672c18
+size 4925359194

pytorch_model-00002-of-00004.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7ec54a2ec76df6fd71f158726797b1744d9dc94f6bfbd3682a0753a260f26d0a
+size 4983183042

pytorch_model-00003-of-00004.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7dc0f73ed03e0e85ccdfd31c2cdbd01a73bd28cc55978e7d06778a50db952c49
+size 4907652664

pytorch_model-00004-of-00004.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d15760408a807be3102337cc34d3bd25406bf0d4eec9f6517757bd59b60a6694
+size 3535621544

pytorch_model.bin.index.json

@@ -0,0 +1,793 @@
+{
+  "metadata": {
+    "total_size": 18351511744
+  },
+  "weight_map": {
+    "lm_head.weight": "pytorch_model-00004-of-00004.bin",
+    "model.lm_head.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.embed_tokens.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.0.input_layernorm.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.0.mlp.experts.0.down_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.0.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.0.mlp.experts.1.down_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.0.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.0.mlp.experts.2.down_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.0.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.0.mlp.gating_layer.gate.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.0.mlp.gating_layer.gate.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.0.post_attention_layernorm.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.0.self_attn.o_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.0.self_attn.qkv_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.1.input_layernorm.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.1.mlp.experts.0.down_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.1.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.1.mlp.experts.1.down_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.1.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.1.mlp.experts.2.down_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.1.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.1.mlp.gating_layer.gate.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.1.mlp.gating_layer.gate.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.1.post_attention_layernorm.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.1.self_attn.o_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.1.self_attn.qkv_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.10.input_layernorm.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.10.mlp.experts.0.down_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.10.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.10.mlp.experts.1.down_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.10.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.10.mlp.experts.2.down_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.10.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.10.mlp.gating_layer.gate.bias": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.10.mlp.gating_layer.gate.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.10.post_attention_layernorm.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.10.self_attn.o_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.10.self_attn.qkv_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.11.input_layernorm.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.11.mlp.experts.0.down_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.11.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.11.mlp.experts.1.down_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.11.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.11.mlp.experts.2.down_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.11.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.11.mlp.gating_layer.gate.bias": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.11.mlp.gating_layer.gate.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.11.post_attention_layernorm.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.11.self_attn.o_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.11.self_attn.qkv_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.12.input_layernorm.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.12.mlp.experts.0.down_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.12.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.12.mlp.experts.1.down_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.12.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.12.mlp.experts.2.down_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.12.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.12.mlp.gating_layer.gate.bias": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.12.mlp.gating_layer.gate.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.12.post_attention_layernorm.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.12.self_attn.o_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.12.self_attn.qkv_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.13.input_layernorm.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.13.mlp.experts.0.down_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.13.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.13.mlp.experts.1.down_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.13.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.13.mlp.experts.2.down_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.13.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.13.mlp.gating_layer.gate.bias": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.13.mlp.gating_layer.gate.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.13.post_attention_layernorm.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.13.self_attn.o_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.13.self_attn.qkv_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.14.input_layernorm.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.14.mlp.experts.0.down_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.14.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.14.mlp.experts.1.down_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.14.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.14.mlp.experts.2.down_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.14.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.14.mlp.gating_layer.gate.bias": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.14.mlp.gating_layer.gate.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.14.post_attention_layernorm.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.14.self_attn.o_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.14.self_attn.qkv_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.15.input_layernorm.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.15.mlp.experts.0.down_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.15.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.15.mlp.experts.1.down_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.15.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.15.mlp.experts.2.down_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.15.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.15.mlp.gating_layer.gate.bias": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.15.mlp.gating_layer.gate.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.15.post_attention_layernorm.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.15.self_attn.o_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.15.self_attn.qkv_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.16.input_layernorm.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.16.mlp.experts.0.down_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.16.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.16.mlp.experts.1.down_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.16.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.16.mlp.experts.2.down_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.16.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.16.mlp.gating_layer.gate.bias": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.16.mlp.gating_layer.gate.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.16.post_attention_layernorm.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.16.self_attn.o_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.16.self_attn.qkv_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.17.input_layernorm.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.17.mlp.experts.0.down_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.17.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.17.mlp.experts.1.down_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.17.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.17.mlp.experts.2.down_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.17.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.17.mlp.gating_layer.gate.bias": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.17.mlp.gating_layer.gate.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.17.post_attention_layernorm.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.17.self_attn.o_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.17.self_attn.qkv_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.18.input_layernorm.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.18.mlp.experts.0.down_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.18.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.18.mlp.experts.1.down_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.18.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.18.mlp.experts.2.down_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.18.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.18.mlp.gating_layer.gate.bias": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.18.mlp.gating_layer.gate.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.18.post_attention_layernorm.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.18.self_attn.o_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.18.self_attn.qkv_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.19.input_layernorm.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.19.mlp.experts.0.down_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.19.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.19.mlp.experts.1.down_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.19.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.19.mlp.experts.2.down_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.19.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.19.mlp.gating_layer.gate.bias": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.19.mlp.gating_layer.gate.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.19.post_attention_layernorm.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.19.self_attn.o_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.19.self_attn.qkv_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.2.input_layernorm.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.2.mlp.experts.0.down_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.2.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.2.mlp.experts.1.down_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.2.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.2.mlp.experts.2.down_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.2.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.2.mlp.gating_layer.gate.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.2.mlp.gating_layer.gate.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.2.post_attention_layernorm.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.2.self_attn.o_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.2.self_attn.qkv_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.20.input_layernorm.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.20.mlp.experts.0.down_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.20.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.20.mlp.experts.1.down_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.20.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.20.mlp.experts.2.down_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.20.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.20.mlp.gating_layer.gate.bias": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.20.mlp.gating_layer.gate.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.20.post_attention_layernorm.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.20.self_attn.o_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.20.self_attn.qkv_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.21.input_layernorm.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.21.mlp.experts.0.down_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.21.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.21.mlp.experts.1.down_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.21.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.21.mlp.experts.2.down_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.21.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.21.mlp.gating_layer.gate.bias": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.21.mlp.gating_layer.gate.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.21.post_attention_layernorm.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.21.self_attn.o_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.21.self_attn.qkv_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.22.input_layernorm.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.22.mlp.experts.0.down_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.22.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.22.mlp.experts.1.down_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.22.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.22.mlp.experts.2.down_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.22.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.22.mlp.gating_layer.gate.bias": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.22.mlp.gating_layer.gate.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.22.post_attention_layernorm.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.22.self_attn.o_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.22.self_attn.qkv_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.23.input_layernorm.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.23.mlp.experts.0.down_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.23.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.23.mlp.experts.1.down_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.23.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.23.mlp.experts.2.down_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.23.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.23.mlp.gating_layer.gate.bias": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.23.mlp.gating_layer.gate.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.23.post_attention_layernorm.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.23.self_attn.o_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.23.self_attn.qkv_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.24.input_layernorm.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.24.mlp.experts.0.down_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.24.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.24.mlp.experts.1.down_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.24.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.24.mlp.experts.2.down_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.24.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.24.mlp.gating_layer.gate.bias": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.24.mlp.gating_layer.gate.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.24.post_attention_layernorm.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.24.self_attn.o_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.24.self_attn.qkv_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.25.input_layernorm.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.25.mlp.experts.0.down_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.25.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.25.mlp.experts.1.down_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.25.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.25.mlp.experts.2.down_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.25.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.25.mlp.gating_layer.gate.bias": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.25.mlp.gating_layer.gate.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.25.post_attention_layernorm.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.25.self_attn.o_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.25.self_attn.qkv_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.26.input_layernorm.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.26.mlp.experts.0.down_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.26.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.26.mlp.experts.1.down_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.26.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.26.mlp.experts.2.down_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.26.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.26.mlp.gating_layer.gate.bias": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.26.mlp.gating_layer.gate.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.26.post_attention_layernorm.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.26.self_attn.o_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.26.self_attn.qkv_proj.weight": "pytorch_model-00003-of-00004.bin",
+    "model.model.layers.27.input_layernorm.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.27.mlp.experts.0.down_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.27.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.27.mlp.experts.1.down_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.27.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.27.mlp.experts.2.down_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.27.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.27.mlp.gating_layer.gate.bias": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.27.mlp.gating_layer.gate.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.27.post_attention_layernorm.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.27.self_attn.o_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.27.self_attn.qkv_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.28.input_layernorm.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.28.mlp.experts.0.down_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.28.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.28.mlp.experts.1.down_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.28.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.28.mlp.experts.2.down_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.28.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.28.mlp.gating_layer.gate.bias": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.28.mlp.gating_layer.gate.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.28.post_attention_layernorm.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.28.self_attn.o_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.28.self_attn.qkv_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.29.input_layernorm.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.29.mlp.experts.0.down_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.29.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.29.mlp.experts.1.down_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.29.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.29.mlp.experts.2.down_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.29.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.29.mlp.gating_layer.gate.bias": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.29.mlp.gating_layer.gate.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.29.post_attention_layernorm.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.29.self_attn.o_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.29.self_attn.qkv_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.3.input_layernorm.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.3.mlp.experts.0.down_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.3.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.3.mlp.experts.1.down_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.3.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.3.mlp.experts.2.down_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.3.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.3.mlp.gating_layer.gate.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.3.mlp.gating_layer.gate.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.3.post_attention_layernorm.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.3.self_attn.o_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.3.self_attn.qkv_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.30.input_layernorm.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.30.mlp.experts.0.down_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.30.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.30.mlp.experts.1.down_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.30.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.30.mlp.experts.2.down_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.30.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.30.mlp.gating_layer.gate.bias": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.30.mlp.gating_layer.gate.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.30.post_attention_layernorm.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.30.self_attn.o_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.30.self_attn.qkv_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.31.input_layernorm.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.31.mlp.experts.0.down_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.31.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.31.mlp.experts.1.down_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.31.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.31.mlp.experts.2.down_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.31.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.31.mlp.gating_layer.gate.bias": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.31.mlp.gating_layer.gate.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.31.post_attention_layernorm.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.31.self_attn.o_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.31.self_attn.qkv_proj.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.layers.4.input_layernorm.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.4.mlp.experts.0.down_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.4.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.4.mlp.experts.1.down_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.4.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.4.mlp.experts.2.down_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.4.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.4.mlp.gating_layer.gate.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.4.mlp.gating_layer.gate.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.4.post_attention_layernorm.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.4.self_attn.o_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.4.self_attn.qkv_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.5.input_layernorm.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.5.mlp.experts.0.down_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.5.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.5.mlp.experts.1.down_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.5.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.5.mlp.experts.2.down_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.5.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.5.mlp.gating_layer.gate.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.5.mlp.gating_layer.gate.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.5.post_attention_layernorm.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.5.self_attn.o_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.5.self_attn.qkv_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.6.input_layernorm.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.6.mlp.experts.0.down_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.6.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.6.mlp.experts.1.down_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.6.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.6.mlp.experts.2.down_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.6.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.6.mlp.gating_layer.gate.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.6.mlp.gating_layer.gate.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.6.post_attention_layernorm.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.6.self_attn.o_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.6.self_attn.qkv_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.7.input_layernorm.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.7.mlp.experts.0.down_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.7.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.7.mlp.experts.1.down_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.7.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.7.mlp.experts.2.down_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.7.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.7.mlp.gating_layer.gate.bias": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.7.mlp.gating_layer.gate.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.7.post_attention_layernorm.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.7.self_attn.o_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.7.self_attn.qkv_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.layers.8.input_layernorm.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.8.mlp.experts.0.down_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.8.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.8.mlp.experts.1.down_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.8.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.8.mlp.experts.2.down_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.8.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.8.mlp.gating_layer.gate.bias": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.8.mlp.gating_layer.gate.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.8.post_attention_layernorm.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.8.self_attn.o_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.8.self_attn.qkv_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.9.input_layernorm.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.9.mlp.experts.0.down_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.9.mlp.experts.0.gate_up_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.9.mlp.experts.1.down_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.9.mlp.experts.1.gate_up_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.9.mlp.experts.2.down_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.9.mlp.experts.2.gate_up_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.9.mlp.gating_layer.gate.bias": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.9.mlp.gating_layer.gate.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.9.post_attention_layernorm.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.9.self_attn.o_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.layers.9.self_attn.qkv_proj.weight": "pytorch_model-00002-of-00004.bin",
+    "model.model.norm.weight": "pytorch_model-00004-of-00004.bin",
+    "model.model.vision_embed_tokens.glb_GN": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.embeddings.class_embedding": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.embeddings.patch_embedding.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.embeddings.position_embedding.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.0.layer_norm1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.0.layer_norm1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.0.layer_norm2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.0.layer_norm2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.0.mlp.fc1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.0.mlp.fc1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.0.mlp.fc2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.0.mlp.fc2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.0.self_attn.k_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.0.self_attn.k_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.0.self_attn.out_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.0.self_attn.out_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.0.self_attn.q_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.0.self_attn.q_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.0.self_attn.v_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.0.self_attn.v_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.1.layer_norm1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.1.layer_norm1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.1.layer_norm2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.1.layer_norm2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.1.mlp.fc1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.1.mlp.fc1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.1.mlp.fc2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.1.mlp.fc2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.1.self_attn.k_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.1.self_attn.k_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.1.self_attn.out_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.1.self_attn.out_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.1.self_attn.q_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.1.self_attn.q_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.1.self_attn.v_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.1.self_attn.v_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.10.layer_norm1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.10.layer_norm1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.10.layer_norm2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.10.layer_norm2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.10.mlp.fc1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.10.mlp.fc1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.10.mlp.fc2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.10.mlp.fc2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.10.self_attn.k_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.10.self_attn.k_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.10.self_attn.out_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.10.self_attn.out_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.10.self_attn.q_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.10.self_attn.q_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.10.self_attn.v_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.10.self_attn.v_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.11.layer_norm1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.11.layer_norm1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.11.layer_norm2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.11.layer_norm2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.11.mlp.fc1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.11.mlp.fc1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.11.mlp.fc2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.11.mlp.fc2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.11.self_attn.k_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.11.self_attn.k_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.11.self_attn.out_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.11.self_attn.out_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.11.self_attn.q_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.11.self_attn.q_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.11.self_attn.v_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.11.self_attn.v_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.12.layer_norm1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.12.layer_norm1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.12.layer_norm2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.12.layer_norm2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.12.mlp.fc1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.12.mlp.fc1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.12.mlp.fc2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.12.mlp.fc2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.12.self_attn.k_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.12.self_attn.k_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.12.self_attn.out_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.12.self_attn.out_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.12.self_attn.q_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.12.self_attn.q_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.12.self_attn.v_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.12.self_attn.v_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.13.layer_norm1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.13.layer_norm1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.13.layer_norm2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.13.layer_norm2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.13.mlp.fc1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.13.mlp.fc1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.13.mlp.fc2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.13.mlp.fc2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.13.self_attn.k_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.13.self_attn.k_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.13.self_attn.out_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.13.self_attn.out_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.13.self_attn.q_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.13.self_attn.q_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.13.self_attn.v_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.13.self_attn.v_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.14.layer_norm1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.14.layer_norm1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.14.layer_norm2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.14.layer_norm2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.14.mlp.fc1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.14.mlp.fc1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.14.mlp.fc2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.14.mlp.fc2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.14.self_attn.k_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.14.self_attn.k_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.14.self_attn.out_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.14.self_attn.out_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.14.self_attn.q_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.14.self_attn.q_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.14.self_attn.v_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.14.self_attn.v_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.15.layer_norm1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.15.layer_norm1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.15.layer_norm2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.15.layer_norm2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.15.mlp.fc1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.15.mlp.fc1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.15.mlp.fc2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.15.mlp.fc2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.15.self_attn.k_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.15.self_attn.k_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.15.self_attn.out_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.15.self_attn.out_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.15.self_attn.q_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.15.self_attn.q_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.15.self_attn.v_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.15.self_attn.v_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.16.layer_norm1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.16.layer_norm1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.16.layer_norm2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.16.layer_norm2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.16.mlp.fc1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.16.mlp.fc1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.16.mlp.fc2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.16.mlp.fc2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.16.self_attn.k_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.16.self_attn.k_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.16.self_attn.out_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.16.self_attn.out_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.16.self_attn.q_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.16.self_attn.q_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.16.self_attn.v_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.16.self_attn.v_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.17.layer_norm1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.17.layer_norm1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.17.layer_norm2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.17.layer_norm2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.17.mlp.fc1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.17.mlp.fc1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.17.mlp.fc2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.17.mlp.fc2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.17.self_attn.k_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.17.self_attn.k_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.17.self_attn.out_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.17.self_attn.out_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.17.self_attn.q_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.17.self_attn.q_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.17.self_attn.v_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.17.self_attn.v_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.18.layer_norm1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.18.layer_norm1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.18.layer_norm2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.18.layer_norm2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.18.mlp.fc1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.18.mlp.fc1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.18.mlp.fc2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.18.mlp.fc2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.18.self_attn.k_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.18.self_attn.k_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.18.self_attn.out_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.18.self_attn.out_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.18.self_attn.q_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.18.self_attn.q_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.18.self_attn.v_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.18.self_attn.v_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.19.layer_norm1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.19.layer_norm1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.19.layer_norm2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.19.layer_norm2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.19.mlp.fc1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.19.mlp.fc1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.19.mlp.fc2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.19.mlp.fc2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.19.self_attn.k_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.19.self_attn.k_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.19.self_attn.out_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.19.self_attn.out_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.19.self_attn.q_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.19.self_attn.q_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.19.self_attn.v_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.19.self_attn.v_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.2.layer_norm1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.2.layer_norm1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.2.layer_norm2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.2.layer_norm2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.2.mlp.fc1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.2.mlp.fc1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.2.mlp.fc2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.2.mlp.fc2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.2.self_attn.k_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.2.self_attn.k_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.2.self_attn.out_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.2.self_attn.out_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.2.self_attn.q_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.2.self_attn.q_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.2.self_attn.v_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.2.self_attn.v_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.20.layer_norm1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.20.layer_norm1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.20.layer_norm2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.20.layer_norm2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.20.mlp.fc1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.20.mlp.fc1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.20.mlp.fc2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.20.mlp.fc2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.20.self_attn.k_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.20.self_attn.k_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.20.self_attn.out_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.20.self_attn.out_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.20.self_attn.q_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.20.self_attn.q_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.20.self_attn.v_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.20.self_attn.v_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.21.layer_norm1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.21.layer_norm1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.21.layer_norm2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.21.layer_norm2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.21.mlp.fc1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.21.mlp.fc1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.21.mlp.fc2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.21.mlp.fc2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.21.self_attn.k_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.21.self_attn.k_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.21.self_attn.out_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.21.self_attn.out_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.21.self_attn.q_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.21.self_attn.q_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.21.self_attn.v_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.21.self_attn.v_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.22.layer_norm1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.22.layer_norm1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.22.layer_norm2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.22.layer_norm2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.22.mlp.fc1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.22.mlp.fc1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.22.mlp.fc2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.22.mlp.fc2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.22.self_attn.k_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.22.self_attn.k_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.22.self_attn.out_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.22.self_attn.out_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.22.self_attn.q_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.22.self_attn.q_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.22.self_attn.v_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.22.self_attn.v_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.23.layer_norm1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.23.layer_norm1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.23.layer_norm2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.23.layer_norm2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.23.mlp.fc1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.23.mlp.fc1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.23.mlp.fc2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.23.mlp.fc2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.23.self_attn.k_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.23.self_attn.k_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.23.self_attn.out_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.23.self_attn.out_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.23.self_attn.q_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.23.self_attn.q_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.23.self_attn.v_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.23.self_attn.v_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.3.layer_norm1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.3.layer_norm1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.3.layer_norm2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.3.layer_norm2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.3.mlp.fc1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.3.mlp.fc1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.3.mlp.fc2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.3.mlp.fc2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.3.self_attn.k_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.3.self_attn.k_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.3.self_attn.out_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.3.self_attn.out_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.3.self_attn.q_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.3.self_attn.q_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.3.self_attn.v_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.3.self_attn.v_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.4.layer_norm1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.4.layer_norm1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.4.layer_norm2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.4.layer_norm2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.4.mlp.fc1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.4.mlp.fc1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.4.mlp.fc2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.4.mlp.fc2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.4.self_attn.k_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.4.self_attn.k_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.4.self_attn.out_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.4.self_attn.out_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.4.self_attn.q_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.4.self_attn.q_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.4.self_attn.v_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.4.self_attn.v_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.5.layer_norm1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.5.layer_norm1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.5.layer_norm2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.5.layer_norm2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.5.mlp.fc1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.5.mlp.fc1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.5.mlp.fc2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.5.mlp.fc2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.5.self_attn.k_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.5.self_attn.k_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.5.self_attn.out_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.5.self_attn.out_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.5.self_attn.q_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.5.self_attn.q_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.5.self_attn.v_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.5.self_attn.v_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.6.layer_norm1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.6.layer_norm1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.6.layer_norm2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.6.layer_norm2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.6.mlp.fc1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.6.mlp.fc1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.6.mlp.fc2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.6.mlp.fc2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.6.self_attn.k_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.6.self_attn.k_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.6.self_attn.out_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.6.self_attn.out_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.6.self_attn.q_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.6.self_attn.q_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.6.self_attn.v_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.6.self_attn.v_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.7.layer_norm1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.7.layer_norm1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.7.layer_norm2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.7.layer_norm2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.7.mlp.fc1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.7.mlp.fc1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.7.mlp.fc2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.7.mlp.fc2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.7.self_attn.k_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.7.self_attn.k_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.7.self_attn.out_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.7.self_attn.out_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.7.self_attn.q_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.7.self_attn.q_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.7.self_attn.v_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.7.self_attn.v_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.8.layer_norm1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.8.layer_norm1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.8.layer_norm2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.8.layer_norm2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.8.mlp.fc1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.8.mlp.fc1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.8.mlp.fc2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.8.mlp.fc2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.8.self_attn.k_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.8.self_attn.k_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.8.self_attn.out_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.8.self_attn.out_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.8.self_attn.q_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.8.self_attn.q_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.8.self_attn.v_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.8.self_attn.v_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.9.layer_norm1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.9.layer_norm1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.9.layer_norm2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.9.layer_norm2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.9.mlp.fc1.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.9.mlp.fc1.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.9.mlp.fc2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.9.mlp.fc2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.9.self_attn.k_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.9.self_attn.k_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.9.self_attn.out_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.9.self_attn.out_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.9.self_attn.q_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.9.self_attn.q_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.9.self_attn.v_proj.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.encoder.layers.9.self_attn.v_proj.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.post_layernorm.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.post_layernorm.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.pre_layrnorm.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_processor.vision_model.pre_layrnorm.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_projection.0.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_projection.0.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_projection.2.bias": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.img_projection.2.weight": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.sub_GN": "pytorch_model-00001-of-00004.bin",
+    "model.model.vision_embed_tokens.wte.weight": "pytorch_model-00001-of-00004.bin"
+  }
+}