| tags: | |
| - quantized | |
| - 4-bit | |
| - AWQ | |
| - autotrain_compatible | |
| - endpoints_compatible | |
| - text-generation-inference | |
| license: apache-2.0 | |
| language: | |
| - en | |
| base_model: Vezora/Mistral-22B-v0.1 | |
| model_creator: Vezora | |
| model_name: Mistral-22B-v0.1 | |
| model_type: mistral | |
| pipeline_tag: text-generation | |
| inference: false | |
| # Vezora/Mistral-22B-v0.1 AWQ | |
| - Model creator: [Vezora](https://huggingface.co/Vezora) | |
| - Original model: [Mistral-22B-v0.1](https://huggingface.co/Vezora/Mistral-22B-v0.1) | |
| ## Model Summary | |
| This model is not an moe, it is infact a 22B parameter dense model! | |
| Just one day after the release of **Mixtral-8x-22b**, we are excited to introduce our handcrafted experimental model, **Mistral-22b-V.01**. This model is a culmination of equal knowledge distilled from all experts into a single, dense 22b model. This model is not a single trained expert, rather its a compressed MOE model, turning it into a dense 22b mode. This is the first working MOE to Dense model conversion. | |
| ## How to use | |
| **GUANACO PROMPT FORMAT** YOU MUST USE THE GUANACO PROMPT FORMAT SHOWN BELOW. Not using this prompt format will lead to sub optimal results. | |
| - This model requires a specific chat template, as the training format was Guanaco this is what it looks like: | |
| - "### System: You are a helpful assistant. ### Human###: Give me the best chili recipe you can ###Assistant: Here is the best chili recipe..." |