amokrov commited on
Commit
66b3dc0
·
verified ·
1 Parent(s): 76f4165

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -45
README.md CHANGED
@@ -8,9 +8,6 @@ base_model:
8
  ---
9
  # bge-base-en-v1.5-int8-ov
10
 
11
- > [!WARNING]
12
- > **Disclaimer**: This model is provided for evaluation purposes only. Performance, accuracy, and stability may vary. Use at your own discretion.
13
-
14
  * Model creator: [BAAI](https://huggingface.co/BAAI)
15
  * Original model: [bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5)
16
 
@@ -22,56 +19,19 @@ This is [bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) model c
22
 
23
  ## Quantization Parameters
24
 
25
- The quantization was performed using the next code:
26
-
27
- ```
28
- from functools import partial
29
-
30
- from transformers import AutoTokenizer
31
-
32
- from optimum.intel import OVConfig, OVModelForFeatureExtraction, OVQuantizationConfig, OVQuantizer
33
-
34
-
35
- MODEL_ID = "OpenVINO/bge-base-en-v1.5-fp16-ov"
36
- base_model_path = "bge-base-en-v1.5"
37
- int8_ptq_model_path = "bge-base-en-v1.5-int8"
38
-
39
- model = OVModelForFeatureExtraction.from_pretrained(MODEL_ID)
40
- model.save_pretrained(base_model_path)
41
 
42
- tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
43
- tokenizer.save_pretrained(base_model_path)
44
-
45
-
46
- quantizer = OVQuantizer.from_pretrained(model)
47
-
48
- def preprocess_function(examples, tokenizer):
49
- return tokenizer(examples["sentence"], padding="max_length", max_length=384, truncation=True)
50
-
51
-
52
- calibration_dataset = quantizer.get_calibration_dataset(
53
- "glue",
54
- dataset_config_name="sst2",
55
- preprocess_function=partial(preprocess_function, tokenizer=tokenizer),
56
- num_samples=300,
57
- dataset_split="train",
58
- )
59
-
60
- ov_config = OVConfig(quantization_config=OVQuantizationConfig())
61
-
62
- quantizer.quantize(ov_config=ov_config, calibration_dataset=calibration_dataset, save_directory=int8_ptq_model_path)
63
- tokenizer.save_pretrained(int8_ptq_model_path)
64
- ```
65
 
66
- For more information on quantization, check the [OpenVINO model optimization guide](https://docs.openvino.ai/2025/openvino-workflow/model-optimization-guide/quantizing-models-post-training.html).
67
 
68
 
69
  ## Compatibility
70
 
71
  The provided OpenVINO™ IR model is compatible with:
72
 
73
- * OpenVINO version 2025.1.0 and higher
74
- * Optimum Intel 1.24.0 and higher
75
 
76
 
77
  ## Running Model Inference with [Optimum Intel](https://huggingface.co/docs/optimum/intel/index)
 
8
  ---
9
  # bge-base-en-v1.5-int8-ov
10
 
 
 
 
11
  * Model creator: [BAAI](https://huggingface.co/BAAI)
12
  * Original model: [bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5)
13
 
 
19
 
20
  ## Quantization Parameters
21
 
22
+ Weight compression was performed using `nncf.compress_weights` with the following parameters:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
23
 
24
+ * mode: **INT8_ASYM**
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
25
 
26
+ For more information on quantization, check the [OpenVINO model optimization guide](https://docs.openvino.ai/2025/openvino-workflow/model-optimization-guide/weight-compression.html).
27
 
28
 
29
  ## Compatibility
30
 
31
  The provided OpenVINO™ IR model is compatible with:
32
 
33
+ * OpenVINO version 2025.3.0 and higher
34
+ * Optimum Intel 1.25.2 and higher
35
 
36
 
37
  ## Running Model Inference with [Optimum Intel](https://huggingface.co/docs/optimum/intel/index)