Upload bigcode/starcoder ctranslate fp16 weights

README.md

@@ -264,9 +264,9 @@ Speedup inference while reducing memory by 2x-4x using int8 inference in C++ on
 
 quantized version of [bigcode/starcoder](https://huggingface.co/bigcode/starcoder)
 ```bash
-pip install hf-hub-ctranslate2>=2.0.8 
+pip install hf-hub-ctranslate2>=2.0.8 ctranslate2>=3.14.0
 ```
-Converted on 2023-05-30 using
+Converted on 2023-05-31 using
 ```
 ct2-transformers-converter --model bigcode/starcoder --output_dir /home/michael/tmp-ct2fast-starcoder --force --copy_files merges.txt tokenizer.json README.md tokenizer_config.json vocab.json generation_config.json special_tokens_map.json .gitattributes --quantization float16 --trust_remote_code
 ```
@@ -290,7 +290,8 @@ model = GeneratorCT2fromHfHub(
 )
 outputs = model.generate(
     text=["How do you call a fast Flan-ingo?", "User: How are you doing? Bot:"],
-    max_length=64
+    max_length=64, 
+    include_prompt_in_result=False
 )
 print(outputs)
 ```
@@ -356,7 +357,7 @@ print(tokenizer.decode(outputs[0]))
 Fill-in-the-middle uses special tokens to identify the prefix/middle/suffix part of the input and output:
 
 ```python
-input_text = "<fim-prefix>def print_hello_world():\n    <fim-suffix>\n    print('Hello world!')<fim-middle>"
+input_text = "<fim_prefix>def print_hello_world():\n    <fim_suffix>\n    print('Hello world!')<fim_middle>"
 inputs = tokenizer.encode(input_text, return_tensors="pt").to(device)
 outputs = model.generate(inputs)
 print(tokenizer.decode(outputs[0]))