add bf16 inference

README.md

@@ -177,6 +177,27 @@ So, the sum of 100, 520, and 60 is 680.
 """
 ```
 
+### BF16 Inference with Intel Extension for Transformers and Intel Extension for Pytorch
+```python
+from transformers import AutoTokenizer, TextStreamer
+import torch
+from intel_extension_for_transformers.transformers import AutoModelForCausalLM
+import intel_extension_for_pytorch as ipex
+
+model_name = "Intel/neural-chat-7b-v3-1"
+prompt = "Once upon a time, there existed a little girl,"
+
+tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
+inputs = tokenizer(prompt, return_tensors="pt").input_ids
+streamer = TextStreamer(tokenizer)
+
+model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16)
+model = ipex.optimize(model.eval(), dtype=torch.bfloat16, inplace=True, level="O1", auto_kernel_selection=True)
+
+outputs = model.generate(inputs, streamer=streamer, max_new_tokens=300)
+```
+
+
 ### INT4 Inference with Transformers and Intel Extension for Transformers
 ```python
 from transformers import AutoTokenizer, TextStreamer