Spaces:

Mikunono
/

Rally_ChatBot

Paused

Mikunono commited on Mar 19, 2024

Commit

2079330

verified ·

1 Parent(s): 27a0fd9

Update app.py

Files changed (1) hide show

app.py CHANGED Viewed

@@ -12,7 +12,7 @@ model = AutoModelForCausalLM.from_pretrained(model_name_or_path,
                                              device_map="auto",
                                              trust_remote_code=True,
                                              revision="main",
-                                             quantization_config=QuantizationConfig(disable_exllama=True)
                                              )
 tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, use_fast=True)

                                              device_map="auto",
                                              trust_remote_code=True,
                                              revision="main",
+                                             #quantization_config=QuantizationConfig(disable_exllama=True)
                                              )
 tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, use_fast=True)