File size: 13,284 Bytes
0b9851a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 |
{
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/home/salman/salman/minomni_sn21/omega-v2v/console/backend/venv/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
" from .autonotebook import tqdm as notebook_tqdm\n",
"/home/salman/salman/minomni_sn21/omega-v2v/console/backend/venv/lib/python3.10/site-packages/torch/nn/utils/weight_norm.py:143: FutureWarning: `torch.nn.utils.weight_norm` is deprecated in favor of `torch.nn.utils.parametrizations.weight_norm`.\n",
" WeightNorm.apply(module, name, dim)\n"
]
}
],
"source": [
"from server import lm"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"from server import tok"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"import torch"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m2025-07-17 20:59:03.022\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36moutetts.models.hf_model\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m20\u001b[0m - \u001b[1m🔄 Using patched RepetitionPenaltyLogitsProcessor -> RepetitionPenaltyLogitsProcessorPatch | penalty_last_n: 64\u001b[0m\n"
]
}
],
"source": [
"\n",
"rr = \"\"\"I'm trying to come up with a funny name for my new goldfish. He's orange with a white spot on his head and he's pretty energetic. Got any silly suggestions?\"\"\"\n",
"\n",
"inputs = tok(rr, return_tensors=\"pt\").to(lm.device)\n",
"\n",
"with torch.inference_mode():\n",
" out_ids = lm.generate(\n",
" **inputs,\n",
" max_new_tokens=500,\n",
" do_sample=True,\n",
" temperature=0.2,\n",
" repetition_penalty=1.11,\n",
" top_k=100,\n",
" top_p=0.95,\n",
" )\n",
"\n",
"resp = tok.decode(\n",
" out_ids[0][inputs.input_ids.shape[-1] :], skip_special_tokens=True\n",
" )"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"\" I've got a few, but they aren't very catchy. The one I like the best is just gonna be called fish. It's kinda long and it's kinda boring. Oh, I thought you were gonna give me some name for the goldfish. I'm just kidding. Yeah. So, you know, it's really easy to take care of a goldfish. We have a big tank, and, we're both in the same house. So it's not like, oh, where are my three goldfish? You know, it's just, oh, how many goldfish do you have? It's, like, four or five. But, we only have room for one person to be a goldfish keeper. So that is hard, especially when it's, like, 20 degrees outside and you're trying to keep a fish at home. Right? Yeah. That's difficult. And with the tank being this size, you don't really feel bad about taking him out. You know, you just kinda get a little more nervous because you know you're gonna be doing a big fish transfer if you have that big of a tank and all that stuff. But Mhmm. It's much easier to take care of the goldfish at home. So I wouldFor the rest of us simple folks, we worry about somebody stealing our password. To you, you laugh about it because you know how to do that with your eyes closed, right, with the technology you've created. So nowadays, you talk to certain investors, so where do hide your passwords? I don't want to really say, but I hide my passwords in my notes section on my phone. Oh shoot. Okay. Where do you hide your passwords? I write it on a piece of paper. Where do you hide your password? I have it on file on my computer. Where do you hide your password? I have it on an Excel spreadsheet, right? And all these places you go through. And so now there's a business model for apps that you put your passwords in and they protect your password. If it's so easy to break into softwares to get my password, How can I trust an app to restore all my password? Is there anywhere you trust to restore your passwords? So let's imagine that I want your password. I'm gonna make a website for Iranian American fans of Atlas Shrugged, and I'm gonna send you an email with a,\""
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"resp"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'All right. Good afternoon, everybody. Welcome to Friday afternoon. Appreciate you all coming. Really pleased today to be able to host the students to to COVID. Great. Correct me if I get it wrong. From the University of Wisconsin,'"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"resp"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"ename": "ValueError",
"evalue": "Cannot use chat template functions because tokenizer.chat_template is not set and no template argument was passed! For information about writing templates and setting the tokenizer.chat_template attribute, please see the documentation at https://huggingface.co/docs/transformers/main/en/chat_templating",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mValueError\u001b[0m Traceback (most recent call last)",
"Cell \u001b[0;32mIn[6], line 5\u001b[0m\n\u001b[1;32m 1\u001b[0m messages \u001b[38;5;241m=\u001b[39m [\n\u001b[1;32m 2\u001b[0m {\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mrole\u001b[39m\u001b[38;5;124m\"\u001b[39m: \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124msystem\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mcontent\u001b[39m\u001b[38;5;124m\"\u001b[39m: \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mYou are a concise assistant that answers in short paragraphs.\u001b[39m\u001b[38;5;124m\"\u001b[39m},\n\u001b[1;32m 3\u001b[0m {\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mrole\u001b[39m\u001b[38;5;124m\"\u001b[39m: \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124muser\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mcontent\u001b[39m\u001b[38;5;124m\"\u001b[39m: \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mExplain rotary positional embeddings briefly.\u001b[39m\u001b[38;5;124m\"\u001b[39m},\n\u001b[1;32m 4\u001b[0m ]\n\u001b[0;32m----> 5\u001b[0m prompt_ids \u001b[38;5;241m=\u001b[39m \u001b[43mtok\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mapply_chat_template\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m 6\u001b[0m \u001b[43m \u001b[49m\u001b[43mmessages\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m 7\u001b[0m \u001b[43m \u001b[49m\u001b[43madd_generation_prompt\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43;01mTrue\u001b[39;49;00m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;66;43;03m# appends the assistant header the model should complete\u001b[39;49;00m\n\u001b[1;32m 8\u001b[0m \u001b[43m \u001b[49m\u001b[43mreturn_tensors\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mpt\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\n\u001b[1;32m 9\u001b[0m \u001b[43m)\u001b[49m\u001b[38;5;241m.\u001b[39mto(lm\u001b[38;5;241m.\u001b[39mdevice)\n",
"File \u001b[0;32m~/salman/minomni_sn21/omega-v2v/console/backend/venv/lib/python3.10/site-packages/transformers/tokenization_utils_base.py:1621\u001b[0m, in \u001b[0;36mPreTrainedTokenizerBase.apply_chat_template\u001b[0;34m(self, conversation, tools, documents, chat_template, add_generation_prompt, continue_final_message, tokenize, padding, truncation, max_length, return_tensors, return_dict, return_assistant_tokens_mask, tokenizer_kwargs, **kwargs)\u001b[0m\n\u001b[1;32m 1618\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m tokenizer_kwargs \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[1;32m 1619\u001b[0m tokenizer_kwargs \u001b[38;5;241m=\u001b[39m {}\n\u001b[0;32m-> 1621\u001b[0m chat_template \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mget_chat_template\u001b[49m\u001b[43m(\u001b[49m\u001b[43mchat_template\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mtools\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m 1623\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m return_assistant_tokens_mask \u001b[38;5;129;01mand\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m re\u001b[38;5;241m.\u001b[39msearch(\u001b[38;5;124mr\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m\\\u001b[39m\u001b[38;5;124m{\u001b[39m\u001b[38;5;124m\\\u001b[39m\u001b[38;5;124m%\u001b[39m\u001b[38;5;124m-?\u001b[39m\u001b[38;5;124m\\\u001b[39m\u001b[38;5;124ms*generation\u001b[39m\u001b[38;5;124m\\\u001b[39m\u001b[38;5;124ms*-?\u001b[39m\u001b[38;5;124m\\\u001b[39m\u001b[38;5;124m%\u001b[39m\u001b[38;5;124m\\\u001b[39m\u001b[38;5;124m}\u001b[39m\u001b[38;5;124m\"\u001b[39m, chat_template):\n\u001b[1;32m 1624\u001b[0m logger\u001b[38;5;241m.\u001b[39mwarning_once(\n\u001b[1;32m 1625\u001b[0m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mreturn_assistant_tokens_mask==True but chat template does not contain `\u001b[39m\u001b[38;5;124m{\u001b[39m\u001b[38;5;132;01m% g\u001b[39;00m\u001b[38;5;124meneration \u001b[39m\u001b[38;5;124m%\u001b[39m\u001b[38;5;124m}` keyword.\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 1626\u001b[0m )\n",
"File \u001b[0;32m~/salman/minomni_sn21/omega-v2v/console/backend/venv/lib/python3.10/site-packages/transformers/tokenization_utils_base.py:1789\u001b[0m, in \u001b[0;36mPreTrainedTokenizerBase.get_chat_template\u001b[0;34m(self, chat_template, tools)\u001b[0m\n\u001b[1;32m 1787\u001b[0m chat_template \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mchat_template\n\u001b[1;32m 1788\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[0;32m-> 1789\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(\n\u001b[1;32m 1790\u001b[0m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mCannot use chat template functions because tokenizer.chat_template is not set and no template \u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 1791\u001b[0m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124margument was passed! For information about writing templates and setting the \u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 1792\u001b[0m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mtokenizer.chat_template attribute, please see the documentation at \u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 1793\u001b[0m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mhttps://huggingface.co/docs/transformers/main/en/chat_templating\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 1794\u001b[0m )\n\u001b[1;32m 1796\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m chat_template\n",
"\u001b[0;31mValueError\u001b[0m: Cannot use chat template functions because tokenizer.chat_template is not set and no template argument was passed! For information about writing templates and setting the tokenizer.chat_template attribute, please see the documentation at https://huggingface.co/docs/transformers/main/en/chat_templating"
]
}
],
"source": [
"messages = [\n",
" {\"role\": \"system\", \"content\": \"You are a concise assistant that answers in short paragraphs.\"},\n",
" {\"role\": \"user\", \"content\": \"Explain rotary positional embeddings briefly.\"},\n",
"]\n",
"prompt_ids = tok.apply_chat_template(\n",
" messages,\n",
" add_generation_prompt=True, # appends the assistant header the model should complete\n",
" return_tensors=\"pt\"\n",
").to(lm.device)\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.17"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
|