Commit
·
012990e
1
Parent(s):
90dc0a3
Update README.md
Browse files
README.md
CHANGED
|
@@ -2,9 +2,6 @@
|
|
| 2 |
language: multilingual
|
| 3 |
datasets:
|
| 4 |
- mc4
|
| 5 |
-
tags:
|
| 6 |
-
- summarization
|
| 7 |
-
- translation
|
| 8 |
|
| 9 |
license: apache-2.0
|
| 10 |
---
|
|
@@ -40,10 +37,10 @@ loss = model(input_ids, labels=labels).loss # forward pass
|
|
| 40 |
For batched inference & training it is however recommended using a tokenizer class for padding:
|
| 41 |
|
| 42 |
```python
|
| 43 |
-
from transformers import T5ForConditionalGeneration,
|
| 44 |
|
| 45 |
model = T5ForConditionalGeneration.from_pretrained('google/byt5-base')
|
| 46 |
-
tokenizer =
|
| 47 |
|
| 48 |
model_inputs = tokenizer(["Life is like a box of chocolates.", "Today is Monday."], padding="longest", return_tensors="pt")
|
| 49 |
labels = tokenizer(["La vie est comme une boîte de chocolat.", "Aujourd'hui c'est lundi."], padding="longest", return_tensors="pt").input_ids
|
|
|
|
| 2 |
language: multilingual
|
| 3 |
datasets:
|
| 4 |
- mc4
|
|
|
|
|
|
|
|
|
|
| 5 |
|
| 6 |
license: apache-2.0
|
| 7 |
---
|
|
|
|
| 37 |
For batched inference & training it is however recommended using a tokenizer class for padding:
|
| 38 |
|
| 39 |
```python
|
| 40 |
+
from transformers import T5ForConditionalGeneration, AutoTokenizer
|
| 41 |
|
| 42 |
model = T5ForConditionalGeneration.from_pretrained('google/byt5-base')
|
| 43 |
+
tokenizer = AutoTokenizer.from_pretrained('google/byt5-base')
|
| 44 |
|
| 45 |
model_inputs = tokenizer(["Life is like a box of chocolates.", "Today is Monday."], padding="longest", return_tensors="pt")
|
| 46 |
labels = tokenizer(["La vie est comme une boîte de chocolat.", "Aujourd'hui c'est lundi."], padding="longest", return_tensors="pt").input_ids
|