Add seq length
Browse files
README.md
CHANGED
|
@@ -121,6 +121,7 @@ See sample configs in [configs](configs) folder or [examples](examples) for quic
|
|
| 121 |
datasets:
|
| 122 |
- path: vicgalle/alpaca-gpt4 # local or huggingface repo
|
| 123 |
type: alpaca # format from earlier
|
|
|
|
| 124 |
```
|
| 125 |
|
| 126 |
- loading
|
|
@@ -333,7 +334,12 @@ Add below flag to train command above
|
|
| 333 |
|
| 334 |
## Common Errors 🧰
|
| 335 |
|
| 336 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 337 |
|
| 338 |
## Contributing 🤝
|
| 339 |
|
|
|
|
| 121 |
datasets:
|
| 122 |
- path: vicgalle/alpaca-gpt4 # local or huggingface repo
|
| 123 |
type: alpaca # format from earlier
|
| 124 |
+
sequence_len: 2048 # max token length / prompt
|
| 125 |
```
|
| 126 |
|
| 127 |
- loading
|
|
|
|
| 334 |
|
| 335 |
## Common Errors 🧰
|
| 336 |
|
| 337 |
+
> Cuda out of memory
|
| 338 |
+
|
| 339 |
+
Please reduce any below
|
| 340 |
+
- `micro_batch_size`
|
| 341 |
+
- `eval_batch_size`
|
| 342 |
+
- `sequence_len`
|
| 343 |
|
| 344 |
## Contributing 🤝
|
| 345 |
|