| # Optimizers | |
| The following optimizers are supported: | |
| - **torch**: | |
| - `Adam` | |
| - `AdamW` | |
| - **bitsandbytes**: | |
| - `Adam` | |
| - `AdamW` | |
| - `Adam8Bit` | |
| - `AdamW8Bit` | |
| > [!NOTE] | |
| > Not all optimizers have been tested with all models/parallel settings. They may or may not work, but this will gradually improve over time. | |