Add human-friendly README

README.md

@@ -1,188 +1,122 @@
 ---
-library_name: transformers
+language: en
+license: mit
 tags:
-- generated_from_trainer
-model-index:
-- name: nano-gpt-from-scratch
-  results: []
+- text-generation
+- gpt2
+- transformers
+- custom-tokenizer
+datasets:
+- wikitext
 ---
 
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
-
-# nano-gpt-from-scratch
-
-This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
-It achieves the following results on the evaluation set:
-- Loss: 3.6938
-
-## Model description
-
-More information needed
-
-## Intended uses & limitations
-
-More information needed
-
-## Training and evaluation data
-
-More information needed
-
-## Training procedure
-
-### Training hyperparameters
-
-The following hyperparameters were used during training:
-- learning_rate: 0.0005
-- train_batch_size: 8
-- eval_batch_size: 8
-- seed: 42
-- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
-- lr_scheduler_type: linear
-- num_epochs: 10
-
-### Training results
-
-| Training Loss | Epoch  | Step  | Validation Loss |
-|:-------------:|:------:|:-----:|:---------------:|
-| 5.9543        | 0.0748 | 200   | 5.9705          |
-| 5.599         | 0.1496 | 400   | 5.5920          |
-| 5.3875        | 0.2244 | 600   | 5.4046          |
-| 5.3405        | 0.2992 | 800   | 5.2639          |
-| 5.1997        | 0.3740 | 1000  | 5.1478          |
-| 5.088         | 0.4488 | 1200  | 5.0422          |
-| 5.0339        | 0.5236 | 1400  | 4.9466          |
-| 4.8631        | 0.5984 | 1600  | 4.8712          |
-| 4.7945        | 0.6731 | 1800  | 4.8020          |
-| 4.7787        | 0.7479 | 2000  | 4.7387          |
-| 4.676         | 0.8227 | 2200  | 4.6780          |
-| 4.616         | 0.8975 | 2400  | 4.6380          |
-| 4.6444        | 0.9723 | 2600  | 4.5994          |
-| 4.6359        | 1.0471 | 2800  | 4.5643          |
-| 4.5646        | 1.1219 | 3000  | 4.5283          |
-| 4.5776        | 1.1967 | 3200  | 4.4985          |
-| 4.5616        | 1.2715 | 3400  | 4.4715          |
-| 4.5367        | 1.3463 | 3600  | 4.4492          |
-| 4.4906        | 1.4211 | 3800  | 4.4217          |
-| 4.4006        | 1.4959 | 4000  | 4.4068          |
-| 4.5386        | 1.5707 | 4200  | 4.3741          |
-| 4.3777        | 1.6455 | 4400  | 4.3616          |
-| 4.4397        | 1.7203 | 4600  | 4.3360          |
-| 4.3095        | 1.7951 | 4800  | 4.3158          |
-| 4.3144        | 1.8699 | 5000  | 4.2964          |
-| 4.3759        | 1.9447 | 5200  | 4.2789          |
-| 4.3907        | 2.0194 | 5400  | 4.2606          |
-| 4.3318        | 2.0942 | 5600  | 4.2496          |
-| 4.2696        | 2.1690 | 5800  | 4.2383          |
-| 4.2866        | 2.2438 | 6000  | 4.2168          |
-| 4.2801        | 2.3186 | 6200  | 4.2040          |
-| 4.3416        | 2.3934 | 6400  | 4.1900          |
-| 4.357         | 2.4682 | 6600  | 4.1744          |
-| 4.1895        | 2.5430 | 6800  | 4.1598          |
-| 4.1719        | 2.6178 | 7000  | 4.1497          |
-| 4.2181        | 2.6926 | 7200  | 4.1386          |
-| 4.2341        | 2.7674 | 7400  | 4.1248          |
-| 4.1929        | 2.8422 | 7600  | 4.1126          |
-| 4.2205        | 2.9170 | 7800  | 4.0989          |
-| 4.1556        | 2.9918 | 8000  | 4.0861          |
-| 4.1201        | 3.0666 | 8200  | 4.0812          |
-| 4.1829        | 3.1414 | 8400  | 4.0691          |
-| 4.1198        | 3.2162 | 8600  | 4.0647          |
-| 4.1811        | 3.2909 | 8800  | 4.0487          |
-| 4.0946        | 3.3657 | 9000  | 4.0418          |
-| 4.098         | 3.4405 | 9200  | 4.0332          |
-| 4.1762        | 3.5153 | 9400  | 4.0212          |
-| 4.1506        | 3.5901 | 9600  | 4.0164          |
-| 4.0208        | 3.6649 | 9800  | 4.0045          |
-| 4.1121        | 3.7397 | 10000 | 4.0025          |
-| 4.0627        | 3.8145 | 10200 | 3.9930          |
-| 3.8746        | 3.8893 | 10400 | 3.9823          |
-| 4.0966        | 3.9641 | 10600 | 3.9739          |
-| 3.9827        | 4.0389 | 10800 | 3.9707          |
-| 3.9926        | 4.1137 | 11000 | 3.9619          |
-| 4.0307        | 4.1885 | 11200 | 3.9533          |
-| 4.0668        | 4.2633 | 11400 | 3.9469          |
-| 4.0161        | 4.3381 | 11600 | 3.9390          |
-| 4.0391        | 4.4129 | 11800 | 3.9330          |
-| 3.9875        | 4.4877 | 12000 | 3.9257          |
-| 4.0578        | 4.5625 | 12200 | 3.9204          |
-| 3.9203        | 4.6372 | 12400 | 3.9126          |
-| 4.0048        | 4.7120 | 12600 | 3.9071          |
-| 4.0535        | 4.7868 | 12800 | 3.9003          |
-| 3.9751        | 4.8616 | 13000 | 3.8971          |
-| 3.9949        | 4.9364 | 13200 | 3.8921          |
-| 3.8623        | 5.0112 | 13400 | 3.8873          |
-| 3.9523        | 5.0860 | 13600 | 3.8841          |
-| 3.9892        | 5.1608 | 13800 | 3.8773          |
-| 3.8098        | 5.2356 | 14000 | 3.8734          |
-| 3.93          | 5.3104 | 14200 | 3.8704          |
-| 3.9913        | 5.3852 | 14400 | 3.8621          |
-| 3.9875        | 5.4600 | 14600 | 3.8599          |
-| 4.0528        | 5.5348 | 14800 | 3.8533          |
-| 3.9126        | 5.6096 | 15000 | 3.8485          |
-| 4.0453        | 5.6844 | 15200 | 3.8422          |
-| 3.8323        | 5.7592 | 15400 | 3.8367          |
-| 3.9323        | 5.8340 | 15600 | 3.8336          |
-| 3.8529        | 5.9088 | 15800 | 3.8309          |
-| 3.8784        | 5.9835 | 16000 | 3.8198          |
-| 3.9129        | 6.0583 | 16200 | 3.8198          |
-| 3.8845        | 6.1331 | 16400 | 3.8169          |
-| 3.7315        | 6.2079 | 16600 | 3.8141          |
-| 3.8735        | 6.2827 | 16800 | 3.8041          |
-| 3.8643        | 6.3575 | 17000 | 3.8056          |
-| 3.871         | 6.4323 | 17200 | 3.7992          |
-| 3.9184        | 6.5071 | 17400 | 3.8034          |
-| 3.8936        | 6.5819 | 17600 | 3.7936          |
-| 3.8913        | 6.6567 | 17800 | 3.7887          |
-| 3.8066        | 6.7315 | 18000 | 3.7871          |
-| 3.843         | 6.8063 | 18200 | 3.7838          |
-| 3.9506        | 6.8811 | 18400 | 3.7783          |
-| 3.7878        | 6.9559 | 18600 | 3.7754          |
-| 3.877         | 7.0307 | 18800 | 3.7725          |
-| 3.8744        | 7.1055 | 19000 | 3.7680          |
-| 3.8774        | 7.1803 | 19200 | 3.7669          |
-| 3.8095        | 7.2550 | 19400 | 3.7645          |
-| 3.8195        | 7.3298 | 19600 | 3.7638          |
-| 3.9152        | 7.4046 | 19800 | 3.7587          |
-| 3.8205        | 7.4794 | 20000 | 3.7546          |
-| 3.7736        | 7.5542 | 20200 | 3.7520          |
-| 3.8518        | 7.6290 | 20400 | 3.7499          |
-| 3.8738        | 7.7038 | 20600 | 3.7444          |
-| 3.8431        | 7.7786 | 20800 | 3.7438          |
-| 3.7439        | 7.8534 | 21000 | 3.7388          |
-| 3.9137        | 7.9282 | 21200 | 3.7376          |
-| 3.7964        | 8.0030 | 21400 | 3.7338          |
-| 3.7578        | 8.0778 | 21600 | 3.7363          |
-| 3.786         | 8.1526 | 21800 | 3.7312          |
-| 3.7942        | 8.2274 | 22000 | 3.7285          |
-| 3.8115        | 8.3022 | 22200 | 3.7272          |
-| 3.8123        | 8.3770 | 22400 | 3.7248          |
-| 3.8411        | 8.4518 | 22600 | 3.7245          |
-| 3.8563        | 8.5266 | 22800 | 3.7208          |
-| 3.8441        | 8.6013 | 23000 | 3.7179          |
-| 3.8005        | 8.6761 | 23200 | 3.7152          |
-| 3.6688        | 8.7509 | 23400 | 3.7147          |
-| 3.8191        | 8.8257 | 23600 | 3.7098          |
-| 3.8469        | 8.9005 | 23800 | 3.7087          |
-| 3.8563        | 8.9753 | 24000 | 3.7079          |
-| 3.7861        | 9.0501 | 24200 | 3.7058          |
-| 3.7502        | 9.1249 | 24400 | 3.7046          |
-| 3.6903        | 9.1997 | 24600 | 3.7032          |
-| 3.7698        | 9.2745 | 24800 | 3.7019          |
-| 3.6781        | 9.3493 | 25000 | 3.7021          |
-| 3.7236        | 9.4241 | 25200 | 3.6995          |
-| 3.7578        | 9.4989 | 25400 | 3.6996          |
-| 3.6315        | 9.5737 | 25600 | 3.6985          |
-| 3.6605        | 9.6485 | 25800 | 3.6965          |
-| 3.7707        | 9.7233 | 26000 | 3.6958          |
-| 3.7968        | 9.7981 | 26200 | 3.6953          |
-| 3.8238        | 9.8728 | 26400 | 3.6939          |
-| 3.7834        | 9.9476 | 26600 | 3.6938          |
-
-
-### Framework versions
-
-- Transformers 4.57.0
-- Pytorch 2.8.0
-- Datasets 4.0.0
-- Tokenizers 0.22.1
+# 🤖 Nano GPT - Built From Scratch
+
+Hey there! Welcome to my tiny language model. I built this GPT from scratch as a learning project, and honestly, it was pretty fun watching it learn to generate text!
+
+## What is this?
+
+This is a super small GPT-2 style language model that I trained on my laptop. It's not going to write your essays or solve world hunger, but it's a cool demonstration of how these language models actually work under the hood.
+
+Think of it as a baby GPT - it can generate text, but don't expect Shakespeare. More like... an enthusiastic toddler who just learned to talk.
+
+## Model Stats
+
+- **Parameters**: ~1,065,728 (yes, that's million with an M, not billion!)
+- **Layers**: 4 transformer layers
+- **Embedding Size**: 128 dimensions
+- **Attention Heads**: 4 heads
+- **Context Length**: 128 tokens
+- **Vocab Size**: 2000 tokens
+- **Training Data**: WikiText-2 (5,000 samples)
+- **Training Time**: 10 epochs on my laptop
+
+## Quick Start
+
+Want to try it out? Here's how:
+
+```python
+from transformers import pipeline
+
+# Load the model
+generator = pipeline('text-generation', model='Tanaybh/nano-gpt-from-scratch')
+
+# Generate some text
+output = generator(
+    "The meaning of life is",
+    max_new_tokens=30,
+    do_sample=True,
+    temperature=0.8
+)
+
+print(output[0]['generated_text'])
+```
+
+## Example Output
+
+I gave it the prompt: "**The **"
+
+And it generated:
+
+> The × 60 munitions, and injuries were found in the taxonomy in the south, the east of the
+
+Not bad for a tiny model trained in a few hours, right?
+
+## Training Details
+
+I trained this model from scratch using:
+- Custom BPE tokenizer (trained on the same data)
+- GPT-2 architecture (just way smaller)
+- AdamW optimizer with a learning rate of 0.0005
+- Batch size of 8
+- Trained for 10 epochs
+
+The whole thing runs on a regular laptop - no fancy GPU clusters needed!
+
+## Limitations
+
+Let's be real here:
+- This model is TINY. Like, really tiny. It has 1,065,728 parameters vs GPT-3's 175 billion.
+- It was only trained on 5,000 Wikipedia samples, so its knowledge is... limited.
+- It might generate weird or nonsensical text sometimes. That's part of the charm!
+- Maximum context length is only 128 tokens, so don't expect long conversations.
+- It's a base model with no instruction tuning, so it just continues text rather than following commands.
+
+## Why I Made This
+
+I wanted to understand how language models work by building one myself. Sure, I could've just fine-tuned a pre-trained model, but where's the fun in that? This project taught me about:
+- Tokenizer training
+- Transformer architecture
+- Training dynamics
+- How LLMs actually generate text
+
+Plus, now I can say I trained a language model from scratch on my laptop. Pretty cool, right?
+
+## Future Improvements
+
+Some things I might try:
+- Train on more data (maybe the full WikiText dataset)
+- Experiment with different model sizes
+- Try different tokenizer configurations
+- Add instruction tuning
+- Fine-tune it for specific tasks
+
+## License
+
+MIT - Feel free to use this however you want! Learn from it, break it, improve it. That's what it's here for.
+
+## Acknowledgments
+
+Built with:
+- 🤗 Hugging Face Transformers
+- PyTorch
+- The WikiText dataset
+- Too much coffee ☕
+
+---
+
+**Note**: This is a learning project and experimental model. Use it for fun and education, not production systems!
+
+If you found this interesting or helpful, feel free to star the repo or reach out. Always happy to chat about ML stuff!
+
+*Last updated: October 05, 2025*