openai-community

/

gpt2

like 3k

Follow

OpenAI community 1.67k

This is a small version of the weights.

The current safetensor model duplicates the embeddings instead of simply using tie_weights.

@julien-c
@sgugger

so to make sure i understand! This new conversion removes the duplicated embeddings?

Yes !

It works by just copying the PT weights. (Instead of using AutoModelForCausalLM.from_pretrained("gpt2"); model.save_pretrained(..) which I think it what you might have used.
save_pretrained with the head, saves the weights with an extra transformer. layer, and saves the head too.

makes sense, thanks! merging