Transformer Explainer

thecatsaton

Generating...

Temperature

1.0•Balanced

Step 1 of 5: Input Embeddings

Input Embeddings

Convert tokens to embeddings

• Tokenize input

• Look up embeddings

• Create vector representation

Generate positional encodings

Combine embeddings and position

Attention

Pending

Add & Norm

Pending

Feed Forward

Pending

Output

Pending

Input Embeddings + PositionProcessing...

Masked Multi-Head Attention

Add & Norm

Feed Forward Network

Linear + Softmax