Best on a bigger screen. Everything here works on a phone, and you are welcome to read on. But these pages are built around interactive diagrams, and a screen this size struggles to show one at a readable size with its controls beside it. If you can, come back on a laptop — that is where the tools are meant to be used.
This tool enables users to visualize the attention mechanisms of a BERT model, showing which words are emphasized during Masked Language Model training for sentence understanding.
Input text
Masked input text
Attention Score Calculation
Input Sample :
Attention Heatmap
Head 0
Epoch #: 1/5000
Input Sample :
Output
✓ You worked through Attention Heatmap
That is 1 of 31 topics explored. Keep the thread going — BERT Explorer builds directly on this one.