Quiz 29: Attention Score

Evaluate your understanding of attention mechanisms, scaled dot-product attention, and how models determine relevance in sequences.

📚 Related Learning Tools:

🔧 Interactive Visualization

Check out these interactive tools to learn more before taking the quiz!

Next Quiz All Quizzes

1. What does the attention mechanism allow a Transformer model to do?

Reduce sequence length

Forget earlier inputs

Generate random outputs

Focus on specific parts of the input sequence

2. What are the three main components used in scaled dot-product attention?

LSTM, GRU, CNN

Tokens, Layers, Embeddings

Convolutions, Pooling, Padding

Keys, Queries, Values

3. Why is the dot product between Query and Key scaled?

To normalize dimensions for positional encoding

To prevent extremely large values that impact softmax

To reduce training time

To ensure all values are integers

4. What is the role of the softmax function in the attention mechanism?

It drops random elements from vectors

It stores attention vectors for reuse

It multiplies all attention scores

It transforms scores into probabilities

5. What happens after applying softmax to the attention scores?

The sequence is reversed

The output is one-hot encoded

The weights are multiplied by the value vectors

The inputs are encoded again

🔒