Quiz: Position Embedding

1. Why are position embeddings necessary in Transformer models?

To give the model information about word order

To replace token embeddings

To improve output formatting

To reduce model parameters

2. Which of the following functions are used in standard position embeddings?

Exponential and logarithmic

Tangent and cotangent

Square and root

Sine and cosine

3. What is the main property of sine and cosine that makes them useful for position encoding?

They match token vocabulary length

They are bounded between -100 and 100

They are periodic and allow unique encoding of positions

They are easy to store in memory

4. How are position embeddings combined with word embeddings in Transformer models?

By concatenating them

By adding them element-wise

By sorting them

By multiplying them

5. What aspect of language understanding do position embeddings help capture?

Vocabulary frequency

Sentence complexity

Word order and structural relationships

Character-level spelling

6. What is the typical dimension of a position embedding in practice?

Same as word embedding dimension

Dependent on the number of tokens in vocabulary

Always 3

Double the number of attention heads

7. In which layer are position embeddings applied in Transformer architectures?

Final classification layer

At the input stage with token embeddings

At the output projection layer

Before the feed-forward network

8. What is a key benefit of relative position embeddings compared to absolute ones?

They are easier to visualize in 3D

They don’t require sine/cosine functions

They reduce model size

They allow models to generalize better across sequences of different lengths

9. Which NLP task especially benefits from capturing the relative order of tokens?

Character counting

Named entity recognition

Stop word removal

Machine translation

10. What kind of plot can help visualize how position embeddings shift with word order?

Pie chart

3D scatter plot

Confusion matrix

Bar chart

Quiz 28: Position Embedding