Quiz: BERT Explorer

1. What does BERT stand for?

Basic Encoding and Recursive Translation

Binary Entity Recognition Transformer

Bidirectional Encoder Representations from Transformers

Back-End Recurrent Transformer

2. What makes BERT different from traditional left-to-right language models?

It reads text bidirectionally to capture full context

It processes words in random order

It uses convolutional layers instead of attention

It only uses tokens from the first sentence

3. What are the two pre-training tasks used by BERT?

Sentiment prediction and summarization

Masked Language Modeling and Next Sentence Prediction

Grammar correction and translation

Word embedding and decoding

4. Which token is used to classify the sentence pair in BERT?

[MASK]

[CLS]

[PAD]

[SEP]

5. What is the purpose of the [MASK] token during training?

To identify padding locations

To mark a word that the model should predict

To initialize a new sentence

To separate two segments

6. In BERT’s architecture, what role does the Multi-Head Attention mechanism play?

It helps the model focus on different parts of the input simultaneously

It outputs class probabilities

It encodes token frequency

It replaces position embeddings

7. What is the output of the 'IsNext' classification task in BERT?

A count of sentence pairs

A flag indicating if one sentence follows another

A list of part-of-speech tags

An integer index of masked tokens

8. What happens to embedding vectors after many training epochs?

They are discarded after each epoch

They become identical across all tokens

They stabilize and encode richer semantic meaning

They decrease in dimensionality

9. Why are residual connections and layer normalization used in BERT?

To stabilize learning and avoid vanishing gradients

To reduce vocabulary size

To speed up translation

To align token indices

10. How does BERT generate final predictions for masked tokens?

By averaging all tokens

Using a lookup table from the first encoder

By passing encoded vectors through a decoder and applying softmax

By matching frequency distributions

Quiz 31: BERT Explorer