Hugging Face Agent Course Practice Test

Autoregressive generation in language models is about predicting the next token given the tokens seen so far. The model learns a conditional distribution for each position: P(next token | all previously generated tokens). Generation proceeds step by step, choosing a token and feeding it back as context for predicting the following one, often by sampling or selecting the most probable option. This sequential factorization is what makes the process autoregressive: each step depends on the history of past tokens, not on future ones. This differs from the other ideas: global optimization over all tokens would imply optimizing the entire sequence at once rather than predicting step by step; generating images isn’t specific to language modeling; and classifying inputs is about assigning a label rather than producing a coherent token sequence.