huggingface의 AutoModel 클래스

from transformers import AutoModel

checkpoint = "distilbert-base-uncased-finetuned-sst-2-english"
model = AutoModel.from_pretrained(checkpoint)

 

This architecture contains only the base Transformer module: given some inputs, it outputs what we’ll call hidden states, also known as features. For each model input, we’ll retrieve a high-dimensional vector representing the contextual understanding of that input by the Transformer model.

 

While these hidden states can be useful on their own, they’re usually inputs to another part of the model, known as the head. 

 

 

what is head?

An additional component, usually made up of one or a few layers, to convert the transformer predictions to a task-specific output.

Adaptation heads, also known simply as heads, come up in different forms: language modeling heads, question answering heads, sequence classification heads...