如何在微调中使用 XLMRoberta，

Question

微调代码时遇到两个问题。我试图使用 X_1 和 X_2 来回归。语料库中有不同的语言。

HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/xlm-roberta-base/resolve/main/tf_model.h5

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
/tmp/ipykernel_33/2123064688.py in <module>
     55 # )
     56 
---> 57 model = TFXLMRobertaForSequenceClassification.from_pretrained('xlm-roberta-base',num_labels=1)

OSError: Can't load weights for 'xlm-roberta-base'. Make sure that:

- 'xlm-roberta-base' is a correct model identifier listed on 'https://huggingface.co/models'
  (make sure 'xlm-roberta-base' is not a path to a local directory with something else, in that case)

- or 'xlm-roberta-base' is the correct path to a directory containing a file named one of tf_model.h5, pytorch_model.bin.

这是我的代码：

tokenizer = XLMRobertaTokenizerFast.from_pretrained('xlm-roberta-base')

train_encoding = tokenizer(X_train_1,X_train_2,truncation=True,padding=True)
val_encoding = tokenizer(X_val_1,X_val_2,truncation=True,padding=True)

train_dataset = tf.data.Dataset.from_tensor_slices(
    (dict(train_encoding),y_train)
)
val_dataset = tf.data.Dataset.from_tensor_slices(
    (dict(val_encoding),y_val)
)

model = TFXLMRobertaForSequenceClassification.from_pretrained('xlm-roberta-base',num_labels=1)

Answer 1

在深入研究 huggingface 变形金刚之前，您最好了解几件事。

使用 huggingface 的转换器的首选库是 PyTorch。
对于几个广泛使用的模型，您可能会在旁边找到 Tensorflow 版本，但并非适用于所有模型。
幸运的是，有一些方法可以将 pt 个检查点转换为 tf，反之亦然。

最后如何修复代码：

# switching to pytorch
tokenizer = XLMRobertaTokenizerFast.from_pretrained('xlm-roberta-base')
model = XLMRobertaForSequenceClassification.from_pretrained('xlm-roberta-base')
# using un-official checkpoints
model = TFXLMRobertaForSequenceClassification.from_pretrained('jplu/tf-xlm-roberta-base',num_labels=1)
# converting pt checkpoint to tensorflow (not recommended!)

如何在微调中使用 XLMRoberta，

how to use XLMRoberta in fine-tuning ,

python

nlp