Python: BERT Error - Some weights of the model checkpoint at were not used when initializing BertMod

最新推荐文章于 2025-06-21 10:49:07 发布

Obolicaca

最新推荐文章于 2025-06-21 10:49:07 发布

阅读量1.2w

点赞数 9

CC 4.0 BY-SA版权

分类专栏： pytorch 文章标签： python bert 自然语言处理

本文链接：https://blog.csdn.net/Obolicaca/article/details/123336942

pytorch 专栏收录该内容

10 篇文章

订阅专栏

本文探讨了在使用transformers库时，遇到BERT模型初始化警告的情况，解释了原因，如任务转移或架构差异，并提供了设置不显示警告的方法。解决方法提示无需担心，只需针对下游任务进行训练即可。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

在调用transformers预训练模型库时出现以下信息：

Some weights of the model checkpoint at bert-base-multilingual-cased were not used when initializing BertForTokenClassification_: ['cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.bias', 'cls.seq_relationship.bias', 'cls.predictions.decoder.weight', 'cls.predictions.transform.LayerNorm.bias', 'cls.seq_relationship.weight']
- This IS expected if you are initializing BertForTokenClassification_ from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BertForTokenClassification_ from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of BertForTokenClassification_ were not initialized from the model checkpoint at bert-base-multilingual-cased and are newly initialized: ['classifier.weight', 'classifier.bias']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

解决方法：

这不是一个错误，但这个警告意味着在你的训练过程中，你没有使用pooler来计算损失。因此，如果是这种情况，则无需担心。

可以通过以下方式设置不显示此警告：

from transformers import logging

logging.set_verbosity_warning()

或者在在训练后加载模型，则不会看到此错误消息