Unrecognized configuration class error
Hi,
I get the following error when trying to load the model:
model = AutoModelForMaskedLM.from_pretrained(
"togethercomputer/m2-bert-80M-2k",
trust_remote_code=True
)
Unrecognized configuration class <class 'transformers_modules.togethercomputer.m2-bert-80M-2k.3e061b6d18dd7fab5df5047432edca801fe89e5d.configuration_bert.BertConfig'> for this kind of AutoModel: AutoModelForMaskedLM.
Model type should be one of AlbertConfig, BartConfig, BertConfig, BigBirdConfig, CamembertConfig, ConvBertConfig, Data2VecTextConfig, DebertaConfig, DebertaV2Config, DistilBertConfig, ElectraConfig, ErnieConfig, EsmConfig, FlaubertConfig, FNetConfig, FunnelConfig, IBertConfig, LayoutLMConfig, LongformerConfig, LukeConfig, MBartConfig, MegaConfig, MegatronBertConfig, MobileBertConfig, MPNetConfig, MvpConfig, NezhaConfig, NystromformerConfig, PerceiverConfig, QDQBertConfig, ReformerConfig, RemBertConfig, RobertaConfig, RobertaPreLayerNormConfig, RoCBertConfig, RoFormerConfig, SqueezeBertConfig, TapasConfig, Wav2Vec2Config, XLMConfig, XLMRobertaConfig, XLMRobertaXLConfig, XmodConfig, YosoConfig.
Do you have any idea why this could be?
My transformers version:
Name: transformers
Version: 4.28.1
Summary: State-of-the-art Machine Learning for JAX, PyTorch and TensorFlow
Home-page: https://github.com/huggingface/transformers
Author: The Hugging Face team (past and future) with the help of all our contributors (https://github.com/huggingface/transformers/graphs/contributors)
Author-email: transformers@huggingface.co
License: Apache 2.0 License
Location: /data/nasif12/modules_if12/SL7/i12g/anaconda/envs/m2-mixer/lib/python3.9/site-packages
Requires: filelock, huggingface-hub, numpy, packaging, pyyaml, regex, requests, tokenizers, tqdm
Required-by: mosaicml-streaming
Hi, thanks for the bug report. It's a similar bug to this: https://huggingface.co/togethercomputer/m2-bert-80M-8k/discussions/1
I think the fix is related to adding a few bits in the source files, similar to this commit on the retrieval models: https://huggingface.co/togethercomputer/m2-bert-80M-2k-retrieval/commit/7be65169100fb4bb2c7ab8f2d1c798870f8f1425
I'll try to get the fix in this week.