Changing llama_v2_70b to correct config class

[JasonMts]

The model llama_v2_70b fails to install with ValueError: Unrecognized configuration class because it has been listed as the wrong config class. It should be listed as a Causal Language Model instead.

[atalman]

Hi @JasonMts could you please include link to a failure and to fixed workflow ?

[JasonMts]

Hi @atalman

With torch 2.6.0:

Error message log before fix:

python benchmark/install.py llama_v2_70b --canary -v

checking packages numpy, torch, torchvision, torchaudio are installed, generating constaints...OK
running setup for /home/user/benchmark/torchbenchmark/canary_models/llama_v2_70b...Traceback (most recent call last):
  File "/home/user/benchmark/torchbenchmark/canary_models/llama_v2_70b/install.py", line 7, in <module>
    cache_model(model_name)
  File "/home/user/benchmark/torchbenchmark/util/framework/huggingface/patch_hf.py", line 15, in cache_model
    download_model(name)
  File "/home/user/benchmark/torchbenchmark/util/framework/huggingface/basic_configs.py", line 310, in download_model
    model = model_cls.from_config(config, **kwargs)
  File "/home/user/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 440, in from_config
    raise ValueError(
ValueError: Unrecognized configuration class <class 'transformers.models.llama.configuration_llama.LlamaConfig'> for this kind of AutoModel: AutoModelForMaskedLM.
Model type should be one of AlbertConfig, BartConfig, BertConfig, BigBirdConfig, CamembertConfig, ConvBertConfig, Data2VecTextConfig, DebertaConfig, DebertaV2Config, DistilBertConfig, ElectraConfig, ErnieConfig, EsmConfig, FlaubertConfig, FNetConfig, FunnelConfig, IBertConfig, LayoutLMConfig, LongformerConfig, LukeConfig, MBartConfig, MegaConfig, MegatronBertConfig, MobileBertConfig, MPNetConfig, MraConfig, MvpConfig, NezhaConfig, NystromformerConfig, PerceiverConfig, QDQBertConfig, ReformerConfig, RemBertConfig, RobertaConfig, RobertaPreLayerNormConfig, RoCBertConfig, RoFormerConfig, SqueezeBertConfig, TapasConfig, Wav2Vec2Config, XLMConfig, XLMRobertaConfig, XLMRobertaXLConfig, XmodConfig, YosoConfig.
FAIL
Error for /home/user/benchmark/torchbenchmark/canary_models/llama_v2_70b:

None

Traceback (most recent call last):
  File "/home/user/benchmark/install.py", line 127, in <module>
    raise RuntimeError("Failed to complete setup")
RuntimeError: Failed to complete setup

After fix, model installs but does not fit into a runner with 120 GB memory so can’t show success.

python benchmark/install.py llama_v2_70b --canary -v

checking packages numpy, torch, torchvision, torchaudio are installed, generating constaints...OK
running setup for /home/user/benchmark/torchbenchmark/canary_models/llama_v2_70b...FAIL
Error for /home/user/benchmark/torchbenchmark/canary_models/llama_v2_70b:

None

Traceback (most recent call last):
  File "/home/user/benchmark/install.py", line 127, in <module>
    raise RuntimeError("Failed to complete setup")
RuntimeError: Failed to complete setup

Memory usage with htop:

[facebook-github-bot]

@atalman has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.