Misc. bug: ERROR:hf-to-gguf:Model Ministral3ForCausalLM is not supported

[vbooka1]

Name and Version

llama.cpp b7376

Operating systems

Linux

Which llama.cpp modules do you know to be affected?

llama-quantize

Command line

python convert_hf_to_gguf.py --outtype q8_0 --verbose Devstral-2-123B-Instruct-2512/

Problem description & steps to reproduce

Hello, I can not quantize Devstral-2-123B-Instruct-2512. I have installed the latest transformers version 4.57.3 from the stable repo and got this error:

$ python convert_hf_to_gguf.py --outtype q8_0 --verbose Devstral-2-123B-Instruct-2512/
INFO:hf-to-gguf:Loading model: Devstral-2-123B-Instruct-2512
WARNING:hf-to-gguf:Failed to load model config from Devstral-2-123B-Instruct-2512: The checkpoint you are trying to load has model type `ministral3` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.

You can update Transformers with the command `pip install --upgrade transformers`. If this does not work, and the checkpoint is very new, then there may not be a release version that supports this model yet. In this case, you can get the most up-to-date code by installing Transformers from source with the command `pip install git+https://github.com/huggingface/transformers.git`
WARNING:hf-to-gguf:Trying to load config.json instead
INFO:hf-to-gguf:Model architecture: Ministral3ForCausalLM
ERROR:hf-to-gguf:Model Ministral3ForCausalLM is not supported

Then I have installed unstable transformers version 5.0.0dev0 from the unstable repo using the command above pip install git+https://github.com/huggingface/transformers.git and got a similar error, although less verbose this time:

$ python convert_hf_to_gguf.py --outtype q8_0 --verbose Devstral-2-123B-Instruct-2512/
INFO:hf-to-gguf:Loading model: Devstral-2-123B-Instruct-2512
INFO:hf-to-gguf:Model architecture: Ministral3ForCausalLM
ERROR:hf-to-gguf:Model Ministral3ForCausalLM is not supported

I was using script convert_hf_to_gguf.py from llama.cpp build version b7342 which should support the Mistral 3 architecture because on huggingface.co there are gguf files successfully converted using older versions, for example this page https://huggingface.co/bartowski/mistralai_Devstral-2-123B-Instruct-2512-GGUF says "Using llama.cpp release b7335 for quantization." and b7335 is older than mine b7342.

Then I have downloaded the latest llama.cpp build b7376 but its version of convert_hf_to_gguf.py also returns the same error.

If I add a "--mistral-format" option then the conversion script shows warning "WARNING:gguf.gguf_writer:Duplicated key name 'general.architecture', overwriting it with new value 'llama' of type STRING" and produces an incorrect file with size 8 MB

I guess I'm doing something wrong, please tell how to quantize the latest Mistral models correctly.

First Bad Commit

No response

Relevant log output

[CISC]

Using --mistral-format is the correct way, but did you download the consolidated safetensors?

[vbooka1]

did you download the consolidated safetensors?

oops, I've thought that these files are redundand and downloaded only "model-000*-of-00027.safetensors". Could I create the "consolidated.*.safetensors" files out of "model.*.safetensors"? I do not want to download another 100 GB over crappy rural ADSL

[CISC]

did you download the consolidated safetensors?

oops, I've thought that these files are redundand and downloaded only "model-000*-of-00027.safetensors". Could I create the "consolidated..safetensors" files out of "model..safetensors"? I do not want to download another 100 GB over crappy rural ADSL

Unfortunately you need the consolidated ones for --mistral-format, the model ones are converted tensors for compatibility, but support for those has not been added to the conversion script.

If you don't want to download again you might want to look into adding support. :) It might be as simple as registering Ministral3 arch to the Mistral3Model class, not sure:

llama.cpp/convert_hf_to_gguf.py

Lines 2838 to 2839 in 4d5ae24

	@ModelBase.register("Mistral3ForConditionalGeneration")
	class Mistral3Model(LlamaModel):

[vbooka1]

If you don't want to download again you might want to look into adding support. :) It might be as simple as registering Ministral3 arch to the Mistral3Model class, not sure:

Yes it seems that simply adding "Ministral3ForCausalLM" has worked, thank you!

--- convert_hf_to_gguf.py      2025-12-13 08:33:53.096850913 -0500
+++ ../src/llama.cpp-b7376/convert_hf_to_gguf.py        2025-12-12 17:58:53.000000000 -0500
@@ -188,7 +188,7 @@

             return tensors

-        prefix = "model" # if not self.is_mistral_format else "consolidated"
+        prefix = "model" if not self.is_mistral_format else "consolidated"
         part_names: set[str] = set(ModelBase.get_model_part_names(self.dir_model, prefix, ".safetensors"))
         is_safetensors: bool = len(part_names) > 0
         if not is_safetensors:
@@ -2835,7 +2835,7 @@
         return []


-@ModelBase.register("Mistral3ForConditionalGeneration", "Ministral3ForCausalLM")
+@ModelBase.register("Mistral3ForConditionalGeneration")
 class Mistral3Model(LlamaModel):
     model_arch = gguf.MODEL_ARCH.MISTRAL3

I did not test the model thoroughly though, anyway this is not a bug so I am closing this issue.

[CISC]

I did not test the model thoroughly though, anyway this is not a bug so I am closing this issue.

If you can verify the model works, feel free to submit a PR registering the arch, would be most appreciated.

[vbooka1]

If you can verify the model works, feel free to submit a PR registering the arch, would be most appreciated.

It works but generates bad quality responses and loops, I'm afraid it's not my sampler settings but something else is required to properly convert the model besides simply adding "Ministral3ForCausalLM" on line 2838.
Or perhaps it is caused by converting "model-" files instead of "consolidated-".

[CISC]

If you can verify the model works, feel free to submit a PR registering the arch, would be most appreciated.

It works but generates bad quality responses and loops, I'm afraid it's not my sampler settings but something else is required to properly convert the model besides simply adding "Ministral3ForCausalLM" on line 2838. Or perhaps it is caused by converting "model-" files instead of "consolidated-".

Maybe, could also be related to #18006 or #18013