nvfp4_gguf_quant/README.md

These scripts require llama.cpp and Convert-to-Quant.

hf, the Hugging Face CLI command is also suggested, but not required.

For quantizations or conversions, try:

• hf_to_nvfp4_quant (huggingface structure to GGUF)
• gguf_to_nvfp4_quant (GGUF to GGUF)
• safetensors_to_nvfp4_quant (ComfyUI style safetensors to GGUF (untested))
• hf_to_nvfp4_safetensors (huggingface structure to ComfyUI style safetensors)

For model info, try:

• hf_info (huggingface structure)
• gguf_info (GGUF structure)

---

Example:

hf download coder3101/gemma-4-12B-it-heretic --local-dir gemma-4-12B-it-heretic
hf_to_nvfp4_gguf -o gemma-4-12B-it-heretic_nvfp4 gemma-4-12B-it-heretic

This would create a NVFP4 GGUF for Ollama/llama.cpp from https://huggingface.co/coder3101/gemma-4-12B-it-heretic

The same directory can be quantized to one ComfyUI NVFP4 safetensors file:

hf_to_nvfp4_safetensors -o gemma-4-12B-it-heretic-NVFP4.safetensors gemma-4-12B-it-heretic

Conversions between formats without quantization is available in the custom commands with --no-quant.

Currently the only supported modes are "Quantize to NVFP4" or "Copy without quantizing" (--no-quant). I may add other quantization options in the future.