nvfp4_gguf_quant/README.md

Convert-to-Quant (ctq) is installed for normal quantizations. hf, the Hugging Face CLI command, and llama.cpp's compiled binaries are also available.

For GGUF quantizations, try:

• hf_to_nvfp4_quant (huggingface structure to GGUF)
• gguf_to_nvfp4_quant (GGUF to GGUF)
• safetensors_to_nvfp4_quant (ComfyUI style safetensors to GGUF (untested))
• hf_to_nvfp4_safetensors (huggingface structure to ComfyUI style safetensors)

For model info, try:

• hf_info (huggingface structure)
• gguf_info (GGUF structure)

---

Example:

hf download coder3101/gemma-4-12B-it-heretic --local-dir gemma-4-12B-it-heretic
hf_to_nvfp4_gguf -o gemma-4-12B-it-heretic_nvfp4 gemma-4-12B-it-heretic

This would create a NVFP4 GGUF for Ollama/llama.cpp from https://huggingface.co/coder3101/gemma-4-12B-it-heretic

The same directory can be quantized to one ComfyUI NVFP4 safetensors file:

hf_to_nvfp4_safetensors -o gemma-4-12B-it-heretic-NVFP4.safetensors gemma-4-12B-it-heretic

Conversions between formats without quantization is available in the custom commands with --no-quant.