add readme and fix docker.sh
This commit is contained in:
1 parent
3674fe921b
commit
be97590146
2 files changed
+31
No files matched your search
@@ -0,0 +1,28 @@
|
||||
Convert-to-Quant (`ctq`) is installed for normal quantizations.
|
||||
`hf`, the Hugging Face CLI command, and llama.cpp's compiled binaries are also available.
|
||||
|
||||
For GGUF quantizations, try:
|
||||
- `hf_to_nvfp4_quant` (huggingface structure to GGUF)
|
||||
- `gguf_to_nvfp4_quant` (GGUF to GGUF)
|
||||
- `safetensors_to_nvfp4_quant` (ComfyUI style safetensors to GGUF (untested))
|
||||
- `hf_to_nvfp4_safetensors` (huggingface structure to ComfyUI style safetensors)
|
||||
|
||||
For model info, try:
|
||||
- `hf_info` (huggingface structure)
|
||||
- `gguf_info` (GGUF structure)
|
||||
---
|
||||
|
||||
Example:
|
||||
|
||||
```bash
|
||||
hf download coder3101/gemma-4-12B-it-heretic --local-dir gemma-4-12B-it-heretic
|
||||
hf_to_nvfp4_gguf -o gemma-4-12B-it-heretic_nvfp4 gemma-4-12B-it-heretic
|
||||
```
|
||||
|
||||
This would create a NVFP4 GGUF for Ollama/llama.cpp from `https://huggingface.co/coder3101/gemma-4-12B-it-heretic`
|
||||
|
||||
The same directory can be quantized to one ComfyUI NVFP4 safetensors file:
|
||||
|
||||
`hf_to_nvfp4_safetensors -o gemma-4-12B-it-heretic-NVFP4.safetensors gemma-4-12B-it-heretic`
|
||||
|
||||
Conversions between formats without quantization is available in the custom commands with `--no-quant`.
|
||||
Reference in new issue
Block a user