add readme and fix docker.sh

This commit is contained in:
zefie committed 2026-11-04 16:49:59 -05:00
1 parent 3674fe921b
commit be97590146
2 files changed
+31

No files matched your search

+28
View File
@@ -0,0 +1,28 @@
Convert-to-Quant (`ctq`) is installed for normal quantizations.
`hf`, the Hugging Face CLI command, and llama.cpp's compiled binaries are also available.
For GGUF quantizations, try:
- `hf_to_nvfp4_quant` (huggingface structure to GGUF)
- `gguf_to_nvfp4_quant` (GGUF to GGUF)
- `safetensors_to_nvfp4_quant` (ComfyUI style safetensors to GGUF (untested))
- `hf_to_nvfp4_safetensors` (huggingface structure to ComfyUI style safetensors)
For model info, try:
- `hf_info` (huggingface structure)
- `gguf_info` (GGUF structure)
---
Example:
```bash
hf download coder3101/gemma-4-12B-it-heretic --local-dir gemma-4-12B-it-heretic
hf_to_nvfp4_gguf -o gemma-4-12B-it-heretic_nvfp4 gemma-4-12B-it-heretic
```
This would create a NVFP4 GGUF for Ollama/llama.cpp from `https://huggingface.co/coder3101/gemma-4-12B-it-heretic`
The same directory can be quantized to one ComfyUI NVFP4 safetensors file:
`hf_to_nvfp4_safetensors -o gemma-4-12B-it-heretic-NVFP4.safetensors gemma-4-12B-it-heretic`
Conversions between formats without quantization is available in the custom commands with `--no-quant`.
+3
View File
@@ -8,6 +8,7 @@ get_latest() {
latest_image=$(docker images --format '{{.CreatedAt}}\t{{.Repository}}:{{.Tag}}' zefie/nvfp4_quant \ latest_image=$(docker images --format '{{.CreatedAt}}\t{{.Repository}}:{{.Tag}}' zefie/nvfp4_quant \
| sort -r \ | sort -r \
| cut -f2- \ | cut -f2- \
| grep -v latest \
| head -n 1) | head -n 1)
if [[ -z "$latest_image" ]]; then if [[ -z "$latest_image" ]]; then
@@ -44,7 +45,9 @@ elif [ "${1}" == "push" ]; then
# push dated one first, then latest # push dated one first, then latest
"${0}" mklatest "${0}" mklatest
IMAGE=$(get_latest) IMAGE=$(get_latest)
echo "Pushing ${IMAGE}"
docker push "${IMAGE}" docker push "${IMAGE}"
echo "Pushing zefie/nvfp4_quant:latest"
docker push "zefie/nvfp4_quant:latest" docker push "zefie/nvfp4_quant:latest"
else else
echo "Usage: ${0} [build|run|mklatest]" echo "Usage: ${0} [build|run|mklatest]"