182 lines
5.3 KiB
Groff
182 lines
5.3 KiB
Groff
.TH OLLAMA_NVFP4 1 "November 2026" "nvfp4_gguf_quant" "User Commands"
|
|
.SH NAME
|
|
ollama_nvfp4 \- inspect Ollama sha256 model stores and quantize models to NVFP4 GGUF
|
|
.SH SYNOPSIS
|
|
.B ollama_nvfp4
|
|
[\fB\-\-models\fR \fIDIR\fR]
|
|
.I subcommand
|
|
[\fIoptions...\fR]
|
|
.br
|
|
.B ollama_nvfp4
|
|
[\fB\-\-models\fR \fIDIR\fR]
|
|
.B \-\-info
|
|
\fImodel\fR
|
|
[\fIgguf_info_args...\fR]
|
|
.SH DESCRIPTION
|
|
.B ollama_nvfp4
|
|
is a management and quantization utility for local \fBOllama\fR model installations.
|
|
Ollama stores model artifacts as content-addressed blobs (\fIblobs/sha256-<digest>\fR)
|
|
referenced by JSON manifests (\fImanifests/\fR).
|
|
.PP
|
|
.B ollama_nvfp4
|
|
resolves these hashes into human-readable model names, displays model capabilities and
|
|
parameters, exports native GGUF weights alongside reconstructed \fBModelfile\fRs,
|
|
and directly quantizes models to NVIDIA FP4 (NVFP4) format. Quantized models can
|
|
optionally be registered straight back into Ollama's local library.
|
|
.PP
|
|
The generated \fBModelfile\fR preserves the original prompt template, stop sequences,
|
|
system message, renderer, and parser settings so that thinking/reasoning modes and
|
|
generation behavior remain intact.
|
|
.SH GLOBAL OPTIONS
|
|
.TP
|
|
.BI \-\-models " " DIR
|
|
Path to the Ollama models store directory containing \fIblobs/\fR and \fImanifests/\fR.
|
|
If not provided, the location is resolved via the \fBOLLAMA_MODELS\fR environment variable,
|
|
\fI/models/ollama\fR, or \fI~/.ollama/models\fR.
|
|
.TP
|
|
.BR \-h ", " \-\-help
|
|
Show command help and exit.
|
|
.SH SUBCOMMANDS
|
|
.SS list
|
|
.B ollama_nvfp4
|
|
[\fB\-\-models\fR \fIDIR\fR]
|
|
.B list
|
|
[\fB\-\-full\-scan\fR]
|
|
[\fB\-\-ctx\-scan\fR]
|
|
.PP
|
|
Display every manifest in the store as a readable model entry with family, quantization type,
|
|
and capability badges (chat, tool call, thinking, vision).
|
|
.TP
|
|
.B \-\-full\-scan
|
|
Read the underlying GGUF header of each model blob. This is slower on large stores but reveals
|
|
capabilities and vision encoders not declared in manifest metadata.
|
|
.TP
|
|
.B \-\-ctx\-scan
|
|
Scan parameter blobs and report models whose Modelfile explicitly configures \fInum_ctx\fR.
|
|
.SS show
|
|
.B ollama_nvfp4
|
|
[\fB\-\-models\fR \fIDIR\fR]
|
|
.B show
|
|
\fImodel\fR
|
|
.PP
|
|
Display detailed information for a specified model, including manifest layers, sha256 blob
|
|
hashes, component media types, GGUF architecture metadata, context length, and system settings.
|
|
The \fImodel\fR argument can be a full name (e.g. \fIllama3.2:latest\fR) or a unique substring.
|
|
.SS export
|
|
.B ollama_nvfp4
|
|
[\fB\-\-models\fR \fIDIR\fR]
|
|
.B export
|
|
[\fB\-o\fR \fIDIR\fR]
|
|
[\fB\-\-modelfile\-only\fR]
|
|
[\fB\-\-overwrite\fR|\fB\-\-force\fR]
|
|
\fImodel\fR
|
|
.PP
|
|
Export the model's underlying GGUF weight blob and generate a fully configured \fBModelfile\fR
|
|
into the destination directory (default: \fI./<model>\fR).
|
|
.TP
|
|
.BI \-o " " DIR ", " \-\-output " " DIR
|
|
Output directory path.
|
|
.TP
|
|
.B \-\-modelfile\-only
|
|
Write only the Modelfile without copying the GGUF or vision projector blobs.
|
|
.TP
|
|
.BR \-\-overwrite ", " \-\-force
|
|
Replace existing files in the output directory.
|
|
.SS quant
|
|
.B ollama_nvfp4
|
|
[\fB\-\-models\fR \fIDIR\fR]
|
|
.B quant
|
|
[\fB\-o\fR \fIDIR\fR]
|
|
[\fB\-\-name\fR \fINAME\fR]
|
|
[\fB\-\-install\fR]
|
|
[\fB\-\-float\-only\fR]
|
|
[\fB\-\-context\-length\fR \fILEN\fR]
|
|
[\fIquant_opts...\fR]
|
|
\fImodel\fR
|
|
.PP
|
|
Quantize an Ollama model to NVFP4 GGUF format and generate an accompanying Modelfile.
|
|
.TP
|
|
.BI \-o " " DIR ", " \-\-output " " DIR
|
|
Output directory (default: \fI./<model>-NVFP4\fR).
|
|
.TP
|
|
.BI \-\-name " " NAME
|
|
Model tag used when registering with \fB\-\-install\fR (default: \fI<model>:nvfp4\fR).
|
|
.TP
|
|
.B \-\-install
|
|
Automatically register the quantized model back into Ollama using \fIollama create\fR.
|
|
.TP
|
|
.B \-\-float\-only
|
|
Refuse to requantize a model if its source blob is already quantized (e.g. \fIQ4_K_M\fR).
|
|
Quantizing from float16/BF16 produces superior NVFP4 fidelity.
|
|
.TP
|
|
.BI \-\-context\-length " " LEN
|
|
Override maximum context length stored in the GGUF.
|
|
.TP
|
|
.I quant_opts
|
|
All standard learned-rounding and execution options supported by \fBgguf_to_nvfp4_gguf\fR
|
|
are supported: \fB\-\-simple\fR, \fB\-\-optimizer\fR, \fB\-\-lr\fR, \fB\-\-num\-iter\fR,
|
|
\fB\-\-scale\-optimization\fR, \fB\-\-device\fR, \fB\-\-checkpoint\-dir\fR, etc.
|
|
.SS \-\-info
|
|
.B ollama_nvfp4
|
|
[\fB\-\-models\fR \fIDIR\fR]
|
|
.B \-\-info
|
|
\fImodel\fR
|
|
[\fIgguf_info_args...\fR]
|
|
.PP
|
|
Resolve \fImodel\fR to its sha256 GGUF weight blob in the store and invoke
|
|
.BR gguf_info (1)
|
|
directly on the file. Any subsequent flags (such as \fB\-\-metadata\fR or \fB\-\-tensors\fR)
|
|
are forwarded directly to \fBgguf_info\fR.
|
|
.SH ENVIRONMENT VARIABLES
|
|
.TP
|
|
.B OLLAMA_MODELS
|
|
Path to the default Ollama models directory. Overridden by \fB\-\-models\fR.
|
|
.SH EXAMPLES
|
|
.TP
|
|
.B List installed models with capability summary:
|
|
.RS
|
|
.nf
|
|
ollama_nvfp4 list
|
|
.fi
|
|
.RE
|
|
.TP
|
|
.B Inspect detailed manifest and parameters of a model:
|
|
.RS
|
|
.nf
|
|
ollama_nvfp4 show llama3.2:3b
|
|
.fi
|
|
.RE
|
|
.TP
|
|
.B Run gguf_info on an Ollama model's weight blob:
|
|
.RS
|
|
.nf
|
|
ollama_nvfp4 --info llama3.2:3b --metadata
|
|
.fi
|
|
.RE
|
|
.TP
|
|
.B Export a model and its Modelfile:
|
|
.RS
|
|
.nf
|
|
ollama_nvfp4 export llama3.2:3b -o ./llama3-export
|
|
.fi
|
|
.RE
|
|
.TP
|
|
.B Quantize an Ollama FP16 model to NVFP4 and install it:
|
|
.RS
|
|
.nf
|
|
ollama_nvfp4 quant --install llama3.2:3b-instruct-fp16
|
|
.fi
|
|
.RE
|
|
.SH EXIT STATUS
|
|
.TP
|
|
.B 0
|
|
Success.
|
|
.TP
|
|
.B 1
|
|
An error occurred (e.g. model not found, invalid store directory, or quantization failure).
|
|
.SH SEE ALSO
|
|
.BR gguf_info (1),
|
|
.BR gguf_to_nvfp4_gguf (1),
|
|
.BR hf_to_nvfp4_gguf (1)
|
|
|