131 lines
4.5 KiB
Groff
131 lines
4.5 KiB
Groff
.TH CONVERT_LLAMA_GGML_TO_GGUF 1 "November 2026" "nvfp4_gguf_quant" "User Commands"
|
|
.SH NAME
|
|
convert_llama_ggml_to_gguf \- convert legacy GGMLv1/v2/v3 model files to GGUF format
|
|
.SH SYNOPSIS
|
|
.B convert_llama_ggml_to_gguf
|
|
\fB\-i\fR \fIINPUT\fR
|
|
\fB\-o\fR \fIOUTPUT\fR
|
|
[\fB\-\-name\fR \fINAME\fR]
|
|
[\fB\-\-desc\fR \fIDESC\fR]
|
|
[\fB\-\-gqa\fR \fIN\fR]
|
|
[\fB\-\-eps\fR \fIEPS\fR]
|
|
[\fB\-c\fR|\fB\-\-context\-length\fR \fILEN\fR]
|
|
[\fB\-m\fR|\fB\-\-model\-metadata\-dir\fR \fIDIR\fR]
|
|
[\fB\-\-vocab\-dir\fR \fIDIR\fR]
|
|
[\fB\-\-vocabtype\fR \fITYPE\fR]
|
|
[\fB\-\-verbose\fR]
|
|
[\fB\-h\fR|\fB\-\-help\fR]
|
|
.SH DESCRIPTION
|
|
.B convert_llama_ggml_to_gguf
|
|
converts legacy binary models in GGML formats (\fBGGML\fR, \fBGGMF\fR, and \fBGGJT\fR /
|
|
\fBGGMLv3\fR) into the modern \fBGGUF\fR format used by current versions of
|
|
\fBllama.cpp\fR.
|
|
.PP
|
|
The tool memory-maps the input file, parses model hyperparameters (embedding dimensions,
|
|
layer counts, head counts, rotary frequencies, and quantization type), reverses
|
|
legacy tensor dimension transpositions where needed, and writes standard GGUF metadata
|
|
and tensor tables.
|
|
.PP
|
|
.B Note:
|
|
This conversion script is provided as a best-effort migration tool for archiving or
|
|
recovering older checkpoints. Whenever feasible, it is strongly recommended to convert
|
|
directly from original Hugging Face source weights using \fBconvert_hf_to_gguf\fR(1).
|
|
.SH REQUIRED ARGUMENTS
|
|
.TP
|
|
.BI \-i " " INPUT ", " \-\-input " " INPUT
|
|
Path to the input legacy GGML binary file (e.g. \fIggml-model-q4_0.bin\fR).
|
|
.TP
|
|
.BI \-o " " OUTPUT ", " \-\-output " " OUTPUT
|
|
Destination path for the converted GGUF model file (e.g. \fImodel-q4_0.gguf\fR).
|
|
.SH OPTIONS
|
|
.TP
|
|
.BI \-\-name " " NAME
|
|
Set the model name metadata attribute in the GGUF header.
|
|
.TP
|
|
.BI \-\-desc " " DESC
|
|
Set the model description text metadata in the GGUF header.
|
|
.TP
|
|
.BI \-\-gqa " " N
|
|
Grouped-query attention (GQA) factor. Defaults to \fB1\fR (standard multi-head attention).
|
|
For LLaMA 2 70B models, this must be set to \fB8\fR unless overridden via
|
|
\fB\-\-model\-metadata\-dir\fR.
|
|
.TP
|
|
.BI \-\-eps " " EPS
|
|
Root Mean Square (RMS) normalization epsilon value. Defaults to \fB5.0e-06\fR.
|
|
Typically \fB1e-6\fR for LLaMA 1 and OpenLLaMA, and \fB1e-5\fR for LLaMA 2.
|
|
.TP
|
|
.BI \-c " " LEN ", " \-\-context\-length " " LEN
|
|
Maximum context length in tokens. Defaults to \fB2048\fR (typical for LLaMA 1). For
|
|
LLaMA 2 models, set to \fB4096\fR.
|
|
.TP
|
|
.BI \-m " " DIR ", " \-\-model\-metadata\-dir " " DIR
|
|
Path to a directory containing original Hugging Face (\fIconfig.json\fR, \fItokenizer.model\fR)
|
|
or Meta PyTorch (\fIparams.json\fR) files. When specified, exact vocabulary items,
|
|
token scores, special tokens, GQA ratios, context limits, and RMS epsilon values are
|
|
loaded directly from the original configuration, overriding command-line heuristic flags.
|
|
.TP
|
|
.BI \-\-vocab\-dir " " DIR
|
|
Directory containing \fItokenizer.model\fR when stored separately from the model metadata
|
|
directory. Only meaningful when \fB\-\-model\-metadata\-dir\fR is also specified.
|
|
.TP
|
|
.BI \-\-vocabtype " " TYPE
|
|
Vocabulary loader types to attempt (comma-separated list, default: \fBspm,hfft\fR). Only
|
|
meaningful in conjunction with \fB\-\-model\-metadata\-dir\fR.
|
|
.TP
|
|
.B \-\-verbose
|
|
Increase logging verbosity, printing debug information about tensors and hyperparameter
|
|
offsets.
|
|
.TP
|
|
.BR \-h ", " \-\-help
|
|
Display command usage summary and exit.
|
|
.SH FORMAT COMPATIBILITY
|
|
.B convert_llama_ggml_to_gguf
|
|
recognizes the following historical format magic identifiers:
|
|
.TP
|
|
.B GGML (magic 'lmgg')
|
|
Version 1 format. Does not contain vocabulary scores; token scores default to 0.0 unless
|
|
\fB\-\-model\-metadata\-dir\fR is provided.
|
|
.TP
|
|
.B GGMF (magic 'fmgg')
|
|
Version 2 format. Adds vocabulary scores to token strings.
|
|
.TP
|
|
.B GGJT / GGMLv3 (magic 'tjgg')
|
|
Version 3 format with 32-byte tensor data alignment. Supports all GGML quantization
|
|
types up to Q6_K.
|
|
.SH EXAMPLES
|
|
.TP
|
|
.B Basic conversion of a legacy LLaMA 1 Q4_0 GGML file:
|
|
.RS
|
|
.nf
|
|
convert_llama_ggml_to_gguf -i llama-7b-q4_0.bin -o llama-7b-q4_0.gguf
|
|
.fi
|
|
.RE
|
|
.TP
|
|
.B Convert a LLaMA 2 70B GGML file with appropriate GQA and context flags:
|
|
.RS
|
|
.nf
|
|
convert_llama_ggml_to_gguf -i llama-2-70b-q4_0.bin -o llama-2-70b-q4_0.gguf \\
|
|
--gqa 8 --eps 1e-5 -c 4096
|
|
.fi
|
|
.RE
|
|
.TP
|
|
.B High-accuracy conversion using original Hugging Face metadata directory:
|
|
.RS
|
|
.nf
|
|
convert_llama_ggml_to_gguf -i model-q4_0.bin -o model-q4_0.gguf \\
|
|
-m ./original-hf-model-dir/
|
|
.fi
|
|
.RE
|
|
.SH EXIT STATUS
|
|
.TP
|
|
.B 0
|
|
Success.
|
|
.TP
|
|
.B 1
|
|
An error occurred (e.g. corrupt header, unsupported dimension, missing metadata).
|
|
.SH SEE ALSO
|
|
.BR convert_hf_to_gguf (1),
|
|
.BR convert_lora_to_gguf (1),
|
|
.BR gguf_info (1)
|
|
|