.TH CONVERT_LORA_TO_GGUF 1 "November 2026" "nvfp4_gguf_quant" "User Commands"
.SH NAME
convert_lora_to_gguf \- convert Hugging Face PEFT LoRA adapters to GGUF format
.SH SYNOPSIS
.B convert_lora_to_gguf
[\fB\-\-outfile\fR \fIOUTFILE\fR]
[\fB\-\-outtype\fR {\fBf32\fR|\fBf16\fR|\fBbf16\fR|\fBq8_0\fR|\fBauto\fR}]
[\fB\-\-base\fR \fIBASE\fR]
[\fB\-\-base\-model\-id\fR \fIID\fR]
[\fB\-\-trust\-remote\-code\fR]
[\fB\-\-bigendian\fR]
[\fB\-\-no\-lazy\fR]
[\fB\-\-dry\-run\fR]
[\fB\-\-verbose\fR]
[\fB\-h\fR|\fB\-\-help\fR]
\fIlora_path\fR
.SH DESCRIPTION
.B convert_lora_to_gguf
converts fine-tuned Hugging Face PEFT LoRA (Low-Rank Adaptation) adapter checkpoints
into GGUF adapter files compatible with \fBllama.cpp\fR runtime adapter injection
(\fB\-\-lora\fR) and offline merging (\fBllama\-export\-lora\fR).
.PP
LoRA checkpoints store low-rank weight matrices \fIA\fR and \fIB\fR rather than full model
weights. To convert these tensors correctly into GGUF format with proper tensor names,
dimensions, and architecture attributes,
.B convert_lora_to_gguf
requires access to the configuration of the base model on which the adapter was trained.
Base model metadata is automatically retrieved from Hugging Face Hub using the
\fIbase_model_name_or_path\fR key inside \fIadapter_config.json\fR, or supplied locally via
\fB\-\-base\fR.
.PP
The tool also features full support for:
.IP \(bu 2
\fBMergekit LoRAs\fR: Adapters created with \fImergekit\-extract\-lora\fR, including adapted
token embedding weights and normalization layers.
.IP \(bu 2
\fBaLoRA (Activation-aware LoRA)\fR: Automatic tokenization of \fIinvocation_string\fR into
GGUF \fIadapter.alora_invocation_tokens\fR metadata.
.IP \(bu 2
Both SafeTensors (\fIadapter_model.safetensors\fR) and legacy PyTorch
(\fIadapter_model.bin\fR) checkpoint formats.
.SH POSITIONAL ARGUMENTS
.TP
.I lora_path
Directory containing the Hugging Face PEFT LoRA adapter. Must contain
\fIadapter_config.json\fR along with \fIadapter_model.safetensors\fR or
\fIadapter_model.bin\fR.
.SH OPTIONS
.TP
.BI \-\-outfile " " OUTFILE
Destination path for the output GGUF file. Defaults to an automatically named file
inside the adapter directory. If the substring \fI{ftype}\fR is present in the path, it is
dynamically replaced with the chosen output format name.
.TP
.BI \-\-outtype " " TYPE
Output tensor precision format. Choices are:
.RS
.TP
.B f32
(Default) 32-bit single-precision floating point. Recommended for preserving adapter
fidelity.
.TP
.B f16
16-bit half-precision IEEE float.
.TP
.B bf16
16-bit Brain Floating Point.
.TP
.B q8_0
Quantize adapter weights to 8-bit integer format.
.TP
.B auto
Select the highest-fidelity 16-bit floating point format based on the first loaded
tensor.
.RE
.TP
.BI \-\-base " " BASE
Path to a local directory containing configuration files (\fIconfig.json\fR,
\fItokenizer.json\fR) of the base model. Note that only configuration and tokenizer
files are required; full model weights are not loaded.
.TP
.BI \-\-base\-model\-id " " ID
Hugging Face model repository ID of the base model (e.g.
\fImeta-llama/Llama-3.2-1B-Instruct\fR). If provided, overrides \fB\-\-base\fR and
fetches configuration directly from Hugging Face Hub.
.TP
.B \-\-trust\-remote\-code
Permit execution of custom modeling and tokenizer code hosted in remote Hugging Face
repositories when downloading base model configuration.
.TP
.B \-\-bigendian
Target big-endian architecture when serializing tensor headers and raw data.
.TP
.B \-\-no\-lazy
Disable lazy tensor loading, computing and staging all modified tensors in memory
before writing.
.TP
.B \-\-dry\-run
Simulate the conversion process and log all planned tensor operations and metadata
without writing any output files to disk.
.TP
.B \-\-verbose
Enable verbose debug logging output.
.TP
.BR \-h ", " \-\-help
Display command usage summary and option descriptions, then exit.
.SH EXAMPLES
.TP
.B Convert a local LoRA adapter with automatic base model download:
.RS
.nf
convert_lora_to_gguf ./my-llama3-lora/
.fi
.RE
.TP
.B Convert a LoRA adapter using a local base model directory:
.RS
.nf
convert_lora_to_gguf --base ./Meta-Llama-3-8B/ ./my-llama3-lora/
.fi
.RE
.TP
.B Convert with explicit output path in 16-bit float:
.RS
.nf
convert_lora_to_gguf --outtype f16 --outfile ./llama3-adapter-f16.gguf \\
    ./my-llama3-lora/
.fi
.RE
.TP
.B Use the converted adapter in llama.cpp:
.RS
.nf
llama-cli -m llama-3-8b-Q4_K_M.gguf --lora llama3-adapter-f16.gguf \\
    -p "Hello, world!"
.fi
.RE
.SH EXIT STATUS
.TP
.B 0
Success.
.TP
.B 1
Conversion failed (e.g. missing adapter files, base model configuration mismatch, or
unsupported architecture).
.SH SEE ALSO
.BR convert_hf_to_gguf (1),
.BR convert_hf_to_gguf_update (1),
.BR convert_llama_ggml_to_gguf (1),
.BR gguf_to_nvfp4_gguf (1),
.BR hf_to_nvfp4_gguf (1),
.BR gguf_info (1)