nvfp4_gguf_quant/man/man1/ctq.1

.TH CTQ 1 "November 2026" "convert_to_quant" "User Commands"
.SH NAME
ctq \- convert safetensors model weights to scaled FP8, NVFP4, INT8, and MXFP8 formats
.SH SYNOPSIS
.B ctq
\fB\-i\fR \fIINPUT\fR
[\fB\-o\fR \fIOUTPUT\fR]
[\fB\-\-comfy_quant\fR]
[\fB\-\-simple\fR]
[\fB\-\-device\fR \fIDEVICE\fR]
[\fB\-\-low\-memory\fR]
[\fB\-\-nvfp4\fR|\fB\-\-int8\fR|\fB\-\-mxfp8\fR]
[\fIoptions...\fR]
.SH DESCRIPTION
.B ctq
(\fBconvert_to_quant\fR) quantizes neural network weights stored in \fI.safetensors\fR
format into scaled lower-precision representations, primarily targeting
\fBComfyUI\fR and hardware-accelerated inference runtimes.
.PP
By default, \fBctq\fR performs Scaled FP8 quantization using SVD-guided learned rounding
(Prodigy optimizer) and per-tensor scaling. It supports multiple quantization formats:
.IP \(bu 2
\fBFP8\fR (Float8 E4M3 / E5M2) with tensor-wise, row-wise, or block-wise scaling.
.IP \(bu 2
\fBNVFP4\fR (NVIDIA 4-bit floating point, E2M1 with dual-level block scaling).
.IP \(bu 2
\fBINT8\fR block-wise quantization with optional group-wise Hadamard rotation (\fIConvRot\fR).
.IP \(bu 2
\fBMXFP8\fR (Microscaling FP8) and Hybrid MXFP8 formats.
.PP
Quantization error can optionally be extracted into a companion Error Correction LoRA
adapter (\fB\-\-extract\-lora\fR) to restore high-precision fidelity.
.SH OPTIONS
.SS Standard Options
.TP
.BI \-i " " INPUT ", " \-\-input " " INPUT
Path to the input \fI.safetensors\fR weight file (required).
.TP
.BI \-o " " OUTPUT ", " \-\-output " " OUTPUT
Path to the output \fI.safetensors\fR destination file. Auto-generated if omitted.
.TP
.BR \-\-comfy_quant ", " \-\-comfy\-quant
Write ComfyUI-compatible quantization tensors and metadata (\fI.comfy_quant\fR).
.TP
.B \-\-simple
Skip SVD learned rounding and apply fast, direct grid quantization.
.TP
.BR \-fpmm ", " \-\-full_precision_matrix_mult
Add \fIfull_precision_matrix_mult=True\fR to \fI.comfy_quant\fR metadata.
.TP
.B \-\-heur
Skip layers with poor quantization characteristics (aspect ratio, size).
.TP
.BI \-\-device " " DEVICE
Execution device (e.g. \fBcuda\fR, \fBcuda:0\fR, \fBcpu\fR). Overrides auto-detection.
.TP
.BR \-lm ", " \-\-low\-memory
Stream tensor loading to minimize RAM consumption (recommended for models exceeding 50% available memory).
.TP
.BI \-\-verbose " " {DEBUG,VERBOSE,NORMAL,MINIMAL}
Logging verbosity level (default: \fBNORMAL\fR).
.TP
.BI \-\-manual_seed " " SEED
Random seed for reproducibility. Set to \-1 for non-deterministic execution.
.TP
.B \-\-save\-quant\-metadata
Write quantization metadata directly into safetensors file header (\fI_quantization_metadata\fR).
.SS Quantization Formats & Modes
.TP
.B \-\-nvfp4
Quantize to NVIDIA FP4 (E2M1) with dual-level block scaling (Blackwell architecture).
.TP
.B \-\-int8
Quantize to INT8 block-wise format instead of FP8.
.TP
.B \-\-convrot
Enable group-wise Hadamard rotation (ConvRot) for INT8 row-wise quantization to mitigate activation outliers.
.TP
.BI \-\-convrot\-group\-size " " SIZE
Group size for ConvRot (must be a power of 4, e.g. 64, 256, 1024; default: 256).
.TP
.B \-\-mxfp8
Quantize to MXFP8 (Microscaling FP8) format.
.TP
.BI \-\-scaling_mode " " {tensor,row,block}
FP8 scaling granularity: \fBtensor\fR (single scale), \fBrow\fR (per-row), or \fBblock\fR (2D tile). Default: \fBtensor\fR.
.TP
.BI \-\-block_size " " SIZE
Block size for block-wise scaling (default: 128).
.SS Learned Rounding & Optimization
.TP
.BI \-\-optimizer " " {prodigy,adamw,radam,original}
Optimization algorithm for learned rounding (default: \fBprodigy\fR).
.TP
.BI \-\-num_iter " " N
Optimization iterations per tensor (default: 4000).
.TP
.BI \-\-lr " " LR
Initial learning rate for AdamW/RAdam/Original (default: 1.0).
.TP
.BI \-\-lr_schedule " " {plateau,exponential,adaptive}
Learning rate decay schedule (default: \fBplateau\fR).
.TP
.B \-\-use_speed
Enable Prodigy's \fIuse_speed\fR acceleration parameter.
.TP
.BI \-\-top_p " " P
Cumulative energy proportion of principal components for SVD (default: 0.2).
.TP
.BI \-\-min_k " " K ", " \-\-max_k " " K
Minimum and maximum SVD rank bounds (default: 256 and 1280).
.TP
.B \-\-full_matrix
Compute full SVD instead of low-rank SVD.
.TP
.BI \-\-scale\-optimization " " {fixed,iterative,joint,dualround}
Block scale tuning strategy (default: \fBfixed\fR).
.SS Error Correction LoRA
.TP
.B \-\-extract\-lora
Extract quantization residual error into a separate LoRA adapter file.
.TP
.BI \-\-lora\-rank " " RANK
Rank for extracted LoRA layers (default: 32).
.TP
.BI \-\-lora\-target " " REGEX
Regex matching layers to target for LoRA extraction.
.TP
.BI \-\-lora\-output " " PATH
Output file path for the extracted LoRA adapter (\fI.safetensors\fR).
.SS Model Filters
.B ctq
includes pre-tuned layer exclusion filters for common architecture families:
.TP
.BR \-\-gemma4 ", " \-\-qwen_vlm ", " \-\-t5xxl ", " \-\-wan ", " \-\-hunyuan ", " \-\-ltxv2 ", " \-\-qwen ", " \-\-zimage
Keep architecture-critical layers (embeddings, layernorms, vision towers, cross-attention) in native precision.
.SH EXAMPLES
.TP
.B Standard FP8 quantization for ComfyUI:
.RS
.nf
ctq -i model.safetensors -o model-fp8.safetensors --comfy_quant
.fi
.RE
.TP
.B NVFP4 quantization using learned rounding:
.RS
.nf
ctq -i model.safetensors -o model-nvfp4.safetensors --nvfp4 --scale-optimization joint
.fi
.RE
.TP
.B Fast INT8 conversion with ConvRot outlier mitigation:
.RS
.nf
ctq -i model.safetensors --int8 --convrot --simple -o model-int8.safetensors
.fi
.RE
.TP
.B Quantize with Error Correction LoRA extraction:
.RS
.nf
ctq -i model.safetensors --nvfp4 --extract-lora --lora-rank 32 --lora-output model-ec-lora.safetensors
.fi
.RE
.SH SEE ALSO
.BR hf_to_nvfp4_safetensors (1),
.BR safetensors_to_nvfp4_gguf (1),
.BR heretic (1)