.TH HERETIC 1 "November 2026" "heretic-llm" "User Commands"
.SH NAME
heretic \- automated directional ablation and refusal suppression for large language models
.SH SYNOPSIS
.B heretic
\fB\-\-model\fR \fIMODEL\fR
[\fB\-\-quantization\fR \fIMETHOD\fR]
[\fB\-\-n\-trials\fR \fIN\fR]
[\fB\-\-export\-strategy\fR \fISTRATEGY\fR]
[\fB\-\-save\-directory\fR \fIDIR\fR]
[\fB\-\-upload\-repo\-id\fR \fIREPO\fR]
[\fB\-\-reproduce\fR \fIFILE\fR]
[\fIoptions...\fR]
.SH DESCRIPTION
.B heretic
(\fBheretic-llm\fR) is an automated optimization and directional ablation tool for
modifying and un-aligning Large Language Models (LLMs). It isolates refusal vectors
and behavioral directions across internal residual streams and attention/MLP layers,
optimizing modifications via genetic multi-objective search (Optuna Pareto front analysis).
.PP
The tool balances refusal suppression ("abliteration" / "decensoring") against model capability
preservation by evaluating candidate modifications against built-in language benchmarks
(MMLU, GSM8K, IFEval, AGIEval, BBH, etc.).
.PP
Modified models can be saved to disk as merged safetensors checkpoints or adapter weights,
or uploaded directly to Hugging Face with complete reproducibility logs (\fIreproduce.json\fR).
.SH OPTIONS
.SS Model & Input
.TP
.BI \-\-model " " MODEL
Hugging Face model repository ID (e.g. \fIgoogle/gemma-2-9b-it\fR) or local filesystem path (required).
.TP
.BI \-\-model\-commit " " HASH
Specific Git commit hash of the Hugging Face model repository.
.TP
.BI \-\-quantization " " {NONE,BNB_4BIT}
Quantization mode used when loading model weights into VRAM: \fBNONE\fR (native precision) or \fBBNB_4BIT\fR (4-bit bitsandbytes). Default: \fBNONE\fR.
.TP
.BI \-\-dtypes " " LIST
List of PyTorch data types to attempt when loading tensors (default: \fI['auto', 'float16', 'bfloat16', 'float32']\fR).
.TP
.BI \-\-system\-prompt " " STR
System prompt passed to the model during evaluation (default: \fI"You are a helpful assistant."\fR).
.SS Execution & Memory
.TP
.BI \-\-device\-map " " MAP
Device allocation strategy passed to Hugging Face Accelerate (default: \fBauto\fR).
.TP
.BI \-\-max\-memory " " DICT
Maximum VRAM/RAM limits per device (e.g. \fI'{"0": "22GB", "cpu": "64GB"}'\fR).
.TP
.BR \-\-offload\-outputs\-to\-cpu ", " \-\-no\-offload\-outputs\-to\-cpu
Offload intermediate residual tensors and logits to CPU RAM immediately to reduce peak VRAM usage (default: enabled).
.TP
.BI \-\-batch\-size " " N
Batch size for evaluation sequences. Set to 0 for automatic detection (default: 0).
.TP
.BI \-\-max\-batch\-size " " N
Upper bound during automatic batch size discovery (default: 128).
.TP
.BI \-\-max\-response\-length " " N
Maximum token generation length during prompt scoring.
.SS Optimization & Genetic Search
.TP
.BI \-\-n\-trials " " N
Total number of optimization trials to explore across the Pareto front.
.TP
.BI \-\-n\-startup\-trials " " N
Number of initial random trials before heuristic exploration begins.
.TP
.BI \-\-seed " " SEED
Random seed for reproducibility.
.TP
.BI \-\-study\-checkpoint\-dir " " DIR
Directory where Optuna study progression and intermediate trials are saved.
.TP
.BI \-\-scorers " " JSON
Custom or overriding scoring configurations (loss metrics, refusal detectors, perplexity monitors).
.TP
.BI \-\-modifiers " " JSON
Custom modification operators (directional projection, orthogonalization, rank cuts).
.SS Benchmarks & Evaluation
.TP
.BI \-\-benchmarks " " LIST
List of evaluation benchmarks to verify model integrity. Supported benchmarks include:
\fIagieval\fR, \fIbbh\fR, \fIcommonsense_qa\fR, \fIeq_bench\fR, \fIgsm8k\fR, \fIhellaswag\fR, \fIifeval\fR, \fImmlu\fR, \fImmlu_pro\fR, \fIpiqa\fR, \fIwinogrande\fR.
.TP
.BI \-\-evaluate\-model " " MODEL
Evaluate an existing model relative to the base model instead of running search trials.
.SS Export & Publishing
.TP
.BI \-\-export\-strategy " " {MERGE,ADAPTER}
Model export strategy: \fBMERGE\fR (bake modifications directly into output safetensors weights)
or \fBADAPTER\fR (export as differential weight delta).
.TP
.BI \-\-save\-directory " " DIR
Local directory path where modified safetensors weights and tokenizer files are written.
.TP
.BI \-\-max\-shard\-size " " SIZE
Maximum file size for exported safetensors shards (default: \fI5GB\fR).
.TP
.BI \-\-upload\-repo\-id " " REPO
Hugging Face repository ID to upload the final modified model to.
.TP
.B \-\-upload\-repo\-private
Make the destination Hugging Face repository private.
.TP
.BI \-\-upload\-tags " " LIST
Tags added to the Hugging Face model card (default: \fI['uncensored', 'decensored', 'abliterated']\fR).
.SS Reproduction
.TP
.BI \-\-reproduce " " PATH_OR_URL
Load parameters from an existing \fIreproduce.json\fR file or URL and reproduce the exact model modification.
.TP
.BI \-\-collect\-reproducibles " " DIR
Download all \fIreproduce.json\fR files from public Heretic model repositories on Hugging Face into \fIDIR\fR for archival.
.SH EXAMPLES
.TP
.B Run automated abliteration search on a local model:
.RS
.nf
heretic --model ./meta-llama-3-8b-instruct --n-trials 50 --save-directory ./llama3-abliterated
.fi
.RE
.TP
.B Optimize with 4-bit base weights to reduce VRAM:
.RS
.nf
heretic --model mistralai/Mistral-7B-Instruct-v0.3 --quantization BNB_4BIT --save-directory ./mistral-heretic
.fi
.RE
.TP
.B Reproduce an existing published Heretic model:
.RS
.nf
heretic --model google/gemma-2-9b-it --reproduce https://huggingface.co/owner/repo/raw/main/reproduce.json --save-directory ./gemma2-reproduced
.fi
.RE
.SH SEE ALSO
.BR ctq (1),
.BR hf_to_nvfp4_gguf (1)