143 lines
5.5 KiB
Groff
143 lines
5.5 KiB
Groff
.TH HERETIC 1 "November 2026" "heretic-llm" "User Commands"
|
|
.SH NAME
|
|
heretic \- automated directional ablation and refusal suppression for large language models
|
|
.SH SYNOPSIS
|
|
.B heretic
|
|
\fB\-\-model\fR \fIMODEL\fR
|
|
[\fB\-\-quantization\fR \fIMETHOD\fR]
|
|
[\fB\-\-n\-trials\fR \fIN\fR]
|
|
[\fB\-\-export\-strategy\fR \fISTRATEGY\fR]
|
|
[\fB\-\-save\-directory\fR \fIDIR\fR]
|
|
[\fB\-\-upload\-repo\-id\fR \fIREPO\fR]
|
|
[\fB\-\-reproduce\fR \fIFILE\fR]
|
|
[\fIoptions...\fR]
|
|
.SH DESCRIPTION
|
|
.B heretic
|
|
(\fBheretic-llm\fR) is an automated optimization and directional ablation tool for
|
|
modifying and un-aligning Large Language Models (LLMs). It isolates refusal vectors
|
|
and behavioral directions across internal residual streams and attention/MLP layers,
|
|
optimizing modifications via genetic multi-objective search (Optuna Pareto front analysis).
|
|
.PP
|
|
The tool balances refusal suppression ("abliteration" / "decensoring") against model capability
|
|
preservation by evaluating candidate modifications against built-in language benchmarks
|
|
(MMLU, GSM8K, IFEval, AGIEval, BBH, etc.).
|
|
.PP
|
|
Modified models can be saved to disk as merged safetensors checkpoints or adapter weights,
|
|
or uploaded directly to Hugging Face with complete reproducibility logs (\fIreproduce.json\fR).
|
|
.SH OPTIONS
|
|
.SS Model & Input
|
|
.TP
|
|
.BI \-\-model " " MODEL
|
|
Hugging Face model repository ID (e.g. \fIgoogle/gemma-2-9b-it\fR) or local filesystem path (required).
|
|
.TP
|
|
.BI \-\-model\-commit " " HASH
|
|
Specific Git commit hash of the Hugging Face model repository.
|
|
.TP
|
|
.BI \-\-quantization " " {NONE,BNB_4BIT}
|
|
Quantization mode used when loading model weights into VRAM: \fBNONE\fR (native precision) or \fBBNB_4BIT\fR (4-bit bitsandbytes). Default: \fBNONE\fR.
|
|
.TP
|
|
.BI \-\-dtypes " " LIST
|
|
List of PyTorch data types to attempt when loading tensors (default: \fI['auto', 'float16', 'bfloat16', 'float32']\fR).
|
|
.TP
|
|
.BI \-\-system\-prompt " " STR
|
|
System prompt passed to the model during evaluation (default: \fI"You are a helpful assistant."\fR).
|
|
.SS Execution & Memory
|
|
.TP
|
|
.BI \-\-device\-map " " MAP
|
|
Device allocation strategy passed to Hugging Face Accelerate (default: \fBauto\fR).
|
|
.TP
|
|
.BI \-\-max\-memory " " DICT
|
|
Maximum VRAM/RAM limits per device (e.g. \fI'{"0": "22GB", "cpu": "64GB"}'\fR).
|
|
.TP
|
|
.BR \-\-offload\-outputs\-to\-cpu ", " \-\-no\-offload\-outputs\-to\-cpu
|
|
Offload intermediate residual tensors and logits to CPU RAM immediately to reduce peak VRAM usage (default: enabled).
|
|
.TP
|
|
.BI \-\-batch\-size " " N
|
|
Batch size for evaluation sequences. Set to 0 for automatic detection (default: 0).
|
|
.TP
|
|
.BI \-\-max\-batch\-size " " N
|
|
Upper bound during automatic batch size discovery (default: 128).
|
|
.TP
|
|
.BI \-\-max\-response\-length " " N
|
|
Maximum token generation length during prompt scoring.
|
|
.SS Optimization & Genetic Search
|
|
.TP
|
|
.BI \-\-n\-trials " " N
|
|
Total number of optimization trials to explore across the Pareto front.
|
|
.TP
|
|
.BI \-\-n\-startup\-trials " " N
|
|
Number of initial random trials before heuristic exploration begins.
|
|
.TP
|
|
.BI \-\-seed " " SEED
|
|
Random seed for reproducibility.
|
|
.TP
|
|
.BI \-\-study\-checkpoint\-dir " " DIR
|
|
Directory where Optuna study progression and intermediate trials are saved.
|
|
.TP
|
|
.BI \-\-scorers " " JSON
|
|
Custom or overriding scoring configurations (loss metrics, refusal detectors, perplexity monitors).
|
|
.TP
|
|
.BI \-\-modifiers " " JSON
|
|
Custom modification operators (directional projection, orthogonalization, rank cuts).
|
|
.SS Benchmarks & Evaluation
|
|
.TP
|
|
.BI \-\-benchmarks " " LIST
|
|
List of evaluation benchmarks to verify model integrity. Supported benchmarks include:
|
|
\fIagieval\fR, \fIbbh\fR, \fIcommonsense_qa\fR, \fIeq_bench\fR, \fIgsm8k\fR, \fIhellaswag\fR, \fIifeval\fR, \fImmlu\fR, \fImmlu_pro\fR, \fIpiqa\fR, \fIwinogrande\fR.
|
|
.TP
|
|
.BI \-\-evaluate\-model " " MODEL
|
|
Evaluate an existing model relative to the base model instead of running search trials.
|
|
.SS Export & Publishing
|
|
.TP
|
|
.BI \-\-export\-strategy " " {MERGE,ADAPTER}
|
|
Model export strategy: \fBMERGE\fR (bake modifications directly into output safetensors weights)
|
|
or \fBADAPTER\fR (export as differential weight delta).
|
|
.TP
|
|
.BI \-\-save\-directory " " DIR
|
|
Local directory path where modified safetensors weights and tokenizer files are written.
|
|
.TP
|
|
.BI \-\-max\-shard\-size " " SIZE
|
|
Maximum file size for exported safetensors shards (default: \fI5GB\fR).
|
|
.TP
|
|
.BI \-\-upload\-repo\-id " " REPO
|
|
Hugging Face repository ID to upload the final modified model to.
|
|
.TP
|
|
.B \-\-upload\-repo\-private
|
|
Make the destination Hugging Face repository private.
|
|
.TP
|
|
.BI \-\-upload\-tags " " LIST
|
|
Tags added to the Hugging Face model card (default: \fI['uncensored', 'decensored', 'abliterated']\fR).
|
|
.SS Reproduction
|
|
.TP
|
|
.BI \-\-reproduce " " PATH_OR_URL
|
|
Load parameters from an existing \fIreproduce.json\fR file or URL and reproduce the exact model modification.
|
|
.TP
|
|
.BI \-\-collect\-reproducibles " " DIR
|
|
Download all \fIreproduce.json\fR files from public Heretic model repositories on Hugging Face into \fIDIR\fR for archival.
|
|
.SH EXAMPLES
|
|
.TP
|
|
.B Run automated abliteration search on a local model:
|
|
.RS
|
|
.nf
|
|
heretic --model ./meta-llama-3-8b-instruct --n-trials 50 --save-directory ./llama3-abliterated
|
|
.fi
|
|
.RE
|
|
.TP
|
|
.B Optimize with 4-bit base weights to reduce VRAM:
|
|
.RS
|
|
.nf
|
|
heretic --model mistralai/Mistral-7B-Instruct-v0.3 --quantization BNB_4BIT --save-directory ./mistral-heretic
|
|
.fi
|
|
.RE
|
|
.TP
|
|
.B Reproduce an existing published Heretic model:
|
|
.RS
|
|
.nf
|
|
heretic --model google/gemma-2-9b-it --reproduce https://huggingface.co/owner/repo/raw/main/reproduce.json --save-directory ./gemma2-reproduced
|
|
.fi
|
|
.RE
|
|
.SH SEE ALSO
|
|
.BR ctq (1),
|
|
.BR hf_to_nvfp4_gguf (1)
|
|
|