Latest commit
History
Folders and files
Repository files navigation
Laplace’s Scissors
Rotate an LLM’s output matrix to compute edits while preserving protected answers.
Rust · Triton · CUDA / HIP · Readout rotation
Quick start · Commands · Implementation and tests · Math and design
Laplace’s Scissors computes rotation edits for an LLM’s original output matrix. probe constructs numerical inputs from model weights and traces directions with weak output discrimination, cancellation across layers, and the associated weight coordinates. diagnose uses cases with expected answer strings to find plane or joint rotations that repair failed answers while protecting successful ones. Each objective and protection condition is checked with actual GPU outputs. Scaling analysis applies the same edit policy across checkpoints to measure deviations from a reference curve and the amount recovered.
flowchart LR A[Original checkpoint] --> B[Observe hidden states] B --> C[Protected readout rotation] C --> D[GPU validation] D --> E[Accepted proposal] E --> F[Generate or export]
Commands
Quick start
Environment
Prepare a Rust toolchain, a Python environment that can import Triton 3.8, and a CUDA or HIP driver. Python provides the Triton compilation bridge. Rust handles model loading, tokenization, execution, edit computation, and result storage.
export LAPLACE_CACHE= " ${XDG_CACHE_HOME :- $HOME / .cache} /laplaces-scissors " export LAPLACE_PYTHON=python3 cargo build --release
.cargo/config.toml places Cargo build artifacts outside the project. LAPLACE_CACHE stores models, kernels, and run results. Select the backend and device index with LAPLACE_BACKEND=auto|cuda|hip and LAPLACE_DEVICE . Set LAPLACE_DRIVER_LIBRARY for the driver library path, or ROCM_PATH / HIP_PATH for the ROCm SDK. The HIP backend also checks the selected Python environment for the rocm-sdk-core package. Device names and execution targets come from the driver.
Fetch a model and prepare inputs
cargo run --release -- fetch Qwen/Qwen3-0.6B cargo run --release -- inspect Qwen/Qwen3-0.6B --out " $LAPLACE_CACHE /graph.json " cargo run --release -- cases --count 12 --seed 7 --out " $LAPLACE_CACHE /cases.jsonl "
Model inputs accept organization/model , organization/model@revision , a Hugging Face model URL, or a local safetensors model directory. Hub inputs are pinned to a commit SHA, with SHA256 recorded for each file. Use --revision to select a revision and HF_TOKEN for authentication.
Check an answer from the original model:
cargo run --release -- generate Qwen/Qwen3-0.6B \ --prompt ' What is 2 + 2? Reply with the number only. '
Case format
Write custom cases as JSONL. expected is the zero-based index of the correct entry in choices . The choices specify the expected answer; the model generates an answer string for prompt . See the example below and the example JSONL .
{ "id" : " sum " , "prompt" : " What is 2 + 2? Reply with the number only. " , "choices" :[ " 3 " , " 4 " , " 5 " ], "expected" : 1 , "oracle" :{ "sum" : 4 }}
The original chat template builds the prompt, whose token prefix stays fixed. The expected answer is encoded separately and followed by EOS to form the target token sequence. The answer body must survive a tokenizer round trip exactly. When the original generation ends with EOS and exactly matches the expected string, every prefix of that generation is protected. For failed cases, each target answer token and EOS must score above every competing token in the vocabulary.
Compute and apply edits
Numerical probes · Answer repair · Checkpoint export · Scaling analysis
Numerical probes · probe
probe takes model weights and an execution graph. Seeded selections of embedding rows form numerical prefixes, a reference vector, and an orthonormal basis. Two vectors with the same norm as the reference vector are injected directly at the decoder input. Responses are compared with Jeffreys divergence, the symmetric KL divergence between output distributions. A weak direction has low output discrimination between equal-sized injections under the recorded numerical input distribution. Model acquisition for this command loads safetensors and configuration files.
cargo run --release -- probe Qwen/Qwen3-0.6B \ --directions 12 --prefix-length 3 --seed 7 --probe-angle 0.05 \ --max-angle-degrees 3 --coordinates 12 --intervention-fraction 0.1 \ --out " $LAPLACE_CACHE /runs/probe " cargo run --release -- probe Qwen/Qwen3-0.6B \ --inputs " $LAPLACE_CACHE /runs/probe/inputs.json " \ --proposal " $LAPLACE_CACHE /runs/probe/proposal.json " \ --out " $LAPLACE_CACHE /runs/probe-replay "
--directions sets the maximum dimension of the search basis constructed from weights. The actual dimension follows the hidden width and the rank of embedding differences. --prefix-length sets the number of numerical vectors before the injection position. --probe-angle is the spherical angle used to construct the two inputs, in radians. The minimum eigenvector of a small response Gram matrix proposes a direction; measurements through the actual decoder compare its divergence with that of the existing basis directions.
Residual branches from the selected pair of runs provide signed layer contributions. The branch with the largest negative contribution supplies --coordinates weight coordinates and attention-head totals. --intervention-fraction controls diagnostic interventions that change each selected coordinate’s magnitude in both directions. Each intervention reruns from the fixed numerical prefix. Signed contributions and measured intervention effects are recorded separately. --coordinates 0 performs layer tracing; --intervention-fraction 0 performs divergence tracing and protected rotation.
The edit is a readout rotation that protects the original output matrix’s row Gram matrix, the midpoint of the two hidden states, and reference and control states. --max-angle-degrees , --max-edit-norm , and --tolerance set bounds on angle, weight change, and protected-output error. Among three candidates within the angle bound, the program saves an edit that increases measured divergence and satisfies the protection tolerance. Candidates apply actual GPU rotation and readout to observed hidden states. --inputs loads saved inputs; protected_hidden can hold additional protected vectors from the same model. Adding --proposal reruns the saved input pair through the full decoder.
Probe outputs are manifest.json , inputs.json , trace.json , transport.json , report.json , status.json , and an accepted proposal.json . Replay also records replay-frames.json . Results include input and model hashes, actual hardware and precision, divergence, layers and weight coordinates, preservation errors, numerical residuals, and execution counts by stage. Proposal version 3 stores the divergence objective and injection-input hash. Reapplication verifies weight and execution-configuration hashes.
Answer repair · diagnose
cargo run --release -- diagnose Qwen/Qwen3-0.6B \ --cases " $LAPLACE_CACHE /cases.jsonl " \ --max-planes 8 --max-angle-degrees 120 --margin 0.05 \ --max-new-tokens 64 --context 2048 --time-budget 20m \ --out " $LAPLACE_CACHE /runs/rotation " cargo run --release -- generate Qwen/Qwen3-0.6B \ --prompt ' What is 2 + 2? Reply with the number only. ' \ --proposal " $LAPLACE_CACHE /runs/rotation/proposal.json "
--max-planes sets the number of rotation planes to investigate. --max-angle-degrees bounds the rotation angle, and --margin sets the minimum logit gap between each target token and every competitor. --tolerance bounds each protected-output change divided by 1 + |original logit| ; its default is 0.00002 . The inequalities include a numerical allowance derived from this tolerance and the observed logit magnitude. For each plane, merged forbidden intervals yield the angle with the smallest weight change. Candidates are checked with actual GPU readout in order of weight change.
After checking plane candidates, a joint response computation adjusts multiple targets together. --joint-steps sets the number of adjustment steps and defaults to 12; setting it to 0 runs the plane candidates. --joint-rank sets the maximum joint rotation basis dimension and defaults to 64. --max-angle-degrees also bounds the joint rotation’s largest principal angle. --max-edit-norm bounds the Frobenius norm of the output-weight change. Acceptance requires the full-vocabulary margin, protected-output tolerance, target answer and EOS, and preservation of originally successful generation paths.
Joint search computes and shares the output basis and cost Gram matrix once. When twice the response count is smaller than the rotation dimension, it solves the Cayley system in that smaller space. Candidate scores use the compressed output basis on the GPU. After search, a rotation completion minimizes weight change while preserving observations. The original GPU readout checks that the completion stays within angle and cost bounds and preserves every prefix’s token choice, satisfied margins, and protected-output tolerance. completion in candidates.json records costs before and after completion and whether it was applied. computation in report.json records compressed-output and original-validation call counts and timings by computation stage.
--max-new-tokens bounds generated tokens, including EOS. --context must accommodate the prompt plus that generation allowance. Omitting --cases generates symbolic problems using --count and --seed . Diagnosis uses the original template with thinking disabled. generate --thinking selects the thinking format.
The time budget starts with model loading after file acquisition and covers computation and attribution. The output directory contains manifest.json , status.json , baseline.json , candidates.json , report.json , and accepted proposal.json and attribution.json files. Reports record the model revision, input token paths, weight dtypes, hardware, measured validation values, and square-root and readout call counts. Status values are complete for completion, partial for budget expiration, and failed for execution errors.
A proposal contains either orthonormal plane vectors and a half-angle coordinate, or an orthonormal joint basis and a small orthogonal matrix. At runtime, the final hidden vector is rotated before multiplication by the original output matrix. In models with shared embeddings, rotation applies at readout calls. Reapplying an answer-repair proposal checks hashes for weights, configuration, and tokenizer files, plus the output-operation contract. generate --proposal reads proposal versions 1, 2, and 3.
Output-row attribution
--attribution-rows sets the number of output rows to report and defaults to 6. For each row, results include high-contribution coordinates, the loss after reverting that row’s edit, the total contribution sum, and integration error. Contributions follow the specified straight-line readout intervention path and retain negative values. Setting the option to 0 saves the generation proposal and validation results.
Checkpoint export · export
export saves a new checkpoint with the proposal applied to the output matrix. It preserves original tensors and writes the edited lm_head.weight in FP32. It unties embedding and readout weights and saves configuration, execution graph, and shard index files. The output matrix uses CPU F64 block multiplication; original-model loading and contract checks use the shared model runtime. Run the exported model using its directory path.
cargo run --release -- export Qwen/Qwen3-1.7B \ --proposal " $LAPLACE_CACHE /edits/proposal.json " \ --out " $LAPLACE_CACHE /exports/qwen3-1.7b-edited " cargo run --release -- generate " $LAPLACE_CACHE /exports/qwen3-1.7b-edited " \ --prompt ' What is 2 + 2? Reply with the number only. '
materialization.json records the original revision and hashes, applied proposal, measured weight-change norm, and FP32 storage rounding error. The configuration’s default loading dtype is FP32, while decoder tensors keep their original storage dtypes. The runtime reads each tensor’s stored dtype.
Scaling analysis · scale
Provide a scaling series as a JSON manifest. The model paths below are relative to the manifest directory. Hugging Face model IDs and revision fields are also supported.
{ "axis" : " training_compute " , "conditions" : " Same model family, training data, and evaluation inputs; vary the recorded resource axis " , "reference" : " Loss-decay exponent estimated on an independent reference interval " , "exponent" : 0.5 , "max_edit_norm" : 10 , "cases" : " repair.jsonl " , "evaluation" : " evaluation.jsonl " , "checkpoints" : [ { "model" : " models/small " , "resource" : 1 }, { "model" : " models/medium " , "resource" : 2 }, { "model" : " models/large " , "resource" : 4 } ] }
cargo run --release -- scale --manifest " $LAPLACE_CACHE /series.json " \ --context 2048 --max-new-tokens 64 --out " $LAPLACE_CACHE /runs/scale "
The fixed exponent and first two checkpoints anchor the power-law curve used to measure deviations at larger scales. List resource values in increasing order. Every checkpoint uses the same max_edit_norm and diagnosis options. The CLI’s --max-edit-norm overrides the manifest budget. cases supplies repair JSONL; evaluation supplies evaluation JSONL with separate prompts. Evaluation loss is the negative log probability of the expected answer, including EOS, divided by the answer’s UTF-8 byte count and then averaged across cases.
scale-report.json stores measured loss curves, recovered deviation, output-weight contributions by checkpoint, and row-reversion effects. Coordinates from different checkpoints are labeled with their model and tensor row and column. Edits use readout rotations after the fixed decoder. The time budget applies to repair and evaluation for each checkpoint.
Implementation and tests
The execution graph supports separate Q/K/V projections, causal grouped-query attention, RMSNorm and LayerNorm, basic RoPE, SiLU/GELU/ReLU-family gated MLPs, and linear readout. Stored weights use F32/F16/BF16, execution states use F32, and plane and inequality computations use F64. Standard dense-decoder configuration builds the graph; laplace.graph.json can specify tensor connections explicitly. Required operations and stored tensor shapes are validated at loading time.
Run checks
cargo fmt --check cargo test export LAPLACE_TEST_CACHE= " $LAPLACE_CACHE /rotation-tests " cargo test --test gpu --test model_rotation --test injection -- --ignored --nocapture --test-threads=1 cargo test --test compiler -- --ignored --nocapture export LAPLACE_TEST_MODEL= " $LAPLACE_CACHE /models/Qwen--Qwen3-0.6B/selected-commit " cargo test --release --test qwen_reference -- --ignored --nocapture
Default tests check open-interval boundaries and singleton solutions, protected rotations, joint responses, contribution sums and scaling deviations, and finite-difference identities for softmax, RMSNorm, and LayerNorm. Full-model GPU tests check agreement between prefix validation and regeneration, original hashes, KV rewind, shared readout, and scaling analysis over three synthetic checkpoints. Numerical-injection tests check agreement between token and direct-input paths including embedding scaling, separation of coordinate interventions, divergence-proposal storage, and full-decoder replay. The compiler test builds CUDA code objects from collected operation contracts. The Qwen test compares the original forward pass with an independent Rust full-context calculation. Run checks appropriate to the changed operations and store results in an external test cache.
Contributing
Use Issues for reproducible bugs and Discussions for questions and research ideas. For execution issues, include the command, model revision, input case, backend, precision, and relevant error output. Review run artifacts before sharing them; they contain input prompts and environment details.
For code changes, follow AGENTS.md , explain the affected computation contract, and run checks for the changed operations. Add focused regression coverage for numerical counterexamples. Keep build artifacts, model files, and execution reports in external cache directories. Pull requests should describe the behavior change and the checks performed.
License
Released under the MIT License . Model checkpoints are distributed by their original publishers under their respective licenses.
See DESIGN.md for computation contracts and core equations, and AGENTS.md for project working rules.
원본 LLM의 출력 행렬을 회전해, 보호할 답변을 유지하며 수정안을 계산합니다.
Rust · Triton · CUDA / HIP · Readout rotation
Laplace’s Scissors는 원본 LLM의 출력 행렬에 적용할 회전 수정안을 계산한다. probe 는 모델 가중치에서 수치 입력을 만들고 출력 구별도가 낮은 방향, 층별 상쇄와 관련 가중치 좌표를 추적한다. diagnose 는 정답 문자열을 가진 사례에서 성공한 답변을 보호하면서 실패한 답변을 복구할 평면 또는 공동 회전을 구한다. 실제 GPU 출력으로 각 목적과 보호 조건을 확인한다. 규모 분석은 여러 checkpoint에 같은 수정 정책을 적용해 기준 곡선의 편차와 회복량을 계산한다.
기능 안내
빠른 시작
실행 환경
Rust toolchain, Triton 3.8을 불러올 수 있는 Python, CUDA 또는 HIP 드라이버를 준비한다. Python은 Triton 컴파일 연결을 담당한다. 모델 적재·tokenizer·실행·수정안 계산·결과 저장은 Rust가 수행한다.
export LAPLACE_CACHE= " ${XDG_CACHE_HOME :- $HOME / .cache} /laplaces-scissors " export LAPLACE_PYTHON=python3 cargo build --release
Cargo 빌드 경로는 .cargo/config.toml 에서 프로젝트 밖으로 지정한다. LAPLACE_CACHE 에는 모델·커널·실행 결과를 저장한다. LAPLACE_BACKEND=auto|cuda|hip , LAPLACE_DEVICE 로 backend와 장치 번호를 선택할 수 있다. 드라이버 경로는 LAPLACE_DRIVER_LIBRARY , ROCm SDK 경로는 ROCM_PATH 또는 HIP_PATH 로 지정한다. HIP backend는 선택한 Python 환경의 rocm-sdk-core 패키지도 조회한다. 장치 이름과 실행 대상은 드라이버에서 읽는다.
모델 확보와 입력 준비
cargo run --release -- fetch Qwen/Qwen3-0.6B cargo run --release -- inspect Qwen/Qwen3-0.6B --out " $LAPLACE_CACHE /graph.json " cargo run --release -- cases --count 12 --seed 7 --out " $LAPLACE_CACHE /cases.jsonl "
모델 입력은 조직/모델 , 조직/모델@revision , Hugging Face 모델 URL 또는 safetensors 모델 폴더다. Hub 입력은 commit SHA로 고정하며 파일별 SHA256을 기록한다. --revision 으로 revision을 지정할 수 있다. 인증은 HF_TOKEN 을 사용한다.
cargo run --release -- generate Qwen/Qwen3-0.6B \ --prompt ' What is 2 + 2? Reply with the number only. '
사례 형식
사용자 사례는 JSONL로 작성한다. expected 는 choices 의 0부터 시작하는 정답 위치다. choices 는 정답 선택용 입력이며, 모델은 prompt 에 대한 답변 문자열을 생성한다. 아래 사례와 예제 JSONL 을 참고한다.
{ "id" : " sum " , "prompt" : " What is 2 + 2? Reply with the number only. " , "choices" :[ " 3 " , " 4 " , " 5 " ], "expected" : 1 , "oracle" :{ "sum" : 4 }}
모델의 원본 chat template로 prompt를 만들고, 그 prompt token 접두사를 고정한다. 정답 문자열을 별도로 인코딩한 뒤 EOS를 붙여 목표 token 열을