This post may be useful to you if you:

• Develop, maintain, or test a programming language, especially one targeting smart contracts
• Do security audits on new compilers or runtime environments
• Do security audits of smart contracts and want to include additional checks on compiler correctness that may be an additional attack vector
• Own a smart contract project and want to understand the compiler-level attack surface

The post is organized as follows:

1. Background — scope, introduction of miscompilation errors and oracles used
2. Verified mutations — the executable semantics, what makes a mutation sound, and the hand-written corpus of identities by category
3. Synthesizing mutations — automatically synthesizing sound mutations with Lean feedback
4. Testing harness — program generation and mutation, input generation
5. Findings – some real-world findings found using this approach, including bug bounty findings
6. Conclusion — summary and further work

If you're experienced, here's the whole approach in essence; dive into the interesting parts later:

1. Build a Lean4 eDSL implementing a subset of the target language's semantics.
2. Hand-write textbook identities and prove them sound in that semantics — we will consider arithmetic over the 256-bit word ring, linear/poly MBA, some QF_BV, and ad-hoc identities that hit common cryptographic builtins and peephole optimizers.
3. Use the executable semantics to synthesize more sound mutations (bounded, with ad-hoc filters): generator + bottom-up search + Lean kernel as an oracle — simpler than counterexample-based approaches.
4. Run the fuzzing pipeline, leveraging both differential and metamorphic testing.

It is not that hard to build, and — implemented properly — it does find bugs in corner cases that other fuzz-testing tools miss.

Background​

The previous post targeted compiler crashes and quality issues only.

This part targets more tricky bugs: semantic drifts and miscompilations. This means that the implementation of the smart-contract/program is correct and matches the specification, but because of the bug in the compiler, the actual implementation behaves differently. The targeted bugs may be located in the compiler, corresponding tooling like linkers and the runtime environment.

Miscompilation bugs​

The most popular early article on incorrect compilation is "Reflections on Trusting Trust" by Ken Thompson in 1984 [1]. While the whole idea has been known for years, these bugs are better known nowadays in Web3, where they can easily affect smart contracts with real money.

Miscompilations are invisible to smart-contract auditors – the source code is correct, but the actual implementation diverges because of the bug in the trusted base. Worse, they affect all the contracts uploaded on-chain — any contract containing the specific pattern may misbehave or just imply some interesting behavior.

The only exploited bugs so far are the Vyper non-reentrancy hack ($69.3M) and the Cetus hack ($213M) – not in the compiler but related to unexpected semantics in the math library. Other bugs could have caused losses but were fixed before being exploited, such as the TSTORE-poison codegen bug in solc and the billion-dollar Move verifier bug.

That's why compiler bugs are worth hunting.

Oracles​

The main problem when hunting miscompilations is how to detect when the program miscompiles. When working with ICE, you get the oracle for free: if the compiler panicked, you got a bug. With miscompilations you have to understand how the compiled program should behave and what the correct behavior is.

In fuzzing, this problem is solved by the oracle: a part of the tool that decides whether a program's output is correct. There are different approaches, but we will consider the ones we use: differential testing and metamorphic testing.

Differential testing​

The key idea of differential testing: execute another implementation of SUT with the same inputs and compare the results.

For compiler testing we could use different strategies, to name some:

• Different implementation of the same language for a different platform (e.g. resolc for Polkadot and solc for Ethereum)
• The same compiler with different options (e.g. different optimization passes enabled)
• Different underlying platform (e.g. solc-compiled EVM bytecode may be executed on evmone and revm)
• Emit semantically equivalent code for similar languages and test the execution (e.g. TON supports two languages with nearly identical semantics)
• Create a concrete interpreter for differential testing (common approach used e.g. in Solidity and Vyper)
• Use the previous version of the same compiler to hunt regression errors
• Dump IR after different optimization passes and prove its equivalence using SMT (essentially Alive2 is about this [2]; for EVM there is HEVM, which could be used for the same task)

Differential testing is simple and does not require any elaboration. It is free when you already have a second implementation meant to share the same semantics. But the question is how to generate meaningful and diverse enough programs to find bugs. This is a part of what the metamorphic relation brings to us.

Metamorphic testing​

Metamorphic testing relies on a metamorphic relation (MR): a rule for how changing the input should change the output, so you can catch a bug without knowing the expected result.

Metamorphic testing, then, is testing through MR: run the program on MR-related inputs and flag a bug if the outputs violate the relation. For compiler testing this means: apply some mutations that create a program with the same observable behavior, but with different code.

This solves two problems:

• Program generation – the program generator may be utterly simple; MR does the rest
• The oracle problem – we don't need to know the exact result, only that MR holds

Scope​

We will focus on a simple subset of the actually used Lean eDSL to limit the scope and target arithmetic-only bugs. These naturally trigger optimizer bugs, especially in peephole optimizers and constant folding, and some infrastructure in the compilation pipeline was also affected by the demonstrated findings.

Now consider how we can define semantics and build the mutations over it, because that's the core of the tool.

Verified mutations​

The main component of the tool is a framework in Lean that implements:

• Executable semantics – generic enough to be extended to a specific language's semantics.
• A library of hand-written mutations.

Lean4​

While similar approaches have been implemented in different works, people usually prefer SMT for similar projects.

Lean was chosen because:

• That's a full-blown functional language with typeclasses, polymorphism and meta-programming which enables us to create a reusable framework.
• Lean provides tactics beyond bv_decide which help in proving some mutations.
• Lean usage became less painful with LLMs available: while the core of the tool and the EVM part are hand-written, LLMs enable us to prove some parts of the source code, automating hard work efficiently, and help to scale the existing project to other languages and semantics efficiently.

Lean is really nice as a programming language, and it is usable not only for formal verification but even for simpler projects like this.

Executable semantics​

Consider a heavily simplified version of the executable semantics in Lean. While it looks like a plzoo cosplay, it is enough to demonstrate the main idea of the approach.

inductive Uop | neg
inductive Bop | add | sub | mul
inductive E (n : Nat) where
  | var : Fin n → E n
  | lit : BitVec 256 → E n
  | un  : Uop → E n → E n
  | bin : Bop → E n → E n → E n

def denU : Uop → BitVec 256 → BitVec 256
  | .neg, a => -a
def denB : Bop → BitVec 256 → BitVec 256 → BitVec 256
  | .add, a, b => a + b
  | .sub, a, b => a - b
  | .mul, a, b => a * b

def den {n : Nat} : E n → (Fin n → BitVec 256) → BitVec 256
  | .var i,     ρ => ρ i
  | .lit c,     _ => c
  | .un o a,    ρ => denU o (den a ρ)
  | .bin o a b, ρ => denB o (den a ρ) (den b ρ)

A mutation rewrites a subexpression lhs into rhs. For this total den, one kernel-checked condition makes it sound: den lhs ρ = den rhs ρ for every ρ — that is exactly what ok certifies. The full semantics is partial (division can diverge), where a second condition is also needed: lhs/rhs must mention the same variables. Otherwise x * 0 → 0 breaks when x := 1 / 0 — x * 0 and 0 are value-equal, but dropping x hides the divergence. So, verified here means any divergence we find later is a real miscompile, not a bad rewrite.

The identities considered below are not hard to prove: bv_decide and ring close most, simp and rfl the definitional ones. decide and omega are not for the identities themselves — they handle other cases, like control flow and index bounds.

The SoundMut type​

In code, a mutation is that rewrite and its proof. fn tries to rewrite a node (none if nothing to match for this pattern); ok certifies that if it fires, den is preserved:

structure SoundMut where
  fn : {n : Nat} → E n → Option (E n) -- try to rewrite a node
  ok : ∀ {n} (e e' : E n) (ρ : Fin n → BitVec 256),
         fn e = some e' → den e ρ = den e' ρ -- ...preserving den

fn only looks at one node, but the harness applies it at every subterm of the program — and ok lifts to soundness at each of those positions.

Now consider a couple of hand-written arithmetic mutations to show how it works, starting from the simplest.

Bittwiddling hacks and nat ring operations​

These are simple textbook identities. Each becomes a SoundMut: fn matches the pattern and rewrites it; ok proves den is preserved.

Here is an example:

-- x * 2  ->  x + x   (strength reduction)
def mul2addFn {n} : E n → Option (E n)
  | .bin .mul a (.lit 2) => some (.bin .add a a)
  | _ => none
def mul2add : SoundMut where
  fn := mul2addFn
  ok := by intro n e e' ρ h; unfold mul2addFn at h; split at h
           · injection h with h; subst h; simp only [den, denB]; bv_decide
           · exact absurd h (by simp)

Bit-level identities (two's complement -x = ~x + 1, De Morgan, …) plug in the same way once E is extended with the bitwise operators.

Good sources of such mutations to add are:

• Warren's Hacker's Delight
• TAOCP
• Bit Twiddling Hacks By Sean Eron Anderson

Mixed-boolean arithmetic​

Mixed-boolean arithmetic (MBA) is a class of expressions that mix arithmetic operators (+, -, *) with bitwise ones (&, |, ^, ~) over fixed-width integers. A good introduction to it is the series of blog posts by Justus Polzin.

Typically, MBA is used in the process of obfuscation and deobfuscation. Obfuscation is defined as a process which consists in transforming a program in order to make its analysis difficult and costly, while preserving its observable behavior. Essentially, it does what we need for metamorphic testing – creates identities.

MBA identities are exactly the obfuscated-looking shape a stacked mutant turns into. The linear ones (with constant coefficients) are hand-written and bit-blast cheaply with bv_decide:

-- a + b  ->  (a ^ b) + 2*(a & b)   (addition as xor + carry)
def addXorFn {n} : E n → Option (E n)
  | .bin .add a b => some (.bin .add (.bin .xor a b) (.bin .mul (.bin .and a b) (.lit 2)))
  | _ => none
def addXor : SoundMut where
  fn := addXorFn
  ok := by intro n e e' ρ h; unfold addXorFn at h; split at h
           · injection h with h; subst h; simp only [den, denB]; bv_decide
           · exact absurd h (by simp)

-- a ^ b  ->  (a | b) - (a & b)
def xorOrAndFn {n} : E n → Option (E n)
  | .bin .xor a b => some (.bin .sub (.bin .or a b) (.bin .and a b))
  | _ => none
def xorOrAnd : SoundMut where
  fn := xorOrAndFn
  ok := by intro n e e' ρ h; unfold xorOrAndFn at h; split at h
           · injection h with h; subst h; simp only [den, denB]; bv_decide
           · exact absurd h (by simp)

Polynomial MBA (products of variables) blows up bv_decide — the variable multipliers don't enable bit-blasting — so those are bridged into ring form and proven with ring, mostly via synthesis.

QF_BV​

QF_BV (quantifier-free bit-vector logic) covers comparisons and predicates lowered to bit operations — the part bv_decide settles directly by bit-blasting.

The hand-written corpus implements the standard lowerings; comparisons return a 0/1 word, for example:

-- a <s 0  ->  (a >>> 255) & 1   (sign-bit extract)
def slt0Fn {n} : E n → Option (E n)
  | .bin .slt a (.lit 0) => some (.bin .and (.bin .lshr a (.lit 255)) (.lit 1))
  | _ => none
def slt0 : SoundMut where
  fn := slt0Fn
  ok := by intro n e e' ρ h; unfold slt0Fn at h; split at h
           · injection h with h; subst h; simp only [den, denB]; bv_decide
           · exact absurd h (by simp)

Domain-specific mutations​

Some mutations are implemented based on the systems that were tested and the underlying stack. For example:

• Cryptographic primitives (e.g. EVM-like mulmod which actually got a real-world finding)
• Optimization-specific patterns from Solidity compilers, especially the Yul backend
• LLVM InstCombine — if you catch some bug in LLVM on the way — it may also affect contract compilers, like the known g++ bug causing ICE in the Solidity compiler

An example from the InstCombine corpus — LLVM's own peephole rewrites like this:

-- InstCombine:  x + x  ->  x << 1   (strength reduction)
def addSelfFn {n} : E n → Option (E n)
  | .bin .add a b => if a = b then some (.bin .shl a (.lit 1)) else none
  | _ => none
def addSelf : SoundMut where
  fn := addSelfFn
  ok := by intro n e e' ρ h; unfold addSelfFn at h; split at h
           · split at h
             · rename_i hab; injection h with h; subst h; subst hab
               simp only [den, denB]; bv_decide
             · exact absurd h (by simp)
           · exact absurd h (by simp)

While there are many ideas on how to include common mutations, to have a really exhaustive corpus of mutations, you should generate verified mutations automatically. Consider how.

Synthesizing mutations​

Hand-writing identities does not scale — the useful mutations hide in corner cases that are hard to fully cover by hand. The solution is to have a script generate mutations and let the Lean kernel decide which proposals are real. The enumerator script suggesting mutations is completely untrusted, and the worst it can do is emit a theorem that fails to compile. Only sound mutations are added to the corpus.

The whole idea is bottom-up search:

1. Mirror — a Python copy of the den semantics used as a fast oracle.
2. Enumerate — create terms over a, b, c and a constant menu (integer type boundaries, EC field primes with their sparse coefficients, shift/mask amounts) up to the specified depth, simplest-first.
3. Fingerprint — evaluate each term on a few adversarial valuations (e.g. corner triples that always probe 0 / -1 / intMin / intMax, plus seeded random) and hash the result vector. Terms that land in the same bucket agree on every probe.
4. Conjecture — every collision between two structurally different terms becomes a candidate identity. Agreement on a few dozen points is not a proof; it is a guess delegated to the kernel.

It is simpler than the counter-example-based SMT approaches for synthesis used in some works, but works great and solves its task.

Concretely, the loop is small — here it is against the den defined earlier (the enumerator and emitter cut for brevity):

MOD = 1 << 256
wrap = lambda x: x & (MOD - 1)                 # 256-bit two's complement

# a Python mirror of the (simplified) Lean `den` above
def den(t, env):
    match t:
        case ('var', i):    return env[i]
        case ('lit', c):    return c
        case ('neg', a):    return wrap(-den(a, env))
        case ('add', a, b): return wrap(den(a, env) + den(b, env))
        case ('sub', a, b): return wrap(den(a, env) - den(b, env))
        case ('mul', a, b): return wrap(den(a, env) * den(b, env))

# fingerprint = den on a fixed battery of valuations (corners + seeded random)
VALS = [(0, 0, 0), (1, 1, 1), (MOD - 1, 1 << 255, 0), *seeded_random_triples(40)]
fp = lambda t: tuple(den(t, v) for v in VALS)

# enumerate terms simplest-first; a fingerprint collision is a candidate identity
bank = {}
for t in enumerate_terms(maxdepth=4):
    f = fp(t)
    if f in bank and bank[f] != t:
        emit_lean_theorem(bank[f], t)          # equal on every probe -> conjecture lhs = rhs
    else:
        bank.setdefault(f, t)

Each candidate is routed to the tactic that can actually close it and emitted as a Lean theorem the kernel re-checks:

• bv_decide for bitwise, shift, and comparison goals — bit-blasting, cheap unless a large multiplier is involved.
• ring for linear and field arithmetic, where 256-bit numerals make bit-blasting explode. ring can't see bitwise operators, so polynomial MBA identities first cross one-off bridges proven once by bv_decide — x ||| y = x + y - (x &&& y), x <<< k = x * 2^k, ... — that rewrite the bitwise atoms into +, -, *; then ring finishes.

Comparisons are schema-driven (curated) rather than blind-enumerated, which would just flood the buckets with trivia.

A proven value-fact ∀ ρ, den lhs ρ = den rhs ρ is still only pointwise equality of two fixed patterns. mkSynthMut lifts it into a real SoundMut that fires at any subterm, depth, and substitution:

-- emitted by the synthesizer, then re-checked by the kernel
def slhs_0007 : E 2 := (.un .not (.bin .and (.var 0) (.var 1)))
def srhs_0007 : E 2 := (.bin .or (.un .not (.var 0)) (.un .not (.var 1)))
theorem sval_0007 : ∀ ρ, den slhs_0007 ρ = den srhs_0007 ρ := by
  intro ρ; simp only [slhs_0007, srhs_0007, den, denB, denU, Option.some.injEq]; bv_decide
def m_synth_0007 : SoundMut := mkSynthMut "synth_0007" slhs_0007 srhs_0007 sval_0007 (by decide)

A simple structural matcher proposes a substitution σ. The rewrite fires only if DecidableEq confirms subst σ lhs == e — the matcher proposes, the certificate disposes. From there liftValue carries the value-fact into any context, and an occurrence check (by decide) makes sure a diverging argument aborts both sides the same way. The resulting mutation is the same as a hand-written one: same SoundMut type, same machine-checked guarantee.

Testing harness​

When we have a library of verified mutations, we can generate a program, apply mutations, and run the differential harness with concrete inputs to ensure the SUT and oracle agree on results. Consider each step in more detail.

Program generation​

The program generation logic is absolutely minimal. Typically, the generator creates a couple of pure functions containing a single return, and optionally a couple of constants.

For this seed only one constant and one function were generated. Otherwise, that's just a minimal pure expression which uses inputs to generate some deterministic result. The next step is to apply mutations to it.

Applying mutations​

We choose a random number of mutations to apply, choose mutations, stack them and apply. Here is an example result:

Here four mutations were applied:

• De Morgan: ~((~a0) | (~(~K1))) = a0 & K1
• dead multiply-by-zero terms: ... * int256(0)
• 64-bit mask + shift-split-merge: & 0xffffffffffffffff | (... >> 64 << 64)
• XOR self-cancellation against 2^128+1: ... ^ C ^ C

After this, we compile the original program and the mutant for the target (for example, Revive resolc) and execute both programs on the fuzzing harness.

Running harness​

The fuzzing harness here is a simple wrapper over the runtime environment that deploys the compiled artifact and executes given inputs over it.

The only challenge there is to choose the best inputs that trigger corner-cases. At the same time, we don't want to have lots of inputs, because it slows down the pipeline.

So, when choosing inputs, the harness:

1. Uses a fixed set of combinations likely to break it: 0, 1, -1, and the int256 bounds MIN = -2^255, MAX = 2^255-1.
2. Adds a few inputs picked to cover behaviors of the mutant the fixed set misses — its overflow points, operator corners, etc.
3. Adds a couple of pseudo-random inputs for diversity.

The process is just differential fuzzing with concrete values. It beats other differential tools only in the quality and quantity of mutations applied. Let's look at a few findings it made.

Findings​

We will consider three bugs found just to show what the findings look like and briefly discuss others and projects that were clean as expected.

Polkadot Revive and ZKsync: mulmod miscompilation​

resolc is a Solidity compiler for Polkadot Revive. It compiles Solidity to RISC-V using LLVM, to be executed in pallet-revive on Polkadot parachains. That's a production compiler that produces real-world contracts executed on-chain, and is itself tested by a differential fuzzing test suite.

There are a few bugs found by the tool: some were reported privately, some – publicly, but we will focus on a single bug that affects corner-cases of mulmod. It was reported through the Parity bug bounty program, received a bug bounty for the finding, and was fixed in the 1.2.0 release.

A correct mulmod(a, b, m) always returns a value strictly less than m. Revive routes every mulmod/addmod with m >= 2^128 — all prime-field, elliptic-curve and ZK code, BN254 included — through a hand-written 256-bit long-division helper that is algorithmically wrong: for a class of operands it silently returns a value >= m, with no revert.

Calling check(2**159, 2**256 - (2**54 + 1), 2**255 + 1):

With m = 5789604…819969, revive returns exactly correct + m: the result simply exceeds the modulus. Over the BN254 scalar field the same defect instead keeps the result < m but with the wrong residue, so it propagates silently through later field arithmetic — which is how it breaks a Bulletproofs inner-product argument when both mulmod operands are attacker-supplied.

ZKsync: the same bug​

The interesting story about the implementation is that it is based on the ZKsync Era LLVM compiler, which has had the same bug for years.

It affects some live contracts, including the ZKsync ecosystem and some dependent projects. But either it is benign, or the affected contracts already have zero TVL – emerging crypto ecosystems move fast, and this bug cannot have any impact there anymore and likely won't be fixed.

Solang: Constant fold << miscompile​

Solang's sema-stage constant folder evaluates <</>> with unbounded precision and narrows to the type width only at the end of the expression, while solc masks after every step. So (uint16(0xffff) << 8) >> 8 folds to 65535 instead of 255: when such a constant gates a check, a require that passes on solc/EVM silently reverts on Solang — no diagnostic, every target affected (Polkadot, Solana, Stellar).

Additionally, the tool highlighted a number of ICEs, including interesting findings like allocation of 10 TiB RAM on compilation, which were not found in the coverage-guided fuzzing campaign.

TON: Miscompile in the shape of a 20-year-old JDK bug​

TON languages – FunC and its new version Tolk – are a goldmine of bugs. Two reasons for that:

1. Overengineered runtime – the underlying TVM contains a rich set of instructions – complicated and redundant.
2. Peephole optimizers – Both compilers try to optimize the generated bytecode because of (1), so they have to deal with many corner cases specific to the platform — that's the main source of bugs (as well as constant folding).

That's why compiler bugs are not in the scope of the bug bounty and even the most creepy bugs take months to be fixed.

The full-blown integration of the Lean framework would be overkill for such a target. It was enough to call an LLM to port some hand-written identities from Lean to Python/z3. This found 19 bugs in total – 13 miscompilations and 6 ICEs – in 2 hours of execution.

An interesting finding among these is the miscompile in the shape of a 20-year-old JDK bug in the TON constant folder:

Otherwise, the compiler changes control flow, miscompiles in constant folding – the corner cases are unhandled.

Here is the whole 1000-line script reproducing some trivial identities that did the job: github.com/jubnzv/tolk-less.

Other findings​

The tool found 15 miscompilations across different projects (not including TON). Some were reported privately and are accepted, still on triage, or ghosted; some were reported publicly and mentioned alongside the blogpost as a demonstration.

But it is not possible to find bugs where there are no bugs. The original Solidity compiler solc did not miscompile on the corpus of identities. The tool did not find anything in the Vyper stable backend, but there were findings like vyper#5012 in the experimental backend, which is not production-ready.

And it would be strange to find anything in audited-to-death projects — in the end these run the multi-billion-dollar infrastructure and should not have such errors.

Conclusion​

The post described practical approaches to hunting miscompilations in production compilers — bugs often invisible to simpler program generators and fuzzing tools.

The choice of Lean as an implementation language is justified by language features that let us prove some identities more conveniently, support identities beyond SMT capabilities, and create an extensible framework scalable to other compilers.

While the differential and metamorphic testing approaches are neither complicated nor exhaustive, they become quite efficient when carefully crafted. 1600+ verified mutations stacked together do the job, reaching the most subtle corner cases.

The hand-written mutations library enables us to write rules that trigger specific compiler cases and scale to other platforms when applicable. Mutation synthesis with filters and bounds lets us extend the library for free, using only the executable semantics.

Some real-world findings were demonstrated to prove the feasibility of the approach implemented that way.

Further work​

We did not consider high-level language constructs such as control flow, storage operations on mappings/arrays/structs, and anything beyond arithmetic and constants, to keep the scope minimal. Other things like overflow semantics and type widths are generalized by the tool leveraging Lean polymorphism.

There are a couple of approaches to program generation and mutation which work well with the current work:

1. Some control flow semantics could be integrated into the described pipeline (with known limitations). The easy wins are core Lean's ite and let-bindings; others are Moggi monads to express statement-level mutation.
2. The typical approach is a bottom-up type-driven generator similar to one used in ghc testing [3]. It could be adapted for imperative languages as well, typically using Arbitrary-like functionality for AST/IR generation. Verified mutations could be applied to generated programs.
3. Dead [4] and live [5] EMI mutations – while these mostly challenge bugs in optimizers, which are typically intentionally not very aggressive in smart-contract compilers, the technique may improve the overall coverage.
4. Some identities and techniques described in obfuscation literature, beyond MBA, are a good source of ideas for new mutations. The idea is the same – keep the observable behavior of the program. Eyrolles' thesis [6] gives a good overview to start with.

References​

1. Thompson – Reflections on Trusting Trust (1984)
2. Lopes et al. – Alive2: Bounded Translation Validation for LLVM (2021)
3. Pałka et al. – Testing an optimising compiler by generating random lambda terms (2011)
4. Le et al. – Compiler Validation via Equivalence Modulo Inputs (2014)
5. Sun et al. – Finding Compiler Bugs via Live Code Mutation (2016)
6. Eyrolles – Obfuscation with Mixed Boolean-Arithmetic Expressions: reconstruction, analysis and simplification tools (2017)

This post may be useful to you if you:

• Develop, maintain, or test a programming language, especially one targeting smart contracts
• Do structure-aware fuzzing against real-world targets

The post is organized as follows:

1. Background — related work, existing approaches, and what makes small-language fuzzing different from C/C++
2. Fuzzing harness and configuration — harness design, fuzzer orchestration, tuning for compiler targets
3. Custom mutators — leveraging LLMs and tree-sitter grammars in AFL++ mutators
4. Corpus and dictionaries — corpus collection, minimization, dictionary construction
5. Triage workflow — deduplication, minimization assisted by tools and LLM, and report filing
6. Evaluation — all targets, all results, consolidated
7. Conclusion and further work — summarizes the post, notes what comes next, lists the published tools

Background​

Fuzzing is one of the approaches for finding bugs in compilers. While it does not provide correctness guarantees, it enables you to uncover hidden bugs by generating corner cases that users rarely trigger. Compilers are particularly good targets – they process complex structured input through multiple transformation passes with internal invariants and assumptions.

In the simplest case, the goal is to find compiler crashes – internal compiler errors (ICE). This is easy, because you don't have to write a fuzzing oracle – just execute the compilation pipeline on fuzz data and collect crashes. This post focuses only on ICE; other kinds of errors will be covered in the later part.

The issues found that way have a low risk for end users – these bugs may crash the tooling (e.g. analyzers, LSP) or the compiler itself, preventing the user from writing planned code and messing up the development process. They don't affect the running program.

The standard fuzzing technique when source code is available is coverage-guided fuzzing. Popular fuzzers operate at byte and bit level — but compilers accept structured input. Pushing random bytes will only hit lexer/parser errors and is far too ineffective to reach later passes. That's why grammar-aware fuzzing exists.

Key idea of grammar-aware fuzzing: generate syntactically correct programs that likely pass the lexer/parser and hit internals of the compiler. This way, we challenge later passes like the typechecker, semantic analysis, and codegen – trying to violate some invariants and assumptions the compiler developers made.

While challenging the lexer/parser is easy, it was intentionally skipped for all the compilers. In small teams and small languages, nobody really cares if input containing 5000 sequential ( symbols will crash the parser. This kind of issue is very common and could be easily found, but it is not worth the time to report or fix, because no sane user will ever write code like this.

Most existing research on grammar-aware compiler fuzzing targets C/C++ compilers. Some of these approaches transfer to smart-contract languages, some do not. Here is what makes these targets different:

• Few optimization passes – smart-contract languages focus on correctness, not runtime speed, and are developed by small teams. Program generators (e.g. CSmith or YARPGen) or EMI mutators (like Hermes [7] or XDead [8]) that target miscompilations from aggressive optimizations are of limited use here.
• Simpler execution environments – smart-contract languages target smart-contracts, not general-purpose computing. This means fewer codegen paths and a simpler runtime, which limits the surface for deep codegen bugs.
• Rust as implementation language – many of these compilers are written in Rust, which determines the tooling (cargo-fuzz, AFL++ Rust bindings) and the crash patterns we target: panics, unprotected unwraps, index-out-of-bounds.
• Low popularity – fewer real-world examples are available, which limits corpus collection and approaches that rely on injecting existing code snippets into the fuzzing process [1].
• Often poor documentation – approaches leveraging language documentation or specification [3] are limited, though they work when teams explicitly care about good docs.
• Tree-sitter grammars available – smart-contract languages typically ship tree-sitter grammars for tooling (IDE extensions, syntax highlighting), while ANTLR4 grammars are rare. This makes tools leveraging tree-sitter work well out of the box.

The fuzzing campaign for ICE requires the following parts to be implemented:

• Fuzzing harness – executes the compilation pipeline on fuzz inputs and collects crashes. Needed to run fuzzers in persistent mode and filter out benign panics like stack overflows from parser bugs.
• Custom mutators – implement grammar-aware mutation rules on top of AFL++. Default byte-level mutators can't generate structurally valid programs, so custom mutators are what actually get past the parser.
• Corpus – a collection of seed programs that mutations are derived from. All grammar-aware mutators operate on these inputs, so corpus quality directly determines mutation quality.
• Fuzzing dictionaries – lists of language-specific tokens fed to default mutators (if used). Help byte-level mutations produce valid-looking fragments instead of pure noise.

Fuzzing harness and configuration​

A fuzzing harness is a program that sets up fuzzers in persistent mode to receive and process fuzz inputs looking for crashes. Additionally, it sorts out benign panics, like stack overflows typically caused by lexer/parser bugs.

The main fuzzer used is AFL++. It is the most mature, provides an API to write custom mutators, and has the best configuration options. Meanwhile, honggfuzz and libFuzzer use different mutation algorithms, which increases coverage when combined with AFL++.

In some campaigns, honggfuzz and libFuzzer were executed in a single thread and were supplementary; the main work was done by AFL++.

While AFL++ provides an option to sync with foreign fuzzers, you'll still need to implement different harness binaries for each fuzzer.

multifuzz: unified orchestration​

To simplify configuration and orchestration of multiple fuzzers, a lightweight orchestrator called multifuzz was implemented. It solves three tasks:

• Unified Rust API to configure all three fuzzers in a single config
• Explicit configuration for all the fuzzers – all env variables and fuzzer arguments must be described explicitly in the config, zero hidden options
• CLI to manage individual fuzzing instances: start, stop, restart

Overall, it adds a zero-overhead configuration layer that sets up a single harness for all three fuzzers and manages them at runtime. Everything is 100% explicit – the tool does not introduce any fancy defaults, so you have to read the documentation.

Here is an example configuration used to fuzz the Sui Move compiler that shows how multiple workers with different options may be configured.

It is optional. Alternatively, you could achieve the same results writing a Makefile or custom scripts and/or running tmux sessions for each fuzzer worker.

Fuzzers configuration​

To achieve the best fuzzer performance for grammar-aware testing, the following options were used:

• Selective instrumentation – used to focus fuzzing on specific places in the source code, like recently added features in the compiler. The approach used AFL++ partial instrumentation and is well described in [2] and the documentation.
• No complex byte level mutators were used – cmp-log (or redqueen [6]), fuzzers involving symbolic execution, and Angora (which uses taint traces from inputs) were all skipped, since they were designed to target bit/byte-level mutations. For grammar-aware fuzzing this does not give much benefit, and considering the execution overhead, it slows down the fuzzer.
• Timeouts – compilers are slow, and some mutations may generate code that increases compilation time, e.g. by hitting constant evaluation or generating many entries. The timeout was typically set around 1000ms – enough to keep the corpus clean and avoid cluttering it with useless inputs.
• Memory limits – some targets eat RAM; a special case is Cairo, which uses Salsa – an incremental computation library with its own cache. Other projects may also consume a lot of memory when dealing with large generated inputs. The -m option is required.

Otherwise, the fuzzing process relies mostly on custom mutators. Fuzzer configuration follows the AFL++ documentation.

Custom mutators​

Recent versions of AFL++ provide Rust API bindings to write custom mutators, which simplify development — smart-contract languages are often written in Rust, so you can trigger their internals (e.g. parser, AST) directly from the custom mutator.

Ad-hoc custom mutator​

The first attempt was simple: after reading the experiment conducted by Alex Groce [13], the idea was to create a Move-specific AFL++ custom mutator. The result is a small mutator written in C that swaps common language symbols (e.g. { and [), replaces and deletes code blocks, and provides some Move-specific mutations. It uses the custom mutator API and does not fork AFL++.

The problems with this approach:

1. To be generic enough to target all C-style-syntax compilers, it has to sacrifice language-specific patterns
2. It relies heavily on a good corpus
3. It is too focused on havoc-style mutations without respecting program structure

afl-ts: Tree-sitter based AFL++ mutator​

Instead of mutating bytes, we could mutate the AST directly. Tree-sitter grammars give you typed nodes to swap, delete, and splice — preserving program structure.

A similar tool and approach already exist in the Rust ecosystem: tree-splicer is used by tree-crasher and ice-maker to find ICEs in the rustc compiler. Similar mutation algorithms are applied in multiple grammar-aware fuzzing projects, which typically use ANTLR4 grammars, uncommon among smart-contract languages. However, tree-splicer is a standalone tool, not an AFL++ custom mutator.

afl-ts mutator integrates grammar-aware mutations into AFL++ as a custom mutator. It is fully configurable via environment variables and works with any tree-sitter grammar built with modern tree-sitter. Instead of tweaking the mutator to add the language as needed in tree-splicer, the user just points to the grammar shared library via the TS_GRAMMAR env variable; the language function name can be set via TS_LANG_FUNC, but the tool can usually deduce it from the grammar filename.

Here is the complete table of mutations it conducts:

Weights represent the probability of each mutation being applied.

Typically, corpus files grow a little when using ts-ins a lot, but not significantly, because the addition must add some coverage to be kept by AFL++.

This mutator alone found lots of bugs; most of the Solang and Solidity findings came from it.

The quality of the tree-sitter grammar matters — grammars producing too many ERROR nodes on valid input degrade mutation quality. Here are the grammars used:

MetaMut-style mutators​

Beyond tree-sitter mutations, we want language-specific operations that test semantic and codegen passes — without hand-writing them. MetaMut solves this.

The MetaMut paper [5] describes an approach to generating language-specific mutations using LLMs. While the experiment in the paper focused on C and C++ compilers, it can be applied to Rust-based smart-contract languages as well.

We will consider the MetaMut-style mutator developed for Sui Move: MetaMove. While the approach is applicable to other languages, we will focus on Move, which contains 884 unique mutators plus all the scripts needed to demonstrate the approach. While the core idea is similar to MetaMut, the implementation differs in several ways.

From the implementation perspective, it consists of these components:

1. Rust mutator library – a small Rust library that lets the model create custom mutators using the compiler's AST without reading the whole compiler codebase on each step. It contains a simplified AST and some logic to call custom mutators.
2. Script to invent new mutators – combines mutating operations (swap, toggle, ...) with all available AST elements, generates descriptions of how each mutation should work, and saves the results.
3. Script to implement new mutators – takes the descriptions generated by the previous script and the AST from the library, and calls the model to generate mutations with compilation feedback.
4. Script to verify the generated mutators – checks whether all of them can be applied and whether they generate syntactically valid code.

The experiment used Sonnet 4.6 to invent and generate the mutators.

Consider the implementation and differences from the original approach in greater detail.

Rust mutator library​

The library wraps the Move parser, walks the AST, collecting target categories (expressions, if/match/loop, let bindings, function calls, etc.), and exposes a single MuAstContext that each mutator operates on. Each generated mutator implements a MoveMutator trait with four methods: name(), description(), needs() returning a bitmask of required AST targets, and mutate() that edits the source via byte-offset rewriting — no AST-to-source serializer needed.

Pre-filtering by needs() is the key efficiency trick: the fuzz loop computes the available target kinds once per input, and only mutators whose needs() overlap with those kinds get invoked. A minimal example:

impl MoveMutator for SwapBinOp {
    fn name(&self) -> &'static str { "SwapBinOp" }
    fn description(&self) -> &'static str { "Swap a binary operator with a compatible one" }
    fn needs(&self) -> u32 { TK_BINOP }
    fn mutate(&self, ctx: &mut MuAstContext) -> bool {
        let binop = ctx.pick_random_binop()?;
        let replacement = ctx.compatible_op(binop.kind);
        ctx.replace_text(binop.loc, replacement);
        true
    }
}

Inventing mutators with LLM​

The invent phase produces (Name, Description) pairs — each named {Action}{Structure} (e.g. SwapBinOp, ToggleMutability) — that feed the implementation phase. The prompt combines two catalogs: 15 generic actions from the paper plus Move-specific AST structures (BinOp, Match, Ability, Visibility, ModuleDef, etc.).

Mutation actions:

swap      — Replace one element with a compatible alternative
remove    — Delete an element from the program
add       — Insert a new element into the program
duplicate — Copy an element and insert the copy nearby
negate    — Invert or negate an element's meaning
modify    — Change an element's value or property
inline    — Replace a reference with the thing it refers to
wrap      — Surround an element with a new construct
unwrap    — Remove a surrounding construct, keeping inner content
reorder   — Change the order of sibling elements
lift      — Move an element to an outer/higher scope
sink      — Move an element to an inner/lower scope
split     — Break one element into two separate ones
merge     — Combine two elements into one
toggle    — Flip a boolean-like property on/off

These are generic enough to apply to smart-contract languages as-is. The prompt explicitly asks for syntactically valid mutations only — anything that fails to parse is rejected later in the Validating phase.

The LLM occasionally hallucinates impossible combinations (e.g. negate ModuleDef) and invents descriptions to fit. That's fine for fuzzing: the mutator still changes program structure and opens new paths, and the Validating phase catches anything that doesn't actually modify code or produces invalid syntax. Here's how negate ModuleDef got interpreted:

"Find two module NAME { ... } declarations in the same file and swap their identifiers, breaking fully-qualified callers and exercising the resolver's duplicate-symbol / shadowing paths."

Differences from the paper:

• Batched generation — 8 operations per target per prompt, saves tokens
• Caching and skip logic for batches that failed in the first iteration
• A configuration option to prioritize specific target structures (e.g. recently-added enum/match for Sui Move)

Implementing mutators​

The implement phase turns each (Name, Description) pair into a compiled Rust mutator registered in the driver. The prompt contains the μAST API reference and a reference implementation (SwapBinOp). The LLM returns Rust code as text — it has no filesystem access. The generation script writes each response to src/mutators/{name}.rs, runs cargo check, and on failure sends the code plus compiler error back for a refinement pass (up to 10 rounds). Mutators that still don't compile are dropped.

Mutator quality varies — many are simple, some are hallucinated. That's fine at scale: each target project has 700–1000 combination ideas to invent mutators, and the Validating phase filters the ones that don't actually modify code or generate garbage. About 7% of mutators needed manual fixes after validation to become useful.

Some mutators are primitive but effective. WrapExpressionStmt, generated for Leo, found 4 ICEs despite its simplicity:

//! WrapExpressionStmt: Wrap an expression statement in an assert or call.
impl LeoMutator for WrapExpressionStmt {
    fn name(&self) -> &'static str { "WrapExpressionStmt" }

    // Generated by the model in the Invent phase
    fn description(&self) -> &'static str {
        "Wraps an expression statement in an assert or redundant call to \
         test type-checking and circuit generation on nested expressions"
    }

    fn mutate(&self, ctx: &mut MuAstContext) -> bool {
        // ... pick an expression from one of the statements if available

        // Wrapping the expression found
        let wrapped = match ctx.rand_index(10) {
            0 => format!("assert_eq({}, {});", expr, expr),
            1 => format!("assert_neq({}, 0u32);", expr),
            2 => format!("assert({} == {});", expr, expr),
            3 => format!("let {} = {};", ctx.generate_unique_name("_w"), expr),
        };
        ctx.replace_text(target, &wrapped);
        true
    }
}

Validating mutators​

The implement phase only guarantees compilation — not useful behavior. The validation script runs each registered mutator against clean source files from the corpus and classifies the output via move-check:

1. Sample N compilable files from the corpus (no parser/lexer errors in baseline)
2. Apply each mutator to K files with different seeds (parallel workers)
3. Classify resulting errors by move-check category:
   • category 1 (parser/lexer) → invalid syntax, mutator rejected
   • categories 2-4 (name resolution, unbound variables, type errors) → acceptable, these are exactly the passes we want to test
4. Flag mutators that never apply (always no-op, wasting CPU) — an example: a generated top-level-declaration mutator that looked for use among function statements

The script also highlights gaps in the corpus – if a mutation never applies, the needed construction is likely missing from the corpus.

Conclusion​

Having hundreds of LLM-generated mutators challenging semantic and codegen passes almost for free is a big win for compiler fuzzing — it opens new paths and increases coverage without hand-writing a program generator or spending time on custom coverage-guided tooling.

It works best when combined with other grammar-aware mutators like the tree-sitter splice mutator, which adds randomness and uncovers more subtle cases.

Built-in AFL++ mutators​

AFL++ ships with several custom mutators in its distribution. Multiple mutators can be stacked via AFL_CUSTOM_MUTATOR_LIBRARY — different mutation algorithms hit different paths, so combining a grammar-aware mutators with byte-level alternatives increases overall coverage.

autotokens​

A grammar-free token fuzzer that splits input into tokens and shuffles them with different strategies. It learns its token pool from the -x dictionary and CMPLOG, and mutates below the grammar level. Useful as a lightweight complement to afl-ts — it picks up on tokens the grammar-aware mutator does not know (e.g. identifiers present in the corpus but not captured in the AST).

radamsa​

radamsa is a general-purpose byte-level fuzzer with several strategies that transfer to compiler fuzzing:

• sed-tree-stutter — generates deeply nested expressions (e.g. f(g(h(f(g(h(f(g(h(x))))))))))), often crashing parser stack depth and occasionally triggering typechecker errors. afl-ts implements the same strategy at the grammar level; radamsa operates at byte level and is more aggressive.
• rand-as-count — appends large A-strings, useful for hitting integral-type boundaries and array length checks
• Byte-level glyph injection — adds "interesting" symbols (unicode glyphs, control bytes) that crash the lexer/parser. Lexer/parser bugs are out of scope here, but the strategy may be useful if you target them.
• Boundary literal injection — swaps numeric values with edge cases (0, MAX, negative, large numbers) to stress integer overflow paths. afl-ts's ts-lit already does this at the grammar level.

Caveats:

• Most radamsa output is parser/lexer noise that should be filtered during triage
• Very large inputs slow the harness (e.g. constant evaluation on huge numbers)

Not the primary mutator, but running it on one worker adds corpus diversity.

Other grammar-aware fuzzers​

There are other open-source fuzzers and custom mutators that may be used to improve the fuzzing campaign. Some of them are integrated to AFL++ while some could be used as an external fuzzers (AFL++ -F flag).

Some fuzzers from papers and open-source projects represent ideas that overlap with the approaches used here. They are mentioned because they may be useful if you are writing your own tooling, or they may be more suitable for the language you are targeting:

• ATNwalk – provides grammar-aware mutations and has built-in AFL++ integration, but is not convenient to use since it requires a quality ANTLR4 grammar.
• Gramatron – grammar-aware fuzzer that operates on grammar automata, which was used for fuzzing an experimental language for the TON blockchain; the main issue was the automaton generation algorithm. Grammars can be generated using this script or manually, but they must be very minimal. This is acceptable for dynamically typed languages like JavaScript as described in the paper [9], but for statically typed blockchain languages the grammars blow up the fixpoint algorithm that generates the automaton.
• Fuzz4All – uses LLM-based generation to fuzz compilers [12]. A good option to extend the corpus or run separately alongside the coverage-guided fuzzer.
• Kitten – an ANTLR4-based program generator that recently found 328 bugs in common compilers. It uses grammar-aware mutations similar to tree-splicer or afl-ts, with additional strategies like rarity-weighted target selection, kleene-targeted mutations, and top-down grammar generation powered by ANTLR4 [10].
• IssueMut – the same idea as MetaMut, but previous findings are used as a source of mutations [11].

These are interesting sources of related mutation strategies that may be used to improve the tooling.

Corpus and dictionaries​

The corpus feeds the mutators. Grammar-aware mutators like afl-ts and MetaMut-style splice, swap, and delete subtrees from corpus entries. Byte-level mutators like radamsa and autotokens extract tokens from the same files. Corpus quality directly determines mutation quality — mutators can only produce what they can see.

A good corpus is small, diverse, and covers a broad surface of the language. Small because havoc and custom splice-style mutators run faster on small files, and oversized entries slow the whole campaign. Diverse because grammar-aware mutators only splice what's present in the corpus — missing language constructs stay unreachable. The tension between "small" and "diverse" is resolved by minimization: collect broadly, then trim to the smallest set that still covers the same paths (discussed in the next subsection).

Collecting corpus files​

The most straightforward way to seed the corpus is to collect source files somehwere, remove large and slow inputs and minimize the corpus.

To collect the initial corpus you could start with:

• compiler's test suite and examples
• projects on GitHub
• datasets, e.g. verifier/scan projects often provides information, there are decompiled sui-packages for Sui Move or Zellic dataset of Ethereum contracts

The initial metric to evaluate the corpus coverage is AFL++ stats and code coverage you could get with llvm-cov/gcov.

tsgen: tree-sitter-based generation​

Sometimes you get low coverage even after seeding with existing open-source code for the compiler. This happens when the language is powerful enough that not all of its features are actively used, or when new features have just been introduced.

This was the case for Cairo fuzzing. To cover the gaps, a small utility was created: tsgen.

It generates a seed corpus directly from a tree-sitter grammar.json. The generator walks the grammar recursively — at each CHOICE node it picks an alternative, at each REPEAT it picks a count, and at each terminal it samples from a dictionary (optionally augmented by identifiers and literals harvested from real source files). A min-depth pre-pass prevents infinite recursion through self-referential rules (expression → binary_op → expression → ...), and generated programs are validated with the compiled parser to drop anything that doesn't parse.

After generating the corpus, run afl-cmin on it. Practical results: 150k generated Solidity files reduced to ~1300 unique seeds under 1024 bytes each; for the Cairo grammar it was ~700 seeds. That's a lot of seeds for free — the process takes less time than exploring the corpus from the ground up with grammar-aware mutators, and harvesting identifiers from real source files gives better diversity.

LLM-based seed generation with coverage feedback​

Another option: generate seeds with an LLM.

Obvious starting points:

• Explore previous findings for the repo and ask the LLM to generate seeds based on them.
• Explore typically bug-prone code based on experience — optimizations, code generation, IR transformations — and ask the LLM to generate code that triggers specific places: constant folding, pattern matching compilation, etc.
• Explore documentation and specification; generate seeds targeting rarely-used or tricky constructions.
• Use git blame on reachable code paths to generate seeds targeting recently introduced changes.

This works well when you are just starting the campaign and seeding the corpus. After that, its usefulness is limited, because grammar-aware fuzzers will hit most of the paths anyway.

To avoid wasting time and tokens on already-covered paths, involve code coverage: look at which paths are not yet triggered and write a simple script that asks the LLM to generate seeds for specific gaps, then check coverage again in a feedback loop.

While code coverage is not a good metric, it at least lets you make sure your corpus doesn't have complete gaps.

Additionally, approaches like WhiteFox[3] that leverage language documentation for fuzzing were successfully applied for TON. But this requires good documentation and does not scale well when testing multiple compilers.

Another idea that worked: compile your corpus and execute what gets compiled. This reveals bytecode opcodes not covered by the corpus. For example, for Sui Move there were a few extremely rare opcodes related to pattern matching that were absent from 400k decompiled contracts and from the initial corpus, but got covered later this way.

Identifier renaming with tree-sitter​

A problem the fuzzer encounters when working with a generated corpus and grammar-aware mutations is the rate of semantic errors. Mutations often shuffle identifiers and code structure, producing lots of "undeclared variable" errors that do not let the fuzzer open new paths.

The solution: write a script that renames all identifiers to deterministic names and saves the result to a separate renamed corpus. Here is a 50-line script doing this with tree-sitter-solidity.

The simplest approach that works: name all identifiers with a uniform pattern, e.g. v0, v1, ... Save this corpus and let afl-ts (in particular the ts-bank mutation) find the errors.

The approach is straightforward and a similar one is used in the generation routine of CodeAlchemist — a program generator for fuzzing JavaScript engines [4] — for the same purpose.

Additionally, you may want to seed stdlib identifiers or language keywords to challenge the semantic passes.

Dictionaries​

A fuzzing dictionary is a list of tokens that the fuzzer inserts into inputs during mutations. All three fuzzers support them: AFL++ via -x, honggfuzz via --dict, and libFuzzer via -dict. A custom dictionary for the language must be used if you run AFL++ without AFL_CUSTOM_MUTATOR_ONLY = 1 — this enables the havoc pass to add meaningful constructions to the code.

Ideas for initial dictionary setup:

• Language's grammar or parser implementation
• Common patterns from documentation and examples
• Names of standard functions and language elements (e.g. possible modifiers or values of pragma)
• {, [ and similar symbols – these often give interesting results
• Constructions involved in previous crashes (find in regression unit tests and/or GitHub search for previous ICEs)
• AFL_LLVM_DICT2FILE — auto-extracts string comparisons from the target at compile time. Useful as a supplement, but for compiler fuzzing a hand-crafted dictionary from the language grammar is more effective

Focus on keeping entries atomic; avoid long constructions. For example, instead of let a = &mut x add &mut and let separately – havoc and grammar-aware mutators will figure it out by combining them with existing identifiers and operations.

After running the corpus for a while, it is a good idea to:

• Check coverage – typically you can find operators/constructions that are hit most rarely – add them to the dictionary
• Include constructions from findings made by the fuzzer

Triage workflow​

The typical output of a fuzzer is a number of crash and hang (timeout) files — usually big files with lots of irrelevant garbage. Additionally, some crashes are duplicated despite the AFL++ deduplication mechanism, because the same bug may be caused by different syntactic constructions leading to different triggering paths. The goal of triaging is to remove duplicated crashes first, and then minimize the remaining files to a minimal reproducible example (MRE) to report.

The suggested approach:

1. Deduplicate crashes with a triage script that analyzes the backtrace of crash/hang callsites
2. Minimize the results — manually, using tooling, or with LLMs
3. Report filing

Each stage is described below.

Deduplication​

A long campaign produces hundreds of crash files for a handful of underlying bugs. Most of them are duplicates of the same panic triggered by different inputs, plus a tail of "benign" panics (stack overflows from parser bugs, intentional TODO errors, etc.) that should not be reported. A short script handles the filtering and grouping.

The algorithm:

1. Collect crash inputs from all AFL++ output dirs across workers
2. Replay each crash against the harness with backtrace enabled
3. Filter out benign panics by matching known patterns (unimplemented features, stack overflows, etc.)
4. Extract the throw location from the backtrace
5. Normalize the panic message — strip identifiers, source locations, numbers, etc.
6. Group and cache crashes to avoid reporting old bugs again

While the script seems easy to implement with LLMs, make sure it works correctly — especially backtrace parsing and deduplication logic — to avoid losing valid bugs.

An example implementation of such a script for a Solidity fuzzing campaign is available as a gist.

Minimization​

When minimizing a crash report manually, use the delta debugging technique — a classic troubleshooting approach.

Among the tools, perses and treereduce can help — both provide grammar-aware reduction. afl-tmin is not a good fit here, because it operates at the bit/byte level and knows nothing about the grammar.

But typically it is not worth your time — you can safely delegate it to an LLM without any extra commands. Two things to watch for: tell the model not to report ASTs recovered after parsing errors; on weird-looking source, it sometimes gives up without reproducing the bug.

Report filing​

After deduplication and minimization, you need to check for duplicates against existing issues and write a report.

LLMs work great here, but you need a good prompt:

• Ask the model to check for duplicates with gh.
• Always ask it to reproduce with the real compiler, not with the fuzzing harness.
• Write very concise reports without root cause analysis or suggested fixes — this avoids hallucinations. Don't write anything you did not check by yourself.

First, triage a few reports by yourself, then write a CLAUDE.md triage guide for the model to follow. The complete move-fuzz triage/minimization/reporting prompt is available in the move-fuzz repo.

Evaluation​

Here are the results of the campaign. All the findings goes beyond lexer and parser, and triggered by later compilation passes. The findings for Sui Move, Leo, and Cairo are comfirmed and almost all were fixed. Solang and Solidity bugs are under triage at the moment of publishsing (Apr 2026).

Here is the complete table:

The campaign was run on a 2019 Intel i7 U-series and did not take that much time. The goal was to verify the approach, not to find all possible bugs, because running the infrastructure takes resources. These findings were mostly the result of initial corpus generation and quality mutators, and relied on coverage-guided path exploration much less.

Here is the concrete configuration used in fuzzing campaigns:

• Sui Move and Leo were fuzzed mostly with crafted MetaMut-style mutators with additional afl-ts instances. Default AFL++ mutations with custom dicts were applied for some workers. honggfuzz and libfuzzer workers with dicts were applied in 1 thread for some time.
• Solidity and Solang both were fuzzed with afl-ts workers mostly because solidity has a large corpus of contracts (e.g. Zellic dataset, lots of regression tests for previous findings in the Solidity repo) and good up-to-date tree-sitter grammar. Default AFL++ mutations with custom dicts were applied for some workers.
• Cairo – a MetaMut-style mutators with only mutations for rare constructions and afl-ts. AFL++ mutations were disabled – since the fuzzing campaign there has extremely low stability and often hits memory limits because of Salsa, any extra executions are expensive because they clutter the corpus making the fuzzing less effective. Thus, only grammar-aware mutations were applied there.

While most of the bugs were found for Sui Move, the approach is maybe more developed there — Sui Move was used as the initial target for fuzzing, as a follow-up to previous work, so the tooling was matured on it before being applied to the other compilers.

There is no precise statistics which custom mutators give the best results nor comparision, while most finding were made by custom MetaMut-style mutations and the afl-ts mutator. This is not a paper evaluating a simple mutator – if your goal is also to find bugs in production code, you should use all the approaches giving you the result with low effort and proven result; combining multiple fuzzers for better corpus diversity and quickier path findng.

Challenges​

Some challenges encountered during these campaigns:

• Corpus growth with big files — if you start without a good initial corpus and enable ts-add, the corpus accumulates oversized entries that slow down the whole campaign. Minimize early and aggressively.
• Stability of stateful compilers — Cairo uses Salsa, an incremental computation library. While convenient for tooling development, it complicates fuzzing: the fuzzing state has to be reset every N iterations to avoid OOM, and the MetaMut-style mutator has to be tweaked accordingly. Move and Leo are mostly stable; any minor issues are likely caused by map type usage, but they don't affect the campaign.
• Tree-sitter grammar quality — the whole pipeline (corpus generation, afl-ts, renaming script) relies heavily on the grammar parsing valid source without ERROR nodes. While afl-ts tries to recover from ERROR nodes by inserting syntactically valid code via its ts-chaos strategy, it is more efficient to run on a clean grammar.
• Reproducibility across versions — compilers move fast. An ICE found at HEAD may already be fixed by the time someone triages the report, or the minimizer may shift the behavior to a different internal error. Pin the submodule version in the harness and include the exact commit in every report.

Conclusion and further work​

This blogpost shares experience setting up a cheap, fast fuzzing campaign for a non-mainstream language to find ICE. The approach and tooling are reproducible for any compiler.

Two new AFL++ grammar-aware mutators are introduced: afl-ts mutator that works with any tree-sitter grammar, and a MetaMut-style LLM-generated mutator that produces hundreds of language-specific operations from a few prompts. Both proved effective in finding ICE.

Corpus and dictionary setup is covered with practical advice: collect broadly, minimize aggressively, mix manual dictionary entries with AFL_LLVM_DICT2FILE auto-generation. Helper tools (tsgen, validation scripts) are included.

Minimization and triage are LLM-assisted: a CLAUDE.md triage guide handles bucketing and MRE generation, while afl-cmin and perses (or an LLM directly) shrink test cases. Concise prompts without root cause analysis reduce hallucination.

It is like experience sharing – before digging into tools and literature this setup took a couple of weeks; with the approach described here, it takes 1-2 days to get real findings.

We intentionally don't consider approaches to testing that require more time and effort to implement. Oracles, miscompilation, and implementation/specification mismatch errors – these techniques are out of scope and will be described in the next part, since this post is already large as fuck.

Projects discussed​

A list of small utilities, mutators, and tools recently published and used in the project:

• jubnzv/afl-ts – grammar-aware AFL++ mutator leveraging tree-sitter
• jubnzv/tsgen – utility for seeding fuzzing corpora using tree-sitter grammars
• jubnzv/multifuzz – unified configuration, orchestration, and Rust API for AFL++/honggfuzz/libFuzzer — no implicit settings or overhead
• nowarp/move-fuzz – fuzzer and mutators for Sui Move
  • Fuzzing harness, dictionaries, and scripts
  • MetaMut-style mutator with 884 custom mutations for Sui Move
  • Ad-hoc Move mutator
• Many ad-hoc scripts demonstrating the approaches — see the post.

Not published yet:

• Leo: fuzzing harness with utilities and MetaMut-style mutator
• Cairo: fuzzing harness with utilities and MetaMut-style mutator
• Solidity and Solang: fuzzing harness with utilities
• Any experiments beyond the scope of the described techniques

If you work on any of the compilers mentioned, reach out — happy to share repo access.

References​

1. Li et al – Boosting Compiler Testing by Injecting Real-World Code (2024)
2. Paaßen et al – Targeted Fuzzing for Unsafe Rust Code: Leveraging Selective Instrumentation (2025)
3. Yang et al – WhiteFox: White-Box Compiler Fuzzing Empowered by Large Language Models (2023)
4. Park et al – Fuzzing JavaScript Engines with Aspect-preserving Mutation (2020)
5. Ou et al – The Mutators Reloaded: Fuzzing Compilers with Large Language Model Generated Mutation Operators (2024)
6. Aschermann et al – REDQUEEN: Fuzzing with Input-to-State Correspondence (2019)
7. Sun et al – Finding compiler bugs via live code mutation (2016)
8. Tu et al – Beyond a Joke: Dead Code Elimination Can Delete Live Code (2024)
9. Srivastava et al – Gramatron: Effective Grammar-Aware Fuzzing (2021)
10. Xie et al – Kitten: A Simple Yet Effective Baseline for Evaluating LLM-Based Compiler Testing Techniques (2025)
11. Liu et al – Bug Histories as Sources of Compiler Fuzzing Mutators (2025)
12. Xia et al – Fuzz4All: Universal Fuzzing with Large Language Models (2024)
13. Groce et al – Making No-Fuss Compiler Fuzzing Effective (2022)

The blog post contains the following sections:

1. Static analysis + LLM: description of the approach
2. Skry: Design & implementation: analyzer pipeline, key features, and what makes it different
3. Evaluation: findings on real-world contracts, detection accuracy, reproduced audit findings
4. Conclusion: current use cases and future work

The current project state is a proof-of-concept. It does find issues in real-world projects demonstrating low false-positive ratio, but needs more work to make its analysis more precise and capable.

Static analysis + LLM​

LLMs are already used in smart-contract security. Typical usage is straightforward: provide source code as input and ask the model to identify potential vulnerabilities.

Some real issues have been found this way, but practical experimentation exposes several limitations:

1. Noise: without strict scoping, the model reasons about irrelevant code and paths.
2. Cost: large contexts and RAG-style setups significantly increase inference cost.
3. Non-determinism: results are fuzzy and difficult to reproduce.
4. High false-positive rate: models tend to over-report issues without semantic grounding.

Using LLMs alone is not effective for systematic vulnerability detection. The issue is not the model itself, but the lack of structure, constraints, and analysis scope. As a result, approaches that combine deterministic methods with LLMs have been proposed. In static analysis + LLM systems, existing tools integrate LLMs into the analysis pipeline either to extend detection logic [1] [2] or to reduce the false-positive rate [3].

In these approaches, static analysis performs the core reasoning and defines the analysis scope, while LLMs are used only for properties that cannot be reliably inferred statically, such as semantic intent or project-specific logic. The goal is not to replace static analysis, but to extend it where classic techniques rely on approximations or heuristics.

Skry: Design & implementation​

Skry is a static program analyzer that uses LLMs to reduce reliance on heuristics and overapproximations when dealing with bugs that are difficult to express using classic static analysis alone. LLMs are used only for data classification and limited semantic reasoning about smart contract constructs. Bug detection and soundness remain the responsibility of the static analyzer.

Internally, the analyzer collects information about the contract as Datalog-style facts. These facts are later queried using a small eDSL based on Hy macros. This design allows both structural and semantic information to be reused across analyses and rules.

The overall pipeline architecture is shown below:

The implementation is based on Python, with Hy used for the eDSL. Choosing Python is intellectually violent, but sufficient for the proof-of-concept version when set up carefully. Python simplifies integration with tree-sitter for parsing, LLM APIs, and provides quality testing and debugging support. It also simplifies future integration with external tooling that may be worth considering, such as probabilistic Datalog engines or SMT solvers.

The analyzer is:

• source-level and currently supports Sui Move only,
• interprocedural and cross-module for taint and dataflow analysis,
• path-insensitive in the current version.

The following sections describe specific components in more detail, including the rule system, fact representation, LLM-based classification, and detection of access control and centralization risks.

Scope and focus​

Skry is intentionally focused on a narrow set of security patterns that are difficult to express using classic static analysis:

• Access control: capability misuse, missing authorization, pause bypass, generic type safety.
• Centralization: admin drain patterns, missing audit events, immutable configuration, single-step ownership.
• Structural checks: double initialization, missing transfers, duplicated branches, weak randomness.

Structural issues come for free when building a static analyzer and are included when detected. Access-control and centralization issues depend on semantic properties such as what a capability represents, who owns what, where privilege boundaries are, and what the project intends. Traditional tools usually cannot model this with confidence and either guess based on heuristics or ignore them.

Skry's detection logic is deterministic. LLM-based classification is used only to extract the minimal semantic information in a constrained scope needed to reason about access control patterns where static analysis alone cannot.

Static analysis and code facts​

The analyzer implements a classic monotone framework and interprocedural taint analysis, propagating information across Move modules and packages.

In addition to classic IRs common in static analyzers, the tool stores extracted information about the source code as Datalog-style facts. This is done for extensibility: users can access these facts directly from the eDSL to create new rules, regardless of whether the information is purely structural or “semantic” data gathered from LLM classification. This approach can also be used to generate project-specific rules or to cover common vulnerability patterns.

The code facts are defined in src/core/facts.py, and represent the structural and semantic information. The goal of such a separation is flexibility: it enables the user or the model to combine these facts to build custom rules. Without changing the analyzer itself or using it as a framework.

A typical dump of code facts produced via --dump-facts=<DIR> contains information about each function and struct in the project, including LLM classifications and project categories. This output can be used for debugging and for designing new project-specific rules. It prints the code fragments of all the modules with the relevant facts. Here is a small example:

Rules and structural filters​

The main detection logic is implemented in a small macro-based eDSL. Hy is chosen for dual interoperability with Python and its macro capabilities, which allow rules to be expressed in a concise format. This makes rules easier to read and, if needed, easier to generate via an LLM for a specific project.

After collecting structural information, the eDSL is used to directly access these facts, along with utilities to manipulate and combine them.

Currently, the tool supports 45 rules.

Here is one of the available rules:

;; ---------------------------------------------------------------------------
;; centralized-reward-distribution - Admin picks lottery/game winners
;; ---------------------------------------------------------------------------
;; Gaming project where admin-controlled function distributes rewards to
;; admin-chosen recipients. No verifiable on-chain randomness - users must
;; trust the admin to be fair.
;; Impact: Unfair game - legitimate players never win.
(defrule centralized-reward-distribution
  :severity :medium
  :categories [:centralization :fairness]
  :description "Admin-controlled reward distribution - no verifiable winner selection"
  :match (fun :public)
  :filter (and
            (project-category? "gaming" facts ctx)        ;; Gaming/lottery/gambling project
            (checks-sender*? f facts ctx)                 ;; Admin-gated (transitive sender check)
            (transfers-from-shared-object? f facts ctx)   ;; Extracts from shared pool
            (has-param-type? f "address" facts ctx)       ;; Has address param = admin picks recipient
            (not (transfers-from-sender? f facts ctx))    ;; Not user withdrawing own funds
            (not (is-init? f facts ctx))))

Here, the rule file uses helper functions that combine existing code facts to make them more convenient to use. The naming is literal: functions ending with ? return booleans, and * indicates transitive behavior.

LLM classification​

LLM classification is used in three cases:

1. Project-wide feature classification: for example, the presence of a global pause, versioning, and project categories. This information is used to adjust specific rules.
2. Data classification: for each struct, the tool determines whether it contains sensitive data, configuration parameters, or protocol invariants, and whether it is intended to be owned by a privileged user.
3. Rule double-checking: in the :classify section for a subset of rules, to reduce the false-positive rate by handling subtle Move patterns or intentional design decisions.

The generated prompts are based on Jinja2 templates and are available in src/prompts/.

Here is the real-world example prompt generated from the data sensitivity classification template:

LLM output:

[
  {
    "field": "LockerCap::liquidation_ratio",
    "reason": "economic",
    "confidence": 0.85
  },
  {
    "field": "LockerCap::price_with_discount_ratio",
    "reason": "economic",
    "confidence": 0.90
  },
  {
    "field": "LockerCap::inactivation_delay",
    "reason": "availability",
    "confidence": 0.75
  }
]

Which is correctly used to generate the following semantic facts:

• liquidation_ratio → economic (manipulate = unfair liquidations)
• price_with_discount_ratio → economic (manipulate = steal funds via discounts)
• inactivation_delay → availability (manipulate = trap lockers or let them escape early)

The model also correctly ignored counters, timestamps, data structures – because of the reduced scope and concrete prompt.

Access control and centralization risks detection​

In Sui Move, access control is expressed through ownership of capability objects. A capability is a first-class object that grants permission to perform a restricted operation. Functions require specific capability types as parameters, and only callers that own the corresponding object can invoke those operations. Capabilities are typically created during initialization and transferred explicitly, making access control decisions explicit and traceable in the code.

The tool implements a simple IR based on code facts that tracks capabilities in a graph. A Mermaid dump of this IR (accessible through --dump-cap-graph=<DIR>) looks like this:

Skry models this access control structure using a capability graph derived from code facts. The graph captures which addresses own which capabilities, which functions require those capabilities, and which objects are mutated as a result. This representation makes privilege boundaries, capability hierarchies, and sensitive state transitions explicit and analyzable.

In addition, the Move ownership model distinguishes between shared and owned objects. This distinction is critical for access control analysis, as shared objects enable global access patterns, while owned objects enforce per-address authority.

Evaluation​

Evaluation Setup​

Data​

We evaluated the tool on a set of real-world Sui contracts with source code available on GitHub.

The following criteria were used to select projects:

• smart contracts only (no libraries),
• only Sui Move; Aptos and other Move-based ecosystems were excluded,
• production-quality projects: no forks, tutorials, or hackathon artifacts,
• primarily targeting Move 2024, with a small number of older projects included.

In total, we identified 94 projects matching these criteria.

In addition, audit reports from prior security assessments were used to reproduce historical findings.

Approach​

The evaluation approach was to reproduce critical access control issues on historical code from large Sui projects and to test non-critical rules on a collected set of contracts.

To evaluate the tool on large, production-ready projects, we reintroduced findings previously reported in audits by top firms. This was done by applying targeted mutations to production code that matched those findings and verifying that the tool detected them.

The evaluation focuses only on manually verified, non-exploitable issues. These findings are used to demonstrate the tool’s detection capabilities. All cases were reviewed manually to confirm that they either reflect intentional design decisions that warrant manual validation or originate from historical source code. No new exploitable vulnerabilities are claimed.

Models and the cost of evaluation​

For evaluation purposes, two models were used:

• DeepSeek — chosen for its inexpensive API and precise adherence to prompt instructions.
• Opus 4.5 — provides the best accuracy and demonstrates strong "knowledge" of Move; available via Claude Code and callable from the CLI, making it cheaper than API-based usage.

In addition to API-based execution, the tool supports multiple modes, including a manual mode for prompt debugging and integration with Claude Code.

The typical number of prompts and overall cost depend on the project. Larger codebases and a higher number of potentially sensitive functions result in more prompts. In practice, small projects (around three Move files) require approximately seven prompts, while large production projects require around 30–40 queries.

The tool also uses caching: all LLM prompts and responses are cached and reused in subsequent executions unless a fresh run is explicitly forced.

Critical access control issues​

These rules detect high-impact access control and capability-handling issues that are typically exploitable and unacceptable in production code.

These rules include:

• unprotected-pause – Access control issue where a user can manipulate the global lock mechanism (critical).
• sensitive-internal-public-exposure – Internal helper exposed as public.
• generic-type-mismatch – Generic type parameter used without validation.
• arbitrary-recipient-drain – Transfer to a user-controlled address without authorization.
• missing-authorization – Entry function reaches a dangerous sink without an authorization check.
• user-asset-write-without-ownership – Write to user assets without ownership proof.
• missing-destroy-guard – Capability destruction without authorization.
• capability-takeover – Capability can be acquired by an unauthorized address.
• capability-leak-via-store – Capability stored in a shared object field.
• phantom-type-mismatch – Capability guard uses a different phantom type than the target.
• test-only-missing – Public function returns a privileged capability without #[test_only].
• duplicated_branch_condition – Same branch condition appears multiple times.

These issues have critical severity. A valid finding in production code is likely exploitable, and therefore such issues are not expected to appear in audited projects.

To validate the tool, previously reported audit findings were reintroduced, and the analyzer was tested to ensure it detects them. The evaluated protocols are production-grade, large, and have mature, audited codebases. The critical issues listed below were present in earlier audits and are not deployed in live systems; the purpose here is tool evaluation.

Reproduced findings include:

This validation approach reintroduces known security issues from audits, runs the analyzer on the modified code, and checks that the expected warnings are produced. This approach is used because unaudited source code for large production projects is generally no longer available.

To validate these warnings, the following mutations were applied.

Pause and version usage anomalies​

A pause or global lock is a common pattern in smart contracts and is typically used for security purposes. The absence of a pause mechanism may indicate a centralization issue, a missing check, or a valid design decision. In most cases, such issues have medium severity and should be mentioned in audits. Only in rare cases does a missing pause check lead to severe or unexpected behavior.

In the current implementation, there are three related rules. The pause-check-missing and version-check-missing rules detect anomalies in version and global pause usage. If a public function omits a check that is consistently applied in similar functions, it is reported.

While an unprotected version check in Move upgradable contracts is considered a critical vulnerability, missing pause checks are often intentional design decisions or non-exploitable bugs.

An example of such a warning is shown below:

[HIGH][pause-check-missing][sources/treasury.move:419:5] in function 'multisig_treasury::treasury::create_simple_proposal'

The treasury has an is_frozen state that is checked in create_emergency_proposal, but create_proposal and create_simple_proposal never check it. As a result, this is highlighted by the rule as a pause check anomaly. In this specific case, it may be a deliberate design choice if the freeze mechanism is intended to block only the emergency fast-track.

Centralization risks​

Centralization risks correspond to intentional design decisions or missing features where privileged user(s) have excessive control over critical project logic.

Below are some example rules and real-world findings.

single-step-ownership​

The rule highlights single-step ownership transfers.

In the source code, this appears as the following pattern:

public fun admin_transfer(ac AdminCap, recipient: address) {
    transfer::transfer(ac, recipient);
}

The issue with this pattern is that if the current admin provides an incorrect address (e.g., due to a typo, phishing attack, or copy-paste error), the admin capability is irrecoverably lost, with no mechanism to cancel or correct the transfer.

Here are example findings highlighting similar patterns:

[HIGH][single-step-ownership][liquid_staking/sources/ownership.move:33:5] in function 'liquid_staking::ownership::transfer_owner'

[HIGH][single-step-ownership][liquid_staking/sources/ownership.move:47:5] in function 'liquid_staking::ownership::transfer_operator'

Both functions transfer_operator and transfer_owner implement the single-step ownership pattern. These can be improved by using a two-step transfer pattern: the current owner calls, for example, offer(new_addr) to set a pending recipient, and the new owner then calls claim() to accept. This ensures the recipient address is valid and controlled, and allows cancellation in case of a mistake.

centralized-reward-distribution​

Detects admin-controlled reward distribution in projects classified as "gaming", where there is no verifiable winner selection.

An example warning for a lottery project:

[MEDIUM][centralized-reward-distribution][sources/core.move:147:5] in function 'rtmtree::longshot_jackpot::goal_shot'

An admin-gated function decides who receives rewards and when. The player address is passed as a parameter, but execution is controlled by the admin. A malicious admin can refuse to distribute rewards to legitimate winners.

admin-bypasses-pause​

Detects a possible centralization risk when an admin can bypass the global lock mechanism.

Here is an example warning:

[INFO][admin-bypasses-pause][lending_core/sources/pool.move:228:5] in function 'lending_core::pool::withdraw_treasury'

withdraw_treasury requires the PoolAdminCap capability. A pause mechanism exists via the Storage.paused field, which is checked through when_not_paused() in user-facing lending functions. There is no pause check here: withdraw_treasury does not take Storage as a parameter and therefore cannot check the pause state. This creates a centralization risk: an admin can drain the treasury even when the protocol is paused.

The severity is informational, reflecting the fact that this may be an intentional design decision, while still requiring additional attention.

missing-admin-event​

Generates an informative warning when critical protocol changes may require emitting an event. This is similar to Slither’s missing event detector, but relies on LLM classification.

Some example informational warnings that require manual validation:

[INFO][missing-admin-event][sources/patience.move:160:5] in function 'patience::patience::withdraw_fee' - extracts value. Emit event with: amount

[INFO][missing-admin-event][sources/patience.move:166:5] in function 'patience::patience::admin_transfer' - transfers to user-controlled address. Emit event with: recipient

[INFO][missing-admin-event][contracts/core/sources/prize_pool.move:161:1] in function 'anglerfish::prize_pool::claim_protocol_fee' - extracts value. Emit event with: amount

[INFO][missing-admin-event][contracts/core/sources/prize_pool.move:172:1] in function 'anglerfish::prize_pool::claim_treasury_reserve' - extracts value. Emit event with: amount

Protocol configuration parameters and invariants​

In Sui Move, all data is expressed in structs. Configuration parameters may be either:

• mutable configuration parameters — values expected to change during the contract’s lifecycle. Examples: fee_rate, loyalty_address.
• immutable protocol invariants — parameters set once during initialization. Examples: protocol_fee_bps, decimals (e.g., to manipulate specific tokens).

Some detectors rely on LLM-based classification and highlight anomalies that should be checked manually:

missing-mutable-config-setter​

The rule relies on LLM classification for struct fields and static analysis that propagates data. If the model detects that a struct field represents a configuration value that is expected to be mutable, the analysis checks whether any setters exist for it.

Some example real-world findings:

[MEDIUM][missing-mutable-config-setter][core/sources/satlayer_pool.move:55:1] field 'satlayer_core::satlayer_pool::Vault.min_deposit_amount'

The Vault struct has admin setters for similar configuration fields:

• staking_cap → set_staking_cap
• withdrawal_cooldown → update_withdrawal_time
• is_paused → toggle_vault_pause
• caps_enabled → set_caps_enabled
• min_deposit_amount → no setter

This makes the finding valid: min_deposit_amount is correctly classified as a configuration field and may require a setter, unless this is an intentional design decision.

[MEDIUM][missing-mutable-config-setter][sources/curve.move:42:5] field 'pumpfun::curve::Configurator.swap_fee'

The admin has setters for all other Configurator fields, but not swap_fee. This looks like an oversight correctly classified and highlighted by the analyzer. If the swap fee needs adjustment, the admin has no way to change it.

mutable-protocol-invariant​

The rule is similar to missing-mutable-config-setter, but detects the opposite pattern: if the LLM classifies a field as an immutable protocol invariant set once during initialization, it should never be changed.

Some examples of real-world findings include:

[HIGH][mutable-protocol-invariant][sources/lootbox.move:118:5] in function 'suigar::lootbox::edit_lootbox' writes to invariant 'suigar::lootbox::LootBox.reward_amounts'

[HIGH][mutable-protocol-invariant][sources/lootbox.move:118:5] in function 'suigar::lootbox::edit_lootbox' writes to invariant 'suigar::lootbox::LootBox.reward_probabilities'

This means that the LLM classified two fields of LootBox as protocol invariants that should be immutable, but they are modified in edit_lootbox. If the contract were to separate purchase and reveal into two transactions, an admin could change reward distributions between these steps, causing users to receive different payouts than expected at purchase time. While this may be an intentional design decision or a centralization risk, it warrants medium severity.

Evaluation Results​

The problem with precision evaluation is that non-exploitable findings often resemble intentional design decisions, even if these don't follow security best practices. As a result, it is impossible to provide an exact number for non-critical warnings.

For small projects with 3–4 modules, the tool typically generates 0 to 5 warnings, including some unclear cases that require manual review, some true positives, and some false positives. Overall, it is not noisy. For the largest production codebases, the tool generates around 40 warnings, including a number of valid concerns that deserve discussion during a security assessment.

Key insights on false positives:

• Many false positives are related to limitations in the current (proof-of-concept) analyzer's implementation; a more mature codebase would significantly reduce the rate.
• LLM misclassification does occur, but can be mitigated by providing more focused context and requesting explicit reasoning steps, followed by prompt refinement. While 100% accuracy or guarantees are not possible, precision can be improved at the cost of higher inference overhead.

Challenges​

The key challenges we encountered include:

• It is impossible to distinguish design decisions from valid findings without direct contact with project owners, which is especially relevant for non-critical findings.
• In many cases, there is no source code available to reproduce previous audit findings. Projects are often renamed, removed, or do not publish the audited code, making historical verification difficult.
• Most evaluated projects are mature and well audited, so access control issues are relatively rare.
• Cost is a minor concern. Having Claude Code installed allowed us to run the tool on around 100 contracts, occasionally hitting usage limits.

Conclusion​

The tool is still proof-of-concept. The approach has been tested and it works. The analyzer produces valid warnings that belong in a security assessment for Sui Move smart contracts. However, the analysis is not yet comprehensive or systematic and requires further work.

Skry is a static analyzer at its core – this is where most engineering effort belongs. The current version shows feasibility, not precision. Improving accuracy requires improving the analysis engine itself. Move is a relatively large language, and supporting more real-world patterns will require additional engineering effort.

Current use cases​

Despite its proof-of-concept status, the analysis is relatively cheap and already usable in practice:

• Bug finding before deployment: can be used as an additional check before audit and deployment.
• Audit assistance: the primary use case. The tool highlights issues for manual validation and can surface potential audit findings.
• Centralization risk validation: can be used to assess trust assumptions and administrative control when evaluating a project.

Future work​

The core of the tool is the static analysis engine. Improving it will improve both static precision and LLM-based classification by enabling more specific and constrained prompts.

Some important areas for future work include:

• Advanced Sui Move pattern support: for example, OTW-based access control, address-based access control patterns (more common in Aptos, but sometimes present in Sui in the wild), and more comprehensive tainted data propagation covering all language constructs.
• IR improvement: the current IR is relatively simplistic and expresses only the information required for access control detection.
• Path sensitivity: full symbolic execution for Move is complex but feasible; introducing simple sink-protection facts and dominance relations in the IR is a reasonable starting point.
• Object lifecycle tracking: as an extension of dataflow analysis to cover additional patterns.
• Execution optimization: routines involving fact processing and interprocedural, cross-module analysis can be optimized to improve performance.
• Dependency management: understanding external dependencies is important; the implementation may require integration with the build system to obtain actual dependency sources.

While improving the analysis engine will make the analyzer more accurate and effective, new rule categories beyond access control and governance can also be introduced:

• Variable misuse: using variables of the correct type in an incorrect context (e.g., incorrect argument ordering or checking properties of the wrong variable, as in this example). Papers such as this one cover this topic in detail. A proper implementation would require IR improvements to reduce the number of LLM requests.
• Oracle patterns: while less widespread compared to access control issues, oracle API misuse still appears in audit findings and is worth covering.
• Flash loan and slippage issues: although Sui smart contract design protects against many flash loan attacks, some patterns still appear in audits and could be addressed.
• Cross-project pattern extraction: while invariant checking is typically handled by a different class of tools, the existing eDSL could be leveraged to generate new rules using LLMs, based on patterns extracted from existing projects via a RAG-style approach combined with code mutation.

References​

1. Xia et al – AuditGPT: Auditing Smart Contracts with ChatGPT
2. Sun et al – GPTScan: Detecting Logic Vulnerabilities in Smart Contracts by Combining GPT with Program Analysis
3. Li et al – The Hitchhiker’s Guide to Program Analysis, Part II: Deep Thoughts by LLMs

This is where static analysis comes in. It's a technique that examines your code before deployment to automatically detect potential vulnerabilities. While no automated tool can guarantee security, static analysis can identify common pitfalls early in development.

This post:

• Explores static analysis capabilities and limitations for smart contracts security.
• Shows how this fits into TON security landscape through Misti.

Understanding static program analysis enables you to add an additional layer of automated security verification to your development process, catching some vulnerabilities before they reach production.

Static Analysis for Web3 and TON​

Static Program Analysis 101​

Security tooling is an essential part of modern smart contract development, serving as the first line of defense against vulnerabilities. While manual code review remains crucial, automated analysis tools can systematically identify classes of bugs that would be tedious and error-prone to catch by hand.

Static program analysis examines code without executing it. The classic approach used in program analysis and compiler design is the Monotone Framework [4]. This builds abstract models of your program using control flow graphs (CFGs) and data flow analysis to reason about program behavior without the undecidability of analyzing all possible runtime scenarios.

The analysis pipeline could be illustrated like this:

In essence, the approach is straightforward: we read and analyze the code structure to identify potential security vulnerabilities, without executing the code itself.

There are also different analysis techniques that exist, which can be used to explore concrete paths of execution, prove properties on a limited domain of program, prove program properties with more accuracy, and so on, but the core issue is always the same: static undecidability.

Application to TON Smart Contracts​

Smart contracts are just programs executing in blockchain. They have some differences, but fundamentally the same techniques of classic program analysis can be applied to smart contract analysis.

Being a dynamic field, web3 security is currently actively developing, new approaches are being tested, but still, that's a wild west: tools' impact on actual bug finding is suboptimal [1]. Despite that, we have a huge set of approaches described in papers and applied in commercial/free tools, we know about past bugs typical for other blockchains, and this knowledge can be extrapolated to TON.

TON creates its own unique architectural issues, while most of the generic web3 bugs from research and practical experience are still valid for TON. It has a runtime environment based on a stack machine and a couple of imperative languages without advanced language design solutions, which makes it quite similar to existing blockchain environments.

Examples of common security and functionality issues present in TON contracts include [2]:

But the most complicated bugs in TON are related to its unique actor model and asynchronous message-passing [3]:

• Partial execution: state mutations due to asynchronous message passing
• Man in the middle in message flow
• Anything that requires understanding the specification of the system; thus requiring a manual audit

Here's the key point: complex bugs require understanding the system, and sometimes even developers don't fully understand it.

Misti: TON Static Analyzer​

Misti as a source-level analyzer for Tact contracts based on monotone framework that works exactly as described above, as well as combining it with Datalog-based analyses. Certainly it has all the limitations typical for this approach. These are essential and done by design.

Misti covers different categories of security and optimization issues:

• Cell storage issues
• Resource exhaustion vectors potentially leading to DoS
• Arithmetic issues
• Unauthorized access to critical functions and contract's state
• Code optimization
• Generic suspicious patterns:things we learn from web3 security in past

Let's consider some concrete case studies of analyses Misti implements for both generic smart contract issues and TON/Tact-specific problems.

Cell storage issues​

TON stores persistent data in Cell structures. Cell is a low-level primitive containing up to 1023 bytes of data and up to 4 references to other cells used to create high-level data structures.

The possible issue with cells arises when access or write operation disrupts these limits. When the user tries either to load non-existing data from a cell or write data/references beyond the specified limits, it leads to CellUnderflow and CellOverflow compute phase exceptions:

let b1 = beginCell();
b1 = b1.storeInt(self.data, 257);
b1 = b1.storeInt(self.balance, 257);
b1 = b1.storeInt(self.owner_data, 257);
// CellOverflow: storing more than 1023 bits
b1 = b1.storeInt(msg.info, 257);

let s1 = beginCell() // Creating a Slice with 1 reference
           .storeRef(c)
           .endCell()
           .asSlice();
let ref1 = s1.loadRef(); // OK
let ref2 = s1.loadRef(); // CellUnderflow

Because of the asynchronous nature of TON, these issues may lead to unexpected message flow disrupting the logic of the contract.

The main issue of detecting these is that in Tact, Cell operations might be used within different data structures like Builder, Slice, Cell, Struct and Message, and might require reasoning about the source code within different function/method calls.

The CellBounds detector tries to handle this by statically inspecting the source code.

Tact-specific issues​

There are plenty of Tact-specific issues covered by Misti. Let's consider some of them with source code examples.

Function arguments in Tact are immutable​

Thus, the developer should not mutate them expecting they'll be changed in the callsite. The ArgCopyMutation detector finds these cases (unless the developer explicitly returns the modified parameter):

fun setA(a: Int, m: map<Int, Int>) {
  // Bad: `m` won't be modified in the callsite
  m.set(self.key, a);
}

Some exit codes are reserved​

Codes from 0 to 255 are reserved by Tact and TON. Thus, the developer should never use them to avoid breaking the expected behavior of the contract. The ExitCodeUsage detector interprets the possible numeric values used in exit codes in order to detect suspicious cases like these:

receive("test") {
  // Bad: Throwing the reserved `128` code
  nativeThrowUnless(128, sender() == self.owner);
}

Don't overlap string receivers values​

Tact has text receivers which accept a particular string as a message. The issue arises when a generic receiver (defined receive()) handles these messages:

contract Test {
  receive("test") {}
  receive(msg: String) {
    // Bad: "test" message should be handles in `receive("test")`
    if (msg == "test") { /*...*/ }
  }
}

This leads to unexpected control flow making some receivers unreachable. The StringReceiversOverlap detector handles this.

Choose better Tact API​

Tact, being a dynamically developed language, can introduce new features making code more effective or deprecate some features or create safer alternatives or potentially dangerous functions.

Examples of these functions include but are not limited to:

1. nativeSendMessage should be replaced with send
2. nativeRandom should be replaced with randomInt
3. Tact provides optimized versions of send: deploy and message

Here is some illustrating code:

contract Test {
  receive() {
    // Bad: Prefer more effective `deploy` function
    let init = initOf A();
    send(SendParameters{ code: init.code, /* ... */ });

    // Bad: Prefer `emptySlice()`
    let s: Slice = emptyCell().asSlice();
  }
}

It might be not-so-easy to follow all the updates within different versions of the Tact compiler; thus Misti covers this. Examples of detectors of this category include PreferredStdlibApi and SuboptimalSend.

Arithmetic issues​

A classic arithmetic issue typical for smart contracts is division before multiplication. The thing is that typically the division operation can leave some remainder that might be missed when using multiplication afterward. If there is no handling of the remainder, the contract might lose user funds or tokens on such operations:

let a: Int = 10;
let b: Int = 3;
let c: Int = 2;
// Bad: Division before multiplication
let result: Int = a / b * c;
}

The DivideBeforeMultiply detector can detect these cases.

Use random properly​

TVM implements some PRG functionality. When using it in Tact, the developer should care about API usage and read the documentation carefully. They always have to initialize seed correctly using either the Tact standard library nativePrepareRandom or some TVM assembly to initialize the seed. EnsurePrgSeed ensures the random seed is set up before accessing randomness features.

Unprotected calls or state changes​

In smart contracts there are always privileged functions which e.g. destroy contract, send funds, and it is possible to change critical parameters through sending messages. All this functionality essentially should be protected from random users to avoid cases like The Parity Wallet Hack.

Here is some code illustrating this for Tact:

receive(s1: Slice) {
  let a = s1.loadAddress();
  // Bad: Anyone could send funds to an arbitrary address
  // The protection would be to `require` a specific sender address
  send(SendParameters{ to: a, /*...*/ });
}

The UnprotectedCall detector can protect your code against such cases.

Generic code issues​

There are plenty of issues typical not only for smart contracts but for normal programs as well. But while using these issues in not safety-critical systems typically doesn't lead to severe damage, in contracts the results might be much worse. Let's consider some case studies:

Be careful with copy-paste​

A typical case is when you need slightly different logic that makes no sense to parametrize to avoid overcomplicating your code. But you should be careful there, not forgetting to update all the branches after copy-pasting and later when refactoring your code:

if (self.lockPeriod > HALF_YEAR) {
  require(sender == self.owner, "Only owner can trade");
} else {
  // Bad: The developer forgot to update the copy-pasted branch
  require(sender == self.owner, "Only owner can trade");
}

The BranchDuplicate detector can find equal branches in code.

Get rid of dead code​

Dead code is not as simple as it sounds. It not only clutters your codebase but often indicates that the developer:

• Forgot to implement the intended logic (e.g. unused constant or write-only field)
• Didn't check the error returning from the function (can lead to control flow anomalies, search for: weird ERC20 attack and read [5] for more)

Examples of dead code detectors in Misti are: NeverAccessedVariables, ReadOnlyVariables, UnusedExpressionResult.

Conclusion​

We've considered the basic information about static program analysis and it's application to the TON security landscape. Now, here are some concrete steps to increase security you should be thinking about.

Increase security of your project​

• Integrate security tools: Use every possibility to make your code secure. Static analysis is a good as a first line of defense, catching common vulnerabilities early and automatically. Add Misti to your CI/CD and integrate it in the development process.
• Apply development practices: testing, design, formal specification or at least documentation. This develops a separate post and might be highlighted in this blog later. Prefer a safe language.
• Set up processes: Security not only about analysis and audits: you should think in advance about security development in incident response processes.
• Do security audit: While static analysis enhances the security process, it cannot replace thorough manual audits. For production contracts, professional security audits remain essential. You could find our contacts on nowarp.io or browse security teams collaborating with TF.

Future Directions in Misti​

There are many possibilities in TON security automation. Our concrete steps in Misti for the next months:

• Implementing IFDS with path-sensitivity tracking in order to improve accuracy of interprocedural taint analysis
• Implement more Tact detectors using advanced static analysis techniques. The concrete roadmap will be available in the GitHub milestones.
• Improve integrability and API to support third-party developers
• Provide better tooling for auditors to actually understand the structure of contracts

Overall, Misti still following the development of the Tact language and improves it support to make development on it more smooth and secure.

References​

1. Smaragdakis et al. – Symbolic Value Analysis for Smart Contracts
2. Zhang et al. - Demystifying Exploitable Bugs in Smart Contracts
3. TON Documentation: Secure Smart Contract Programming
4. Anders Møller and Michael I. Schwartzbach – Static Program Analysis
5. Gan et al. – Why Trick Me: The Honeypot Traps on Decentralized Exchanges