JIT vs AOT¶

anvil supports two execution modes from the same NumericalFunction: JIT (Just-In-Time) for Python calling and AOT (Ahead-Of-Time) for standalone C++ deployment. Both modes use the same codegen pipeline and produce equivalent native code.

Overview¶

flowchart TB
    NF["NumericalFunction"] --> AOT["AOT: generate_module()"]
    NF --> JIT["JIT: fn(x)"]

    AOT --> HPP[".hpp + .cpp<br/>Standalone files"]
    HPP --> Build["Your build system<br/>(CMake, Make, ...)"]
    Build --> Binary["Embedded binary"]

    JIT --> Src["Single .cpp<br/>(self-contained)"]
    Src --> Clang["clang++ -shared -O2"]
    Clang --> Dylib[".dylib/.so"]
    Dylib --> Ctypes["ctypes.CDLL"]
    Ctypes --> Call["fn(np.ndarray) → np.ndarray"]

AOT: generate_module()¶

Produces standalone .hpp and .cpp files that can be compiled into any C++ project:

generate_module("my_module", [fn1, fn2, solver])

• Each function gets its own namespace inside the module namespace
• Multiple functions share the same Buffer<T, Ns...> template
• SQP solvers include the full optimization loop, linking against PIQP
• No Python dependency at runtime

JIT: fn(x)¶

On the first call, the function is compiled to a shared library and loaded via ctypes. Subsequent calls go straight to native code:

sequenceDiagram
    participant User
    participant NF as NumericalFunction
    participant Cache as ~/.cache/anvil/jit/

    User->>NF: fn(x) (first call)
    NF->>NF: generate C++ source
    NF->>NF: SHA256(source)
    NF->>Cache: check for cached .dylib
    alt Cache miss
        NF->>NF: clang++ -shared -O2 → .tmp
        NF->>Cache: atomic rename .tmp → .dylib
    end
    NF->>NF: ctypes.CDLL(dylib)
    NF->>NF: init_ws()
    NF->>User: result (np.ndarray)

    User->>NF: fn(x) (subsequent calls)
    NF->>NF: call native function
    NF->>User: result (np.ndarray)

Caching¶

Compiled shared libraries are cached in ~/.cache/anvil/jit/ keyed by {name}_{sha256(source)}.{dylib|so}. Compilation writes to a .tmp file first, then atomically renames, so concurrent processes don't corrupt the cache.

Workspace lifecycle¶

• NumericalFunction: init_ws() allocates a flat Buffer<char, N> for intermediate buffers. Cleanup via weakref.finalize.
• SQP: init_ws() allocates the full Ws struct (PIQP workspace, QP data, all intermediate buffers). deinit_ws() calls PIQP cleanup and frees all buffers. Also cleaned up via weakref.finalize.

What differs between JIT and AOT¶

The JIT templates (jit_module.j2, jit_shim_*.j2) add extern "C" wrappers around the same C++ code that the AOT templates produce. This is necessary because ctypes can only call C functions, not C++ namespaced functions.