Aila 0.1.7 Release Notes

Aila 0.1.7 moves the published Windows build to Intel oneAPI 2026.1, isolates
the C API inference runtime in a dedicated worker process, and adds structured
chat/tool-calling APIs. It also includes FP8 KV-cache quantization and a set of
Qwen3.5, NF4, vision, ASR, and TTS performance/reliability improvements.

These notes cover the final repository state since tag 0.1.6 (4b238b6).
The experimental MTP work that was reverted before this release is not part of
0.1.7.

Highlights

Isolated Windows C API runtime

• AilaShared.dll is now a lightweight C ABI proxy built without oneAPI
  inference imports.
• Each initialized C API engine owns a dedicated AilaWorker.exe process that
  loads the model and the private oneAPI runtime.
• AILA_RUNTIME_DLL_DIR selects the worker/runtime directory. Absolute paths
  are accepted; relative paths resolve relative to AilaShared.dll and are
  normalized to absolute paths internally.
• Worker startup validates protocol, ABI, build identity, executable path,
  runtime directory, and the isolated child environment. Missing, mismatched,
  timed-out, or crashed workers are reported through the existing engine error
  APIs and are not automatically retried.
• Worker logs are forwarded through the existing global C log callback using a
  bounded, serialized proxy dispatcher.
• Non-Windows builds continue to use in-process inference.

The Windows distribution layout is now:

integration_root/
|-- AilaShared.dll
`-- aila_runtime/
    |-- AilaWorker.exe
    |-- Aila.exe
    `-- <oneAPI and other runtime DLLs>

Structured chat, reasoning, and tool calls

• Added llama.cpp-style Jinja chat rendering with a fixed built-in Qwen3.5
  template plus request/environment overrides.
• Added structured parsing of reasoning, visible content, and Qwen-style tool
  calls.
• Added aila_generate_chat_json, aila_generate_chat_json_ex, and
  aila_generate_chat_json_stream_ex.
• Added append-safe AilaGenConfigV2, including thinking budget and fixed-seed
  fields while preserving the legacy AilaGenConfig layout.
• Added structured streaming events for reasoning deltas, content deltas,
  completed tool calls, warnings, and final metadata/tool calls.
• Added CLI --chat-output-json and --chat-stream-jsonl modes.
• Added warn/strict tool policy validation, tool-choice enforcement, canonical
  final tool calls, and response metadata. Aila formats and parses tool calls;
  callers still execute tools externally and submit tool-result messages in a
  subsequent request.
• Added bounded thinking via request fields, AILA_THINKING_BUDGET,
  --thinking-budget, and the interactive /thinking_budget command.
• Fixed duplicated <think> markers and several partial-marker, orphan-close,
  streaming, and required-tool-choice edge cases.

Runtime and model performance

• Added optional FP8 E4M3 KV-cache quantization through --kv-quant,
  AILA_KV_QUANT, and ASR/TTS/VLM-specific overrides.
• Added Qwen3.5 prompt-prefill chunking and recurrent-state checkpoint controls
  to bound scratch memory and support cache rollback.
• Backported shared backend optimizations for NF4 GEMM/GEMV, convolution,
  safetensors loading, vision weights, and KV-cache handling.
• Added Mimi transpose-convolution vec8 layout and fused decoder
  conv2/residual paths, both enabled by default with environment-variable
  fallbacks.
• Improved state snapshot/restore behavior used by Qwen3.5 context truncation.

Breaking changes and migration

1. Windows deployments must ship the worker and private runtime

Copying only the new AilaShared.dll is no longer sufficient. Deploy
AilaShared.dll and the complete aila_runtime/ directory from the same 0.1.7
distribution. Do not mix AilaShared.dll and AilaWorker.exe from different source revisions or release packages. Starting from 0.1.7, updating to a new release should update both AilaShared.dll and AilaWorker.exe.

Embedding hosts should set the variable before loading the proxy:

import ctypes
import os

os.environ["AILA_RUNTIME_DLL_DIR"] = "aila_runtime"
aila = ctypes.CDLL(r".\AilaShared.dll")

Only expose the directory containing AilaShared.dll to the host DLL loader.
Do not add aila_runtime/ with os.add_dll_directory, and do not append it to
the host PATH; either action reintroduces the oneAPI DLL collision that the
worker architecture is designed to prevent.

An unset/empty AILA_RUNTIME_DLL_DIR still enables the legacy flat-directory
fallback, but that layout does not isolate the private runtime and is not
recommended for Python, ComfyUI, or other embedding hosts.

The worker runs with the runtime directory as its current directory. Embedded C
API callers should pass an absolute model path to aila_engine_init; a relative
model path is otherwise interpreted relative to AILA_RUNTIME_DLL_DIR.

2. TTS streaming now uses an explicit stream handle

The old API returned an integer and used the engine for wait/destroy:

int rc = aila_synthesize_stream(engine, /* ... */);
aila_stream_wait(engine);
aila_stream_destroy(engine);

0.1.7 returns an opaque handle. Store it, wait on it if desired, and always
destroy it:

AilaTTSStream* stream = aila_synthesize_stream(engine, /* ... */);
if (stream != NULL) {
    int rc = aila_stream_wait(stream);
    aila_stream_destroy(stream);
}

Update FFI declarations accordingly:

• aila_synthesize_stream.restype / return type becomes a pointer.
• aila_stream_wait takes AilaTTSStream*, not AilaEngine*.
• aila_stream_destroy takes AilaTTSStream*, not AilaEngine*.

ASR and TTS stream handles retain shared ownership of their proxy engine.
Destroy all ASR streams and wait for or cancel, then destroy, all TTS streams
before destroying AilaEngine; otherwise worker shutdown can be deferred.

3. Published binaries use the oneAPI 2026.1 runtime

Do not replace or merge the DLLs under aila_runtime/ with a 2025.x oneAPI
installation or another application's runtime. Source builds now reject IntelLLVM
older than 2026.1 unless the explicit legacy option is used for the controlled
2025.3 comparison harness.

4. Structured chat is opt-in, but formatting behavior has changed

--messages-json continues to emit raw assistant text by default, and the
legacy generation functions remain available. However, chat rendering now uses
the redesigned Jinja/fixed-template pipeline. Integrations that depended on the
old hand-built ChatML formatting should compare prompts or supply an explicit
request/environment template override.

For new integrations, prefer AilaGenConfigV2 with
aila_generate_chat_json_ex or aila_generate_chat_json_stream_ex. Initialize
it with aila_default_gen_config_v2() or set struct_size and zero all reserved
fields. String pointers in structured stream events are valid only during the
callback and must be copied if retained.

Compatibility summary

• Existing blocking/streaming text generation, ASR, alignment, error, and
  Aila-owned memory-release entry points remain available.
• The legacy AilaGenConfig binary layout is unchanged.
• Windows inference execution is now out-of-process and requires the matching
  worker/runtime distribution.
• TTS streaming FFI declarations and call sites must migrate to
  AilaTTSStream*.
• CLI users launching from an extracted distribution should run
  aila_runtime/Aila.exe (or change into that directory).

Full Changelog: 0.1.6...0.1.7

Fix:

• Fix engine.extractSpeakerEmbedding for the broken Qwen3-TTS Base models' reference audio speaker embedding extraction.

Changes:

• Rename --spk to --ref in CLI
• Rename AILA_SPK_CACHE_DIR to AILA_REF_CACHE_DIR
• Rename reference audio cache file name from <audio_path>.spk.bin to <audio_path>.ref.bin, removal of old caches is recommended!
• Rename speaker_audio_path to reference_audio_path in API

Full Changelog: 0.1.5...0.1.6

Features:

• Add inference support for Qwen3-TTS CustomVoice models and VoiceDesign model (now Aila support all 5 Qwen3-TTS models)
• Add both API and CLI support for Qwen3-TTS CustomVoice&VoiceDesign models
• Add inference support for Qwen3-ForceAligner-0.6B model
• Add both API and CLI support for Qwen3-ForceAligner-0.6B model
• Add bitsandbytes NF4 quantization support for Qwen3-ForceAligner-0.6B (weight is currently available on huggingface Qwen3-ForceAligner-0.6B-BNB-NF4)
• Add streaming support for voice generation

Performance:

• Improve TTS voice generation performance, achieved RTF < 1 on 0.6B models (May furtherly improve in future releases)

Improvements:

• Improve CLI user experience. It is recommended to use PowerShell 7.0+ and with

$OutputEncoding = [System.Text.Encoding]::UTF8
[Console]::OutputEncoding = [System.Text.Encoding]::UTF8
[Console]::InputEncoding = [System.Text.Encoding]::UTF8

appended to your PowerShell Profile file（usually %USERPROFILE%\Documents\PowerShell\Microsoft.PowerShell_profile.ps1) or prepended to your commands

Changes:

• aila_synthesize API changed to 7 params

Full Changelog: 0.1.4...0.1.5

Features:

• Add inference support for Qwen3-TTS Base models (both Qwen3-TTS-12Hz-0.6B-Base BF16 and Qwen3-TTS-12Hz-1.7B-Base BF16), supporting zero-shot voice cloning(the performance may not be optimal, will improve in future releases)
• Add both API and CLI support for Qwen3-TTS Base models
• Add bitsandbytes NF4 quantization support for both Qwen3-ASR-0.6B and Qwen3-ASR-1.7B (weights are currently available on huggingface Qwen3-ASR-0.6B-BNB-NF4 Qwen3-ASR-1.7B-BNB-NF4)
• Add streaming(both input and output) support for audio transcription

Performance:

• Improve ASR transcription performance, achieved 10x faster than realtime speed using 1.7B models

Improvements:

• OpenAI Compatible API JSON format input support for LLM/VLM by using jinja engine from llama.cpp

Credits:

• Thanks to llama.cpp for the jinja engine

Full Changelog: 0.1.3...0.1.4

Features:

• Add support for Qwen3-ASR series models (including both Qwen3-ASR-0.6B BF16 and Qwen3-ASR-1.7B BF16, NF4 quantization support is still under development)
• Native support for Mono or Stereo .mp3 .wav .flac with any sample rate, can handle unsupported audio format(eg. .m4a) by using external ffmpeg (you have to add ffmpeg to your PATH to enable this)
• Update CLI and API for the new audio transcription feature
• WIP: LoRA support for Qwen3 series models for both BF16 and NF4, tested on Qwen3 0.6B and sakura-qwen3-0.6b-lora-demo-v1

Credits:

• Thanks to dr_libs for the native support of .mp3 .wav .flac
• Thanks to ffmpeg for other audio format support

Full Changelog: 0.1.2...0.1.3

Performance:

• Inference

Gain over 10% decode speed up by manually unrolling dequantize operator and packing weight in vec8
For A770 16G : currently best result is prefill 1649 tok/s decode 58 tok/s

• Startup & Warmup

Slightly improve the startup performance by disabling unnecessary oneDNN JIT compilation and reducing the warmup prompt length

Features:

• Add global log level control with 5 log levels

Bug Fix:

• Fix <think> disappearance in CLI's --messages-json output

Full Changelog: 0.1.1...0.1.2

Fix: Fix the COT(Chain Of Thought) in output.
New: Add /think suffix command to force thinking mode.
Change: Add seperate .exe and .dll to releases.

First public release of Aila.
2026.05.02 Fix the baseline path on B580 caused by the hardcoded subgroup size in decode phase. #1