I-Parakeet: Integer-Only Conformer ASR on Mobile NPU

What happened

To resolve key deployment bottlenecks of existing Conformer speech recognition models: large model footprints, and continued reliance on floating-point fallback operations even post-quantization that blocks full mobile NPU compute utilization, research teams built I-Parakeet on NVIDIA’s 0.6B-parameter Parakeet-CTC base model. Using three core optimizations: integer-based core attention redesign, Swish activation approximation, and hierarchical activation value range calibration, the team created a model that runs on smartphone NPUs with zero floating-point dependencies.

Key facts

Base model

NVIDIA Parakeet-CTC (0.6B parameters)

Core features

Pure integer implementation, no floating-point operations, no CPU fallback

Test device

Qualcomm smartphone NPU

WER on LibriSpeech test-other dataset

4.97%

Real-time factor (RTF)

0.048

Speedup relative to CPU baseline

7.5x

Background

Most leading Conformer speech recognition models have large footprints that make edge deployment difficult. Even after quantization compression, most still require floating-point compute fallback for numerically sensitive operations, preventing them from unlocking the full performance potential of integer accelerators like mobile NPUs.

Why it matters

This work unlocks a full-integer deployment path for high-accuracy Conformer speech recognition models on on-device NPUs, lowering the compute barrier for edge speech recognition. For developers, it delivers a reusable pure-integer quantization adaptation framework to deploy high-accuracy ASR without requiring floating-point compute support. For end users, it enables lower-latency, more power-efficient on-device speech recognition that balances strong accuracy with improved privacy.

What to watch

Future watchpoints include the adaptation and rollout of this pure-integer ASR solution across mainstream smartphone platforms, as well as its accuracy in complex real-world speech scenarios.