The Storyteller software ecosystem uses a sophisticated forced alignment algorithm to automatically synchronize ebook text with audiobook narration, enabling immersive reading experiences through the EPUB Media Overlay specification.

Key Points

• Storyteller automates audio-text synchronization by aligning ebook content with audiobook narration without requiring manual user input.
• The system utilizes the Massively Multilingual Speech (MMS) model and Wav2Vec 2.0 to generate CTC emissions for audio processing.
• A "RANSAC’d n-grams" boundary search identifies corresponding text segments between formats, effectively handling missing or reordered chapters.
• The CTC Viterbi algorithm calculates the optimal path for word-level alignment, determining precise start and end times for narration.
• These features are currently available in the Storyteller v3 beta and the stalign tool, with a wider release planned for the near future.

Why it Matters

This technology significantly lowers the barrier for creating immersive, synchronized digital books by removing the need for manual, time-consuming alignment processes. By automating this synchronization, Storyteller enhances accessibility and user engagement for readers who prefer or require audio-visual reinforcement while consuming literature.