PATRYK KULPOK
← BACK TO SYSTEM MAPSYSTEM MAP ENTRY

SYSTEM 07

WHISPER AI ASSISTANT

REAL-TIME SPEECH / TRANSLATION / LOCAL AI

Desktop AI assistant for real-time speech transcription, EN-PL translation and local AI-assisted responses.

PYTHONWHISPERCTRANSLATE2HUGGING FACELOCAL LLMSPEECH RECOGNITIONWINDOWS

SYSTEM IDENTITY

SYSTEM IDWHISPER-AI-07

STATUSWORKING PROTOTYPE

TYPESPEECH / TRANSLATION / LOCAL AI

Real-time desktop speech pipeline combining transcription, translation, question routing and a local language model.

01 / OVERVIEW

PROJECT OVERVIEW

The system captures speech from microphone or system audio input and turns it into structured textual context in near real-time.

Core local flow is: AUDIO to TRANSCRIPTION to LANGUAGE PROCESSING to QUESTION ROUTING to LOCAL AI.

System overview

  1. AUDIO
  2. TRANSCRIPTION
  3. LANGUAGE PROCESSING
  4. QUESTION ROUTING
  5. LOCAL AI

02 / USE CASES

ASSISTANT USE CASES

UC-01

LIVE MICROPHONE TRANSCRIPTION

Capture microphone audio and turn speech into near-real-time transcript segments.

UC-02

SYSTEM AUDIO INPUT

Route desktop loopback audio through the same local speech pipeline.

UC-03

EN / PL TRANSLATION

Process speech through an English and Polish translation workflow.

UC-04

QUESTION DETECTION

Detect complete spoken questions before assembling context for the assistant.

UC-05

LOCAL RESPONSE STREAM

Send question and context to a local Qwen model with streaming and cancellation control.

03 / AUDIO INPUT

AUDIO INPUT

The assistant can accept two local sources and keep them in short timestamped segments before transcription.

Speech signal map

SEG-01

MICROPHONE

local capture channel

SEG-02

SYSTEM AUDIO / LOOPBACK

desktop audio channel

AUDIO INPUT MATRIX
AUDIO BUFFERT-01 / T-02
SPEECH PROCESSINGVOICE FLOW

Input chain

  1. MICROPHONE
  2. SYSTEM AUDIO / LOOPBACK
  3. AUDIO BUFFER
  4. SPEECH PROCESSING

04 / SPEECH PIPELINE

SPEECH PIPELINE

The speech flow starts by segmenting local audio, then passes every segment through Whisper and produces transcript output.

Speech processing

  1. AUDIO INPUT
  2. BUFFER / SEGMENT
  3. WHISPER
  1. TRANSCRIPT
  2. LANGUAGE ROUTING

Transcript branching

TRANSLATION PATH

EN - PL translation can be built through a local CTranslate2 pipeline using Hugging Face models.

A separate manual translation workflow is also available.

QUESTION PATH

Transcript may also be routed directly for question analysis and assistant flow.

This branch prepares context for response composition and local model handoff.

TRANSLATION IS OPTIONAL IN THE PIPELINE

05 / QUESTION ROUTING

QUESTION ROUTING

Question detection is treated as a routing step before local model invocation.

Question routing chain

  1. TRANSCRIPT
  2. QUESTION ANALYSIS
  3. QUESTION COMPLETE
  4. CONTEXT ASSEMBLY
  5. LOCAL LLM
  1. QUESTION DETECTED

    The system marks incoming text as question-intent when the route condition matches.

  2. QUESTION COMPLETE

    Parsed sentence is verified and prepared as a full request signal.

  3. QUESTION CONTENT

    The question content is extracted from the transcript and prepared for downstream context assembly.

  • ADDRESSEE
  • PROFILE CONTEXT REQUIRED

06 / LOCAL AI ASSISTANT

LOCAL AI ASSISTANT

For question-ready transcripts, the assistant builds a request from complete question, conversation context and user context, then generates output through a local language model.

Local assistant flow

  1. COMPLETE QUESTION + CONTEXT
  2. LOCAL QWEN LLM
  3. STREAMING RESPONSE
  • Preload: model context is prepared before request starts.
  • Health check: runtime and input readiness status is verified.
  • Streaming output: tokens are emitted incrementally.
  • Cancellation: generation can be interrupted by user action.
  • Conversation context: prior turns are carried into new prompts.
  • Follow-up questions: prompts can build from previous exchanges.

07 / TECH STACK

TECH STACK

  • Python
  • Whisper
  • CTranslate2
  • Hugging Face Transformers
  • Local LLM / Qwen
  • Speech Recognition
  • NLP
  • Windows

08 / STATUS

PROJECT STATE

SYSTEM ID WHISPER-AI-07

STATE WORKING PROTOTYPE

DEPTH REAL-TIME SPEECH + LOCAL AI SYSTEM