Adding Providers

Step-by-step guide to adding a new STT, LLM, or TTS provider to Voxray

Voxray selects providers at startup from config.json. Adding a new provider means creating a Go package under pkg/services/, implementing the correct interface, and registering the provider in the factory so it can be chosen by name. This guide walks through every step.

Before starting, read pkg/services/interfaces.go and pkg/services/factory.go. The factory is the single file that wires provider names to concrete implementations — most of your registration work happens there.

Service Interfaces

Every provider must satisfy one or more of the following Go interfaces. These are defined in pkg/services/interfaces.go and pkg/services/llmapi/api.go.

LLMService

// LLMService provides chat completion; may stream text frames.
// Defined in pkg/services/llmapi/api.go.
type LLMService interface {
    Chat(ctx context.Context, messages []map[string]any, onToken func(*frames.LLMTextFrame)) error
}

// LLMServiceWithTools is an LLM service that supports registering tools (e.g. from MCP).
type LLMServiceWithTools interface {
    LLMService
    RegisterTool(schema schemas.FunctionSchema, handler ToolHandler)
    ToolsSchema() *schemas.ToolsSchema
}

Chat must stream tokens incrementally by calling onToken for each delta and return nil on success or a wrapped error on failure. Context cancellation must abort the stream and return promptly.

STTService

// STTService transcribes audio to text (batch).
type STTService interface {
    Transcribe(ctx context.Context, audio []byte, sampleRate, numChannels int) ([]*frames.TranscriptionFrame, error)
}

// STTStreamingService extends STTService with real-time streaming transcription.
type STTStreamingService interface {
    STTService
    // TranscribeStream sends TranscriptionFrames (interim and final) to outCh
    // as audio arrives on audioCh, without waiting for the full segment.
    TranscribeStream(
        ctx context.Context,
        audioCh <-chan []byte,
        sampleRate, numChannels int,
        outCh chan<- frames.Frame,
    )
}

Transcribe is the minimum requirement. If the upstream provider offers a streaming WebSocket or gRPC API, also implement STTStreamingService — the pipeline will use it automatically to reduce first-token latency.

TTSService

// TTSService converts text to speech (batch).
type TTSService interface {
    Speak(ctx context.Context, text string, sampleRate int) ([]*frames.TTSAudioRawFrame, error)
}

// TTSStreamingService extends TTSService with incremental audio output.
type TTSStreamingService interface {
    TTSService
    // SpeakStream streams TTSAudioRawFrames to outCh as they are produced,
    // reducing time-to-first-audio.
    SpeakStream(ctx context.Context, text string, sampleRate int, outCh chan<- frames.Frame)
}

---

Steps

---

Provider Checklist

Use this checklist before opening a pull request. Every box must be checked.

Configuration

• No hardcoded API keys or secrets anywhere in the package.
• API key is wired through apiKeyForProvider with both a config.json key and an environment variable fallback.
• All config fields (model, voice, language, region, base URL, etc.) are documented in the PR description and in docs/build/integrations/<provider>.mdx.
• Reasonable defaults are provided for optional fields (model name, sample rate, language, etc.).

Interface compliance

• The struct satisfies the interface at compile time (add var _ services.LLMService = (*LLMService)(nil) if helpful).
• If the provider supports streaming, STTStreamingService or TTSStreamingService is also implemented, not just the batch interface.
• RealtimeService is implemented if the provider offers a realtime/duplex API (and registered in SupportedRealtimeProviders).

Correctness and robustness

• context.Context is passed to every network call; cancellation aborts the operation promptly.
• All errors from the upstream SDK or HTTP response are wrapped with provider context (fmt.Errorf("myprovider: %w", err)).
• No panic in public API paths or on transient provider errors.
• Goroutines launched inside the package are tied to a context.Context and exit when it is cancelled.
• Shared mutable state (if any) is protected by a mutex with documented assumptions.

Observability

• Prometheus metrics (latency histogram, error counter) are recorded consistently with other providers.
• Logging uses pkg/logger and avoids noisy per-token or per-chunk log lines.

Testing

• Unit tests cover at minimum: success path, context cancellation, and upstream error response.
• Mock or recorded fixtures are used so unit tests are offline and deterministic.
• Integration test is added under tests/pkg/services/<provider>/ and is skipped when MYPROVIDER_API_KEY is unset.
• go test ./... passes with no failures and no race conditions (go test -race ./...).

Registration

• Provider constant added to the const block in factory.go.
• Constant appended to all applicable Supported*Providers slices.
• case added in all applicable factory switch statements (NewLLMFromConfig, NewSTTFromConfig, NewTTSFromConfig).
• case added in apiKeyForProvider.
• Provider name is consistent across constant, config key, env var prefix, and documentation.