AI Model Support
Knowably supports multiple LLM providers including OpenAI, Anthropic, and Azure OpenAI. Switch models per agent without rewriting integration code.
Which LLM are you going to use two years from now?
I have no idea. Neither do you. The model landscape shifts every few months. Today's best-in-class is tomorrow's baseline. If your knowledge platform is hard-wired to a single provider, you are making a bet you cannot unwind without rewriting integration code.
I built Knowably to be model-agnostic from day one. Eleven providers out of the box, from OpenAI and Anthropic to self-hosted models via Ollama. You bring your own API keys. You choose which models to use. You switch providers without touching your agents, knowledge bases, or pipeline configuration.
No vendor lock-in. Full control.
Get Started | View Provider List
The Lock-in Problem
Most teams pick a provider early. OpenAI, usually. The integration goes deep. Prompt templates get tuned to one model's behavior. Token counting and cost logic gets hard-coded to one pricing structure.
Then Anthropic releases a new Claude. Or DeepSeek drops a model that is ten times cheaper for your workload. Or a regulation says production data cannot leave your network.
Switching should be straightforward. In practice, it rarely is.
Different steps in a RAG pipeline have different requirements. A fast, cheap model works well for query expansion. A large reasoning model is better for complex multi-source answers. Locking into one provider means you cannot optimize for each step independently.
I wanted Knowably to solve this at the architecture level. So I did.
Supported Providers {#providers}
Eleven providers, configured and ready. Add your API key. Pick your models. Done.
Commercial Cloud Providers
OpenAI - The GPT-4o family, GPT-4 Turbo, and the o-series reasoning models. Model auto-discovery fetches new models as OpenAI releases them.
Anthropic - Claude Opus, Sonnet, and Haiku across the latest generations. Auto-discovery via the Anthropic Models API means new Claude releases appear automatically.
Google Gemini - Google's multimodal model family, accessed through the Generative Language API.
Mistral - European AI provider with strong multilingual performance at competitive pricing.
DeepSeek - High-performance models at a fraction of the cost. Worth evaluating for cost-sensitive workloads.
xAI - Grok models, accessible through a standard API.
Aggregator and Routing Services
OpenRouter - Access hundreds of models from dozens of providers through a single API key. Useful for experimentation or accessing niche models without managing separate accounts.
AWS Bedrock - Enterprise-grade access with IAM-based authentication. No API key required when using the default AWS credential provider chain. Region-configurable for data residency.
Speed-Optimized Inference
Groq - Ultra-low-latency inference on custom hardware. When response speed matters more than model size, Groq delivers sub-second token generation.
Self-Hosted and On-Premise
Ollama - Run open-source models on your own infrastructure. Llama, Mistral, Phi, Gemma - whatever runs on Ollama runs in Knowably. No API key needed. No data leaves your network.
Ollama support exists because some organizations cannot send a single token to an external API. When compliance mandates that production data stays within your perimeter, this gives you a fully self-hosted LLM without sacrificing the platform around it.
Embeddings
VoyageAI - Dedicated embedding models for high-quality vector representations. These are separate from chat models. VoyageAI powers the semantic search layer that makes the RAG pipeline work.
Architecture: Prism PHP
I built on Prism PHP (prism-php/prism) because I did not want to write provider-specific code for every LLM. One interface, any model.
The same code calls OpenAI, Anthropic, Ollama, or any other provider. The only thing that changes is the provider name and model identifier:
Prism::text()
->using('anthropic', 'claude-sonnet-4-5-20250929')
->withSystemPrompt($systemPrompt)
->withPrompt($userPrompt)
->asText();
Switch 'anthropic' to 'openai' and the model to 'gpt-4o'. Everything else stays the same. No adapter code. No provider-specific branches.
For AWS Bedrock, the prism-php/bedrock companion package handles IAM authentication.
If Prism adds a provider, Knowably supports it. Zero application code changes.
Connections and Models
Knowably uses a two-level hierarchy for provider access.
A connection is a provider combined with an API key and optional configuration. Think of it as "our organization's access to Anthropic" or "the engineering team's OpenAI account."
A model is a specific model within that connection. One Anthropic connection might have Claude Opus 4.5, Claude Sonnet 4.5, and Claude Haiku 4.5 - each independently enabled or disabled.
Multiple connections per provider. You can create two OpenAI connections with different API keys. One for the product team, one for internal tools. Separate billing. Separate rate limits. Same provider.
Custom base URLs. Each connection supports a configurable base URL. Point an OpenAI connection to an Azure OpenAI endpoint. Route an Anthropic connection through a private proxy. Point Ollama at a remote server instead of localhost.
Independent activation. Disable a connection and all its models go dark instantly. Enable or disable individual models without removing them.
Dynamic Model Discovery
When you add an OpenAI or Anthropic connection, Knowably does not rely on a hard-coded model list. It queries the provider's API directly.
For OpenAI, we call /v1/models and filter for GPT and o-series models. For Anthropic, the same endpoint filtered for Claude models. New releases appear automatically. No platform update required.
Results are cached for one hour per connection. Clear the cache manually to force a refresh.
If the provider API is unreachable - network issue, invalid key, rate limit - Knowably falls back to a curated static list maintained in configuration. You always have models available, even when discovery is temporarily down.
For other providers (Mistral, Groq, xAI, Gemini, DeepSeek, Ollama, VoyageAI, OpenRouter, Bedrock), models come from the static list. These providers either lack a standardized listing endpoint or their catalog changes infrequently.
Per-Tenant Configuration
Every workspace in Knowably is a fully isolated tenant with its own database. LLM connections are no exception.
Tenant A can use Anthropic exclusively. Tenant B can run a mix of OpenAI and Ollama. Tenant C can connect to AWS Bedrock for compliance. No shared configuration. No platform defaults overriding tenant choices.
Connections, models, usage records, and API keys all live in the tenant's own database. Cross-tenant access is architecturally impossible. Actual database separation, not access control rules.
Each tenant sets its own default model - the model used when no specific model is selected for an agent or conversation. Only enabled models on active connections qualify. One default at a time, enforced at the database level.
Bring Your Own API Keys
Every tenant brings their own keys for every provider they use.
Encrypted at rest. API keys are encrypted using AES-256-CBC before they touch the database. The raw key is decrypted only at the moment of an API call, held in memory briefly, then discarded.
Hidden from output. The api_key field is in the model's $hidden array. Never included in API responses, JSON serialization, or log output. You cannot accidentally leak it through an endpoint.
Validated before activation. When you add a connection, Knowably sends a minimal test request - a simple prompt with a 10-token max response. If the key is invalid, you know immediately.
Custom configuration per connection. Beyond the API key, connections support provider-specific settings: custom base URLs, API versions, organization IDs, and project IDs.
Model Selection and Defaults
Model selection works at two levels.
Tenant default. Set one model as the workspace-wide default. Every new conversation and every agent without a specific assignment uses this model. Change the default and it takes effect immediately for new conversations.
Agent override. Individual agents can specify a different model. Your HR assistant might use a fast, cheap model for simple policy lookups. Your technical research agent might use a larger reasoning model for complex synthesis.
Only enabled models on active connections appear in the selection interface. Display names include the provider for clarity - [anthropic] Claude Sonnet 4.5 versus [openai] GPT-4o - so there is no ambiguity.
Usage Tracking and Cost Management
How much did your LLM calls cost yesterday?
If you cannot answer that per model, per user, and per day, you do not have cost visibility. You have hope.
Per-Request Tracking
Every LLM call records:
- Input tokens and output tokens (tracked separately, because they cost different amounts)
- Total tokens consumed
- Response time in milliseconds
- The specific user, model, and chat message that triggered the call
- Error messages when a call fails
This is not sampling. Every single request is tracked.
Cost Estimation
Knowably maintains a cost map with per-model token pricing for known models - GPT-4o, GPT-4o Mini, GPT-4 Turbo, Claude Opus, Sonnet, and Haiku families. Input and output tokens are priced separately. The cost estimate appears on every usage record.
Daily Analytics
Usage data rolls up into daily summaries: total requests, total tokens, total estimated cost, and error count. Filter by date range. Compare models side by side. Track average response times to spot performance changes after a model update.
Admins see workspace-wide analytics. Regular users see their own usage only.
Security and Governance
Security around LLM access is not just about encrypting keys. It is about controlling who can do what and knowing what happened.
Role-based access control. Separate permissions govern who can create connections, manage models, and view usage data. Not every user needs access to API key management.
Connection validation. Every connection is tested before activation. A misconfigured key cannot silently cause runtime failures.
Tenant isolation. Connections, models, keys, and usage records live in per-tenant databases. There is no query that could cross tenant boundaries.
Audit trail. Every LLM interaction is linked to a specific user and timestamp. "Who ran 50,000 tokens through GPT-4 Turbo on Tuesday?" is a question you can answer.
Integration with the RAG Pipeline
The LLM you choose powers multiple stages of the RAG pipeline. Not just the final answer.
Query decomposition - breaking complex questions into focused sub-queries. Query expansion - generating alternative phrasings to improve retrieval. Chunk filtering - evaluating whether retrieved passages are actually relevant. Answer generation - synthesizing the final response from source documents.
All of these stages use the configured model. Switch from GPT-4o to Claude Sonnet, and the entire pipeline switches. No reconfiguration. The pipeline code never references a specific provider - it asks for the configured model and uses whatever it gets.
Deep dive into the RAG pipeline
Provider Comparison
| Provider | Authentication | Auto-Discovery | Self-Hosted | Embeddings |
|---|---|---|---|---|
| OpenAI | API key | Yes | No (Azure via custom URL) | No |
| Anthropic | API key | Yes | No | No |
| Google Gemini | API key | Static list | No | No |
| Mistral | API key | Static list | No | No |
| DeepSeek | API key | Static list | No | No |
| xAI | API key | Static list | No | No |
| OpenRouter | API key | Static list | No | No |
| AWS Bedrock | IAM / API key | Static list | No | No |
| Groq | API key | Static list | No | No |
| Ollama | None | Static list | Yes | No |
| VoyageAI | API key | Static list | No | Yes |
Getting Started
Adding your first LLM connection takes three steps.
Create a connection. Choose a provider, enter your API key, and optionally configure a custom base URL or API version.
Validate. Knowably tests the connection with a minimal request. Green means ready.
Enable models. Select which models you want available. Set one as the default.
That is it. Your agents can now use the model.
Want to add a second provider? Same three steps. Your agents can pick from any enabled model across any active connection.
Configure your first agent | Book a Demo
FAQ
Can I use multiple providers at the same time?
Yes. Each agent can be assigned a specific model from any active connection. Your HR agent can run on Claude Sonnet while your engineering agent uses GPT-4o.
What happens if a provider goes down?
Failed requests are logged with error details and response times. You can switch the default model to a different provider immediately. There is no automatic failover - I prefer explicit control over silent fallbacks.
Do you support Azure OpenAI?
Yes. Create an OpenAI connection and set the base URL to your Azure OpenAI endpoint. The API is compatible.
Can I run models on my own hardware?
Yes. Ollama integration lets you run any Ollama-compatible model on your own infrastructure. Point the connection URL at your Ollama server. No API key needed. No data leaves your network.
How accurate are the cost estimates?
Based on published per-model token pricing and actual token counts per request. They are estimates, not invoices - but they track closely with provider billing for supported models.