AI Providers

Actyze supports 100+ AI providers through LiteLLM integration, giving you the flexibility to use any LLM service - from major cloud providers to local open-source models - with a consistent, simple configuration.

Overview

Powered by LiteLLM

Actyze uses LiteLLM, an open-source library that provides a unified interface to 100+ LLM providers. This means:

✅ No Vendor Lock-in - Switch providers anytime with just 2 lines of config
✅ Consistent Interface - Same configuration pattern for all providers
✅ Automatic Handling - Authentication, rate limits, and retries handled automatically
✅ Future-Proof - New providers supported as LiteLLM adds them
✅ Local & Cloud - Use cloud APIs or run models locally

Configuration Pattern:

All providers follow this simple 2-line pattern:

PROVIDER_API_KEY=your-api-key
EXTERNAL_LLM_MODEL=provider-model-name

Supported Providers

Popular Cloud Providers

Industry-leading AI services with production-ready APIs.

Provider	Configuration	Models	Best For
Anthropic Claude	`ANTHROPIC_API_KEY`	`claude-sonnet-4-20250514`, `claude-opus-4-20250514`	Best SQL accuracy, complex queries
OpenAI	`OPENAI_API_KEY`	`gpt-4o`, `gpt-4-turbo`, `gpt-3.5-turbo`	General purpose, widely supported
Google Gemini	`GEMINI_API_KEY`	`gemini/gemini-pro`, `gemini/gemini-2.0-flash`	Fast, cost-effective
Perplexity	`PERPLEXITY_API_KEY`	`perplexity/sonar-reasoning-pro`	Real-time data, research
Cohere	`COHERE_API_KEY`	`cohere/command-r-plus`	Enterprise, fine-tuning

Example - Anthropic Claude:

ANTHROPIC_API_KEY=sk-ant-xxxxx
EXTERNAL_LLM_MODEL=claude-sonnet-4-20250514

Enterprise Cloud Providers

Enterprise-grade AI services with SLAs, compliance, and dedicated support.

Provider	Configuration	Models	Best For
Azure OpenAI	`AZURE_API_KEY`, `AZURE_API_BASE`	`azure/gpt-4o`, `azure/your-deployment`	Microsoft Azure customers
AWS Bedrock	`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`	`bedrock/us.anthropic.claude-sonnet-4`, `bedrock/anthropic.claude-v2`	AWS infrastructure
Google Vertex AI	`VERTEXAI_PROJECT`, `VERTEXAI_LOCATION`	`vertex_ai/gemini-pro`	Google Cloud Platform
IBM watsonx.ai	`WATSONX_API_KEY`	`watsonx/meta-llama`	IBM enterprise stack

Example - Azure OpenAI:

AZURE_API_KEY=your-azure-key
AZURE_API_BASE=https://your-resource.openai.azure.com
EXTERNAL_LLM_MODEL=azure/your-deployment-name

Fast & Free Tier Providers

High-performance models with generous free tiers or pay-per-use pricing.

Provider	Configuration	Models	Best For
Groq	`GROQ_API_KEY`	`groq/llama-3.3-70b-versatile`, `groq/mixtral-8x7b`	Ultra-fast inference, testing
Together AI	`TOGETHERAI_API_KEY`	`together_ai/meta-llama/Meta-Llama-3.1`	Open models, low cost
Fireworks AI	`FIREWORKS_API_KEY`	`fireworks_ai/llama-v3-70b`	Fast, open source models
Replicate	`REPLICATE_API_KEY`	`replicate/meta/llama-2-70b`	Model marketplace

Example - Groq:

GROQ_API_KEY=gsk_xxxxx
EXTERNAL_LLM_MODEL=groq/llama-3.3-70b-versatile

Open Source & Local Providers

Run models locally or on your infrastructure - no API costs, complete privacy.

Provider	Configuration	Models	Best For
Ollama	`OLLAMA_API_BASE`	`ollama/llama3`, `ollama/mistral`	Local deployment, privacy
LM Studio	`LM_STUDIO_API_BASE`	`lm_studio/model-name`	Desktop, development
vLLM	`VLLM_API_BASE`	`vllm/model-name`	Self-hosted, GPU clusters
OpenRouter	`OPENROUTER_API_KEY`	`openrouter/auto`	Multiple models, routing
Hugging Face	`HUGGINGFACE_API_KEY`	`huggingface/model-name`	Research, custom models

Example - Ollama (Local):

OLLAMA_API_BASE=http://localhost:11434
EXTERNAL_LLM_MODEL=ollama/llama3
# No API key needed for local Ollama

Specialized Providers

Purpose-built AI services for specific use cases.

Provider	Configuration	Use Case
Anthropic via Bedrock	AWS Bedrock	Enterprise AWS deployments
OpenAI via Azure	Azure OpenAI	Microsoft compliance requirements
Custom OpenAI-compatible	Enterprise gateways	IT-managed AI access
Mistral AI	`MISTRAL_API_KEY`	European AI sovereignty
AI21 Labs	`AI21_API_KEY`	Multilingual, long context

Configuration

Standard Providers

For cloud providers (Anthropic, OpenAI, Gemini, etc.), use this 2-line pattern:

Docker (.env):

# Choose your provider
ANTHROPIC_API_KEY=sk-ant-xxxxx
EXTERNAL_LLM_MODEL=claude-sonnet-4-20250514

# Or use OpenAI
# OPENAI_API_KEY=sk-xxxxx
# EXTERNAL_LLM_MODEL=gpt-4o

# Or use Gemini
# GEMINI_API_KEY=your-key
# EXTERNAL_LLM_MODEL=gemini/gemini-pro

Helm (values.yaml + values-secrets.yaml):

# values.yaml
modelStrategy:
  externalLLM:
    enabled: true
    model: "claude-sonnet-4-20250514"

# values-secrets.yaml
secrets:
  externalLLM:
    apiKey: "sk-ant-xxxxx"

Local/Self-Hosted Models

For local models (Ollama, LM Studio, vLLM), set the base URL:

Docker (.env):

# Ollama running locally
OLLAMA_API_BASE=http://localhost:11434
EXTERNAL_LLM_MODEL=ollama/llama3

# Or vLLM on your server
# VLLM_API_BASE=http://your-server:8000
# EXTERNAL_LLM_MODEL=vllm/mistral-7b

Helm (values.yaml):

modelStrategy:
  externalLLM:
    enabled: true
    model: "ollama/llama3"
    baseUrl: "http://ollama-service:11434"

Enterprise Gateway

For IT-managed AI gateways that proxy multiple providers:

Docker (.env):

# Enterprise gateway configuration
EXTERNAL_LLM_MODE=openai-compatible
EXTERNAL_LLM_BASE_URL=https://llm-gateway.yourcompany.com/v1/chat/completions
EXTERNAL_LLM_API_KEY=your-enterprise-token
EXTERNAL_LLM_MODEL=your-internal-model-name

# Optional: Add custom headers for tracking
EXTERNAL_LLM_AUTH_TYPE=bearer
EXTERNAL_LLM_EXTRA_HEADERS={"X-Department": "engineering", "X-Cost-Center": "analytics"}

Use Cases:

Centralized billing and cost tracking
Compliance and audit logging
Rate limiting and access control
Multi-model routing and fallback
Custom authentication requirements

Model Selection Guide

By Use Case

Production SQL Generation:

Best: claude-sonnet-4-20250514 - Highest accuracy
Fast: gpt-4o - Good balance of speed and accuracy
Budget: gemini/gemini-pro - Cost-effective

Development & Testing:

Local: ollama/llama3 - No API costs
Cloud: groq/llama-3.3-70b-versatile - Free tier, very fast

Enterprise Compliance:

AWS: bedrock/us.anthropic.claude-sonnet-4-20250514 (inference profiles supported)
Azure: azure/gpt-4o
GCP: vertex_ai/gemini-pro

By Performance

Fastest (< 1s response):

Groq (any model)
Gemini Flash
Together AI

Most Accurate:

Claude Opus 4
GPT-4o
Claude Sonnet 4

Best Value:

Gemini Pro (cheapest)
Groq (free tier)
Ollama (self-hosted, no API costs)

Need Another Provider?

Check LiteLLM Documentation

Actyze supports all providers that LiteLLM supports. For the complete, up-to-date list:

📚 LiteLLM Providers Documentation:

https://docs.litellm.ai/docs/providers

This includes:

100+ cloud and local providers
Configuration examples for each
Model names and capabilities
Authentication requirements

Configuration Pattern

All LiteLLM providers follow this pattern:

# 1. Set the provider's API key (if needed)
PROVIDER_API_KEY=your-key

# 2. Set the model name (format: provider/model-name)
EXTERNAL_LLM_MODEL=provider/model-name

# 3. Optional: Set base URL for self-hosted
PROVIDER_API_BASE=http://your-server:port

Example - Using Mistral AI:

Go to: https://docs.litellm.ai/docs/providers/mistral
Get your Mistral API key

Configure:

MISTRAL_API_KEY=your-mistral-key
EXTERNAL_LLM_MODEL=mistral/mistral-large-latest

Request a Provider

If you need a provider not yet supported by LiteLLM:

Check LiteLLM GitHub: https://github.com/BerriAI/litellm/issues
Open an issue requesting the provider
Contact Actyze Support with your use case

We'll work with the LiteLLM team to add support.

Troubleshooting

Provider Not Working

Check configuration:

# Verify environment variables
docker-compose exec nexus env | grep -i "api_key\|llm"

# Or in Kubernetes
kubectl exec -n actyze deployment/dashboard-nexus -- env | grep -i "api_key\|llm"

Common issues:

❌ Wrong model name format (should be provider/model-name)
❌ API key not set or incorrect
❌ Model name doesn't exist for that provider
❌ Rate limit exceeded (check provider dashboard)

LiteLLM Version & Compatibility

Current Version: Actyze uses LiteLLM v1.81.12 (February 2026)

Key Features:

✅ AWS Bedrock inference profiles support (global.*, us.* prefixes)
✅ Bedrock Batches API & Knowledge Bases
✅ Claude Code web search across all providers
✅ 100+ LLM providers with unified interface

Check Compatibility: https://docs.litellm.ai/docs/providers

⚠️ Bedrock Users: If using inference profiles like bedrock/us.anthropic.claude-sonnet-4-20250514, ensure you're on v1.81.12+ for proper support.

Test Provider Directly

Test your API key directly with the provider's API:

# Example: Test Anthropic API key
curl https://api.anthropic.com/v1/messages \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{"model":"claude-sonnet-4-20250514","max_tokens":100,"messages":[{"role":"user","content":"test"}]}'

Best Practices

Production Deployments

Use Enterprise Providers for SLAs and support
- Azure OpenAI, AWS Bedrock, or Vertex AI
Configure Fallbacks in your enterprise gateway
- Primary: Claude Sonnet 4
- Fallback: GPT-4o
Monitor Costs via provider dashboards

Set Rate Limits in .env:

EXTERNAL_LLM_MAX_TOKENS=4096
EXTERNAL_LLM_TEMPERATURE=0.1

Development & Testing

Use Free Tiers for testing:
- Groq (fastest, free tier)
- Ollama (local, no costs)

Switch Easily between providers:

# Test with free Groq
GROQ_API_KEY=gsk_xxxxx
EXTERNAL_LLM_MODEL=groq/llama-3.3-70b-versatile

# Then production with Claude
ANTHROPIC_API_KEY=sk-ant-xxxxx
EXTERNAL_LLM_MODEL=claude-sonnet-4-20250514

Cost Optimization

Start with cheaper models: Gemini Pro or Groq
Use local models for development: Ollama
Cache responses (enabled by default in Actyze)
Monitor token usage via provider dashboards

Next Steps

Quick Start - Get Actyze running
LLM Provider Configuration - Detailed setup examples
Docker Deployment - Deploy with Docker
Helm Deployment - Deploy on Kubernetes

Additional Resources

LiteLLM Documentation - Complete LiteLLM guide
LiteLLM Providers - All supported providers
Model Pricing - Compare provider pricing
GitHub Repository - Actyze source code

Overview​

Supported Providers​

Popular Cloud Providers​

Enterprise Cloud Providers​

Fast & Free Tier Providers​

Open Source & Local Providers​

Specialized Providers​

Configuration​

Standard Providers​

Local/Self-Hosted Models​

Enterprise Gateway​

Model Selection Guide​

By Use Case​

By Performance​

Need Another Provider?​

Check LiteLLM Documentation​

Configuration Pattern​

Request a Provider​

Troubleshooting​

Provider Not Working​

LiteLLM Version & Compatibility​

Test Provider Directly​

Best Practices​

Production Deployments​

Development & Testing​

Cost Optimization​

Next Steps​

Additional Resources​

Overview

Supported Providers

Popular Cloud Providers

Enterprise Cloud Providers

Fast & Free Tier Providers

Open Source & Local Providers

Specialized Providers

Configuration

Standard Providers

Local/Self-Hosted Models

Enterprise Gateway

Model Selection Guide

By Use Case

By Performance

Need Another Provider?

Check LiteLLM Documentation

Configuration Pattern

Request a Provider

Troubleshooting

Provider Not Working

LiteLLM Version & Compatibility

Test Provider Directly

Best Practices

Production Deployments

Development & Testing

Cost Optimization

Next Steps

Additional Resources