Copy page

AI Gateway

Zuplo AI Gateway Getting Started

This guide will walk you through setting up your first AI Gateway project, from initial configuration to making your first LLM request through Zuplo.

Prerequisites

• A Zuplo account (sign up free at zuplo.com)
• API keys for at least one LLM provider (OpenAI, Anthropic, Google Gemini, etc.)
• An application that needs to call LLM APIs

Step 1: Create an AI Gateway Project

1. Log into your Zuplo account
2. Navigate to Projects
3. Click New Project
4. Click AI or MCP Gateway at the bottom of the dialog
5. Give your project a name (for example, "MyCompany AI Gateway")
6. Click Create Project

Your AI Gateway project will be created in seconds. You'll notice the interface includes Apps, Teams, and a setup guide to help you get started.

Step 2: Configure Providers

Providers are the LLM services (like OpenAI or Google Gemini) that your applications will use. You'll configure these once as an administrator, and your team members can use them without needing direct access to provider API keys.

Adding Your First Provider

1. Click Add Provider
2. Select your AI provider (for example, OpenAI)
3. Enter a name for this provider configuration
4. Paste your provider's API key
5. Select which models you want to make available to your teams
6. Click Create

Adding Additional Providers

Repeat the process above to add more providers. This allows your teams to switch between providers (OpenAI, Gemini, etc.) without changing application code.

Example providers you might add:

• OpenAI (for GPT models)
• Google Gemini (for Gemini models)
• Additional providers as they become available

Step 3: Create a Team

Teams allow you to organize users and set hierarchical budget controls. Even if you're starting solo, you'll need at least one team.

Creating Your Root Team

1. Click Create Team
2. Name your team (for example, "Root" or your company name)
3. Choose an icon for easy identification
4. Click Create Team
5. Set organization-wide limits (optional) by selecting the Usage & Limits tab:
   • Budget Limit: Maximum spend per day (for example, $1,000)
   • Rate Limits: Request limits if needed

Creating Sub-Teams (Optional)

For larger organizations, create sub-teams with their own budgets:

1. From your root team, click Create Sub-Team
2. Name the team (for example, "Engineering Team", "Credit Team")
3. Choose an icon
4. Set team-specific limits by clicking on Settings:
   • Daily budgets that are equal to or less than the parent team's limit
   • Example: If root is $1,000/day, a sub-team might be $500/day
5. Click Save Changes

Step 4: Create Applications

Applications represent individual projects or services that will use the AI Gateway. Each app gets its own unique URL and API key.

Creating Your First App

1. Click Apps followed by Create App
2. Configure your app:
   • App Name: Descriptive name (for example, "Tennis Chat", "Customer Support Bot")
   • Team: Select which team owns this app
   • Provider: Choose your LLM provider (for example, OpenAI)
   • Completions: Select the model for chat completions (for example, GPT-4o)
   • Embeddings: Select the model for embeddings (optional)
3. Set application-level budgets:
   • Daily Limit: (for example, $1/day for a hackathon project)
   • Monthly Limit: (for example, $10/month)
4. Enable Semantic Caching (optional):
   • Caches similar prompts to reduce costs and improve performance
   • Best for applications with repeated queries
5. Click Create App

Access Your App Credentials

After creating your application, you'll see:

• API Key: Your Zuplo-managed key. You'll need it to integrate with your application.

Step 5: Integrate with Your Application

Now you'll update your application to use the Zuplo AI Gateway instead of calling LLM providers directly.

In the examples below, we assume that you are using the official Node.js OpenAI SDK and that you have executed npm install openai.

Before: Direct Provider Integration (sample.mjs)

Code
import OpenAI from "openai";

// Old approach - directly calling OpenAI
const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

const completion = await openai.chat.completions.create({
  model: "gpt-4",
  messages: [{ role: "user", content: "Hello!" }],
});

console.log(completion.choices[0].message.content);

After: Using Zuplo AI Gateway (sample.mjs)

Code
import OpenAI from "openai";

// New approach - using Zuplo AI Gateway
const openai = new OpenAI({
  apiKey: process.env.ZUPLO_API_KEY,
  baseURL: "https://your-ai-gateway-url.zuplo.app/v1",
});

const completion = await openai.chat.completions.create({
  model: "gpt-4",
  messages: [{ role: "user", content: "Hello!" }],
});

console.log(completion.choices[0].message.content);

Run the example with node sample.mjs.

What Changed?

1. URL: Replace your provider's URL with your Zuplo Gateway URL
2. API Key: Use your Zuplo API key instead of the provider's key
3. Everything else stays the same: The request format remains compatible with OpenAI's API

Verify Your Setup

Make Your First Request

Send a test request through your gateway:

Code
curl https://your-ai-gateway-url.zuplo.app/v1/chat/completions \
  -H "Authorization: Bearer YOUR_ZUPLO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4",
    "messages": [{"role": "user", "content": "Hello, world!"}]
  }'

Check Your Dashboard

1. Open the Apps tab of your AI Gateway project
2. Click on your app
3. Click on Dashboard to view:
   • Request count
   • Token usage
   • Time to first byte
   • Current spending

You should see your test request appear with token usage and performance metrics.

Next Steps

Now that your AI Gateway is running, explore additional features:

Switch Providers Without Code Changes

1. Go to your app settings
2. Change the Provider dropdown (for example, from OpenAI to Gemini)
3. Select a new model
4. Click Save Changes

Your application will now use the new provider without any code changes.

Common Issues

Issue: "Authentication failed" error

• Solution: Verify you're using your Zuplo API key, not your provider's key

Issue: Budget limit reached immediately

• Solution: Check that sub-team limits don't exceed available budget from parent team

Issue: Semantic caching not working

• Solution: Ensure caching is enabled in your application settings and prompts are similar enough to match