Using Custom AI Providers
Zuplo's AI Gateway supports the addition of custom AI providers. This allows users to route their AI Gateway Apps to self hosted services and models, providing an additional layer of control, security and governance when using these models in production.
Please note that currently only OpenAI-compatible models are supported using custom providers
Adding a Custom AI Provider
To add a custom AI provider to your Zuplo AI Gateway, follow these steps:
-
Open Settings → AI Providers in your AI Gateway project in the Zuplo Portal.
-
Click on the Add Provider button.
-
Select the Custom option from the Custom Providers section of the list
-
Specify a label for the custom provider instance. You can change the label later if required.
-
Specify the API URL of the custom provider you are using.
-
Enter the API Key for the selected provider (if there is no API key required, you can leave this blank).
-
Finally, add the available models that are hosted with your custom provider, along with their type. Optionally, you can add a dollar cost for input and output tokens if you wish to track this via the AI Gateway.
-
Click Create.
Modify, Update or Delete your Custom AI Provider
To modify, update, or delete an existing provider, open Settings → AI Providers and click the Edit or Delete icon next to the custom provider.
Further information can be found in the Managing Providers guide.