Skip to main content
When using OpenAI as an LLM provider on your own key, you can specify custom model identifiers to use models that aren’t in the provider’s built-in list. This is useful for:
  • Using locally hosted models through Ollama (e.g., llama3.1:70b)
  • Using Azure OpenAI deployments with custom names
  • Using newly released models before they’re added to the default list
  • Using fine-tuned or specialized models

​
Requirements

To follow the steps in this guide, you’ll need:
  • Organization Admin permissions in Omni
  • An OpenAI API key

​
Enabling a custom OpenAI model as a provider

Custom models default to a 400,000-token context window.
1
In Omni, navigate to AI Hub > General and click the LLM settings tab.
2
In the Providers section, click the Your API keys tab.
3
Click OpenAI API-compatible to open its settings.
4
Enter your connection details:
  • Base URL - Optional. If your endpoint uses a different base URL (such as for Azure OpenAI or Ollama), enter it in this field.
  • API key - Required. Enter your OpenAI API key.
5
In the Custom models field, enter your model identifiers as a comma-separated list (e.g., llama3.1:70b, gpt-4-turbo-2024-04-09, or your Azure deployment name).
6
In the Simple-task model dropdown, select the LLM to use for light tasks, such as naming a chat and writing summaries. To use a custom model, select Custom model identifier and enter the identifier in the field that appears.
7
Click Save and approve.
8
To run AI features on a custom OpenAI model by default, select one of the models as the organization default.
By default, Omni approves all available LLMs once a provider is successfully configured, which means users can select the LLMs in the chat picker. Use the LLM section to revoke specific LLMs, mark an LLM as test only, and add useful descriptions.

​
Disabling a custom OpenAI model as a provider

In the Your API keys tab, use the toggle next to the OpenAI API-compatible option to disable it as a provider. After Omni saves the changes, a Delete saved key option will display. Check this option and click Save to permanently delete the key. If you decide to re-enable the OpenAI API-compatible option later, you’ll need to add a new API key.