Providers & Models
Administrators use Models → Providers, Model Marketplace, Model Definitions, Group Pricing to connect upstream model services, manage callable models, and configure request routing and billing prices.
For a first integration, follow the order "confirm model definitions → create a provider and select models → check the Model Marketplace and routing → verify with a call". With providers already in place, search for a model directly and adjust its backing, routing, or pricing.
| Page | What it manages | Common actions |
|---|---|---|
| Providers | The connection configuration for an upstream model service | Configure access type, API URL, key, group pricing, and supported models; maintain status |
| Model Marketplace | Availability aggregated per model | Find and copy model IDs, view backing providers and last-24-hour health, enable or disable models, adjust routing |
| Model definitions | A model's base info and billing configuration | Maintain name, display name, description, context, and prices |
| Group Pricing | The billing ratios applied to calls | Create and maintain group pricing; set pricing ratios and what users can select |
Providers
A provider can serve multiple models, and one model can be carried by multiple providers. For example, "Production primary" and "Production backup" can connect to the same service with different upstream URLs or account keys.
Preparing Connection Details
Before connecting, prepare the upstream service's API URL, API key, and the model names it actually supports, and decide which group pricing applies to it.
What you enter here is the upstream provider's API key. The API keys applications use to call OneGateway are created separately.
Creating a Provider
- Open Models → Providers and click New.
- Choose the access type matching the upstream interface. If it's not among the common options, expand More access types and search.
- Fill in the connection details and verify the auto-filled API URL.
| Field | Guidance |
|---|---|
| Provider name | A purpose-distinguishing name such as "Production primary" / "Production backup"; avoid duplicates |
| API key | A valid key issued by the upstream service |
| Group Pricing | The group pricing that applies to this provider; multiple selection supported—these are not organization departments |
| API URL | The service base URL the access type requires; follow the field hint and don't paste the full chat URL |
The access type must follow the interface protocol the upstream provides. When connecting through compatible interfaces or proxy services, also verify the required access type, URL, and key.
Testing the Connection and Selecting Models
After filling in the details, click Test connection. The system probes the upstream's model-list API to check connectivity and discover models.
- Test succeeds with selectable models: Review the models to sync, keep the ones to connect, and create the provider.
- Test succeeds with no selectable models: Check the model definitions on the current gateway instance. Selectable models require a same-named definition on that instance; add definitions first and re-enter the flow if needed.
- Test fails or times out: Verify the access type, URL, key, and network. The page lets you hand-pick models from existing definitions and continue, but a successful creation still requires an actual call to prove availability.
Select at least one model and one group pricing, then click Create provider. Back in the list, check the record's status and supported models.
Connection testing mainly validates the model-list API; it cannot prove every model answers normally, nor that user group pricing and API key restrictions are configured correctly.
Day-to-Day Maintenance
In the provider list, search by name or use filters to locate a record; click the name to view and edit.
| List info | How to use it |
|---|---|
| Type and group pricing | Check the interface access method and the group pricing that applies to the provider |
| Status | Whether the provider is enabled; disabling affects calls to models it carries |
| Priority and weight | Primary/backup order and same-tier split ratio; also adjustable from the Model Marketplace's routing view |
| Used / remaining | Reflowed runtime data; combine with the upstream account to decide top-ups—data may lag |
After updating the URL or key, retest and verify with a real call. To temporarily stop using a provider, first confirm its models have other available carriers, then disable it.
Model Marketplace
The Model Marketplace aggregates providers, prices, and runtime status per model, and is also the entry point for enabling/disabling models and configuring routing.
Reviewing Model Availability
Open Models → Model Marketplace and search by model name, ID, or vendor; combine the All Model Types, All Providers, and status filters (All, Enabled, Disabled) to locate a model.
| Card item | Meaning |
|---|---|
| Display name and model ID | The display name reads well; the model ID is what goes in the request's model field—use the copy entry |
| Providers | The number of providers carrying the model; click for details |
| Context | The context configured in the model definition |
| Input Price / 1M Tokens, Output Price / 1M Tokens | The model's configured prices; note the per-token vs per-request billing unit |
| Health - Last 24 Hours and Availability | Recent health at a glance; gray blocks or -- mean no data to show, not a fault verdict |
"Enabled" in the Model Marketplace means the model has provider backing. Whether a call succeeds still depends on provider status, the upstream service, group pricing configuration, and API key restrictions.
Enabling and Disabling Models
Use the switch on a model card. Read the confirmation prompt before disabling, and make sure applications depending on the model are prepared.
Disabling removes the model from the supported-model scopes of providers on the current instance while preserving the backing relations for recovery. Re-enabling attempts to restore them; afterward verify the providers are enabled and their configuration synced.
Model Routing
Open a model's routing visualization in the Model Marketplace to view its providers, response times, priorities, and weights.
This section covers provider selection for the same model
Priorities and weights here decide which provider serves the same model. To choose different models by request complexity, roll out one model as a replacement for another, or switch to another model when one is unavailable, use the Intelligent Routing plugin. After the plugin rewrites the model, every request still goes through the provider selection described here; every target and backup model used in the plugin's rules must be onboarded and priced here.
Understanding Primary/Backup and Traffic Split
| Setting or marker | Meaning |
|---|---|
| Priority | Higher is preferred; lower values serve as backup |
| Weight | Relative split within the same priority tier; not a fixed percentage |
| Solid line | Primary-tier providers, split by weight within the tier |
| Dashed line | Backup tier, tried on primary failure per the retry policy |
| Gray dashed | Removed providers that do not participate in scheduling |
For example, one model carried by three providers:
| Example provider | Priority | Weight | Intended role |
|---|---|---|---|
| Production A | 10 | 3 | ~3/4 of selections in the primary tier |
| Production B | 10 | 1 | ~1/4 of selections in the primary tier |
| Backup | 0 | 1 | Backup candidate when the primary tier fails |
These ratios illustrate weighted splitting and don't guarantee an exact 3:1 on small samples. Actual scheduling also depends on group pricing configuration, provider availability, and retry policy. After configuring backups, verify switchover against the real retry settings.
Saving Routing Adjustments
- Confirm the model's carriers and current primary/backup layout.
- Adjust the editable providers' priority and weight.
- Click Save, then reopen the routing view to check.
- Make a test call and verify the provider actually used and the result in Request Logs.
Adjustments affect every model the provider carries
Priority and weight belong to the provider. Editing them in one model's routing view applies to all other models it carries after saving. Confirm the full impact before changing.
Model Definitions
Open Models → Model Definitions and search by display name or other criteria. Click a record to maintain it; when onboarding a new model name, click New to add its definition.
Common models need no manual maintenance: the platform ships definitions (base info and prices) for mainstream models, keeps their display names, context, and descriptions up to date automatically, and their prices cannot be modified. Only models missing from the public catalog need to be added manually via New.
Basic Info
| Field | Description |
|---|---|
| Name | The model's identifier; must match the name used in the provider configuration; requests use the corresponding model ID |
| Display name | The readable name shown in the Model Marketplace; changing it does not change the model ID in requests |
| Icon | For display only; never affects the provider protocol or routing |
| Context | The model's context information; fill per actual capability—editing here does not extend the upstream model's ability |
| Description | Purpose, capabilities, and restrictions to help users choose |
Definitions own display and billing information; actual connectivity comes from provider configuration. After creating a definition, still add the model to a provider's supported scope.
Billing Unit and Price Items
In Billing Unit, choose per-token or per-request billing. Per-token settles by token usage; per-request uses a fixed price per call—choose per the service's actual billing.
For per-token billing, enable the price items you need, then fill in values. $/1M tokens means US dollars per million tokens.
| Price item | Matching usage |
|---|---|
| Input price | Input tokens |
| Output price | Output tokens generated by the model |
| Cache read price | Cache-read tokens reported upstream |
| Cache creation price | Cache-creation tokens reported upstream |
| Image input price | Corresponding image input usage |
| Audio input price | Corresponding audio input usage |
| Audio completion price | Corresponding audio output usage; requires enabling the audio input item first |
Fill in actual prices per business requirements, submit or update, then check the Model Marketplace's price display.
A price item's toggle manages the billing configuration, not a model capability. Don't read a disabled item as "free"; actual cost also depends on gateway billing rules, pricing ratios, and upstream-reported usage.
Group Pricing
Group Pricing sets the pricing ratio applied to calls. For example, create group pricing such as standard and discount to apply different ratios to different members or API keys, and select the applicable group pricing on each provider.
Creating and Maintaining Group Pricing
Open OneGateway Group Pricing and create a record with the following configuration.
| Setting | Description |
|---|---|
| Name | Unique within the instance, up to 50 characters; use recognizable lowercase letters, digits, underscores, or hyphens—immutable after creation |
| Description | Explains to members when this group pricing applies |
| Pricing Ratio | Non-negative, default 1; e.g., 0.8 multiplies the applicable billing rules by 0.8 |
| User-selectable | Whether this group pricing is offered to users; when turning it off, also review the group pricing of existing API keys |
| Sort order | Display order of the group pricing |
After creating group pricing, select it on the applicable providers in Providers and check the providers' supported models and status. Creating group pricing alone doesn't connect any model services to it.
Group pricing that is referenced by API keys, providers, or other records cannot be deleted until those references are removed.
Setting a User's Group Pricing
When a user's OneGateway account initializes, the system picks the earliest-created eligible group pricing in the tenant as the default group pricing.
To adjust a user's group pricing:
- Open the target user from the Users list.
- Use the Edit Group Pricing entry and choose the user's group pricing.
- If Selectable Group Pricing appears, configure any other group pricing the user can choose when creating API keys.
- Click Save and verify the updated group pricing and selectable group pricing.
The user's own group pricing is always selectable. If prompted that the account hasn't been initialized, confirm the user is active, has OneGateway roles, and the account provisioning completed.
Assigning Group Pricing to an API Key
When creating or editing an API key, use Group Pricing to choose the pricing that applies to the key. If left empty, the user's group pricing is used.
An API key's callable models are restricted by its available models. If a call reports the model is unavailable, also confirm that a provider serving the model is enabled and has the key's group pricing selected.
After changing a user's group pricing or selectable group pricing, review the API keys still in use—especially keys with group pricing explicitly assigned. Then make a call with each key and confirm in Request Logs that the model, provider, and cost are as expected.
Verification and Troubleshooting
Quick Verification
After onboarding or changes, chain-check the configuration with one real call:
- In Providers, confirm the connection details, supported models, and group pricing, and that the provider is enabled.
- In the Model Marketplace, find the model, confirm an available carrier exists, and copy the model ID.
- Using an API key whose group pricing matches the provider and whose available models include the model, make a simple call per Quickstart—for example, "Introduce yourself in one sentence"—with
modelset to the copied ID. - Check the result, model, provider, token usage, and cost in Request Logs.
- For routing or pricing changes, watch the distribution and cost of subsequent calls in Dashboard & Analytics.
When Configuration Doesn't Take Effect
If results don't change after saving, check in order:
- Confirm the sync result: Check the sync status and error, resolve the cause, and retest only after a successful sync.
- Verify model and provider: Confirm the instance, model ID, and the provider's supported-model scope agree, and the provider is enabled.
- Check call permissions: Verify the API key's group pricing, model restrictions, and available quota.
- Cross-check the logs: Make a new request and inspect the actual model, provider, and error. Validate routing or price changes with fresh requests.