CF-Incidents AI Gateway - Route across a pool of models with model aliases

[ENF News Bot]

AI Gateway now supports model aliases. A model alias is a pool of models that your application calls by one name, such as custom/support-fast. AI Gateway tries the models in the order you set and falls back to the next one if a request fails, or uses smart routing to choose a model for each request.

Because your application only references the alias, you can swap models or change the routing algorithm on your gateway without redeploying. Create and manage aliases on your gateway's Routing page, alongside dynamic routes.

To call an alias through the REST API, use its name as the model. For example, send this body to /accounts/{account_id}/ai/run:

{
	"model": "custom/support-fast",
	"input": {
		"messages": [{ "role": "user", "content": "Reply with a short greeting." }]
	}
}

Include the cf-aig-gateway-id header to select the gateway where you created the alias. Aliases are scoped to individual gateways.

For setup instructions and routing options, refer to Model aliases.

Continue reading...