CF-Incidents AI Gateway - AI Gateway launches Realtime WebSockets API

[ENF News Bot]

We are excited to announce that AI Gateway now supports real-time AI interactions with the new Realtime WebSockets API.

This new capability allows developers to establish persistent, low-latency connections between their applications and AI models, enabling natural, real-time conversational AI experiences, including speech-to-speech interactions.

The Realtime WebSockets API works with the OpenAI Realtime API ↗︎, Google Gemini Live API ↗︎, and supports real-time text and speech interactions with models from Cartesia ↗︎, and ElevenLabs ↗︎.

Here's how you can connect AI Gateway to OpenAI's Realtime API ↗︎ using WebSockets:

OpenAI Realtime API examplejavascript

import WebSocket from "ws";

const url =
	"wss://gateway.ai.cloudflare.com/v1/<account_id>/<gateway>/openai?model=gpt-4o-realtime-preview-2024-12-17";
const ws = new WebSocket(url, {
	headers: {
		"cf-aig-authorization": process.env.CLOUDFLARE_API_KEY,
		Authorization: "Bearer " + process.env.OPENAI_API_KEY,
		"OpenAI-Beta": "realtime=v1",
	},
});

ws.on("open", () => console.log("Connected to server."));
ws.on("message", (message) => console.log(JSON.parse(message.toString())));

ws.send(
	JSON.stringify({
		type: "response.create",
		response: { modalities: ["text"], instructions: "Tell me a joke" },
	}),
);

Get started by checking out the Realtime WebSockets API documentation.

Continue reading...