Prompts
Prompt Type
Text and chat prompts: what each contains and how it joins a request.
Every prompt is either a text prompt or a chat prompt. The type decides what the prompt's content looks like and how the prompt combines with the caller's messages. A text prompt supplies the system message. A chat prompt supplies a short conversation, such as instructions followed by worked examples, that comes before the caller's messages.
| Text prompt | Chat prompt | |
|---|---|---|
| Content | One block of text | An array of system, user and assistant messages |
| Inline prompt | type set to text | type set to chat |
| GitHub prompt | A .md file | A .json file |
| Joins the request as | The system message | Messages placed before the caller's messages |
| Prompt composition | Applies | Not applicable |
Text prompts
A text prompt is a single block of instructions, written as plain text or Markdown, that becomes the system message of each request. Through the API its body is a JSON string; in a GitHub repository it is the full text of a .md file. When the caller also sends a system message, Prompt composition on the router decides whether the two are merged, the router's prompt replaces the caller's, or the caller's is used in place of the router's. See Prompt Binding.
Chat prompts
A chat prompt is an ordered array of messages, each with a role and content. It is used to give the model instructions together with example exchanges that show the expected answers, a technique known as few-shot prompting. Each request places the chat prompt's messages in front of the caller's messages. The caller's own system message, when it sends one, is replaced by the chat prompt.
A request through a router bound to this prompt sends one new customer message:
Chat rules
A chat prompt must form a valid start of a conversation, so that any caller message can follow it. The dashboard, the API and GitHub syncing all check these rules, and a prompt that breaks one is refused when it is saved or synced:
| Rule | Detail |
|---|---|
| Roles | Each message has the role system, user or assistant, in any letter case. |
| System message | At most one system message, and only as the first message. |
| First conversational message | The first message after the system message has the role user. |
| Alternation | User and assistant messages alternate after that. |
| Content | Every message has content with at least one character other than spaces. |
| Length | At least 1 and at most 64 messages. |
| Size | At most 128 KiB for an inline prompt, and 2 MiB for a GitHub file. |
A chat prompt can consist of a single system message. Message content is either a string or an array of content parts, in the same shape as Chat Completions messages.
Choosing a type
A text prompt fits instructions that every request needs, and works with callers that send their own system message through Prompt composition. A chat prompt fits behavior that is easier to show than to describe, such as a classification format or a tone, and it sets the system message for every caller. The type of an inline prompt is fixed when it is created; to change type, create a new prompt and bind routers to it.