AI Chat Copilot

The AI Copilot is in Early Access for customers on an Enterprise Plan. To enable AI Copilot functionality for your account, contact your account manager or email [email protected]

PixieBrix's AI Copilot provides seamless access to conversational AI within any web application and across your devices.

Configuring the Chat Copilot

To register a chat copilot, use the Add AI Copilot brick. The brick has the following input properties:

• Title/Icon: the title for the copilot, displayed to the user on the copilot selection dropdown

• Integration Configuration: the integration configuration for LLM calls. Must be compatible with the OpenAI chat completions API

• System Prompt: optional prompt/instructions for the Copilot

Tool Call Settings

• Allow Brick Use: experimental support for calling PixieBrix bricks as tools

• Allow Browser Use: experimental support for automatically performing browser actions, e.g., form fill, etc.

• MCP Servers: an array of MCP server URLs to connect to for tools and prompts

Security Settings

The following security controls are used to prevent data exfiltration via prompt injection for copilots with access to sensitive information:

• Embedded Content Allowlist: an array of match patterns allowed for embedded content (e.g., images, etc.) in the assistant output. If not provided, defaults to banning embedded content

• Link Allowlist: an array of match patterns allowed for links rendered in assistant output. If not provided, defaults to https://*/*

Extending the AI Chat Copilot

Registering Custom Tools

Custom Tools/Functions enable the AI Copilot to look up information (e.g., for Retrieval Augmented Generation) and take actions.

To register a custom tool, use the Add AI Copilot Tool brick. The brick has following input properties:

• Name: The tool/function's name, e.g., get_weather

• Description: Details on when and how to use the function

• Parameters: JSON schema of type object specifying the tool's input parameters

• Handle: The handler to run. The validated arguments are provided to the handlers as @args

Registering Custom Prompts

Custom Prompts provide a powerful way "to standardize and share common LLM interactions."

To define a Custom Prompt, use Add AI Copilot Prompt brick. The brick has the following input properties:

• Name: Unique identifier for the prompt. Displayed to the user

• Description: Human-readable description

• Arguments (Advanced): array of arguments to the prompt, in the Model Context Protocol structure (not JSON Schema). The AI Copilot will prompt the user to provide the arguments

• Handle: The handler to generate the prompt messages. The prompt arguments are provided to the handler as @args. Use the Copilot Messages brick to easily create a messages array.

Starting a Conversation

To start a conversation from another starter brick and/or mod, use the Start Copilot Conversation brick. The brick has the following input properties:

• Title (Optional): an optional title for the conversation

• Messages: the initial messages for the conversation

If the Copilot is not currently open/displayed, PixieBrix will automatically open the Copilot.

Registering Custom Message Actions

Custom Message Actions enable users to take actions on Assistant Messages. There are some actions built-in to the AI Copilot:

• Copy to Clipboard

• Regenerate Message

Examples of custom actions:

• Provide Feedback (positive/negative)

• Insert at Cursor

• Share Message

To register a custom Message Action, use the Add AI Copilot Message Action brick. The brick has the following input properties:

• Title: The message action's title to display in the Copilot interface

• Icon: The action's icon to display in the Copilot interface

• Handle: the action to run. The message is provided to the handler as @message

Registering Conversation Context Providers

Conversation Context Providers provide additional context to the AI Copilot without being stored in the in the conversation history/context.

Examples of conversation context providers:

• Chat with Page: provide compressed web page context to the Copilot

To register a Conversation Context Provider, Add AI Copilot Conversation Context brick. The brick has the following input properties:

• Title: User-facing title of the action

• Icon: The provider's icon to display in the Copilot interface

• Handle: The handler to generate the context messages. Use the Copilot Messages brick to easily create a messages array.

Messages provided by a conversation context provider, enhance the context in the following way:

• System: system messages are appended to the AI Copilot's System message

• User/Assistant: user/assistant messages a included between the conversation history and the user-provided message

Running Bricks as Tools

The AI Copilot has two built-in tools to support automatically running bricks:

• bricks_search : search all bricks available to the user

• bricks_run : run a brick by registry id and arguments

Brick Tool Limitations

• Bricks are run in the top-level frame. To handle pages with frames, define a custom brick with frame handling in the workshop: Advanced: Workshop

• Bricks with integration configurations cannot be run as tools. To run a brick with an integration as as a tool, register it as a custom tool: Registering Custom Tools. (You will be prompted to choose an integration configuration when activating/deploying the containing mod)

White Labelling and Component Customization

White Labelling is only available on an Enterprise plan. To White Label the AI Copilot, contact your account manager or email [email protected]

The following AI Copilot components can be white labelled/customized:

• Messages Container

• User Message

• Assistant Message

• Tool Call

• Message Action

• Custom Prompt

• Conversation Context Provider