Copy page

AI Microservice

The AI microservice provides an in-app chat assistant that answers questions about Primebrick using RAG (Retrieval-Augmented Generation). It embeds documentation into a pgvector knowledge base, retrieves relevant chunks at query time, and streams responses from a local or cloud LLM.

The service is inference-only: it never calls application tools itself. The agentic loop is owned by the caller (the Frontend) — the AI emits client-tool-call SSE events, the caller executes them against the BE's MCP gateway (/system/mcp/call), and resubmits the results.

• Package: primebrick-ai
• Service code: AI
• Default port: 3004
• Runtime: Bun (dev: bun --hot, prod: bun dist/index.js)
• Database schema: ai (isolated per-service schema)
• Vector extension: pgvector (cosine similarity, HNSW index)
• Declared capabilities: llm_orchestrator, chat_orchestrator, vectorizing_engine

Architecture

Components

RAG pipeline

The RAG (Retrieval-Augmented Generation) pipeline powers documentation-aware answers:

1. Embedding: The user's query is embedded using Xenova/all-MiniLM-L6-v2 (384-dim, CPU/WASM via Transformers.js).
2. Vector search: The embedding is compared against ai.docs_kb using pgvector cosine distance (<=> operator) with an HNSW index.
3. Context injection: The top-k chunks are injected into the LLM prompt as context.
4. LLM response: The LLM generates an answer grounded in the retrieved documentation.

Caller-driven tool loop

The orchestrator does not connect to the Backend's MCP server. Instead, the caller drives the loop end to end:

1. The caller fetches the tool catalog once from the BE: GET /system/mcp/tools (includes a read_only flag per tool).
2. The caller passes tools[] (OpenAI-style definitions) to POST /api/v1/ai/chat.
3. When the LLM requests a tool, the AI emits a client-tool-call SSE event and stops — it does not execute it.
4. The caller executes the tool via POST /system/mcp/call — read-only tools immediately, write tools behind a user confirmation — then resubmits POST /api/v1/ai/chat with the reconstructed history, tool_results[], and no message.
5. Repeat until the model produces a final answer (finish event). The caller bounds the loop (FE: MAX_LOOP_STEPS = 8).

The only server-side tool the AI executes itself is search_docs, a local tool that queries the RAG knowledge base — the LLM calls it when the user asks about product documentation, architecture, APIs, or configuration.

PII redaction

When REDACT_PII=true (recommended for cloud LLMs), the orchestrator redacts personally identifiable information from messages before sending them to the LLM:

• Emails → [EMAIL_1], [EMAIL_2], ...
• Phone numbers → [PHONE_1], ...
• Credit card numbers → [CARD_1], ... (with Luhn validation)

Original messages with real PII are persisted to the database unchanged. Only the LLM sees the redacted version.

HTTP routes

All routes are proxied by the Backend via /ws/ai/*. Authentication is handled by the gateway (headers: x-user-id, Authorization: Bearer <jwt>).

Method	Path	Purpose	Auth
GET	/api/v1/ai/health	AI health check (LLM connectivity + model info)	Gateway
POST	/api/v1/ai/chat	Chat endpoint with SSE streaming	Gateway + rate limit
GET	/api/v1/ai/conversations	List user's conversations	Gateway
POST	/api/v1/ai/conversations	Create new conversation	Gateway
GET	/api/v1/ai/conversations/:uuid	Get conversation with messages	Gateway
PATCH	/api/v1/ai/conversations/:uuid	Update conversation title	Gateway
DELETE	/api/v1/ai/conversations/:uuid	Delete conversation	Gateway
POST	/api/v1/ai/feedback	Submit thumbs up/down on a message	Gateway
GET	/api/v1/ai/feedback	List user's feedback	Gateway
GET	/api/v1/ai/telemetry	Aggregate stats + recent requests	AUTHENTICATED_ADMIN
POST	/api/v1/ai/reindex	Trigger embedding pipeline	AUTHENTICATED_ADMIN
POST	/api/v1/ai/embed	Server-side embedding of query text	AUTHENTICATED_USER
POST	/api/v1/system/docs/search	Semantic search over docs_kb	AUTHENTICATED_USER
POST	/api/v1/system/docs/document	Fetch a doc's chunks by path	AUTHENTICATED_USER

Chat endpoint

POST /api/v1/ai/chat streams the response via Server-Sent Events (SSE) and returns the conversation id in the X-Conversation-UUID header.

Request body — message OR tool_results is required:

Code
{
  "message": "How does RBAC work?",
  "conversation_uuid": "optional-uuid",
  "history": ["full message history incl. assistant tool-call parts"],
  "tools": ["OpenAI-style tool defs from GET /system/mcp/tools"],
  "tool_results": ["results for a tool-loop continuation (no message)"]
}

SSE events:

Event	Payload	Description
text-delta	{ text: string }	Partial text chunk
tool-call	{ toolName, input }	LLM invoked a server-side tool (search_docs)
tool-result	{ toolName, result }	Server-side tool returned a result
client-tool-call	{ toolName, input }	Caller must execute via /system/mcp/call
finish	{ text, tokensIn, tokensOut, steps }	Stream complete
error	{ message }	Error occurred

Rate limiting

The chat endpoint uses a token bucket algorithm per user (identified by x-user-id header). Configuration:

Config key	Default	Description
rate_limit_capacity	10	Max tokens in bucket per user
rate_limit_refill_rate	1	Tokens refilled per second
rate_limit_token_cost	1	Tokens consumed per chat request

If the shared redis_url config is set, rate limiting is distributed across instances. Otherwise, an in-memory token bucket is used (single-instance only).

Database schema

All tables are in the ai schema (microservice isolation).

docs_kb — RAG knowledge base

Column	Type	Description
id	bigint	Auto-increment PK
repo	text	Source repo (backend, frontend, sdk, dal, microservices, api)
path	text	Doc path within the repo
title	text	Doc title (from frontmatter)
chunk_idx	int	Chunk index within the doc
content	text	Chunk text content
embedding	vector(384)	Embedding vector (pgvector)
metadata	jsonb	Additional metadata (source, content_type, heading_path)
content_hash	text	SHA-256 of the exact embedding input: title, heading path, and chunk body

Index: HNSW on embedding with vector_cosine_ops (m=16, ef_construction=64).

ai_conversations — Chat conversations

Column	Type	Description
uuid	uuid	PK
user_uuid	uuid	Owner
title	text	Conversation title
created_at	timestamptz	Creation timestamp
updated_at	timestamptz	Last update timestamp

ai_messages — Chat messages

Column	Type	Description
uuid	uuid	PK
conversation_uuid	uuid	FK → ai_conversations (CASCADE)
role	text	user, assistant
content	jsonb	{ text, tool_calls, tool_results }
tokens_in	int	Input tokens
tokens_out	int	Output tokens
created_at	timestamptz	Creation timestamp

ai_telemetry — Per-request telemetry

Column	Type	Description
id	bigint	Auto-increment PK
uuid	uuid	Unique request ID
conversation_uuid	uuid	Associated conversation
user_uuid	uuid	Requesting user
model	text	LLM model name
tokens_in	int	Total input tokens
tokens_out	int	Total output tokens
steps	int	Reasoning steps (tool call rounds)
tool_calls	int	Total tool calls
rag_hits	int	search_docs calls
latency_ms	int	End-to-end latency
success	boolean	Whether the request succeeded
error	text	Error message (if failed)

ai_feedback — User feedback

Column	Type	Description
id	bigint	Auto-increment PK
uuid	uuid	Unique feedback ID
message_uuid	uuid	Message being rated
user_uuid	uuid	User who rated
rating	text	up or down
comment	text	Optional comment

Embedding pipeline (indexing)

Production source: service docs packs. Every service ships a dist/docs-pack (built by pnpm generate:docs, served at GET /api/v1/system/docs/pack). On service.register with the docs_pack capability the AI pulls the pack and ingests it under the pack's repo key; service.unregister drops that repo's docs. The BE's pack is pulled at startup and on every gateway_online broadcast. Packs are self-contained: they carry openapi.json (the service runtime spec) alongside the MDX docs, so openapi_path is retired.

The filesystem pipeline below is the dev fallback (set docs_fs_pipeline=true). It populates docs_kb from the documentation source files. Run it with:

Code
cd ai && pnpm run index-docs

Flow:

1. Load docs from DOCS_PATH (MDX files) and OPENAPI_PATH (JSON specs).
2. Parse frontmatter with LF or CRLF line endings, extracting the document title, then chunk the body by headings and paragraphs.
3. Build embedding input for each chunk from the document title, heading path, and chunk body. The stored content remains the original chunk body.
4. Compute SHA-256 over that exact embedding input; skip chunks whose content_hash matches the database.
5. Batch embed changed inputs (batch size 16) via the embedding provider.
6. Upsert content, embedding, metadata, and the input hash into docs_kb (ON CONFLICT update).
7. Delete stale chunks (docs removed from source or chunks no longer present).

The pipeline is idempotent — re-running it only re-embeds chunks whose source content, title, or heading path changed.

Environment variables

Variable	Required	Default	Description
DATABASE_URL	Yes	—	PostgreSQL connection string
DATABASE_SCHEMA	Yes	—	Declared schema domain (ai; CSV/JSON array for multi-schema — [0] is owned)

These are the only env-sourced settings (env policy). All runtime configuration lives in ai.config_entries and is read through src/config/ai-config.ts:

Config key	Default	Description
service_base_url	http://localhost:3004	Service URL for BE proxy
llm_base_url	http://localhost:8080/v1	LLM endpoint (OpenAI-compatible)
llm_model	Qwen3-4B-Instruct-2507	LLM model name
llm_api_key	not-required	LLM API key (not needed for local)
embedding_provider	transformers	transformers or openai
embedding_model	Xenova/all-MiniLM-L6-v2	Embedding model
embedding_base_url	—	OpenAI-compatible embeddings URL
embedding_api_key	—	Embeddings API key
embedding_dim	384	Vector dimension (OpenAI provider)
embedding_local_only / embedding_dtype	false / q8	transformers.js runtime options
be_base_url	http://localhost:3001	Backend URL — required by ServiceRegistrar registration, no longer used for MCP
redact_pii	false	true for cloud LLM PII redaction
rate_limit_capacity / rate_limit_refill_rate / rate_limit_token_cost	10 / 1 / 1	Chat token-bucket settings
docs_fs_pipeline	false	Dev fallback: true runs the legacy FS pipeline at startup and enables POST /api/v1/ai/reindex
docs_path	../../primebrick-v3-docs/pages	MDX docs path (fallback only)

| nats_url, service_code, http_port, service_client_shield_key | — | SDK bootstrap settings |

Distributed rate limiting uses the shared redis_url key (BE-owned global config), not a service env var.

LLM runtime

The default LLM is a local mistral.rs container running Qwen3-4B-Instruct-2507 (Q4_K_M quantization). See the Backend's infra/docker-compose.ai-llm.yml for the container configuration.

The LLM provider is pluggable — any OpenAI-compatible endpoint works (OpenAI, Anthropic via proxy, DeepSeek, Groq, etc.) by changing LLM_BASE_URL, LLM_API_KEY, and LLM_MODEL.

Local development

Prerequisites

1. PostgreSQL with pgvector (from infra/docker-compose.postgres.yml)
2. LLM container (from infra/docker-compose.ai-llm.yml) or a cloud LLM
3. Backend running on port 3001 — needed for service registration and for the caller-driven tool loop (/system/mcp/*); the AI itself no longer depends on it to serve chat

Steps

1. Copy .env.example to .env and adjust values
2. Apply migrations: pnpm run db:migrate
3. Index docs: pnpm run index-docs
4. Start dev server: pnpm run dev
5. Verify health: curl http://localhost:3004/health