Introducing Personal AI Harness

A self-hosted personal agent platform with pluggable LLMs, real tools, Docker sandboxes, a secrets vault, reusable Skills and scheduled Agents. It all runs on your own machine.

 

Most AI chat apps stop at conversation. You ask, the model answers, and you go and do the work. I wanted something that could do the work: search the web, read and edit files, call APIs, check my email, run code, and keep doing that on a schedule while I'm away. I also wanted it to run on my hardware, with my choice of model, and without my API keys ending up in a prompt.

That project is PersonalAI. This post covers what it does and how it's built.

What it is

PersonalAI is one ASP.NET Core API, one Next.js web UI and one small broker process that sits on a Docker host. With it you can:

• Chat with local or hosted models. Ollama, OpenAI, Azure OpenAI and DeepSeek all sit behind one IAIProvider interface. Each chat session, skill or agent can use a different provider and model.
• Let the model use tools. There are 11 built-in tools: web_search (via self-hosted SearXNG), read_file, write_file, edit_file, list_files, search_files, http_request, gmail, network_scan, computer_control and sandbox.
• Run arbitrary code safely. Every chat session or agent run gets its own Docker container, with a file browser, port forwarding and downloads.
• Build reusable Skills. A Skill is a saved prompt plus an allowlist of the tools and secrets it may use.
• Schedule Agents. An Agent orchestrates your Skills and runs on demand, once, on an interval or on a cron.
• Keep secrets out of prompts. API keys live in a vault, and the model only ever sees placeholders.
• Optionally lock it down with login through any OAuth2 / OpenID Connect provider (Keycloak, Auth0, Okta, Entra ID, Authentik and others).

The architecture at a glance

┌─────────────────────────┐
│  Browser (Next.js)      │
│  Chat · Agents · Setup  │
└────────────┬────────────┘
             │ HTTP + SSE
┌────────────▼────────────────────────┐
│  ASP.NET Core API                    │
│  ├─ ChatAgentService (agentic loop)  │
│  ├─ SkillExecutionService            │
│  ├─ AgentExecutionService            │
│  ├─ AIProviderRegistry               │
│  ├─ ToolRegistry (11 tools)          │
│  └─ SandboxHostRegistry (WebSocket)  │
└────────────┬────────────────────────┘
             │ SQLite + outbound WebSocket
      ┌──────▼──────────────────┐
      │ Docker host + volumes   │
      │ PersonalAI.SandboxHost  │
      │ (broker, dials out)     │
      └─────────────────────────┘

The stack is .NET 10 and C# 13 with EF Core on SQLite for the backend, Next.js 16 with React 19 and Redux Toolkit for the frontend, and Docker Compose for deployment.

The heart of it: one agentic loop

Chat, Skills and Agents all run the same basic loop:

1. The user's message is saved to the database.
2. The conversation history and tool definitions go to the model.
3. The model's reply streams back to the browser over Server-Sent Events.
4. If the model asks for tools, they run and their results are added to the conversation.
5. The loop goes back to step 2 until the model answers in plain text, hits the iteration cap (25 by default), or pauses for approval.

A few details make this hold up in daily use:

• Human-in-the-loop approval. A file operation outside the workspace, or a sandbox command that matches a dangerous-command pattern, pauses the turn until you approve or deny it.
• Turns run on the server, not on the request. Closing the tab doesn't kill the turn. Reload the page and it re-attaches, replays what has happened so far and continues live. A Stop button cancels the turn cleanly.
• Empty or truncated replies are reported. Reasoning models sometimes spend their whole output budget thinking and return nothing. PersonalAI tells you that happened and how to fix it, instead of showing a blank answer.
• Long conversations stay coherent. When history compaction is on, older messages are folded into a running summary by a side LLM call instead of being dropped.

Skills and Agents: loops inside loops

A Skill is a reusable prompt with an explicit allowlist of tools and secrets. If the model calls a tool outside that list, the call is rejected before it reaches the tool. Skills run unattended, so a call that would normally need approval is rejected with an explanation rather than waiting for a user who isn't there.

An Agent has its own prompt, and its "tools" are your Skills, each exposed as skill_<slug>. That makes it a loop nested inside a loop. Agents can run manually, once, on an interval or on a cron. A background scheduler picks up due agents, and every run is recorded with its status, a summary and a full transcript.

For example, you might have a "summarise unread email" Skill and a "check my site is up" Skill, plus an Agent that runs both every morning at 8:00 and writes up the results.

Secrets the model never sees

Skills declare which secrets they need. The model is told to refer to a secret as {{secret.KEY}} inside tool arguments. The real value is swapped in immediately before the tool runs, so only the placeholder ever appears in the conversation history. The API never sends stored secret values back to the browser either; it only reports whether a value is set.

The Docker sandbox, and why the broker dials out

The file tools are confined to a workspace folder and http_request is URL-allowlisted, but none of the in-process tools can run arbitrary code. The sandbox tool fills that gap with a real Docker container per chat session or per agent run.

The API doesn't SSH into the Docker host. A small broker, PersonalAI.SandboxHost, runs on the Docker host and opens one outbound WebSocket to the API, authenticated with a shared key. Every sandbox operation (create, exec, file operations, destroy) is multiplexed over that one connection.

• The Docker host needs no inbound ports and no SSH keys.
• There is one transport and one auth scheme to secure.
• Sandbox images stay vanilla (python:3.12-slim, node:22-slim and so on). The broker drives Docker from outside, so adding a runtime never means rebuilding an image.

Each sandbox's /work directory is bind-mounted from a per-owner folder on the host. That's what powers the file browser, downloads and zip-a-selection in the UI.

Bring your own model

Every provider implements the same IAIProvider contract: streaming chat with tool calls, model listing and, for Ollama, model pulling. OpenAI and DeepSeek share a base class that handles the OpenAI streaming protocol, including reassembling tool-call fragments into complete calls. Per-model settings such as max output tokens, tool calling, reasoning effort and thinking on/off apply everywhere the model is used: chat, skills, agents and compaction.

In practice you can run a small local Ollama model for cheap scheduled jobs and a hosted model for heavy interactive work, side by side.

Try it

git clone https://github.com/priyanr/PersonalAI.git
cd PersonalAI
cp .env.example .env          # all values optional; defaults work
docker compose up -d --build

Then open:

• Web UI: http://localhost:3000
• API: http://localhost:5080
• SearXNG: http://localhost:8888
• Ollama: http://localhost:11434

If you'd rather run it without Docker for the app itself, you need the .NET 10 SDK and Node.js 22+:

dotnet run --project src/PersonalAI.Api
cd web && npm install && npm run dev

⚠️ A word of caution. PersonalAI gives an LLM real capabilities. It can run code in containers, call HTTP endpoints, scan your LAN and, on Windows, drive your mouse and keyboard. Auth is off by default, so keep it on localhost or turn on OAuth2/OIDC login before you expose it to a network. The sandbox broker mounts the Docker socket, which is root-equivalent on the Docker host, so treat SANDBOX_HOST_KEY like a root password.

Wrapping up

PersonalAI started as "a chat UI for Ollama" and grew into a small agent platform: one reusable agentic loop, a tool system with guardrails, isolated sandboxes, scoped secrets and scheduled automation, all self-hosted. It's backed by over 600 automated tests across the agent loop, controllers, infrastructure, services, tools and sandbox.

The code, setup guide and detailed architecture docs are on GitHub: github.com/priyanr/PersonalAI. Issues and ideas are welcome.