A private, always-on assistant that turns messages, calls, mail, calendar events, and wearable transcripts into searchable memory and approval-ready work.
Built on OpenClaw and hosted on a dedicated Mac mini, this self hosted ai personal assistant keeps the gateway, indexed corpus, audit logs, and original audio under local control. Remote access stays inside a Tailscale tailnet, while the Claude API handles reasoning through owner-controlled credentials.
Self Hosted AI Personal Assistant Security Boundary
This self hosted ai assistant is not bound to a public IP, reverse proxy, or open inbound port. Only approved tailnet identities can reach its dashboard or maintenance endpoints. Each connector receives a separate secret, narrow scope, rate limit, and kill switch, so a compromised integration cannot automatically reach every data source.
Local storage uses FileVault, encrypted backups, service-specific macOS accounts, and append-only audit events. The permission model follows NIST Zero Trust Architecture and tests agent tools against the OWASP Top 10 for Agentic Applications. Review checklists also reference the Verizon 2026 DBIR and IBM 2025 Data Breach Report.
Self Hosted AI Personal Assistant Memory and Routing
OpenClaw iMessage is the primary conversation surface. Microsoft Graph supplies read-only Outlook mail and calendar events, Quo supplies business call records, and Omi supplies nightly wearable transcripts.
Every item enters a common envelope with source, timestamp, participants, address, permissions, confidence, and original-file pointer. Speaker-tagged transcripts are linked to people, companies, properties, claims, and projects. The assistant then produces morning briefs, evening recaps, overdue commitments, and role-specific draft messages without silently sending them.
Self Hosted AI Voice Assistant Capture
The self hosted ai voice assistant pipeline watches call recordings, voicemail events, synced Notes recordings, and one-tap Apple Shortcuts. Audio remains attached to its transcript so a user can verify an extracted commitment against the original recording.
Nightly study jobs transcribe new audio, separate speakers, match known phone numbers and addresses, extract dates and action items, and flag unresolved promises. Cross-call analysis groups repeated objections, recurring project blockers, and missed follow-ups without sending audio to a call-intelligence platform.
Core Features
| Feature | Description |
|---|---|
| Private OpenClaw Gateway | Public exposure creates an unnecessary attack surface; the gateway accepts management traffic only through the private tailnet and approved devices. |
| Multi-Channel Context Inbox | Important context gets scattered across messages, Outlook, calls, and wearable audio; normalized events enter one searchable timeline with source-level permissions. |
| Nightly Transcript Study | Manual call review is easy to postpone; scheduled workers transcribe, identify speakers, link entities, and create dated action items with source references. |
| Entity-Linked Knowledge Map | Names and addresses vary between systems; canonical records connect people, properties, companies, claims, conversations, and active projects. |
| Approval-First Delegation | Autonomous sending creates avoidable risk; team scorecards, lead alerts, permit checks, and reminder drafts stay in approval mode for the first 60 days. |
| Heartbeats and Failure Escalation | Silent connector failure hides missing data; each workflow emits a heartbeat, stores its last success, retries safely, and alerts after two missed intervals. |
| Portable Corpus Export | Proprietary storage can trap operating history; exports include JSON, Markdown, metadata, relationships, and original audio in a documented folder layout. |
Where the n8n Self Hosted AI Assistant Layer Fits
The n8n self-hosting runtime is used only for deterministic connector glue, schedules, retries, and notifications. The n8n self hosted ai assistant layer does not own reasoning, memory policy, permissions, or audit decisions; those remain in the typed application services.
Read-only adapters can pull operational records from Roofr, the SPOTIO API, and the CompanyCam API. Each adapter maps external IDs into the local entity graph and can be disabled without stopping the assistant.
Technical Stack and Selection Rationale
| Component | Role and reason |
|---|---|
| OpenClaw | Provides the local gateway and messaging channel model instead of rebuilding channel orchestration. |
| TypeScript services | Enforces typed event contracts, connector boundaries, approval states, and export schemas. |
| Python workers | Handles transcription, speaker segmentation, extraction pipelines, and batch backfills. |
| PostgreSQL with pgvector | Stores relational entities and semantic embeddings in one locally managed database. |
| n8n | Runs visible schedules and connector retries while remaining replaceable glue. |
| launchd and Docker | Keeps native macOS channel access separate from containerized workers and databases. |
Runtime Acceptance Targets
These are deployment tests, not marketing claims. Channel heartbeats run every 60 seconds; two missed checks trigger an alert. Search should return the top linked records in under 2 seconds at the 95th percentile on a 100,000-chunk test corpus. Nightly ingestion must be restartable, idempotent, and able to resume from the last confirmed file.
Backfill reconciliation compares source counts, imported counts, skipped records, and checksum failures. Restore drills verify that a clean Mac can rebuild the database, reconnect channels, rotate each secret, and reproduce exports from documented steps.
Project Directory
openclaw-private-assistant/
├── apps/
│ ├── gateway/
│ │ ├── src/channels/imessage.ts
│ │ ├── src/approvals/approval-router.ts
│ │ └── src/health/heartbeat-registry.ts
│ ├── dashboard/
│ │ ├── src/pages/open-loops.tsx
│ │ └── src/pages/integration-health.tsx
│ └── workers/
│ ├── transcription/transcribe.py
│ ├── diarization/speaker_tags.py
│ ├── extraction/commitments.py
│ └── backfill/outlook_import.py
├── packages/
│ ├── connectors/
│ │ ├── microsoft_graph.ts
│ │ ├── quo.ts
│ │ ├── omi.ts
│ │ ├── roofr.ts
│ │ ├── spotio.ts
│ │ └── companycam.ts
│ ├── memory/
│ │ ├── entity_resolution.ts
│ │ ├── semantic_search.ts
│ │ └── export_corpus.ts
│ └── security/
│ ├── scopes.ts
│ ├── secret_rotation.ts
│ └── audit_log.ts
├── workflows/n8n/
│ ├── nightly-study.json
│ ├── morning-brief.json
│ └── heartbeat-alerts.json
├── infra/
│ ├── docker-compose.yml
│ ├── tailscale-policy.hujson
│ └── launchd/com.cogworklabs.openclaw.plist
├── docs/
│ ├── handoff.md
│ ├── recovery-runbook.md
│ ├── key-rotation.md
│ └── data-export.md
└── tests/
├── security/
├── connectors/
├── backfill/
└── recovery/
Use Cases
- Start the day with verified context: receive a brief covering today's appointments, unresolved commitments, missed callbacks, and records that failed to ingest.
- Recover a property conversation quickly: search an address to see linked messages, calls, photos, participants, extracted dates, and the original recording.
- Keep team follow-up controlled: generate sales scorecards, lead-response alerts, permit checks, and draft reminders while a manager approves every outbound action.
- Audit missing data before it becomes invisible: view connector health, last successful sync, retry history, source counts, and the exact records held for review.
How to Manage Daily Operations Using Self Hosted AI Personal Assistant
Download & Set Up the Project
Download, set up, and install Self Hosted AI Personal Assistant to get the project running. If you hit any difficulty, contact us here.
Open the Private Dashboard
Connect through Tailscale, open the local dashboard, and verify green status for iMessage, Outlook, voice capture, memory, and nightly study.
Select Sources and Approval Mode
Choose active read-only connectors, set transcript retention, confirm approved senders, and keep team outputs in draft approval mode during the initial operating period.
Run the Daily Brief
Select Generate Brief to receive linked appointments, open commitments, failed ingestions, and draft follow-ups in iMessage and the dashboard review queue.
For connector extensions or deployment changes, CogworkLabs provides custom AI agent integration and ongoing automation maintenance.
FAQs
Does the assistant expose my Mac mini to the public internet?
No. The gateway binds to local interfaces and is reachable remotely only through approved Tailscale identities. Firewall rules, device approval, connector-specific scopes, and separate secrets reduce the blast radius of a stolen credential.
Can I export the assistant's complete memory without proprietary formats?
Yes. The export job writes normalized JSON, readable Markdown, relationship files, checksums, and original audio into a documented directory. A restore test confirms that the exported corpus can rebuild search and entity links on a clean installation.
What happens when a connector or nightly workflow stops working?
Each workflow records start, progress, completion, and heartbeat events. Two missed heartbeat intervals create an alert containing the failed connector, last successful checkpoint, retry count, and the runbook step needed to investigate.
