The System
We're building Community Scale AI at Z-Space. What started as pooled funds to buy motherboards, RAM, and GPUs is now a cluser of machines with connected software supporting local knowledge sharing.
Hardware, Apps, and People
The infrastructure layer is the AI hardware itself: a gateway sitting in front of the inference machine cluster. This isn't one person tinkering with a expensive machine on their desk just for them, and it's not a corporate engineering department choosing vendors for a whole company. What does it mean to share compute power?
The application layer is what gets built on top. We deploy a number of apps over time, connected by atproto and/or OIDC single sign on. They connect into the model router to access the AI compute. Everything is run locally as a small mini-cloud that is on premise.
The next two primitives are users and groups. Every user account is an atproto account with permissioned data support, connected by groups. You can bring your own atmosphere account, or be provisioned a local hosted one. We've implemented the emerging private data spec. Since people work across many different concepts, we have the concept of Workspaces: shared groups across members where you can pool and share data with other members and with agents. With local agents and AI compute, all your data stays local to you. With atproto's account data ownership model, you move across apps and groups and take your data with you.
Proxmox
We use Proxmox as the basis for our system, infrastructure, and app hosting. Each service is its own LXC container. Services talk to each other over an internal bridge that has no route in from the world.
| Role | What runs it |
|---|---|
| Reverse proxy | Caddy. Terminates TLS, and is the public edge. |
| Session broker | Corliss, this app. Sign-in, keys, the console. OIDC gateway. |
| Registry | HappyView, serving the membership lexicons |
| Gateway AI Router | LiteLLM. Every inference request goes through it. |
| Chat | Gel.Chat. We built our own atproto-native agentic chat. |
| Notes | scn-notes, the automerge server behind local first shared notes |
| Database | Postgres, shared by the services above |
| Sessions | Redis |
| Object store | Garage, S3-compatible |
AI Inference
Inference runs on separate hardware: a heterogeneous set of new, used and donated parts.. None of these machines is publicly exposed, the LiteLLM gateway router is the only thing that talks to them.
| Node | Hardware |
|---|---|
| orca | 2× RTX 3090, 48 GB VRAM, with one model pinned across both cards |
| salmon | Intel 1100 MAX, 48 GB VRAM. Researching the performance and drivers for the 1100 MAX series, in single card mode. |
| owl | 1080i, 11 GB VRAM. Dedicated audio transcription models. |
Which models are actually being served, and by which node, is answered live on the API page, accessible to members only.
Which models
Open-weight models only, with a standing preference for ones released from jurisdictions with meaningful transparency law: France, Canada, Switzerland. What each node serves is set at the gateway and changes as better weights land.
The code
One of the project's aims is that another community could stand up a cluster like this one, and that is not possible if the recipe is private. We use ansible for orchestration across Proxmox.
| Repository | What it is |
|---|---|
| zai-ops | Ansible and Proxmox. Builds a cluster from a bare machine and operates it. |
| Corliss | Django. The session broker, the member surface, and this page. |
| scn-member-registry | ATProto lexicons and the code that serves them. The record of who is a member. |
| scn-notes | Rust. The sync server behind shared notes. |
| scn-obsidian | TypeScript. The Obsidian plugin that keeps your workspaces' notes in step with the relay. |
| gel.chat | Agentic chat over ATProto permissioned data. Hosted on Tangled. |
The repository is still called zai-ops
after the project's earlier name. Renaming it would break the one URL that
bootstraps a new cluster, which was not worth the churn.
Read on: what this is, and the team that builds it.