Understanding Allma Studio

The modern AI landscape presents users with a fundamental trade-off: access to powerful large language models in exchange for their data. Every prompt sent to cloud-based AI services like ChatGPT, Claude, or Gemini is processed on remote servers, creating privacy concerns for individuals and compliance nightmares for organizations.

Consider the implications: legal professionals cannot consult AI about sensitive cases, healthcare workers cannot analyze patient data, businesses cannot discuss proprietary strategies, and researchers cannot explore confidential findings. The most transformative technology of our time becomes off-limits for the most sensitive use cases.

Allma Studio was conceived to solve this problem: a full-stack AI application that runs entirely locally, combining the conversational capabilities of modern LLMs with document-grounded RAG responses, all while maintaining complete user privacy and zero cloud dependency.

Allma Studio follows a microservices-inspired monolith architecture, where the application is structured as independent services but deployed as a single unit. This provides the benefits of clean separation while avoiding the complexity of distributed systems.

Click to enlarge

High-level system architecture showing the four-layer design: Orchestration, Presentation, Intelligence, and Infrastructure layers

Key Components

The Presentation Layer

Built with React and Vite, the frontend prioritizes developer experience and user responsiveness. Vite's instant Hot Module Replacement accelerates development cycles, while React's component model enables the rich, interactive chat interface users expect from modern AI applications.

TailwindCSS powers the styling system, providing utility-first classes that enable rapid UI iteration. The dark/light mode toggle uses CSS custom properties and local storage for persistence, respecting system preferences while allowing manual override.

The Intelligence Layer

FastAPI serves as the backend framework, chosen specifically for its async-first architecture and automatic OpenAPI documentation. The async support is critical: when users send messages, the backend must simultaneously query the vector store, construct prompts, and stream responses—all without blocking other requests.

Retrieval-Augmented Generation transforms Allma from a simple chat interface into a knowledge-aware assistant. Users upload their documents, and the system automatically extracts, chunks, embeds, and indexes the content—creating a searchable knowledge base that grounds every response in user-provided context.

Click to enlarge

Complete RAG pipeline showing query processing through response generation with vector search and context assembly

Query-Time Retrieval Flow

When RAG is enabled, each user query triggers a sophisticated retrieval pipeline that enriches the LLM prompt with relevant context:

1. Query Embedding — The user's question is embedded using the same model as document chunks (Nomic Embed Text)
2. Similarity Search — LanceDB performs cosine similarity search to find the top-K most relevant chunks
3. Context Assembly — Retrieved chunks are formatted with source attribution and prepended to the system prompt
4. Prompt Construction — The orchestrator builds a complete prompt with context, instructions, and the user query
5. Streaming Generation — Ollama generates the response token-by-token via Server-Sent Events
6. Source Attribution — Chunk metadata is returned alongside the response for full transparency

The ingestion pipeline transforms raw documents into searchable vector embeddings. This state machine ensures robust handling of various file formats while maintaining UI responsiveness through clear state transitions.

Click to enlarge

State machine showing the document ingestion flow from user upload through indexing with success/failure paths

Ingestion Stages

Supported File Types

The orchestrator is the nervous system of Allma Studio—a central coordinator that manages the flow of data between services, maintains conversation state, and ensures each component receives the context it needs.

Backend Layer Overview

The backend follows a layered architecture with clear separation of concerns:

┌──────────────────────────────────────────────────┐
│              Presentation Layer                  │
│         (Routes / API Endpoints)                 │
├──────────────────────────────────────────────────┤
│              Orchestration Layer                 │
│         (Business Logic Coordinator)             │
├──────────────────────────────────────────────────┤
│               Service Layer                      │
│     (Domain-Specific Business Logic)             │
├──────────────────────────────────────────────────┤
│               Data Access Layer                  │
│    (Database, Vector Store, External APIs)       │
└──────────────────────────────────────────────────┘

Service Coordination

The orchestrator coordinates four primary services:

Route Handlers

LanceDB serves as the persistent vector database, storing document embeddings and enabling fast similarity search. Unlike cloud-based alternatives like Pinecone or Weaviate, LanceDB runs entirely locally with no external dependencies.

Why LanceDB?

• Zero Configuration — Works out of the box with sensible defaults
• Python Native — First-class Python integration with type hints
• Persistent Storage — Survives restarts with configurable data directory
• Metadata Support — Store and filter by arbitrary metadata alongside vectors
• Local-First — No cloud account, API key, or network required
• Fast SIMD — Optimized vector operations using CPU SIMD instructions

Embedding Model Selection

Nomic Embed Text was selected as the embedding model for several reasons:

Allma Studio uses a combination of SQLite for conversation storage and LanceDB for vector embeddings. This hybrid approach optimizes each storage system for its specific use case.

Click to enlarge

Data model showing relationships between Session, Message, Document, and Chunk entities

Key Entities

Key Relationships

• Session → Messages — One session contains many messages (1:N)
• Document → Chunks — One document splits into many chunks (1:N)
• Chunk → Embedding — Each chunk has exactly one vector (1:1)
• Message → Sources — RAG messages reference multiple source chunks (N:N)

AI responses can take several seconds to complete. Without streaming, users would stare at a blank screen—an eternity in modern UX terms. Allma implements true token streaming, displaying each token as it's generated.

Server-Sent Events Implementation

The streaming pipeline uses Server-Sent Events (SSE) to push tokens to the frontend:

# Backend (FastAPI)
async def stream_response():
    async for token in ollama.chat_stream(prompt):
        yield f"data: {json.dumps({'content': token})}\n\n"
    yield f"data: {json.dumps({'done': True, 'sources': sources})}\n\n"

# Frontend (React)
const eventSource = new EventSource('/api/chat');
eventSource.onmessage = (event) => {
    const data = JSON.parse(event.data);
    if (data.done) {
        setSources(data.sources);
    } else {
        appendMessage(data.content);
    }
};

Message Types

Privacy isn't a feature of Allma Studio—it's the foundation. Every architectural decision was made to ensure that sensitive data never leaves the user's machine.

Security Layers

Data Privacy Guarantees

Building a production-quality local AI application surfaced several engineering challenges. Here's how we solved them: