SessionIQ: engineering notes

Problem

A growing music catalog means scattered bounces, MIDI files, mix notes and unfinished decisions. Finding the right file is only part of the problem: understanding what needs attention is another.

Approach

SessionIQ organizes audio, MIDI and production notes into albums and songs. Analysis feeds a retrieval-augmented generation (RAG) assistant, exact metadata tools and production-planning views.

Hybrid RAG with file-level citations

Retrieval-augmented generation combines lexical search and metadata signals with optional ChromaDB vector search. Relevant files ground answers from OpenAI or local Ollama models, with source citations and conversational follow-ups. Without embeddings, lexical retrieval remains available.

Tool-backed answers and streaming

The assistant uses seven library tools to search, filter, calculate statistics, inspect assets, compare tracks and rank projects. Counts and comparisons run against metadata; server-sent events stream question interpretation, tool calls and answer text.

MCP integration for external agents

An optional Model Context Protocol (MCP) server exposes the same seven tools to clients such as Claude Desktop, Cursor and custom agents over stdio. It is read-only, uses in-memory lexical and metadata retrieval, and reads a library snapshot at startup; restart it to pick up new files.

Audio, MIDI and reference analysis

Audio analysis extracts tempo, musical key, peak/RMS levels, integrated LUFS, brightness and beat positions. MIDI analysis extracts notes, pitch and velocity ranges, instruments and tempo. Track similarity includes per-dimension explanations, while reference A/B comparisons show loudness, tone and duration differences.

Creative memory and production planning

Saved decisions, editable preferences and a library-derived producer profile preserve creative context. Project Health checks, extracted tasks, finish-next rankings, weekly digests and Markdown session reports help turn unfinished work into a clear next action.

Local-first workflow

A deterministic answer engine works offline without a model. Ollama, OpenAI, ChromaDB embeddings and local Whisper voice-memo transcription are optional. Folder import, smart collections, tags and background re-analysis keep the library organized.

Quality reports and evaluation

Each answer exposes citation and numeric checks, model provenance, latency, token usage and tools used. Query logs and feedback reveal gaps; labeled retrieval evaluations, Pytest, frontend checks and CI make behavior inspectable. Automated checks do not establish that every claim is correct.

Design decision

Keep retrieval and computation separate. Finding a relevant file is a search problem; identifying the loudest track is a comparison across real metadata. Tool-backed answers handle the latter without asking a model to guess.

Current scope

Designed as a single-user local application with one API process per library. Background jobs do not survive a restart. It is not a publicly hosted multi-user SaaS.