Lyria Studio
A music tool that generates a track from a chat prompt with Google's Lyria RealTime model, then opens it in a studio to split it into sections and regenerate any one of them. I built it during my internship at DevFortress, first as a single Streamlit prototype and then as three parts: a Next.js chat, a FastAPI backend, and a Streamlit studio.
Architecture
The chat is a Next.js page that sends the prompt, duration, BPM, and density to a FastAPI endpoint. The backend opens a WebSocket to Lyria RealTime through the Gemini API, writes the streamed audio into a WAV file, and sends the file back for the chat to play. An Open in Studio link passes the file name and the original prompt to the Streamlit studio, which splits the track into equal sections and regenerates the one I pick with its own prompt, BPM, and density. Pydub stitches each new section into a working copy, and exporting saves the result as a new WAV file to download.
Key decisions
A chat for ideas, a studio for editing
The prototype was one Streamlit app that switched between a simple view and an editor. In V2 I split it: a Next.js chat asks a FastAPI backend for tracks, and the Streamlit studio keeps the editing in Python, next to the audio code. That let me put the chat online by itself on Vercel, while the studio runs locally.
Each section gets its own prompt
The studio splits the track into equal sections, from 1 to 10. A section is regenerated with its own in-painting prompt, BPM and density, so the bridge can turn denser or slower while the rest of the track stays as it was. The prompt starts as the original one, so a small change is a small edit.
Edits never touch the original
A regenerated section is first a candidate that I can play next to the track. Only Commit Change stitches it in, and only into a working copy. Export writes a new WAV, so the generated original is always there to go back to.
The hard part
Turning a live jam model into an editable file
Lyria RealTime is made to play live, like a jam: I send a prompt and PLAY over a WebSocket, and audio keeps coming with no end, because nothing in the request sets a length. To get a track of an exact duration, I count bytes: 24,000 samples a second, two channels, 2 bytes each, so 96,000 bytes a second. Chunks go into a WAV as they arrive, and STOP goes out the moment the count is reached. Regenerating a section in the middle added the seams: a new clip starts and ends cold, so I ask Lyria for the section plus half the crossfade and blend it in at both ends with a 1 to 5 second crossfade.
Security and performance
- The Google API key lives only in an environment variable on the backend. The browser talks to my FastAPI endpoint and never sees the key or the Lyria connection.
- The API has no accounts and accepts only a prompt and three numeric settings. CORS is open to every origin for now, and the README notes it must be locked to the chat's domain in production.