A fully autonomous, voice-driven website editor. Users issue natural language or voice commands (e.g. “Add a testimonials section before the footer”), and the system dynamically maps the intent to DOM-level operations, executes changes, and logs all updates in real-time.
/backend # FastAPI, agent workflow, ChromaDB, speech-to-text, etc.
/talkable # Next.js frontend, live editor, changelog viewer
- 🎙 Voice Command Support — Transcribe audio to actionable commands.
- 🧬 DOM Manipulation — Insert, delete, replace, and style HTML elements via natural language.
- 🧠 ChromaDB RAG — Store and retrieve HTML patterns for context-aware editing.
- 📝 Changelog Generation — Human-readable logs for transparency and undo support.
- 🧰 Feedback & Repair — Automatic detection and repair of layout issues.
- 🚀 Live Editor Frontend — Real-time preview and changelog viewer.
- 🔥 FastAPI Backend — Easy integration with any frontend.
-
Install dependencies:
pip install fastapi uvicorn chromadb bs4 langchain langgraph
-
Configure your Google Gemini API key:
- Edit
app.pyandspeech_to_text_tool.py. - Set your
GOOGLE_API_KEYas an environment variable or hardcode directly.
- Edit
Run the backend server:
uvicorn app:app --reload✅ The FastAPI server will be available at: http://localhost:8000
| Endpoint | Description |
|---|---|
POST /chat |
Main endpoint for user messages and agent replies |
POST /store_html |
Store HTML content in ChromaDB for RAG |
speech_to_textstore_html_in_chromadbretrieve_html_from_chromadbexecute_dom_operationgenerate_changelog_entryfeedback_and_repair
-
Install dependencies:
npm install
-
Configure any environment variables if required.
Run the frontend:
npm run dev✅ The frontend will be available at: http://localhost:3000
✅ Voice input interface ✅ Live preview of the edited site ✅ Real-time changelog viewer ✅ Responsive design for all screen sizes
- User sends a voice or text command via the frontend.
- Backend transcribes (if audio), retrieves context, plans, and executes DOM operations.
- Backend validates and repairs the DOM if needed.
- Backend generates a changelog entry and returns updated DOM and logs.
- Frontend updates the live preview and changelog viewer.
This project is licensed under the MIT License. See the LICENSE file for more details.
Feel free to contribute, raise issues, or suggest enhancements!