Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
86c1f7a
Added AudioStream library for robot test to be able to simulate stream
thestumonkey Nov 22, 2025
b117291
removed * which breaks windows
thestumonkey Nov 22, 2025
2128ea9
Added middleware so server shows API responses in logs and masks the …
thestumonkey Nov 22, 2025
1515d35
removed old task manager
thestumonkey Nov 22, 2025
c974084
enable job filtering by type
thestumonkey Nov 22, 2025
87f4ff5
removed deprecvated processors
thestumonkey Nov 22, 2025
16181e9
Enabled creation and update of users to get tests to pass
thestumonkey Nov 22, 2025
13f890d
added computed properties for conversation model to get active transc…
thestumonkey Nov 22, 2025
2cc3972
moved queue health endpoint, and added better job query functions
thestumonkey Nov 22, 2025
8782c1a
added params to force waiting for chunk queue to be empty
thestumonkey Nov 22, 2025
a766026
ensure segments are updated on transcript versions
thestumonkey Nov 22, 2025
83b9753
added self healing function if redis loses the worker connection
thestumonkey Nov 22, 2025
31adba6
Changed function to get the conversation audio in order to properly auth
thestumonkey Nov 22, 2025
9a56f7e
Thinned out the payload from conversation controller
thestumonkey Nov 22, 2025
23b4543
moved metadatalinking to jobs
thestumonkey Nov 22, 2025
f46bfa8
added memory version addiing
thestumonkey Nov 22, 2025
2c516fd
added lib for mcp and test framework
thestumonkey Nov 22, 2025
537234b
Moved testing framework setup files to own folder
thestumonkey Nov 22, 2025
a277b36
added tagging file to segment tests that run at different speeds
thestumonkey Nov 22, 2025
b4e9777
Added tests to check queues with audio streaming
thestumonkey Nov 22, 2025
2b9031f
Updated robot tests
thestumonkey Nov 22, 2025
1dbdc01
Override ON method in order to return API response messages back to t…
thestumonkey Nov 22, 2025
5963216
Added test to check for failing workers
thestumonkey Nov 22, 2025
b879e2c
Add Robot Framework tests workflow for CI/CD
thestumonkey Nov 22, 2025
8816bc3
testing env
thestumonkey Nov 22, 2025
096ca05
Fix duplicate environment keys in Robot tests workflow
thestumonkey Nov 22, 2025
8347fcf
Fix test environment file location
thestumonkey Nov 22, 2025
912fcb5
Fix service startup issues in Robot tests workflow
thestumonkey Nov 22, 2025
223c16f
Fix test asset path and improve HTML report instructions
thestumonkey Nov 22, 2025
c0d3ef2
Add test audio assets and fix environment variables
thestumonkey Nov 22, 2025
5cff757
Start RQ workers in CI environment
thestumonkey Nov 22, 2025
51ece28
Add service logs output after tests run
thestumonkey Nov 22, 2025
789672a
Add OPENAI_BASE_URL to backend and worker containers
thestumonkey Nov 22, 2025
ada305d
Add worker registration verification step before tests
thestumonkey Nov 22, 2025
5496801
Check if workers container is running and show logs if not
thestumonkey Nov 22, 2025
635301f
Add environment variables to worker verification step
thestumonkey Nov 22, 2025
e9a1e1b
Start workers after backend is healthy, not with infrastructure services
thestumonkey Nov 22, 2025
aee26a3
Add GitHub Pages deployment for HTML test reports
thestumonkey Nov 22, 2025
807526a
Remove robotframework-reporter-action and add deployment summary
thestumonkey Nov 23, 2025
223e79a
Add Docker layer caching and pip caching to speed up builds
thestumonkey Nov 23, 2025
3a3b4cb
Consolidate and standardize test tags with comprehensive guidelines
thestumonkey Nov 23, 2025
68de6d6
Simplify to 11 approved tags only
thestumonkey Nov 23, 2025
cb11767
fixed conversation active_transcript version and tidied test
thestumonkey Nov 23, 2025
875f496
Add PR comment with test results to robot-tests workflow
thestumonkey Nov 23, 2025
679344e
Add PR comment with test results to robot-tests workflow
thestumonkey Nov 23, 2025
97e5575
Refactor robot-tests workflow to follow integration-tests pattern
thestumonkey Nov 24, 2025
d532596
Increase audio chunks for inactivity timeout test to ensure conversat…
thestumonkey Nov 24, 2025
fb5a31b
Revert to old robot-tests.yml workflow with explicit service management
thestumonkey Nov 25, 2025
848f1fc
Fix YAML syntax error in robot-tests.yml and add act testing guide
thestumonkey Nov 25, 2025
f8d06f9
added support to use a folder for audio uploads
thestumonkey Nov 26, 2025
98a8408
Fixed bug where consumer lock wasn't being cleared
thestumonkey Nov 26, 2025
b82a6ea
Added fixture for tests and queue flushing
thestumonkey Nov 26, 2025
64b3785
Fixed all tests
thestumonkey Nov 26, 2025
1d874a5
fixed asynbc issue with flush jobs on github
thestumonkey Nov 26, 2025
0d50fb9
fixed asynbc issue with flush jobs on github
thestumonkey Nov 26, 2025
9e128a5
fixed cancel job to avoid deadlocks
thestumonkey Nov 27, 2025
fdd0aa3
changed find test conversation to find the first (fixture) one
thestumonkey Nov 27, 2025
733aae4
fixed exporting env keys
thestumonkey Nov 27, 2025
09bab4a
added pre-commit hook to run tests
thestumonkey Nov 28, 2025
8d76886
Clear up of test keywords
thestumonkey Nov 28, 2025
3032825
Added pre-commit install steps to setup
thestumonkey Nov 28, 2025
2f4f4ce
fixed speaker recog getting env vars
thestumonkey Nov 28, 2025
eaf011f
added testing user guide
thestumonkey Nov 28, 2025
27d9249
added write access to PR comments
thestumonkey Nov 29, 2025
fcf372c
Cleat test folder
thestumonkey Nov 29, 2025
0ab1b5b
Cleat test folder
thestumonkey Nov 29, 2025
f857923
Merge branch 'robot_tests' into robot2
thestumonkey Nov 29, 2025
f8f35c0
verifying api keys export
thestumonkey Nov 29, 2025
c9c3920
Fix GitHub Actions env var ordering - revert to working structure
thestumonkey Nov 29, 2025
8f1526f
verifying api keys export
thestumonkey Nov 29, 2025
fa8310a
add checks for secrets
thestumonkey Nov 29, 2025
f0f2f67
fixed test env nor starting in dev env
thestumonkey Nov 29, 2025
aa3933a
Update test execution commands to use 'uv' for improved environment m…
AnkushMalaker Dec 7, 2025
f45b48f
Enhance error handling in job status, retrieval, and cancellation rou…
AnkushMalaker Dec 7, 2025
c302312
remove stray file
AnkushMalaker Dec 7, 2025
1b4f802
Merge branch 'main' into robot2
AnkushMalaker Dec 7, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 19 additions & 1 deletion .github/workflows/integration-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,25 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@v4


- name: Verify required secrets
env:
DEEPGRAM_API_KEY: ${{ secrets.DEEPGRAM_API_KEY }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
run: |
echo "Verifying required secrets..."
if [ -z "$DEEPGRAM_API_KEY" ]; then
echo "❌ ERROR: DEEPGRAM_API_KEY secret is not set"
exit 1
fi
if [ -z "$OPENAI_API_KEY" ]; then
echo "❌ ERROR: OPENAI_API_KEY secret is not set"
exit 1
fi
echo "✓ DEEPGRAM_API_KEY is set (length: ${#DEEPGRAM_API_KEY})"
echo "✓ OPENAI_API_KEY is set (length: ${#OPENAI_API_KEY})"
echo "✓ All required secrets verified"

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

Expand Down
55 changes: 53 additions & 2 deletions .github/workflows/robot-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,24 @@ jobs:
- name: Checkout code
uses: actions/checkout@v4

- name: Verify required secrets
env:
DEEPGRAM_API_KEY: ${{ secrets.DEEPGRAM_API_KEY }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
run: |
echo "Verifying required secrets..."
if [ -z "$DEEPGRAM_API_KEY" ]; then
echo "❌ ERROR: DEEPGRAM_API_KEY secret is not set"
exit 1
fi
if [ -z "$OPENAI_API_KEY" ]; then
echo "❌ ERROR: OPENAI_API_KEY secret is not set"
exit 1
fi
echo "✓ DEEPGRAM_API_KEY is set (length: ${#DEEPGRAM_API_KEY})"
echo "✓ OPENAI_API_KEY is set (length: ${#OPENAI_API_KEY})"
echo "✓ All required secrets verified"

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
Expand Down Expand Up @@ -161,6 +179,26 @@ jobs:

echo "✓ All services ready!"

- name: Verify checked out code
working-directory: tests
run: |
echo "Current git commit:"
git log -1 --oneline
echo ""
echo "Test files in current checkout:"
find . -name "*.robot" -type f | head -10
echo ""
echo "Sample of tags in test files:"
grep -h "\[Tags\]" endpoints/*.robot infrastructure/*.robot integration/*.robot 2>/dev/null | head -20 || echo "No tag files found"

- name: Clean previous test results
working-directory: tests
run: |
echo "Cleaning any previous test results..."
rm -rf results
mkdir -p results
echo "✓ Fresh results directory created"

- name: Run Robot Framework tests
working-directory: tests
env:
Expand All @@ -170,8 +208,11 @@ jobs:
OPENAI_MODEL: gpt-4o-mini
DEEPGRAM_API_KEY: ${{ secrets.DEEPGRAM_API_KEY }}
run: |
# Run all tests
make all OUTPUTDIR=results || true
# Run all tests (don't fail workflow to allow artifact upload)
make all OUTPUTDIR=results
TEST_EXIT_CODE=$?
echo "test_exit_code=$TEST_EXIT_CODE" >> $GITHUB_ENV
exit 0 # Don't fail here, we'll fail at the end after uploading artifacts

- name: Show service logs
if: always()
Expand Down Expand Up @@ -343,3 +384,13 @@ jobs:
working-directory: backends/advanced
run: |
docker compose -f docker-compose-test.yml down -v

- name: Fail workflow if tests failed
if: always()
run: |
if [ "${{ env.test_exit_code }}" != "0" ]; then
echo "❌ Tests failed with exit code ${{ env.test_exit_code }}"
exit 1
else
echo "✅ All tests passed"
fi
20 changes: 19 additions & 1 deletion .github/workflows/speaker-recognition-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,25 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@v4


- name: Verify required secrets
env:
HF_TOKEN: ${{ secrets.HF_TOKEN }}
DEEPGRAM_API_KEY: ${{ secrets.DEEPGRAM_API_KEY }}
run: |
echo "Verifying required secrets..."
if [ -z "$HF_TOKEN" ]; then
echo "❌ ERROR: HF_TOKEN secret is not set"
exit 1
fi
if [ -z "$DEEPGRAM_API_KEY" ]; then
echo "❌ ERROR: DEEPGRAM_API_KEY secret is not set"
exit 1
fi
echo "✓ HF_TOKEN is set (length: ${#HF_TOKEN})"
echo "✓ DEEPGRAM_API_KEY is set (length: ${#DEEPGRAM_API_KEY})"
echo "✓ All required secrets verified"

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

Expand Down
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -76,3 +76,9 @@ extras/openmemory-mcp/data/*
backends/advanced/nginx.conf
backends/advanced/Caddyfile

app/ios/Pods
results
log.html
output.xml
report.html
.secrets
24 changes: 24 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,26 @@
repos:
# Local hooks (project-specific checks)
- repo: local
hooks:
# Run Robot Framework endpoint tests before push
- id: robot-framework-tests
name: Robot Framework Tests (Endpoints)
entry: bash -c 'cd tests && make endpoints OUTPUTDIR=.pre-commit-results'
language: system
pass_filenames: false
stages: [push]
verbose: true

# Clean up test results after hook runs
- id: cleanup-test-results
name: Cleanup Test Results
entry: bash -c 'cd tests && rm -rf .pre-commit-results'
language: system
pass_filenames: false
stages: [push]
always_run: true

# Code formatting
- repo: https://github.com/psf/black
rev: 24.4.2
hooks:
Expand All @@ -9,6 +31,8 @@ repos:
hooks:
- id: isort
files: ^backends/advanced-backend/src/.*\.py$

# File hygiene
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
hooks:
Expand Down
32 changes: 25 additions & 7 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,9 +38,6 @@ uv run pytest tests/test_memory_service.py # Single test file
# Environment setup
cp .env.template .env # Configure environment variables

# Setup test environment (optional, for running integration tests)
uv run --with-requirements setup-requirements.txt python setup_test_env.py # Creates .env.test

# Reset data (development)
sudo rm -rf backends/advanced/data/
```
Expand Down Expand Up @@ -71,10 +68,6 @@ cd backends/advanced
# Requires .env file with DEEPGRAM_API_KEY and OPENAI_API_KEY
cp .env.template .env # Configure API keys

# Optional: Setup test environment with test-specific credentials
# (wizard.py prompts for this, or run manually)
uv run --with-requirements setup-requirements.txt python setup_test_env.py

# Run full integration test suite
./run-test.sh

Expand Down Expand Up @@ -408,6 +401,31 @@ For detailed technical documentation, see:
- **[@docs/speaker-recognition.md](docs/speaker-recognition.md)**: Advanced analysis and live inference features
- **[@docs/distributed-deployment.md](docs/distributed-deployment.md)**: Multi-machine deployment with Tailscale

## Robot Framework Testing

**IMPORTANT: When writing or modifying Robot Framework tests, you MUST follow the testing guidelines.**

Before writing any Robot Framework test:
1. **Read [@tests/TESTING_GUIDELINES.md](tests/TESTING_GUIDELINES.md)** for comprehensive testing patterns and standards
2. **Check [@tests/tags.md](tests/tags.md)** for approved tags - ONLY 11 tags are permitted
3. **SCAN existing resource files** for keywords - NEVER write code that duplicates existing keywords
4. **Follow the Arrange-Act-Assert pattern** with inline verifications (not abstracted to keywords)

Key Testing Rules:
- **Check Existing Keywords FIRST**: Before writing ANY test code, scan relevant resource files (`websocket_keywords.robot`, `queue_keywords.robot`, `conversation_keywords.robot`, etc.) for existing keywords
- **Tags**: ONLY use the 11 approved tags from tags.md, tab-separated (e.g., `[Tags] infra audio-streaming`)
- **Verifications**: Write assertions directly in tests, not in resource keywords
- **Keywords**: Only create keywords for reusable setup/action operations AFTER confirming no existing keyword exists
- **Resources**: Always check existing resource files before creating new keywords or duplicating logic
- **Naming**: Use descriptive names that explain business purpose, not technical implementation

**DO NOT:**
- Write inline code without checking if a keyword already exists for that operation
- Create custom tags (use only the 11 approved tags)
- Abstract verifications into keywords (keep them inline in tests)
- Use space-separated tags (must be tab-separated)
- Skip reading the guidelines before writing tests

## Notes for Claude
Check if the src/ is volume mounted. If not, do compose build so that code changes are reflected. Do not simply run `docker compose restart` as it will not rebuild the image.
Check backends/advanced/Docs for up to date information on advanced backend.
Expand Down
24 changes: 24 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ menu: ## Show interactive menu (default)
@echo "================================"
@echo
@echo "📋 Quick Actions:"
@echo " setup-dev 🛠️ Setup development environment (git hooks, pre-commit)"
@echo " setup-k8s 🏗️ Complete Kubernetes setup (registry + infrastructure + RBAC)"
@echo " config 📝 Generate all configuration files"
@echo " deploy 🚀 Deploy using configured mode ($(DEPLOYMENT_MODE))"
Expand Down Expand Up @@ -114,6 +115,29 @@ help: ## Show detailed help for all targets
@echo "🧹 CLEANUP:"
@echo " clean Clean up generated configuration files"

# ========================================
# DEVELOPMENT SETUP
# ========================================

setup-dev: ## Setup development environment (git hooks, pre-commit)
@echo "🛠️ Setting up development environment..."
@echo ""
@echo "📦 Installing pre-commit..."
@pip install pre-commit 2>/dev/null || pip3 install pre-commit
@echo ""
@echo "🔧 Installing git hooks..."
@pre-commit install --hook-type pre-push
@pre-commit install --hook-type pre-commit
@echo ""
@echo "✅ Development environment setup complete!"
@echo ""
@echo "💡 Hooks installed:"
@echo " • Robot Framework tests run before push"
@echo " • Black/isort format Python code on commit"
@echo " • Code quality checks on commit"
@echo ""
@echo "⚙️ To skip hooks: git push --no-verify / git commit --no-verify"

# ========================================
# KUBERNETES SETUP
# ========================================
Expand Down
1 change: 1 addition & 0 deletions backends/advanced/.dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,6 @@
!nginx.conf
!nginx.conf.template
!start.sh
!start-k8s.sh
!start-workers.sh
!Caddyfile
Loading
Loading