feat: add execution traces for tool/skill/MCP tracking (#194) #252

ThanhNguyxn · 2025-12-12T09:13:07Z

Summary

Implements #194 - Show which tools/skills/MCPs ran during a response.

Changes

Database

Migration 008: New execution_traces table with indexes for tracking tool/skill/MCP execution

Backend Services

TraceManager.ts: Service for recording and querying execution traces
TraceRoutes.ts: API endpoints for trace retrieval
Types: ExecutionTraceRow, ExecutionTraceInput interfaces

Frontend

ExecutionTrace: Interface added to viewer types
ObservationCard: Added "traces" toggle and list display for execution traces

Integration

TraceManager and TraceRoutes wired into WorkerService
Auto-recording: Traces are automatically recorded in SessionRoutes when observations are queued

API Endpoints

GET /api/traces                     # Recent traces (limit=100)
GET /api/traces/:session_id         # All traces for session  
GET /api/traces/:session_id/:prompt # Traces for specific prompt

Database Schema

CREATE TABLE execution_traces (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  sdk_session_id TEXT NOT NULL,
  prompt_number INTEGER,
  step_order INTEGER NOT NULL,
  trace_type TEXT CHECK(trace_type IN ('tool', 'skill', 'mcp', 'hook')) NOT NULL,
  name TEXT NOT NULL,
  source TEXT,
  input_summary TEXT,
  output_summary TEXT,
  duration_ms INTEGER,
  created_at TEXT NOT NULL,
  created_at_epoch INTEGER NOT NULL
);

Completed Features

Database schema & migration
Backend services & API
Auto-recording of tool usage
Frontend UI display & toggle

…edotmack#194) - Add migration008 for execution_traces table - Add ExecutionTraceRow and ExecutionTraceInput types - Create TraceManager service for recording/querying traces - Add TraceRoutes API endpoints (/api/traces) This provides the database schema and backend services needed to track which tools, skills, MCPs, and hooks ran during each response. TODO: Wire TraceManager into WorkerService and SessionRoutes for automatic trace recording on each observation.

- Wire TraceManager and TraceRoutes into WorkerService - Add ExecutionTrace interface to frontend types.ts - TraceRoutes API endpoints now available after DB init Full integration of execution traces feature for issue thedotmack#194

- Add getTraceManager() getter to WorkerService - Auto-record execution traces in SessionRoutes.handleObservationsByClaudeId - Add traces toggle button in ObservationCard UI - Fetch and display traces from /api/traces endpoint - Show trace type, name, and duration in UI list Closes thedotmack#194

Copilot

Pull request overview

This PR implements execution trace tracking for tools, skills, MCPs, and hooks during Claude responses. It adds database infrastructure, backend services, API endpoints, and type definitions to record and query which resources were utilized during each interaction.

Key Changes:

New execution_traces database table with indexes for efficient querying by session, prompt, type, and time
TraceManager service for recording and retrieving execution traces with auto-incrementing step counters
REST API endpoints for querying traces at session, prompt, and global levels

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
src/services/sqlite/migrations.ts	Adds migration008 defining the execution_traces table schema with appropriate indexes and foreign key constraints
src/services/sqlite/types.ts	Defines ExecutionTraceRow and ExecutionTraceInput interfaces for type safety; removes trailing whitespace
src/services/worker/TraceManager.ts	Implements trace recording with auto-increment step_order tracking and query methods for retrieving traces
src/services/worker/http/routes/TraceRoutes.ts	Provides three REST API endpoints for querying traces by session, prompt, or recent activity
src/services/worker-service.ts	Integrates TraceManager and TraceRoutes into the worker service initialization flow
src/ui/viewer/types.ts	Adds ExecutionTrace interface for frontend consumption of trace data

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-12T09:19:49Z

src/services/worker/http/routes/TraceRoutes.ts

+        const promptNum = parseInt(prompt_number, 10);
+
+        if (isNaN(promptNum)) {
+            this.badRequest(res, 'prompt_number must be a number');


The error message 'prompt_number must be a number' is not very helpful because it doesn't indicate what the actual invalid value was. Consider including the invalid value in the error message, e.g., prompt_number must be a valid integer, received: ${prompt_number}.

Suggested change

this.badRequest(res, 'prompt_number must be a number');

this.badRequest(res, `prompt_number must be a valid integer, received: ${prompt_number}`);

Copilot · 2025-12-12T09:19:49Z

src/services/worker/http/routes/TraceRoutes.ts

+    private handleGetSessionTraces = this.wrapHandler((req: Request, res: Response): void => {
+        const { session_id } = req.params;
+        const traces = this.traceManager.getTracesForSession(session_id);
+        res.json({ traces, count: traces.length });


The session_id parameter is not validated before being passed to the TraceManager. Consider adding validation to ensure session_id is not empty and is properly formatted. An empty or malformed session_id could lead to unexpected database queries.

Copilot · 2025-12-12T09:19:49Z

src/services/worker/http/routes/TraceRoutes.ts

+        // Get all traces for a session
+        app.get('/api/traces/:session_id', this.handleGetSessionTraces.bind(this));
+
+        // Get traces for a specific prompt
+        app.get('/api/traces/:session_id/:prompt_number', this.handleGetPromptTraces.bind(this));
+
+        // Get recent traces (for debugging)
+        app.get('/api/traces', this.handleGetRecentTraces.bind(this));


Route order is incorrect. The route /api/traces (line 22) should be registered BEFORE /api/traces/:session_id (line 16), otherwise Express will treat the word "traces" in the GET /api/traces request as a session_id parameter and route it to handleGetSessionTraces instead of handleGetRecentTraces. Move the /api/traces route registration to occur before the parameterized routes.

Suggested change

// Get all traces for a session

app.get('/api/traces/:session_id', this.handleGetSessionTraces.bind(this));

// Get traces for a specific prompt

app.get('/api/traces/:session_id/:prompt_number', this.handleGetPromptTraces.bind(this));

// Get recent traces (for debugging)

app.get('/api/traces', this.handleGetRecentTraces.bind(this));

// Get recent traces (for debugging)

app.get('/api/traces', this.handleGetRecentTraces.bind(this));

// Get all traces for a session

app.get('/api/traces/:session_id', this.handleGetSessionTraces.bind(this));

// Get traces for a specific prompt

app.get('/api/traces/:session_id/:prompt_number', this.handleGetPromptTraces.bind(this));

Copilot · 2025-12-12T09:19:50Z

src/services/worker/http/routes/TraceRoutes.ts

+     * GET /api/traces - Get recent traces across all sessions
+     */
+    private handleGetRecentTraces = this.wrapHandler((req: Request, res: Response): void => {
+        const limit = parseInt(req.query.limit as string, 10) || 100;


When the limit query parameter is provided but is not a valid number (e.g., "abc"), parseInt returns NaN and the code falls back to 100. This could be confusing for API users. Consider validating the limit parameter and returning a 400 error if it's provided but invalid, rather than silently using the default.

Suggested change

const limit = parseInt(req.query.limit as string, 10) || 100;

let limit = 100;

if (req.query.limit !== undefined) {

const parsedLimit = parseInt(req.query.limit as string, 10);

if (isNaN(parsedLimit)) {

this.badRequest(res, 'limit must be a valid number');

return;

}

limit = parsedLimit;

}

Copilot · 2025-12-12T09:19:50Z

src/services/sqlite/migrations.ts

+      );
+
+      CREATE INDEX IF NOT EXISTS idx_traces_session ON execution_traces(sdk_session_id);
+      CREATE INDEX IF NOT EXISTS idx_traces_prompt ON execution_traces(sdk_session_id, prompt_number);


Consider creating a composite index on (sdk_session_id, prompt_number, step_order) to optimize the common query pattern for getting traces for a specific prompt in order. The current idx_traces_prompt index on (sdk_session_id, prompt_number) is good but doesn't cover the ORDER BY step_order ASC clause used in getTracesForPrompt.

Suggested change

CREATE INDEX IF NOT EXISTS idx_traces_prompt ON execution_traces(sdk_session_id, prompt_number);

CREATE INDEX IF NOT EXISTS idx_traces_prompt ON execution_traces(sdk_session_id, prompt_number);

CREATE INDEX IF NOT EXISTS idx_traces_prompt_order ON execution_traces(sdk_session_id, prompt_number, step_order);

Copilot · 2025-12-12T09:19:50Z

src/services/worker/TraceManager.ts

+        // Auto-increment step_order per session if not provided
+        let stepOrder = input.step_order;
+        if (stepOrder === undefined || stepOrder === 0) {
+            const currentCount = this.traceCounters.get(input.sdk_session_id) || 0;
+            stepOrder = currentCount + 1;
+            this.traceCounters.set(input.sdk_session_id, stepOrder);
+        }


The comment "Auto-increment step_order per session if not provided" should be updated to clarify the behavior when step_order is 0. Currently it's unclear whether 0 is considered a valid explicit value or treated as "not provided".

Copilot · 2025-12-12T09:19:51Z

src/services/worker/TraceManager.ts

+    private traceCounters: Map<string, number> = new Map(); // session -> step_order counter
+
+    constructor(sessionStore: SessionStore) {
+        this.sessionStore = sessionStore;
+    }
+
+    /**
+     * Record an execution trace
+     */
+    recordTrace(input: ExecutionTraceInput): number {
+        const now = new Date();
+        const nowEpoch = now.getTime();
+
+        // Auto-increment step_order per session if not provided
+        let stepOrder = input.step_order;
+        if (stepOrder === undefined || stepOrder === 0) {
+            const currentCount = this.traceCounters.get(input.sdk_session_id) || 0;
+            stepOrder = currentCount + 1;
+            this.traceCounters.set(input.sdk_session_id, stepOrder);
+        }


The traceCounters Map is not thread-safe. If multiple traces are recorded concurrently for the same session, there could be race conditions where two traces get the same step_order value. Consider using a database-based counter or implementing proper synchronization to ensure step_order uniqueness.

Copilot · 2025-12-12T09:19:51Z

src/services/worker-service.ts

+    // Initialize TraceManager and routes
+    this.traceManager = new TraceManager(this.dbManager.getSessionStore());
+    this.traceRoutes = new TraceRoutes(this.traceManager);
+    this.traceRoutes.setupRoutes(this.app);
+    logger.info('WORKER', 'TraceManager initialized and trace routes registered');


The TraceManager is being initialized before the execution_traces table is created. The migration008 that creates the execution_traces table is defined in migrations.ts but is never actually applied because SessionStore doesn't call it in its constructor. You need to add a method in SessionStore (e.g., ensureExecutionTracesTable()) that applies migration008, and call it in the SessionStore constructor after the other migration calls.

Copilot · 2025-12-12T09:19:51Z

src/services/worker/TraceManager.ts

+
+        // Auto-increment step_order per session if not provided
+        let stepOrder = input.step_order;
+        if (stepOrder === undefined || stepOrder === 0) {


The step_order auto-increment logic has a flaw. When step_order is exactly 0, it's treated the same as undefined and gets auto-incremented. However, 0 could be a valid explicit step_order value. Consider checking only for undefined: if (stepOrder === undefined) instead of if (stepOrder === undefined || stepOrder === 0).

Suggested change

if (stepOrder === undefined || stepOrder === 0) {

if (stepOrder === undefined) {

Copilot · 2025-12-12T09:19:51Z

src/services/sqlite/types.ts

+export interface ExecutionTraceInput {
+  sdk_session_id: string;
+  prompt_number?: number;
+  step_order: number;


The step_order field is marked as required (not optional) in ExecutionTraceInput, but the TraceManager.recordTrace method expects it to be optional and provides auto-increment functionality. This is inconsistent. Consider making step_order optional here (step_order?: number) to match the actual usage pattern in TraceManager.

Suggested change

step_order: number;

step_order?: number;

Address Copilot review suggestion - include the invalid value in the error message when prompt_number is not a number.

ThanhNguyxn added 2 commits December 12, 2025 16:07

feat: complete execution traces integration

7f58e67

- Wire TraceManager and TraceRoutes into WorkerService - Add ExecutionTrace interface to frontend types.ts - TraceRoutes API endpoints now available after DB init Full integration of execution traces feature for issue thedotmack#194

Copilot AI review requested due to automatic review settings December 12, 2025 09:13

Copilot started reviewing on behalf of ThanhNguyxn December 12, 2025 09:13 View session

Copilot AI reviewed Dec 12, 2025

View reviewed changes

fix: improve error message for invalid prompt_number

fcac7e0

Address Copilot review suggestion - include the invalid value in the error message when prompt_number is not a number.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

feat: add execution traces for tool/skill/MCP tracking (#194) #252

feat: add execution traces for tool/skill/MCP tracking (#194) #252

Uh oh!

ThanhNguyxn commented Dec 12, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Dec 12, 2025

Uh oh!

Copilot AI Dec 12, 2025

Uh oh!

Copilot AI Dec 12, 2025

Uh oh!

Copilot AI Dec 12, 2025

Uh oh!

Copilot AI Dec 12, 2025

Uh oh!

Copilot AI Dec 12, 2025

Uh oh!

Copilot AI Dec 12, 2025

Uh oh!

Copilot AI Dec 12, 2025

Uh oh!

Copilot AI Dec 12, 2025

Uh oh!

Copilot AI Dec 12, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

	this.badRequest(res, 'prompt_number must be a number');
	this.badRequest(res, `prompt_number must be a valid integer, received: ${prompt_number}`);

-        const limit = parseInt(req.query.limit as string, 10) || 100;
+        let limit = 100;
+        if (req.query.limit !== undefined) {
+            const parsedLimit = parseInt(req.query.limit as string, 10);
+            if (isNaN(parsedLimit)) {
+                this.badRequest(res, 'limit must be a valid number');
+                return;
+            }
+            limit = parsedLimit;
+        }

	CREATE INDEX IF NOT EXISTS idx_traces_prompt ON execution_traces(sdk_session_id, prompt_number);
	CREATE INDEX IF NOT EXISTS idx_traces_prompt ON execution_traces(sdk_session_id, prompt_number);
	CREATE INDEX IF NOT EXISTS idx_traces_prompt_order ON execution_traces(sdk_session_id, prompt_number, step_order);

	if (stepOrder === undefined \|\| stepOrder === 0) {
	if (stepOrder === undefined) {

Uh oh!

feat: add execution traces for tool/skill/MCP tracking (#194) #252

Are you sure you want to change the base?

feat: add execution traces for tool/skill/MCP tracking (#194) #252

Uh oh!

Conversation

ThanhNguyxn commented Dec 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Database

Backend Services

Frontend

Integration

API Endpoints

Database Schema

Completed Features

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

ThanhNguyxn commented Dec 12, 2025 •

edited

Loading