Skip to content

Refactor: Record Per-Message Latency Data from the Receiver Process #78

@dustinblack

Description

@dustinblack

To provide detailed analysis, the benchmark must be capable of recording the latency of every single message.
With the implementation of the multi-process model and end-to-end latency measurement, the correct latency is
now calculated within the child (receiver) process.

This task is to correctly channel this per-message data into the streaming output files only when explicitly
requested by the user via the --streaming-output-csv or --streaming-output-json flags.

Proposed Solution

The logic for handling per-message latency data needs to be conditional and managed carefully between the
parent and child processes.

  1. Conditional Logic in Parent:

    • The parent process will check if either --streaming-output-csv or --streaming-output-json was provided.
    • If a streaming flag is present: The parent will open the user-specified file and pass its file handle (or
      path) to the child process.
    • If no streaming flag is present: The parent will create a temporary, intermediate file (e.g., in /tmp) and
      pass that file's handle to the child. This file will be invisible to the user.
  2. Child Process (Receiver) Logic:

    • The child process receives a file handle from the parent. It does not need to know whether this is a
      user-requested file or a temporary one.
    • In its main receive loop, after calculating the end-to-end latency for each message, the child process
      will write the result as a new record to the provided file handle.
  3. Parent Process (Finalization) Logic:

    • After the child process completes, the parent needs to calculate the final summary statistics (min, max,
      average, percentiles).
    • It will do this by reading the raw data from the file that the child wrote to (either the user's streaming
      file or the temporary file).
    • If a temporary file was used, the parent process will delete it immediately after the summary statistics
      have been calculated.

This approach ensures that:

  • Streaming files are only created when the user explicitly requests them.
  • The final summary is always calculated correctly, regardless of whether streaming is enabled.
  • Raw per-message data is never displayed on stdout.

Acceptance Criteria

  • When no streaming flags are used, no streaming files are created or left in the user's directory.
  • When a streaming flag is used, the child (receiver) process writes all per-message latency records to the
    specified file.
  • The data written to the streaming files accurately reflects the end-to-end latency calculated by the
    receiver.
  • The final summary report is correctly calculated and displayed in all cases (with or without streaming
    enabled).

Context

This is a high-priority task (P2) for ensuring the benchmark provides detailed and analyzable results without
creating unwanted side effects.

This task is dependent on the completion of:

  1. The multi-process refactoring (Issue Refactor: Switch from Multi-Threaded to Multi-Process Model for All IPC Mechanisms #74)
  2. The end-to-end latency measurement implementation (Issue Implement End-to-End Latency Measurement for All IPC Mechanisms #75)

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Priority 2 (nice to have or refinement)enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions