Skip to content

Skip Docker image validation when Docker is unavailable, add --validate-images flag#26074

Merged
pelikhan merged 6 commits intomainfrom
copilot/remove-docker-dependency-validate
Apr 13, 2026
Merged

Skip Docker image validation when Docker is unavailable, add --validate-images flag#26074
pelikhan merged 6 commits intomainfrom
copilot/remove-docker-dependency-validate

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 13, 2026

gh aw compile --validate returns warnings when Docker is not installed or the daemon is not running (e.g., ubuntu-slim runners). Compile is a source transformation step — image pullability is a runtime concern.

Changes

  • pkg/workflow/docker_validation.go: validateDockerImage() now returns nil instead of an error when Docker CLI is missing or daemon is unresponsive. Debug-level logging preserved. Argument injection guard (- prefix) still runs unconditionally. Accepts a new requireDocker parameter to opt-in to strict validation.
  • --validate-images flag: Added to both compile and validate commands. When specified, container image validation fails with an actionable error if Docker is not installed or the daemon is not running, instead of silently skipping.
  • pkg/workflow/compiler_types.go: Added requireDocker field and SetRequireDocker setter to the Compiler struct.
  • pkg/cli/compile_config.go: Added ValidateImages field to CompileConfig, wired through compile_compiler_setup.go.
  • pkg/workflow/maintenance_workflow.go: The generated validate_workflows job now uses compile --validate --validate-images --verbose, ensuring Docker must be available for container image validation in the agentic maintenance workflow.
  • pkg/workflow/docker_validation_graceful_test.go: Unit tests for graceful degradation path, continued hyphen-prefix rejection, and requireDocker failure paths.

When Docker is available, validation behavior is unchanged. Without --validate-images, container image validation is silently skipped when Docker is unavailable.

Copilot AI and others added 2 commits April 13, 2026 18:15
When Docker is not installed or the daemon is not running, silently skip
container image validation during compile instead of returning errors
that surface as spurious warnings. This removes the hard dependency on
Docker from `gh aw compile --validate`, making it safe to use on
ubuntu-slim runners and other environments without Docker.

Fixes #1454

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/c1be8c98-5f4c-4692-a036-c1c7ebb6fd0c

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Remove Docker dependency from gh aw compile --validate Skip Docker image validation when Docker is unavailable Apr 13, 2026
Copilot AI requested a review from pelikhan April 13, 2026 18:18
@github-actions github-actions bot mentioned this pull request Apr 13, 2026
@pelikhan pelikhan marked this pull request as ready for review April 13, 2026 18:44
Copilot AI review requested due to automatic review settings April 13, 2026 18:44
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates Docker image validation to gracefully skip checks when Docker is unavailable so gh aw compile --validate doesn’t emit warnings on runners/machines without Docker.

Changes:

  • Make validateDockerImage() return nil (skip) when Docker CLI is missing or the daemon is not running.
  • Preserve the hyphen-prefix guard to prevent argument injection regardless of Docker availability.
  • Add unit tests for the graceful-degradation behavior and the unconditional hyphen-prefix rejection.
Show a summary per file
File Description
pkg/workflow/docker_validation.go Switches Docker-missing/daemon-down cases from warning-producing errors to silent skips (debug-log only).
pkg/workflow/docker_validation_graceful_test.go Adds tests for skip behavior when Docker is unavailable and for always-on hyphen-prefix rejection.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (1)

pkg/workflow/docker_validation_graceful_test.go:31

  • These tests are environment-dependent and will be skipped entirely when Docker is installed and the daemon is running, which means the graceful-degradation behavior may never be exercised in CI/dev. To make the test deterministic, consider forcing the 'Docker unavailable' condition (e.g., temporarily setting PATH to an empty/temp dir so exec.LookPath("docker") fails) or using a fake docker binary in a temp dir; if you stub docker info, also reset the cached dockerDaemonOnce/dockerDaemonAvailable between subtests to avoid cross-test leakage.
func TestValidateDockerImage_SkipsWhenDockerUnavailable(t *testing.T) {
	// If docker is not installed or daemon not running, validation should
	// silently pass — no error, no warning.
	if _, lookErr := exec.LookPath("docker"); lookErr != nil {
		err := validateDockerImage("ghcr.io/some/image:latest", false)
		assert.NoError(t, err, "should silently skip when Docker is not installed")
		return
	}
	if !isDockerDaemonRunning() {
		err := validateDockerImage("ghcr.io/some/image:latest", false)
		assert.NoError(t, err, "should silently skip when Docker daemon is not running")
		return
	}

	t.Skip("Docker is available — graceful degradation path not exercised")
  • Files reviewed: 2/2 changed files
  • Comments generated: 2

Comment on lines +1 to +7
//go:build !integration

package workflow

import (
"os/exec"
"testing"
Copy link

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The build constraint for this test file is !integration, but it imports os/exec and calls isDockerDaemonRunning(), which only exists in docker_validation.go (excluded under js/wasm). This will fail to compile for GOOS=js/GOARCH=wasm (or any build with js/wasm tags). Consider updating the build tag to also exclude js and wasm (e.g. !integration && !js && !wasm), or provide wasm-safe stubs for the referenced symbols.

This issue also appears on line 17 of the same file.

Copilot uses AI. Check for mistakes.
Comment thread pkg/workflow/docker_validation.go Outdated
// - If the Docker daemon is not running, validation is silently skipped (debug log only)
// - If an image cannot be pulled due to authentication (private repo), validation passes
// - If an image truly doesn't exist, returns an error
// - Verbose mode provides detailed validation feedback
Copy link

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The header comment says "Verbose mode provides detailed validation feedback", but validateDockerImage itself does not use the verbose parameter (debug logging is controlled by the DEBUG/ACTIONS_RUNNER_DEBUG env vars, and the only verbose output is printed by the caller on success). Consider adjusting this comment to avoid implying that --verbose changes Docker validation logging/behavior here.

Suggested change
// - Verbose mode provides detailed validation feedback
// - Detailed validation logging is available via debug logging when enabled

Copilot uses AI. Check for mistakes.
@github-actions
Copy link
Copy Markdown
Contributor

🧪 Test Quality Sentinel Report

Test Quality Score: 90/100

Excellent test quality

Metric Value
New/modified tests analyzed 3
✅ Design tests (behavioral contracts) 3 (100%)
⚠️ Implementation tests (low value) 0 (0%)
Tests with error/edge cases 3 (100%)
Duplicate test clusters 0
Test inflation detected ⚠️ Yes (65 test lines vs 22 production lines added, ratio ≈ 2.95:1)
🚨 Coding-guideline violations 0

Test Classification Details

Test File Classification Issues Detected
TestValidateDockerImage_SkipsWhenDockerUnavailable pkg/workflow/docker_validation_graceful_test.go:18 ✅ Design Skips when Docker is available (see note below)
TestValidateDockerImage_StillRejectsHyphenWithoutDocker pkg/workflow/docker_validation_graceful_test.go:35 ✅ Design Verifies security invariant; covers error path
TestValidateContainerImages_NoWarningWithoutDocker pkg/workflow/docker_validation_graceful_test.go:45 ✅ Design Verifies higher-level graceful degradation

Notes

i️ TestValidateDockerImage_SkipsWhenDockerUnavailable

This test calls t.Skip() when Docker is available, meaning it only exercises assertions in environments where Docker is absent. This is a legitimate design choice for testing environment-specific degradation paths. However, in a typical CI environment where Docker is present, this test provides zero coverage and effectively becomes a no-op.

Consideration: The test correctly documents why it skips ("Docker is available — graceful degradation path not exercised"), which is good practice. No change needed, but be aware that on Docker-enabled runners this test never exercises the code.

i️ Test Inflation (minor)

65 lines were added to the test file vs. 22 lines added to the production file (ratio ≈ 2.95:1), triggering the inflation threshold. However, this is largely attributable to well-written documentation comments in the test and the fact that the production change was a targeted conditional insertion rather than a large feature addition. The test coverage quality is high despite the metric.


Language Support

Tests analyzed:

  • 🐹 Go (*_test.go): 3 tests — unit (//go:build !integration)
  • 🟨 JavaScript (*.test.cjs, *.test.js): 0 tests

Verdict

Check passed. 0% of new tests are implementation tests (threshold: 30%). All three tests verify behavioral contracts with proper assertion messages, error-path coverage, a correct build tag, and no prohibited mock libraries.


📖 Understanding Test Classifications

Design Tests (High Value) verify what the system does:

  • Assert on observable outputs, return values, or state changes
  • Cover error paths and boundary conditions
  • Would catch a behavioral regression if deleted
  • Remain valid even after internal refactoring

Implementation Tests (Low Value) verify how the system does it:

  • Assert on internal function calls (mocking internals)
  • Only test the happy path with typical inputs
  • Break during legitimate refactoring even when behavior is correct
  • Give false assurance: they pass even when the system is wrong

Goal: Shift toward tests that describe the system's behavioral contract — the promises it makes to its users and collaborators.

🧪 Test quality analysis by Test Quality Sentinel · ● 398.1K ·

Copy link
Copy Markdown
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Test Quality Sentinel: 90/100. Test quality is excellent — 0% of new tests are implementation tests (threshold: 30%). All 3 tests verify behavioral contracts with proper build tags, assertion messages, error-path coverage, and no prohibited mock libraries.

@pelikhan
Copy link
Copy Markdown
Collaborator

@copilot review proposal and comments.

Add a flag to ensure validate fails if docker is not available

Copilot AI and others added 2 commits April 13, 2026 20:28
…e validation

Adds a --validate-images flag to both `compile` and `validate` commands
that requires Docker to be available for container image validation.
Without this flag, validation silently skips when Docker is unavailable.

Also addresses review feedback:
- Fix build tag on test file to exclude js/wasm
- Fix header comment about verbose mode

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/740dc287-2d09-460c-bdf1-c9ba3ecd10ce

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 13, 2026

@copilot review proposal and comments.

Add a flag to ensure validate fails if docker is not available

Added --validate-images flag to both compile and validate commands (ae962f6). When specified, container image validation fails with an actionable error if Docker is not installed or the daemon is not running. Without the flag, validation silently skips (the new default).

Also addressed the reviewer comments: fixed the build tag for js/wasm and corrected the verbose comment.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/graphql
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE sh (http block)
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE node (http block)
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw GOMOD GOMODCACHE o fix.&#34;; \ exitGO111MODULE env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE node (http block)
  • https://api.github.com/repos/actions/upload-artifact/git/ref/tags/v7
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v7 --jq .object.sha GOSUMDB GOWORK 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE sh (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v7 --jq .object.sha GOSUMDB GOWORK 64/bin/go GOINSECURE %H %ct %D (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v7 --jq .object.sha 5233578/b424/_pkGOINSECURE GO111MODULE 64/bin/go GOINSECURE b/gh-aw/pkg/typeenv GOMODCACHE go env H8pU/eNnoa-rIndzGOINSECURE GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE 5233578/b424/impGOPROXY (http block)
  • https://api.github.com/repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b
    • Triggering command: /usr/bin/gh gh api /repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b --jq .object.sha gJ4l/xSWsXgQ5zegGOINSECURE GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE 5233578/b416/impGO111MODULE -c k/gh-aw/gh-aw/pkGOINSECURE k/gh-aw/gh-aw/pkGOMOD 64/bin/go GOSUMDB GOWORK 64/bin/go /opt/hostedtoolcshow (http block)
    • Triggering command: /usr/bin/gh gh api /repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b --jq .object.sha H8pU/eNnoa-rIndzGOINSECURE GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE 5233578/b424/impGOPROXY /hom�� k/gh-aw/gh-aw/pkGOSUMDB k/gh-aw/gh-aw/pkGOWORK 64/bin/go **/*.json --ignore-path ../../../.prettixterm-color /opt/hostedtoolcache/go/1.25.8/xGO111MODULE (http block)
  • https://api.github.com/repos/docker/build-push-action/git/ref/tags/v7
    • Triggering command: /usr/bin/gh gh api /repos/docker/build-push-action/git/ref/tags/v7 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/docker/build-push-action/git/ref/tags/v7 --jq .object.sha -json GO111MODULE r: $owner, name: $name) { hasDiscussionsEnabled } } GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/githubnext/agentics/git/ref/tags/-
    • Triggering command: /usr/bin/gh gh api /repos/githubnext/agentics/git/ref/tags/- --jq .object.sha &#34;prettier&#34; --cheGOSUMDB sh 64/bin/go tierignore (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI changed the title Skip Docker image validation when Docker is unavailable Skip Docker image validation when Docker is unavailable, add --validate-images flag Apr 13, 2026
@pelikhan
Copy link
Copy Markdown
Collaborator

@copilot add validate-images to the agentic maintenance generated validate job

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 13, 2026

@copilot add validate-images to the agentic maintenance generated validate job

Added --validate-images to the maintenance workflow's compile step in maintenance_workflow.go (645630b). The generated agentics-maintenance.yml now runs compile --validate --validate-images --verbose in the validate_workflows job.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/graphql
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw (http block)
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw &#34;$k=&#34;.$v[&#34;localrev-parse (http block)
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw &#34;$k=&#34;.$v[&#34;local--norc (http block)
  • https://api.github.com/repos/actions/upload-artifact/git/ref/tags/v7
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v7 --jq .object.sha (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v7 --jq .object.sha ypass.go ommon.go x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v7 --jq .object.sha /\1/p (http block)
  • https://api.github.com/repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b
    • Triggering command: /usr/bin/gh gh api /repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b --jq .object.sha diff.go export.go x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api /repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b --jq .object.sha (http block)
  • https://api.github.com/repos/docker/build-push-action/git/ref/tags/v7
    • Triggering command: /usr/bin/gh gh api /repos/docker/build-push-action/git/ref/tags/v7 --jq .object.sha (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v0.47.4
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v0.47.4 --jq .object.sha (http block)
  • https://api.github.com/repos/githubnext/agentics/git/ref/tags/-
    • Triggering command: /usr/bin/gh gh api /repos/githubnext/agentics/git/ref/tags/- --jq .object.sha (http block)

If you need me to access, download, or install something from one of these locations, you can either:

@pelikhan pelikhan merged commit 8ca5e8c into main Apr 13, 2026
@pelikhan pelikhan deleted the copilot/remove-docker-dependency-validate branch April 13, 2026 21:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove Docker dependency from gh aw compile --validate

3 participants