Skip to content

Cleanup disk space in image cache push jobs#56198

Merged
amoghrajesh merged 1 commit intoapache:mainfrom
amoghrajesh:enhance-free-disk-space
Sep 29, 2025
Merged

Cleanup disk space in image cache push jobs#56198
amoghrajesh merged 1 commit intoapache:mainfrom
amoghrajesh:enhance-free-disk-space

Conversation

@amoghrajesh
Copy link
Copy Markdown
Contributor

In some recent jobs, we saw some issues such related to disk space: https://github.com/apache/airflow/actions/runs/18071055239/job/51422086979

  #59 loading layer 2f7fcd295b18 61.07MB / 61.07MB 40.3s done
  #59 ERROR: write /usr/python/lib/python3.10/site-packages/scipy/linalg/cython_lapack.cpython-310-x86_64-linux-gnu.so: no space left on device
  
  #56 exporting to docker image format
  #56 sending tarball 74.3s done
  #56 ERROR: rpc error: code = Unknown desc = write /usr/python/lib/python3.10/site-packages/scipy/linalg/cython_lapack.cpython-310-x86_64-linux-gnu.so: no space left on device
  ------
   > exporting to docker image format:
  ------
  ------
   > importing to docker:
  ------
  ERROR: failed to build: failed to solve: rpc error: code = Unknown desc = write /usr/python/lib/python3.10/site-packages/scipy/linalg/cython_lapack.cpython-310-x86_64-linux-gnu.so: no space left on device
Your image build failed. It could be caused by conflicting dependencies.

As a potential improvement, we could reuse of disk cleanup job in these CI workflows too, right after checkout to clean some clutter up.

I do not see side effects because the cleanup step:

  • Removes large unused toolchains (.NET, GraalVM, Android SDK, etc.)
  • Cleans package caches (APT, pip, npm)

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@amoghrajesh amoghrajesh merged commit 1024b65 into apache:main Sep 29, 2025
109 checks passed
github-actions bot pushed a commit that referenced this pull request Sep 29, 2025
(cherry picked from commit 1024b65)

Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
@github-actions
Copy link
Copy Markdown
Contributor

Backport successfully created: v3-1-test

Status Branch Result
v3-1-test PR Link

github-actions bot pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Sep 29, 2025
(cherry picked from commit 1024b65)

Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
potiuk pushed a commit that referenced this pull request Oct 3, 2025
)

(cherry picked from commit 1024b65)

Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
dabla pushed a commit to dabla/airflow that referenced this pull request Oct 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants