{2023.06}[foss/2023a] parallel v20230722 (also: test remove Lmod update cache) by trz42 · Pull Request #310 · NorESSI/software-layer

trz42 · 2024-04-07T15:39:39Z

This PR adds parallel v20230722. It also serves as a test for #309

SPDX license identifier: GPL-3.0-or-later

Missing installations:

1 out of 3 required modules missing:

* parallel/20230722-GCCcore-12.3.0 (parallel-20230722-GCCcore-12.3.0.eb)

We process this PR as follows:

We build parallel and verify if the tarballs do no longer include the Lmod cache updates.
If so, we merge remove Lmod cache update #309 first, and update this PR here.
Finally, we deploy the built tarballs and verify if the Lmod caches are updated on the Stratum 0 during the ingestion.

nessi-bot · 2024-04-07T15:39:43Z

Instance AWS-MC-NESSI is configured to build:

arch x86_64/generic for repo nessi-2022.11-swl-deb10
arch x86_64/generic for repo nessi-2023.06-cl
arch x86_64/generic for repo nessi-2023.06-swl-deb10
arch x86_64/generic for repo nessi-2023.06-swl-deb11
arch x86_64/intel/skylake_avx512 for repo nessi-2022.11-swl-deb10
arch x86_64/intel/skylake_avx512 for repo nessi-2023.06-cl
arch x86_64/intel/skylake_avx512 for repo nessi-2023.06-swl-deb10
arch x86_64/intel/skylake_avx512 for repo nessi-2023.06-swl-deb11
arch x86_64/amd/zen2 for repo nessi-2022.11-swl-deb10
arch x86_64/amd/zen2 for repo nessi-2023.06-cl
arch x86_64/amd/zen2 for repo nessi-2023.06-swl-deb10
arch x86_64/amd/zen2 for repo nessi-2023.06-swl-deb11
arch aarch64/generic for repo nessi-2022.11-swl-deb10
arch aarch64/generic for repo nessi-2023.06-cl
arch aarch64/generic for repo nessi-2023.06-swl-deb10
arch aarch64/generic for repo nessi-2023.06-swl-deb11

nessi-bot · 2024-04-07T15:39:43Z

Instance eX3-NESSI is configured to build:

arch x86_64/amd/zen2 for repo nessi-2023.06-cl
arch x86_64/amd/zen2 for repo nessi-2023.06-swl-deb10
arch x86_64/amd/zen2 for repo nessi-2023.06-swl-deb11
arch aarch64/generic for repo nessi-2023.06-cl
arch aarch64/generic for repo nessi-2023.06-swl-deb10
arch aarch64/generic for repo nessi-2023.06-swl-deb11

nessi-bot · 2024-04-07T15:39:43Z

Instance Fram-NESSI is configured to build:

arch x86_64/generic for repo nessi-2023.06-cl
arch x86_64/generic for repo nessi-2023.06-swl-deb10
arch x86_64/generic for repo nessi-2023.06-swl-deb11
arch x86_64/intel/broadwell for repo nessi-2023.06-cl
arch x86_64/intel/broadwell for repo nessi-2023.06-swl-deb10
arch x86_64/intel/broadwell for repo nessi-2023.06-swl-deb11

nessi-bot · 2024-04-07T15:39:43Z

Instance Saga-NESSI is configured to build:

arch x86_64/intel/skylake_avx512 for repo nessi-2023.06-cl
arch x86_64/intel/skylake_avx512 for repo nessi-2023.06-swl-deb10
arch x86_64/intel/skylake_avx512 for repo nessi-2023.06-swl-deb11
arch x86_64/intel/broadwell for repo nessi-2023.06-cl
arch x86_64/intel/broadwell for repo nessi-2023.06-swl-deb10
arch x86_64/intel/broadwell for repo nessi-2023.06-swl-deb11
arch x86_64/generic for repo nessi-2023.06-cl
arch x86_64/generic for repo nessi-2023.06-swl-deb10
arch x86_64/generic for repo nessi-2023.06-swl-deb11

trz42 · 2024-04-07T15:42:45Z

bot: build inst:Fram-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/broadwell
bot: build inst:Saga-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/skylake
bot: build inst:eX3-NESSI repo:nessi-2023.06-swl-deb11 arch:aarch64/generic
bot: build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:zen2
bot: build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/generic

nessi-bot · 2024-04-07T15:42:48Z

Updates by the bot instance AWS-MC-NESSI (click for details)

received bot command build inst:Fram-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/broadwell from trz42
- expanded format: build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell
received bot command build inst:Saga-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/skylake from trz42
- expanded format: build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake
received bot command build inst:eX3-NESSI repo:nessi-2023.06-swl-deb11 arch:aarch64/generic from trz42
- expanded format: build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:zen2 from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/generic from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic
handling command build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell resulted in:
- no jobs were submitted
handling command build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake resulted in:
- no jobs were submitted
handling command build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic resulted in:
- no jobs were submitted
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2 resulted in:
- submitted job 9177, for details & status see {2023.06}[foss/2023a] parallel v20230722 (also: test remove Lmod update cache) #310 (comment)
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic resulted in:
- submitted job 9178, for details & status see {2023.06}[foss/2023a] parallel v20230722 (also: test remove Lmod update cache) #310 (comment)

nessi-bot · 2024-04-07T15:42:48Z

Updates by the bot instance Saga-NESSI (click for details)

received bot command build inst:Fram-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/broadwell from trz42
- expanded format: build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell
received bot command build inst:Saga-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/skylake from trz42
- expanded format: build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake
received bot command build inst:eX3-NESSI repo:nessi-2023.06-swl-deb11 arch:aarch64/generic from trz42
- expanded format: build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:zen2 from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/generic from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic
handling command build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell resulted in:
- no jobs were submitted
handling command build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake resulted in:
- submitted job 11097101, for details & status see {2023.06}[foss/2023a] parallel v20230722 (also: test remove Lmod update cache) #310 (comment)
handling command build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic resulted in:
- no jobs were submitted
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2 resulted in:
- no jobs were submitted
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic resulted in:
- no jobs were submitted

nessi-bot · 2024-04-07T15:42:48Z

Updates by the bot instance eX3-NESSI (click for details)

received bot command build inst:Fram-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/broadwell from trz42
- expanded format: build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell
received bot command build inst:Saga-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/skylake from trz42
- expanded format: build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake
received bot command build inst:eX3-NESSI repo:nessi-2023.06-swl-deb11 arch:aarch64/generic from trz42
- expanded format: build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:zen2 from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/generic from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic
handling command build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell resulted in:
- no jobs were submitted
handling command build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake resulted in:
- no jobs were submitted
handling command build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic resulted in:
- submitted job 163957, for details & status see {2023.06}[foss/2023a] parallel v20230722 (also: test remove Lmod update cache) #310 (comment)
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2 resulted in:
- no jobs were submitted
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic resulted in:
- no jobs were submitted

nessi-bot · 2024-04-07T15:42:48Z

Updates by the bot instance Fram-NESSI (click for details)

received bot command build inst:Fram-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/broadwell from trz42
- expanded format: build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell
received bot command build inst:Saga-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/intel/skylake from trz42
- expanded format: build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake
received bot command build inst:eX3-NESSI repo:nessi-2023.06-swl-deb11 arch:aarch64/generic from trz42
- expanded format: build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:zen2 from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2
received bot command build inst:AWS-MC-NESSI repo:nessi-2023.06-swl-deb11 arch:x86_64/generic from trz42
- expanded format: build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic
handling command build instance:Fram-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/broadwell resulted in:
- submitted job 5762232, for details & status see {2023.06}[foss/2023a] parallel v20230722 (also: test remove Lmod update cache) #310 (comment)
handling command build instance:Saga-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/intel/skylake resulted in:
- no jobs were submitted
handling command build instance:eX3-NESSI repository:nessi-2023.06-swl-deb11 architecture:aarch64/generic resulted in:
- no jobs were submitted
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:zen2 resulted in:
- no jobs were submitted
handling command build instance:AWS-MC-NESSI repository:nessi-2023.06-swl-deb11 architecture:x86_64/generic resulted in:
- no jobs were submitted

nessi-bot · 2024-04-07T15:42:52Z

New job on instance Fram-NESSI for architecture x86_64-intel-broadwell for repository nessi-2023.06-swl-deb11 in job dir /cluster/projects/nn9992k/pilot.nessi.no/jobs/2024.04/pr_310/5762232

date	job status	comment
Apr 07 15:42:52 UTC 2024	submitted	job id `5762232` awaits release by job manager
Apr 07 15:43:35 UTC 2024	released	job awaits launch by Slurm scheduler
Apr 07 15:44:37 UTC 2024	running	job `5762232` is running
Apr 07 15:47:40 UTC 2024	finished	😁 SUCCESS (click triangle for details) Details ✅ job output file `slurm-5762232.out` ✅ no message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ✅ found message(s) matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts `eessi-2023.06-software-linux-x86_64-intel-broadwell-1712504838.tar.gz` size: 2 MiB (2344154 bytes) entries: 114 modules under 2023.06/software/linux/x86_64/intel/broadwell/modules/all `parallel/20230722-GCCcore-12.3.0.lua` software under 2023.06/software/linux/x86_64/intel/broadwell/software `parallel/20230722-GCCcore-12.3.0` other under 2023.06/software/linux/x86_64/intel/broadwell no other files in tarball
Apr 07 15:47:40 UTC 2024	test result	(no tests yet)
Apr 07 17:47:00 UTC 2024	uploaded	transfer of `eessi-2023.06-software-linux-x86_64-intel-broadwell-1712504838.tar.gz` to S3 bucket succeeded
Apr 07 05:57:17 PM UTC 2024	staged	tarball `eessi-2023.06-software-linux-x86_64-intel-broadwell-1712504838.tar.gz` downloaded to Stratum-0
Apr 07 05:58:30 PM UTC 2024	pr_opened	merge PR https://github.com/NorESSI/2024-staging/pull/112 to approve ingest
Apr 07 07:04:02 PM UTC 2024	approved	👍 tarball `eessi-2023.06-software-linux-x86_64-intel-broadwell-1712504838.tar.gz` approved, see PR https://github.com/NorESSI/2024-staging/pull/112
Apr 07 07:06:23 PM UTC 2024	ingested	🎉 tarball `eessi-2023.06-software-linux-x86_64-intel-broadwell-1712504838.tar.gz` successfully ingested at `2023.06/software/linux/x86_64/intel/broadwell/`

nessi-bot · 2024-04-07T15:42:53Z

New job on instance eX3-NESSI for architecture aarch64-generic for repository nessi-2023.06-swl-deb11 in job dir /home/thomarob/pilot.nessi.no/jobs/2024.04/pr_310/163957

date	job status	comment
Apr 07 03:42:52 PM UTC 2024	submitted	job id `163957` awaits release by job manager
Apr 07 03:43:25 PM UTC 2024	released	job awaits launch by Slurm scheduler
Apr 07 03:44:27 PM UTC 2024	running	job `163957` is running
Apr 07 03:51:35 PM UTC 2024	finished	😁 SUCCESS (click triangle for details) Details ✅ job output file `slurm-163957.out` ✅ no message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ✅ found message(s) matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts `eessi-2023.06-software-linux-aarch64-generic-1712505033.tar.gz` size: 2 MiB (2344787 bytes) entries: 114 modules under 2023.06/software/linux/aarch64/generic/modules/all `parallel/20230722-GCCcore-12.3.0.lua` software under 2023.06/software/linux/aarch64/generic/software `parallel/20230722-GCCcore-12.3.0` other under 2023.06/software/linux/aarch64/generic no other files in tarball
Apr 07 03:51:35 PM UTC 2024	test result	(no tests yet)
Apr 07 05:48:26 PM UTC 2024	uploaded	transfer of `eessi-2023.06-software-linux-aarch64-generic-1712505033.tar.gz` to S3 bucket succeeded
Apr 07 05:57:11 PM UTC 2024	staged	tarball `eessi-2023.06-software-linux-aarch64-generic-1712505033.tar.gz` downloaded to Stratum-0
Apr 07 05:58:21 PM UTC 2024	pr_opened	merge PR https://github.com/NorESSI/2024-staging/pull/109 to approve ingest
Apr 07 06:03:32 PM UTC 2024	approved	👍 tarball `eessi-2023.06-software-linux-aarch64-generic-1712505033.tar.gz` approved, see PR https://github.com/NorESSI/2024-staging/pull/109
Apr 07 06:13:10 PM UTC 2024	ingested	🎉 tarball `eessi-2023.06-software-linux-aarch64-generic-1712505033.tar.gz` successfully ingested at `2023.06/software/linux/aarch64/generic/`

nessi-bot · 2024-04-07T15:42:53Z

New job on instance AWS-MC-NESSI for architecture x86_64-amd-zen2 for repository nessi-2023.06-swl-deb11 in job dir /project/def-nessi/SHARED/jobs/2024.04/pr_310/9177

date	job status	comment
Apr 07 15:42:53 UTC 2024	submitted	job id `9177` awaits release by job manager
Apr 07 15:43:23 UTC 2024	released	job awaits launch by Slurm scheduler
Apr 07 15:48:27 UTC 2024	running	job `9177` is running
Apr 07 15:56:42 UTC 2024	finished	😁 SUCCESS (click triangle for details) Details ✅ job output file `slurm-9177.out` ✅ no message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ✅ found message(s) matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts `eessi-2023.06-software-linux-x86_64-amd-zen2-1712505339.tar.gz` size: 2 MiB (2344415 bytes) entries: 114 modules under 2023.06/software/linux/x86_64/amd/zen2/modules/all `parallel/20230722-GCCcore-12.3.0.lua` software under 2023.06/software/linux/x86_64/amd/zen2/software `parallel/20230722-GCCcore-12.3.0` other under 2023.06/software/linux/x86_64/amd/zen2 no other files in tarball
Apr 07 15:56:42 UTC 2024	test result	(no tests yet)
Apr 07 17:46:58 UTC 2024	uploaded	transfer of `eessi-2023.06-software-linux-x86_64-amd-zen2-1712505339.tar.gz` to S3 bucket succeeded
Apr 07 05:57:13 PM UTC 2024	staged	tarball `eessi-2023.06-software-linux-x86_64-amd-zen2-1712505339.tar.gz` downloaded to Stratum-0
Apr 07 05:58:24 PM UTC 2024	pr_opened	merge PR https://github.com/NorESSI/2024-staging/pull/110 to approve ingest
Apr 07 07:03:59 PM UTC 2024	approved	👍 tarball `eessi-2023.06-software-linux-x86_64-amd-zen2-1712505339.tar.gz` approved, see PR https://github.com/NorESSI/2024-staging/pull/110
Apr 07 07:05:27 PM UTC 2024	ingested	🎉 tarball `eessi-2023.06-software-linux-x86_64-amd-zen2-1712505339.tar.gz` successfully ingested at `2023.06/software/linux/x86_64/amd/zen2/`

nessi-bot · 2024-04-07T15:42:57Z

New job on instance AWS-MC-NESSI for architecture x86_64-generic for repository nessi-2023.06-swl-deb11 in job dir /project/def-nessi/SHARED/jobs/2024.04/pr_310/9178

date	job status	comment
Apr 07 15:42:56 UTC 2024	submitted	job id `9178` awaits release by job manager
Apr 07 15:43:25 UTC 2024	released	job awaits launch by Slurm scheduler
Apr 07 15:48:29 UTC 2024	running	job `9178` is running
Apr 07 15:56:44 UTC 2024	finished	😁 SUCCESS (click triangle for details) Details ✅ job output file `slurm-9178.out` ✅ no message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ✅ found message(s) matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts `eessi-2023.06-software-linux-x86_64-generic-1712505341.tar.gz` size: 2 MiB (2345688 bytes) entries: 114 modules under 2023.06/software/linux/x86_64/generic/modules/all `parallel/20230722-GCCcore-12.3.0.lua` software under 2023.06/software/linux/x86_64/generic/software `parallel/20230722-GCCcore-12.3.0` other under 2023.06/software/linux/x86_64/generic no other files in tarball
Apr 07 15:56:44 UTC 2024	test result	(no tests yet)
Apr 07 17:47:18 UTC 2024	uploaded	transfer of `eessi-2023.06-software-linux-x86_64-generic-1712505341.tar.gz` to S3 bucket succeeded
Apr 07 05:57:15 PM UTC 2024	staged	tarball `eessi-2023.06-software-linux-x86_64-generic-1712505341.tar.gz` downloaded to Stratum-0
Apr 07 05:58:27 PM UTC 2024	pr_opened	merge PR https://github.com/NorESSI/2024-staging/pull/111 to approve ingest
Apr 07 06:46:28 PM UTC 2024	approved	👍 tarball `eessi-2023.06-software-linux-x86_64-generic-1712505341.tar.gz` approved, see PR https://github.com/NorESSI/2024-staging/pull/111
Apr 07 06:47:55 PM UTC 2024	ingested	🎉 tarball `eessi-2023.06-software-linux-x86_64-generic-1712505341.tar.gz` successfully ingested at `2023.06/software/linux/x86_64/generic/`

nessi-bot · 2024-04-07T15:43:03Z

New job on instance Saga-NESSI for architecture x86_64-intel-skylake_avx512 for repository nessi-2023.06-swl-deb11 in job dir /cluster/projects/nn9992k/pilot.nessi.no/jobs/2024.04/pr_310/11097101

date	job status	comment
Apr 07 03:43:02 PM UTC 2024	submitted	job id `11097101` awaits release by job manager
Apr 07 03:43:44 PM UTC 2024	released	job awaits launch by Slurm scheduler
Apr 07 03:44:46 PM UTC 2024	running	job `11097101` is running
Apr 07 03:51:54 PM UTC 2024	finished	😁 SUCCESS (click triangle for details) Details ✅ job output file `slurm-11097101.out` ✅ no message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ✅ found message(s) matching `No missing installations` ✅ found message matching `.tar.gz created!` Artefacts `eessi-2023.06-software-linux-x86_64-intel-skylake_avx512-1712505054.tar.gz` size: 2 MiB (2346064 bytes) entries: 114 modules under 2023.06/software/linux/x86_64/intel/skylake_avx512/modules/all `parallel/20230722-GCCcore-12.3.0.lua` software under 2023.06/software/linux/x86_64/intel/skylake_avx512/software `parallel/20230722-GCCcore-12.3.0` other under 2023.06/software/linux/x86_64/intel/skylake_avx512 no other files in tarball
Apr 07 03:51:54 PM UTC 2024	test result	(no tests yet)
Apr 07 05:47:04 PM UTC 2024	uploaded	transfer of `eessi-2023.06-software-linux-x86_64-intel-skylake_avx512-1712505054.tar.gz` to S3 bucket succeeded
Apr 07 05:57:19 PM UTC 2024	staged	tarball `eessi-2023.06-software-linux-x86_64-intel-skylake_avx512-1712505054.tar.gz` downloaded to Stratum-0
Apr 07 05:58:33 PM UTC 2024	pr_opened	merge PR https://github.com/NorESSI/2024-staging/pull/113 to approve ingest
Apr 07 07:04:04 PM UTC 2024	approved	👍 tarball `eessi-2023.06-software-linux-x86_64-intel-skylake_avx512-1712505054.tar.gz` approved, see PR https://github.com/NorESSI/2024-staging/pull/113
Apr 07 07:06:54 PM UTC 2024	ingested	🎉 tarball `eessi-2023.06-software-linux-x86_64-intel-skylake_avx512-1712505054.tar.gz` successfully ingested at `2023.06/software/linux/x86_64/intel/skylake_avx512/`

…into test_remove_lmod_update_cache

trz42 · 2024-04-07T17:32:46Z

Checklist before starting deployment (setting bot:deploy label):

Check if the SPDX license identifier is provided
Check whether builds for all required architectures succeed (SUCCESS message + reasonably sized tarball)
Check if the PR is up-to-date with the target branch nessi.no-2023.06 in the repository (if not what are the differences)
Assess if all requested changes are sound (checking files changed on GitHub.com)
Verify that all easyconfig/s being built are included with the EB version used (if not why not)
Review changes (if any) needed to get the build(s) succeed (common changes for all architectures, changes for a single architecture, changes because of build environment specifics, etc.)

trz42 · 2024-04-07T17:52:11Z

Target architectures

x86_64-intel-broadwell
x86_64-intel-skylake_avx512
x86_64-amd-zen2
x86_64-generic
aarch64-generic

Note! We want to carefully deploy this step-by-step (tarball-by-tarball) to verify if the changed procedure on the Stratum-0 works as designed (ingesting correctly and creating an updated Lmod cache).

Checklist for deployment/ingestion

at least one tarball for each architecture has been uploaded
at least one tarball for each architecture has been staged
for each staged tarball was a PR for approval/rejection opened
for each architecture one and only one tarball has been approved
for each architecture one and only one tarball was ingested
all tests in the CI have succeeded (rerun failed tests ~ 10 mins after last ingest)
has the package become available via CernVM-FS for all architectures ingested (for lmod cache updates pay attention to timestamp of files)

command & log

command

cd /cvmfs/pilot.nessi.no/versions/2023.06/software/linux ; \
ls -l \
      x86_64/{generic,amd/zen2,intel/[bsc]*}/{software,modules/all}/parallel/* \
      aarch64/generic/{software,modules/all}/parallel/* \
      x86_64/{generic,amd/zen2,intel/[bsc]*}/.lmod/cache/* \
      aarch64/generic/.lmod/cache/*

log after ingest of aarch64/generic

ls: cannot access 'x86_64/generic/software/parallel/*': No such file or directory
ls: cannot access 'x86_64/generic/modules/all/parallel/*': No such file or directory
ls: cannot access 'x86_64/amd/zen2/software/parallel/*': No such file or directory
ls: cannot access 'x86_64/amd/zen2/modules/all/parallel/*': No such file or directory
ls: cannot access 'x86_64/intel/[bsc]*/software/parallel/*': No such file or directory
ls: cannot access 'x86_64/intel/[bsc]*/modules/all/parallel/*': No such file or directory
-rw-r--r-- 1 cvmfs cvmfs 1373592 Apr  7 20:11  aarch64/generic/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  917369 Apr  7 20:11  aarch64/generic/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs 1382346 Apr  7 20:11  aarch64/generic/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  916127 Apr  7 20:11  aarch64/generic/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 20:11  aarch64/generic/.lmod/cache/timestamp
-rw-r--r-- 1 cvmfs cvmfs    1140 Apr  7 17:46  aarch64/generic/modules/all/parallel/20230722-GCCcore-12.3.0.lua
-rw-r--r-- 1 cvmfs cvmfs 1433260 Apr  7 20:11  x86_64/amd/zen2/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  953119 Apr  7 20:11  x86_64/amd/zen2/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs 1443712 Apr  7 20:11  x86_64/amd/zen2/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  953119 Apr  7 20:11  x86_64/amd/zen2/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 20:11  x86_64/amd/zen2/.lmod/cache/timestamp
-rw-r--r-- 1 cvmfs cvmfs 1430887 Apr  7 20:12  x86_64/generic/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  951478 Apr  7 20:12  x86_64/generic/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs 1441339 Apr  7 20:12  x86_64/generic/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  951478 Apr  7 20:12  x86_64/generic/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 20:11  x86_64/generic/.lmod/cache/timestamp
-rw-r--r-- 1 cvmfs cvmfs 1449871 Apr  7 20:12  x86_64/intel/broadwell/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  964606 Apr  7 20:12  x86_64/intel/broadwell/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs 1460323 Apr  7 20:12  x86_64/intel/broadwell/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  964606 Apr  7 20:12  x86_64/intel/broadwell/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 20:12  x86_64/intel/broadwell/.lmod/cache/timestamp
-rw-r--r-- 1 cvmfs cvmfs  679781 Apr  7 20:12  x86_64/intel/cascadelake/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  483261 Apr  7 20:12  x86_64/intel/cascadelake/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs  694693 Apr  7 20:12  x86_64/intel/cascadelake/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  490787 Apr  7 20:12  x86_64/intel/cascadelake/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 20:12  x86_64/intel/cascadelake/.lmod/cache/timestamp
-rw-r--r-- 1 cvmfs cvmfs 1461736 Apr  7 20:12  x86_64/intel/skylake_avx512/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  972811 Apr  7 20:12  x86_64/intel/skylake_avx512/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs 1472188 Apr  7 20:12  x86_64/intel/skylake_avx512/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  972811 Apr  7 20:12  x86_64/intel/skylake_avx512/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 20:12  x86_64/intel/skylake_avx512/.lmod/cache/timestamp

aarch64/generic/software/parallel/20230722-GCCcore-12.3.0:
total 14
dr-xr-xr-x 2 cvmfs cvmfs 4096 Apr  7 17:46 bin
dr-xr-xr-x 3 cvmfs cvmfs 4096 Apr  7 17:46 easybuild
dr-xr-xr-x 6 cvmfs cvmfs 4096 Apr  7 17:46 share

log after ingest of x86_64/generic

ls: cannot access 'x86_64/amd/zen2/software/parallel/*': No such file or directory
ls: cannot access 'x86_64/amd/zen2/modules/all/parallel/*': No such file or directory
ls: cannot access 'x86_64/intel/[bsc]*/software/parallel/*': No such file or directory
ls: cannot access 'x86_64/intel/[bsc]*/modules/all/parallel/*': No such file or directory
-rw-r--r-- 1 cvmfs cvmfs 1373592 Apr  7 20:47  aarch64/generic/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  917369 Apr  7 20:47  aarch64/generic/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs 1373592 Apr  7 20:47  aarch64/generic/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  917369 Apr  7 20:47  aarch64/generic/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 20:47  aarch64/generic/.lmod/cache/timestamp
-rw-r--r-- 1 cvmfs cvmfs    1140 Apr  7 17:46  aarch64/generic/modules/all/parallel/20230722-GCCcore-12.3.0.lua
-rw-r--r-- 1 cvmfs cvmfs 1433260 Apr  7 20:47  x86_64/amd/zen2/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  953119 Apr  7 20:47  x86_64/amd/zen2/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs 1433260 Apr  7 20:47  x86_64/amd/zen2/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  953119 Apr  7 20:47  x86_64/amd/zen2/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 20:47  x86_64/amd/zen2/.lmod/cache/timestamp
-rw-r--r-- 1 cvmfs cvmfs 1432309 Apr  7 20:47  x86_64/generic/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  952718 Apr  7 20:47  x86_64/generic/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs 1430887 Apr  7 20:47  x86_64/generic/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  951478 Apr  7 20:47  x86_64/generic/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 20:47  x86_64/generic/.lmod/cache/timestamp
-rw-rw-r-- 1 cvmfs cvmfs    1139 Apr  7 17:53  x86_64/generic/modules/all/parallel/20230722-GCCcore-12.3.0.lua
-rw-r--r-- 1 cvmfs cvmfs 1449871 Apr  7 20:47  x86_64/intel/broadwell/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  964606 Apr  7 20:47  x86_64/intel/broadwell/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs 1449871 Apr  7 20:47  x86_64/intel/broadwell/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  964606 Apr  7 20:47  x86_64/intel/broadwell/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 20:47  x86_64/intel/broadwell/.lmod/cache/timestamp
-rw-r--r-- 1 cvmfs cvmfs  679781 Apr  7 20:47  x86_64/intel/cascadelake/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  483261 Apr  7 20:47  x86_64/intel/cascadelake/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs  679781 Apr  7 20:47  x86_64/intel/cascadelake/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  483261 Apr  7 20:47  x86_64/intel/cascadelake/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 20:47  x86_64/intel/cascadelake/.lmod/cache/timestamp
-rw-r--r-- 1 cvmfs cvmfs 1461736 Apr  7 20:47  x86_64/intel/skylake_avx512/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  972811 Apr  7 20:47  x86_64/intel/skylake_avx512/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs 1461736 Apr  7 20:47  x86_64/intel/skylake_avx512/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  972811 Apr  7 20:47  x86_64/intel/skylake_avx512/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 20:47  x86_64/intel/skylake_avx512/.lmod/cache/timestamp

aarch64/generic/software/parallel/20230722-GCCcore-12.3.0:
total 14
dr-xr-xr-x 2 cvmfs cvmfs 4096 Apr  7 17:46 bin
dr-xr-xr-x 3 cvmfs cvmfs 4096 Apr  7 17:46 easybuild
dr-xr-xr-x 6 cvmfs cvmfs 4096 Apr  7 17:46 share

x86_64/generic/software/parallel/20230722-GCCcore-12.3.0:
total 14
dr-xr-xr-x 2 cvmfs cvmfs 4096 Apr  7 17:53 bin
dr-xr-xr-x 3 cvmfs cvmfs 4096 Apr  7 17:53 easybuild
dr-xr-xr-x 6 cvmfs cvmfs 4096 Apr  7 17:53 share

log after ingest of x86_64/{amd/zen2,intel/broadwell,intel/skylake_avx512}

-rw-r--r-- 1 cvmfs cvmfs 1373592 Apr  7 21:06 aarch64/generic/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  917369 Apr  7 21:06 aarch64/generic/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs 1373592 Apr  7 21:06 aarch64/generic/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  917369 Apr  7 21:06 aarch64/generic/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 21:06 aarch64/generic/.lmod/cache/timestamp
-rw-r--r-- 1 cvmfs cvmfs    1140 Apr  7 17:46 aarch64/generic/modules/all/parallel/20230722-GCCcore-12.3.0.lua
-rw-r--r-- 1 cvmfs cvmfs 1434685 Apr  7 21:06 x86_64/amd/zen2/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  954361 Apr  7 21:06 x86_64/amd/zen2/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs 1434685 Apr  7 21:06 x86_64/amd/zen2/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  954361 Apr  7 21:06 x86_64/amd/zen2/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 21:06 x86_64/amd/zen2/.lmod/cache/timestamp
-rw-rw-r-- 1 cvmfs cvmfs    1140 Apr  7 17:53 x86_64/amd/zen2/modules/all/parallel/20230722-GCCcore-12.3.0.lua
-rw-r--r-- 1 cvmfs cvmfs 1432309 Apr  7 21:06 x86_64/generic/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  952718 Apr  7 21:06 x86_64/generic/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs 1432309 Apr  7 21:06 x86_64/generic/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  952718 Apr  7 21:06 x86_64/generic/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 21:06 x86_64/generic/.lmod/cache/timestamp
-rw-rw-r-- 1 cvmfs cvmfs    1139 Apr  7 17:53 x86_64/generic/modules/all/parallel/20230722-GCCcore-12.3.0.lua
-rw-r--r-- 1 cvmfs cvmfs 1451317 Apr  7 21:06 x86_64/intel/broadwell/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  965862 Apr  7 21:06 x86_64/intel/broadwell/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs 1451317 Apr  7 21:06 x86_64/intel/broadwell/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  965862 Apr  7 21:06 x86_64/intel/broadwell/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 21:06 x86_64/intel/broadwell/.lmod/cache/timestamp
-rw-rw-r-- 1 cvmfs cvmfs    1147 Apr  7 17:45 x86_64/intel/broadwell/modules/all/parallel/20230722-GCCcore-12.3.0.lua
-rw-r--r-- 1 cvmfs cvmfs  679781 Apr  7 21:06 x86_64/intel/cascadelake/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  483261 Apr  7 21:06 x86_64/intel/cascadelake/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs  679781 Apr  7 21:06 x86_64/intel/cascadelake/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  483261 Apr  7 21:06 x86_64/intel/cascadelake/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 21:06 x86_64/intel/cascadelake/.lmod/cache/timestamp
-rw-r--r-- 1 cvmfs cvmfs 1463197 Apr  7 21:06 x86_64/intel/skylake_avx512/.lmod/cache/spiderT.lua
-rw-r--r-- 1 cvmfs cvmfs  974077 Apr  7 21:06 x86_64/intel/skylake_avx512/.lmod/cache/spiderT.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs 1461736 Apr  7 21:06 x86_64/intel/skylake_avx512/.lmod/cache/spiderT.old.lua
-rw-r--r-- 1 cvmfs cvmfs  972811 Apr  7 21:06 x86_64/intel/skylake_avx512/.lmod/cache/spiderT.old.luac_5.1
-rw-r--r-- 1 cvmfs cvmfs       0 Apr  7 21:06 x86_64/intel/skylake_avx512/.lmod/cache/timestamp
-rw-rw-r-- 1 cvmfs cvmfs    1152 Apr  7 17:47 x86_64/intel/skylake_avx512/modules/all/parallel/20230722-GCCcore-12.3.0.lua

aarch64/generic/software/parallel/20230722-GCCcore-12.3.0:
total 14
dr-xr-xr-x 2 cvmfs cvmfs 4096 Apr  7 17:46 bin
dr-xr-xr-x 3 cvmfs cvmfs 4096 Apr  7 17:46 easybuild
dr-xr-xr-x 6 cvmfs cvmfs 4096 Apr  7 17:46 share

x86_64/amd/zen2/software/parallel/20230722-GCCcore-12.3.0:
total 14
dr-xr-xr-x 2 cvmfs cvmfs 4096 Apr  7 17:53 bin
dr-xr-xr-x 3 cvmfs cvmfs 4096 Apr  7 17:53 easybuild
dr-xr-xr-x 6 cvmfs cvmfs 4096 Apr  7 17:53 share

x86_64/generic/software/parallel/20230722-GCCcore-12.3.0:
total 14
dr-xr-xr-x 2 cvmfs cvmfs 4096 Apr  7 17:53 bin
dr-xr-xr-x 3 cvmfs cvmfs 4096 Apr  7 17:53 easybuild
dr-xr-xr-x 6 cvmfs cvmfs 4096 Apr  7 17:53 share

x86_64/intel/broadwell/software/parallel/20230722-GCCcore-12.3.0:
total 14
dr-xr-xr-x 2 cvmfs cvmfs 4096 Apr  7 17:45 bin
dr-xr-xr-x 3 cvmfs cvmfs 4096 Apr  7 17:45 easybuild
dr-xr-xr-x 6 cvmfs cvmfs 4096 Apr  7 17:45 share

x86_64/intel/skylake_avx512/software/parallel/20230722-GCCcore-12.3.0:
total 14
dr-xr-xr-x 2 cvmfs cvmfs 4096 Apr  7 17:47 bin
dr-xr-xr-x 3 cvmfs cvmfs 4096 Apr  7 17:47 easybuild
dr-xr-xr-x 6 cvmfs cvmfs 4096 Apr  7 17:47 share

poksumdo

All works smoothly -- no Lmod cache updates when building software, but during ingestion at Stratum 0. 🎉

{2023.06}[foss/2023a] parallel v20230722

fca385c

trz42 added NESSI/2023.06 debugging-testing sync-with-EESSI labels Apr 7, 2024

trz42 mentioned this pull request Apr 7, 2024

remove Lmod cache update #309

Merged

trz42 mentioned this pull request Apr 7, 2024

Syncing code-base and CVMFS repository with EESSI (April 2024) #294

Open

51 tasks

Merge branch 'nessi.no-2023.06' of github-trz:NorESSI/software-layer …

0c16970

…into test_remove_lmod_update_cache

trz42 added the bot:deploy label Apr 7, 2024

poksumdo approved these changes Apr 7, 2024

View reviewed changes

poksumdo merged commit deed9bb into NorESSI:nessi.no-2023.06 Apr 7, 2024

Conversation

trz42 commented Apr 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nessi-bot bot commented Apr 7, 2024

Uh oh!

nessi-bot bot commented Apr 7, 2024

Uh oh!

nessi-bot bot commented Apr 7, 2024

Uh oh!

nessi-bot bot commented Apr 7, 2024

Uh oh!

trz42 commented Apr 7, 2024

Uh oh!

nessi-bot bot commented Apr 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nessi-bot bot commented Apr 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nessi-bot bot commented Apr 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nessi-bot bot commented Apr 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nessi-bot bot commented Apr 7, 2024 • edited by trz42 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nessi-bot bot commented Apr 7, 2024 • edited by trz42 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nessi-bot bot commented Apr 7, 2024 • edited by trz42 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nessi-bot bot commented Apr 7, 2024 • edited by trz42 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nessi-bot bot commented Apr 7, 2024 • edited by trz42 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

trz42 commented Apr 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

trz42 commented Apr 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

poksumdo left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Comments

trz42 commented Apr 7, 2024 •

edited

Loading

nessi-bot bot commented Apr 7, 2024 •

edited

Loading

nessi-bot bot commented Apr 7, 2024 •

edited

Loading

nessi-bot bot commented Apr 7, 2024 •

edited

Loading

nessi-bot bot commented Apr 7, 2024 •

edited

Loading

nessi-bot bot commented Apr 7, 2024 •

edited by trz42

Loading

nessi-bot bot commented Apr 7, 2024 •

edited by trz42

Loading

nessi-bot bot commented Apr 7, 2024 •

edited by trz42

Loading

nessi-bot bot commented Apr 7, 2024 •

edited by trz42

Loading

nessi-bot bot commented Apr 7, 2024 •

edited by trz42

Loading

trz42 commented Apr 7, 2024 •

edited

Loading

trz42 commented Apr 7, 2024 •

edited

Loading