Migrate from `yarn` to `pnpm` and drop `lerna` in the process by tdeekens · Pull Request #1931 · commercetools/nodejs

tdeekens · 2025-03-07T09:53:29Z

Summary

This migrates from yarn to pnpm which we adopted across repositories. It's speedier for installs are more strict about dependency resolution.

At the same time we can drop lerna in the process and replace it with manypkg which enforces dependency rules across the repository.

changeset-bot · 2025-03-07T09:53:33Z

🦋 Changeset detected

Latest commit: 8f7de67

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 33 packages

Name	Type
@commercetools/sdk-middleware-correlation-id	Major
@commercetools/sdk-middleware-user-agent	Major
@commercetools/csv-parser-discount-code	Major
@commercetools/customer-groups-exporter	Major
@commercetools/custom-objects-exporter	Major
@commercetools/custom-objects-importer	Major
@commercetools/discount-code-generator	Major
@commercetools/discount-code-exporter	Major
@commercetools/discount-code-importer	Major
@commercetools/personal-data-erasure	Major
@commercetools/sdk-middleware-logger	Major
@commercetools/inventories-exporter	Major
@commercetools/product-json-to-xlsx	Major
@commercetools/sdk-middleware-queue	Major
@commercetools/api-request-builder	Major
@commercetools/product-json-to-csv	Major
@commercetools/sdk-middleware-auth	Major
@commercetools/sdk-middleware-http	Major
@commercetools/category-exporter	Major
@commercetools/csv-parser-orders	Major
@commercetools/csv-parser-price	Major
@commercetools/csv-parser-state	Major
@commercetools/product-exporter	Major
@commercetools/resource-deleter	Major
@commercetools/get-credentials	Major
@commercetools/http-user-agent	Major
@commercetools/price-exporter	Major
@commercetools/state-importer	Major
@commercetools/sync-actions	Major
@commercetools/sdk-client	Major
@commercetools/sdk-types	Major
@commercetools/sdk-auth	Major
@commercetools/integration-tests	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

tdeekens · 2025-03-07T10:54:57Z

.circleci/config.yml

-  - &restore_yarn_cache_node_18
-    keys:
-      - v1-yarn-cache-node_18-{{ .Branch }}-{{ checksum "yarn.lock" }}
-      - v1-yarn-cache-node_18-{{ .Branch }}
-      - v1-yarn-cache-node_18
-
-  - &save_yarn_cache_node_18
-    key: v1-yarn-cache-node_18-{{ .Branch }}-{{ checksum "yarn.lock" }}
-    paths:
-      - ~/.cache/yarn
-
-
-  - &restore_yarn_cache_node_20
-    keys:
-      - v1-yarn-cache-node_20-{{ .Branch }}-{{ checksum "yarn.lock" }}
-      - v1-yarn-cache-node_20-{{ .Branch }}
-      - v1-yarn-cache-node_20
-
-  - &save_yarn_cache_node_20
-    key: v1-yarn-cache-node_20-{{ .Branch }}-{{ checksum "yarn.lock" }}
-    paths:
-      - ~/.cache/yarn
-
-  - &restore_yarn_cache_node_22
-    keys:
-      - v1-yarn-cache-node_22-{{ .Branch }}-{{ checksum "yarn.lock" }}
-      - v1-yarn-cache-node_22-{{ .Branch }}
-      - v1-yarn-cache-node_22
-
-  - &save_yarn_cache_node_22
-    key: v1-yarn-cache-node_22-{{ .Branch }}-{{ checksum "yarn.lock" }}
-    paths:
-      - ~/.cache/yarn
-
-  - &yarn_install
-    name: Installing
-    command: yarn --frozen-lockfile


We can simplify large parts of this:

We don't need a cache per Node.js version. Package managers don't install differently based on Node.js versions

We can migrate this into a command. That way it's native to CircleCI and validated by its language server also locally

tdeekens · 2025-03-07T10:55:11Z

.circleci/config.yml

    working_directory: *working_directory

+commands:
+  pnpm_install:


This is the command to replace everything above.

tdeekens · 2025-03-07T10:55:30Z

.circleci/config.yml

-      - run: *yarn_install
-      - save_cache: *save_yarn_cache_node_18
+      - pnpm_install
+      - run:


I suggest not to build as a post install. It's not needed and frustrating. If we don't do that we must do it here.

tdeekens · 2025-03-07T10:56:01Z

.github/workflows/release.yml

-      - name: Read .nvmrc
-        run: echo ::set-output name=NVMRC::$(cat .nvmrc)
-        id: nvm
-
-      - name: Setup Node (uses version in .nvmrc)
-        uses: actions/setup-node@v2
+      - name: Install pnpm
+        uses: pnpm/action-setup@v4.1.0
        with:
-          node-version: '${{ steps.nvm.outputs.NVMRC }}'
+          run_install: false

-      - name: Get yarn cache
-        id: yarn-cache
-        run: echo "::set-output name=dir::$(yarn cache dir)"
-
-      - uses: actions/cache@v4
+      - name: Setup Node (uses version in .nvmrc)
+        uses: actions/setup-node@v4
        with:
-          path: ${{ steps.yarn-cache.outputs.dir }}
-          key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
-          restore-keys: |
-            ${{ runner.os }}-yarn-


Large parts here can be offloaded to the action-setup.

tdeekens · 2025-03-07T10:56:25Z

.husky/commit-msg

@@ -0,0 +1 @@
+pnpm commitlint --edit $1


Husky has to be updated in the process. It has now files here instead of a global config file.

tdeekens · 2025-03-07T10:57:33Z

package.json

+    "clean": "rm -rf coverage; manypkg exec rimraf lib dist node_modules; rm -rf node_modules",
    "commit": "git-cz",
-    "build": "lerna run build --no-private",
+    "build": "pnpm --recursive run build",


pnpm has this which allows us to drop lerna.

tdeekens · 2025-03-07T10:57:40Z

package.json

-    "packages/*"
-  ],
-  "dependencies": {
+  "resolutions": {


Just formatting.

tdeekens · 2025-03-07T10:58:07Z

package.json

+    "chalk": "3.0.0"
+  },
+  "packageManager": "pnpm@9.15.6",
+  "devDependencies": {


Semi important: root package.json files should not contain deps but devDeps. We use things in there for development.

tdeekens · 2025-03-07T10:58:39Z

scripts/postinstall.sh

@@ -0,0 +1,11 @@
+#!/usr/bin/env bash


Easier to maintain than an inline run script.

tdeekens · 2025-03-07T10:58:58Z

tsconfig.json

    "target": "esnext",
    "allowJs": false,
    "baseUrl": ".",
+    "skipLibCheck": true,


Needed at the moment for one package. I don't see a major downside. I plan to enable this later again.

codecov · 2025-03-07T14:48:32Z

Codecov Report

Attention: Patch coverage is 99.25373% with 1 line in your changes missing coverage. Please review.

Project coverage is 97.76%. Comparing base (245eaf3) to head (8f7de67).
Report is 2 commits behind head on master.

Files with missing lines	Patch %	Lines
packages/sdk-auth/src/auth.js	75.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1931      +/-   ##
==========================================
+ Coverage   94.65%   97.76%   +3.10%     
==========================================
  Files         148      147       -1     
  Lines        5105     4779     -326     
  Branches     1380     1277     -103     
==========================================
- Hits         4832     4672     -160     
+ Misses        270      104     -166     
  Partials        3        3

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copilot

PR Overview

This PR migrates the project from yarn to pnpm while also dropping lerna in favor of a new dependency management strategy. Key changes include updating changesets, CircleCI configurations, GitHub release workflows, and contributing documentation to reflect the migration.

Reviewed Changes

File	Description
.changeset/silver-tips-serve.md	Adds a changeset specifying major version bumps for several packages.
.circleci/config.yml	Replaces yarn commands and cache logic with pnpm alternatives in CI pipelines.
.github/workflows/release.yml	Updates the release workflow to use pnpm for dependency installation, building, and publishing.
CONTRIBUTING.md	Revises installation and command instructions to use pnpm, though still retains a reference to lerna.
husky.config.js	Removes the husky configuration, potentially impacting commit hooks management.

Copilot reviewed 97 out of 97 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

husky.config.js:1

The husky.config.js file has been removed; if commit hooks are still needed, please ensure they are configured elsewhere and update the documentation accordingly.

module.exports = {

CONTRIBUTING.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

islam3zzat

This clearly involved a great deal of effort, thank you so much!

islam3zzat · 2025-03-17T11:25:27Z

.circleci/config.yml

+          command: pnpm test:ci
      - run:
          name: Running test (with coverage report)
-          command: yarn test:ci
+          command: pnpm test:ci


I'm not sure why we used to run the tests twice!

tdeekens commented Mar 7, 2025

View reviewed changes

tdeekens requested review from CarlosCortizasCT and kafis March 7, 2025 10:59

tdeekens force-pushed the yarn-to-pnpm branch 3 times, most recently from c17fb85 to 4269cf9 Compare March 7, 2025 12:51

tdeekens marked this pull request as ready for review March 7, 2025 12:57

Base automatically changed from nodejs-v18 to master March 7, 2025 13:20

tdeekens added 19 commits March 7, 2025 14:25

chore: try running on up-to-date nodejs

33a81de

refactor: drop support for lower nodejs v18

52793b6

chore: update gitignore

30f8681

chore: yarn.lock

44f4340

chore: update jsondiffpatcher

0d8b2e3

chore: try updating rollup

430ec8d

fix: remove yarnrc

dea2db9

chore: add changeset

1068295

chore: lockfile update

bff623d

efactor: replace lerna and yarn with pnpm and manypkg

fe9d327

fix: husky hooks

6f799de

chore: lockfile

618da50

fix: being to stupid to type

bb75671

refactor: no more postbuild

b8947d1

refactor: add integration tests as workspace

60dd899

chore: lint fixes

6caa9d6

fix: testing ci

65fbb34

fix: the transform ignore pattern regex

c8d6096

fix: enable manypkg

6a223fd

tdeekens force-pushed the yarn-to-pnpm branch from 4269cf9 to 6a223fd Compare March 7, 2025 13:28

chore: new lockfile

b14242c

tdeekens added 2 commits March 7, 2025 14:55

fix: integration test setup

8a4f644

fix: test package command

3d2a66f

tdeekens force-pushed the yarn-to-pnpm branch from 05626ee to 3d2a66f Compare March 7, 2025 14:19

chore: update snaps

3f85151

tdeekens force-pushed the yarn-to-pnpm branch from 3bcb803 to 3f85151 Compare March 7, 2025 14:41

tdeekens requested a review from Copilot March 7, 2025 16:04

Copilot AI reviewed Mar 7, 2025

View reviewed changes

CONTRIBUTING.md Outdated Show resolved Hide resolved

Update CONTRIBUTING.md

8f7de67

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

islam3zzat approved these changes Mar 17, 2025

View reviewed changes

tdeekens merged commit ccb7812 into master Mar 17, 2025
9 checks passed

tdeekens deleted the yarn-to-pnpm branch March 17, 2025 12:56

ct-changesets bot mentioned this pull request Mar 18, 2025

Version Packages #1935

Closed

Conversation

tdeekens commented Mar 7, 2025

Summary

Uh oh!

changeset-bot bot commented Mar 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Mar 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

PR Overview

Reviewed Changes

Uh oh!

Uh oh!

islam3zzat left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

changeset-bot bot commented Mar 7, 2025 •

edited

Loading

codecov bot commented Mar 7, 2025 •

edited

Loading