Publish preview builds to NPM instead of GitHub

.github/workflows/publish-preview.yml

@@ -40,13 +40,11 @@ jobs:
         with:
           node-version-file: '.nvmrc'
           cache: yarn
-          registry-url: 'https://npm.pkg.github.com'
-          scope: '@MetaMask'
       - run: yarn --immutable
       - name: Get commit SHA
         id: commit-sha
         run: echo "COMMIT_SHA=$(git rev-parse --short HEAD)" >> "$GITHUB_OUTPUT"
-      - run: yarn prepare-preview-builds ${{ steps.commit-sha.outputs.COMMIT_SHA }}
+      - run: yarn prepare-preview-builds @metamask-previews ${{ steps.commit-sha.outputs.COMMIT_SHA }}
       - run: yarn build
       - name: Publish preview build
         run: yarn publish-previews

constraints.pro

@@ -242,10 +242,6 @@ gen_enforced_field(WorkspaceCwd, 'files', []) :-
 gen_enforced_field(WorkspaceCwd, 'scripts.build:docs', 'typedoc') :-
   WorkspaceCwd \= '.'.
 
-% All published packages must have the same "prepare-manifest:preview" script.
-gen_enforced_field(WorkspaceCwd, 'scripts.prepare-manifest:preview', '../../scripts/prepare-preview-manifest.sh') :-
-  \+ workspace_field(WorkspaceCwd, 'private', true).
-
 % All published packages must have the same "publish:preview" script.
 gen_enforced_field(WorkspaceCwd, 'scripts.publish:preview', 'yarn npm publish --tag preview') :-
   \+ workspace_field(WorkspaceCwd, 'private', true).

docs/contributing.md

@@ -51,81 +51,54 @@ When submitting a pull request for this repo, take some a bit of extra time to f
   - What are the anticipated effects to whichever platform might want to make use of these changes?
   - If there are breaking changes to the API, what do consumers need to do in order to adapt to those changes upon upgrading to them?
 
-## Using packages in other projects during development/testing
+## Testing changes to packages against another project
 
-When developing changes to packages within this repository that a different project depends upon, you may wish to load those changes into the project and test them locally or in CI before publishing proper releases of those packages. To solve that problem, this repository provides a mechanism to publish "preview" versions of packages to GitHub Package Registry. These versions can then be used in the project like any other version, provided the project is configured to use that registry.
+If you have a project that depends on packages from this monorepo, you may wish to load changes to those packages into the project and test them locally or in CI before releasing them. To solve that problem, this repository provides a mechanism to publish "preview" versions of packages. These versions can then be used in the project like any other versions.
 
 ### Publishing preview builds as a MetaMask contributor
 
-If you are a member of the MetaMask organization, you can create branches directly on this repository rather than using a fork. This allows you to use our preview build GitHub Action.
+If you're a member of the MetaMask organization, you can create preview builds for a pull request by posting a comment with the text `@metamaskbot publish-preview`. This starts the `publish-preview` GitHub action; after a few minutes, it should complete and you will see a new comment that lists the newly published packages. You can then [use these versions in your project](#using-preview-builds).
 
-Post a comment on the PR with the text `@metamaskbot publish-preview` (This triggers the `publish-preview` GitHub action). After a few minutes, you will see a new comment indicating that all packages have been published with the format `<package name>-<commit id>`.
+Note two things about each package:
 
-### Publishing preview builds as an independent contributor
-
-If you're a contributor and you've forked this repository, you can create preview versions for a branch by following these steps:
-
-1. Open the `package.json` for each package that you want to publish and change the scope in the name from `@metamask` to `@<your GitHub username>`.
-2. From your local fork of the repository, run `yarn prepare-preview-builds "$(git rev-parse --short HEAD)" && yarn build && yarn publish-previews` to generate preview versions for all packages based on the current branch and publish them to GitHub Package Registry.
-   - Take note of the version that is published; it should look like `1.2.3-e2df9b4` instead of `1.2.3`.
-
-### Using preview builds
+- The name is scoped to `@metamask-previews` instead of `@metamask`.
+- The ID of the last commit in the branch is appended to the version, e.g. `1.2.3-e2df9b4` instead of `1.2.3`.
 
-> **Warning**
->
-> There is a known problem with the preview build workflow. It relies upon you having a local cache of any non-preview `@metamask/`-scoped packages.
->
-> If you encounter problems installing non-preview `@metamask/`-scoped packages when using this workflow, you can work around the problem by first installing dependencies without preview builds enabled (e.g. by temporarily removing the `.npmrc` or unsetting the required environment variables) to install the missing packages. Once they are installed, preview build installations should work (the non-preview `@metamask/`-scoped packages will be found in your local cache).
->
-> See [issue #1075](https://github.com/MetaMask/core/issues/1075) for more details.
+If you make more changes and wish to publish another set of preview builds, post another comment on the pull request with `@metamaskbot publish-preview`.
 
-Preview builds should automatically work in CI on the MetaMask extension and MetaMask mobile repositories, as long as the PR is in draft.
+### Publishing preview builds as an independent contributor
 
-To use preview builds locally, follow these steps:
+If you've forked this repository, you can create preview versions for a branch by following these steps:
 
-1. Navigate to your settings within GitHub and [create a classic access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token#creating-a-personal-access-token-classic). Make sure to give this token the `read:packages` scope.
-2. Follow these steps in the project using the preview builds:
+1. First, since preview build releases are published under an NPM organization, you'll need access to one. If this is not the case, you can either [create a new organization](https://www.npmjs.com/org/create) or [convert your existing username into an organization](https://www.npmjs.com/org/upgrade).
 
-   - **Yarn 1 (classic) or NPM**
+2. Once you've done this, open the `package.json` for each package that you want to publish and change the scope in the name from `@metamask` to `@<NPM_ORG>`, replacing `NPM_ORG` appropriately.
 
-     Add the following in `.npmrc`
+3. Next, run the following commands to generate preview versions for all packages based on the current branch and publish them (again, replacing `NPM_ORG`):
 
-     ```
-     @metamask:registry=https://npm.pkg.github.com
-     //npm.pkg.github.com/:_authToken=<your personal access token>
-     ```
+   ```
+   yarn prepare-preview-builds "@<NPM_ORG>" "$(git rev-parse --short HEAD)" && yarn build && yarn publish-previews
+   ```
 
-   - **Yarn >= 2 (berry):**
+   You should be able to see the published version of each package in the output. Note two things:
 
-     Ensure that the project `.yarnrc.yml` file has the following contents:
+   - The name is scoped to the NPM organization you entered instead of `@metamask`.
+   - The ID of the last commit in the branch is appended to the version, e.g. `1.2.3-e2df9b4` instead of `1.2.3`.
 
-     ```
-     npmRegistries:
-       "https://npm.pkg.github.com":
-         npmAlwaysAuth: true
-         npmAuthToken: "${GITHUB_NPM_TOKEN-}"
+4. Once this command is complete, all preview builds will be published, and you can then [use them in your project](#using-preview-builds).
 
-     npmScopes:
-       metamask:
-         npmRegistryServer: "${METAMASK_NPM_REGISTRY:-https://registry.yarnpkg.com}"
-     ```
+If you make more changes and wish to publish another set of preview builds, run steps 3 and 4 again.
 
-     The `METAMASK_NPM_REGISTRY` environment variable lets you control which registry is used for `@metamask`-scoped packages. Set this environment variable to `https://npm.pkg.github.com` to use preview builds. The `GITHUB_NPM_TOKEN` environment variable is where your token is set (the one created in step 1).
+### Using preview builds
 
-     For example, in Bash or ZSH, you can set both of these environment variables when installing dependencies:
+To use a preview build for a package within a project, open the project's `package.json`, find the dependency that corresponds to the package, and make two changes:
 
-     ```bash
-     GITHUB_NPM_TOKEN=<your personal access token> METAMASK_NPM_REGISTRY=https://npm.pkg.github.com yarn install
-     ```
+- Update `@metamask/` to `@metamask-previews/` (or whatever NPM scope you've chosen to publish under, if you've forked this monorepo)
+- Update the version to match a preview build (e.g. `1.2.3-e2df9b4` instead of `^1.2.3`)
 
-     - It's recommended to use your machine's local keychain to store the token, and retrieve it from there. For example on macOS, you can use:
-       ```bash
-       GITHUB_NPM_TOKEN=$(security find-generic-password -s 'GitHub NPM Token' -w) METAMASK_NPM_REGISTRY=https://npm.pkg.github.com yarn install
-       ```
+Finally, run `yarn install`.
 
-3. Update `package.json` with the new preview build versions
-   - Each preview build package should have a version matching (e.g. `1.2.3-e2df9b4` instead of `~1.2.3`), then run `yarn install`.
-4. Repeat step 3 each time you publish new preview builds.
+If you re-publish preview builds, make sure to return to the project's `package.json` and update the preview versions for the corresponding packages.
 
 ## Releasing
 

package.json

@@ -24,8 +24,8 @@
     "lint:fix": "yarn lint:eslint --fix && yarn lint:misc --write && yarn constraints --fix",
     "lint:misc": "prettier '**/*.json' '**/*.md' '!**/CHANGELOG.md' '!**/CHANGELOG.old.md' '**/*.yml' '!.yarnrc.yml' '!merged-packages/**' --ignore-path .gitignore",
     "prepack": "./scripts/prepack.sh",
-    "prepare-preview-builds": "yarn workspaces foreach --parallel run prepare-manifest:preview",
-    "publish-previews": "yarn workspaces foreach --parallel run publish:preview",
+    "prepare-preview-builds": "./scripts/prepare-preview-builds.sh",
+    "publish-previews": "yarn workspaces foreach --parallel --verbose run publish:preview",
     "setup": "yarn install",
     "test": "yarn workspaces foreach --parallel --verbose run test"
   },

packages/address-book-controller/package.json

@@ -23,7 +23,6 @@
   "scripts": {
     "build:docs": "typedoc",
     "changelog:validate": "../../scripts/validate-changelog.sh @metamask/address-book-controller",
-    "prepare-manifest:preview": "../../scripts/prepare-preview-manifest.sh",
     "publish:preview": "yarn npm publish --tag preview",
     "test": "jest",
     "test:watch": "jest --watch"

packages/announcement-controller/package.json

@@ -23,7 +23,6 @@
   "scripts": {
     "build:docs": "typedoc",
     "changelog:validate": "../../scripts/validate-changelog.sh @metamask/announcement-controller",
-    "prepare-manifest:preview": "../../scripts/prepare-preview-manifest.sh",
     "publish:preview": "yarn npm publish --tag preview",
     "test": "jest",
     "test:watch": "jest --watch"

packages/approval-controller/package.json

@@ -23,7 +23,6 @@
   "scripts": {
     "build:docs": "typedoc",
     "changelog:validate": "../../scripts/validate-changelog.sh @metamask/approval-controller",
-    "prepare-manifest:preview": "../../scripts/prepare-preview-manifest.sh",
     "publish:preview": "yarn npm publish --tag preview",
     "test": "jest",
     "test:watch": "jest --watch"

packages/assets-controllers/package.json

@@ -23,7 +23,6 @@
   "scripts": {
     "build:docs": "typedoc",
     "changelog:validate": "../../scripts/validate-changelog.sh @metamask/assets-controllers",
-    "prepare-manifest:preview": "../../scripts/prepare-preview-manifest.sh",
     "publish:preview": "yarn npm publish --tag preview",
     "test": "jest",
     "test:watch": "jest --watch"

packages/base-controller/package.json

@@ -23,7 +23,6 @@
   "scripts": {
     "build:docs": "typedoc",
     "changelog:validate": "../../scripts/validate-changelog.sh @metamask/base-controller",
-    "prepare-manifest:preview": "../../scripts/prepare-preview-manifest.sh",
     "publish:preview": "yarn npm publish --tag preview",
     "test": "jest",
     "test:watch": "jest --watch"

packages/composable-controller/package.json

@@ -23,7 +23,6 @@
   "scripts": {
     "build:docs": "typedoc",
     "changelog:validate": "../../scripts/validate-changelog.sh @metamask/composable-controller",
-    "prepare-manifest:preview": "../../scripts/prepare-preview-manifest.sh",
     "publish:preview": "yarn npm publish --tag preview",
     "test": "jest",
     "test:watch": "jest --watch"

packages/controller-utils/package.json

@@ -23,7 +23,6 @@
   "scripts": {
     "build:docs": "typedoc",
     "changelog:validate": "../../scripts/validate-changelog.sh @metamask/controller-utils",
-    "prepare-manifest:preview": "../../scripts/prepare-preview-manifest.sh",
     "publish:preview": "yarn npm publish --tag preview",
     "test": "jest",
     "test:watch": "jest --watch"

packages/ens-controller/package.json

@@ -23,7 +23,6 @@
   "scripts": {
     "build:docs": "typedoc",
     "changelog:validate": "../../scripts/validate-changelog.sh @metamask/ens-controller",
-    "prepare-manifest:preview": "../../scripts/prepare-preview-manifest.sh",
     "publish:preview": "yarn npm publish --tag preview",
     "test": "jest",
     "test:watch": "jest --watch"

packages/gas-fee-controller/package.json

@@ -23,7 +23,6 @@
   "scripts": {
     "build:docs": "typedoc",
     "changelog:validate": "../../scripts/validate-changelog.sh @metamask/gas-fee-controller",
-    "prepare-manifest:preview": "../../scripts/prepare-preview-manifest.sh",
     "publish:preview": "yarn npm publish --tag preview",
     "test": "jest",
     "test:watch": "jest --watch"

packages/keyring-controller/package.json

@@ -23,7 +23,6 @@
   "scripts": {
     "build:docs": "typedoc",
     "changelog:validate": "../../scripts/validate-changelog.sh @metamask/keyring-controller",
-    "prepare-manifest:preview": "../../scripts/prepare-preview-manifest.sh",
     "publish:preview": "yarn npm publish --tag preview",
     "test": "jest",
     "test:watch": "jest --watch"

packages/logging-controller/package.json

@@ -23,7 +23,6 @@
   "scripts": {
     "build:docs": "typedoc",
     "changelog:validate": "../../scripts/validate-changelog.sh @metamask/logging-controller",
-    "prepare-manifest:preview": "../../scripts/prepare-preview-manifest.sh",
     "publish:preview": "yarn npm publish --tag preview",
     "test": "jest",
     "test:watch": "jest --watch"

packages/message-manager/package.json

@@ -23,7 +23,6 @@
   "scripts": {
     "build:docs": "typedoc",
     "changelog:validate": "../../scripts/validate-changelog.sh @metamask/message-manager",
-    "prepare-manifest:preview": "../../scripts/prepare-preview-manifest.sh",
     "publish:preview": "yarn npm publish --tag preview",
     "test": "jest",
     "test:watch": "jest --watch"

packages/network-controller/package.json

@@ -23,7 +23,6 @@
   "scripts": {
     "build:docs": "typedoc",
     "changelog:validate": "../../scripts/validate-changelog.sh @metamask/network-controller",
-    "prepare-manifest:preview": "../../scripts/prepare-preview-manifest.sh",
     "publish:preview": "yarn npm publish --tag preview",
     "test": "jest",
     "test:watch": "jest --watch"

packages/notification-controller/package.json

@@ -23,7 +23,6 @@
   "scripts": {
     "build:docs": "typedoc",
     "changelog:validate": "../../scripts/validate-changelog.sh @metamask/notification-controller",
-    "prepare-manifest:preview": "../../scripts/prepare-preview-manifest.sh",
     "publish:preview": "yarn npm publish --tag preview",
     "test": "jest",
     "test:watch": "jest --watch"

packages/permission-controller/package.json

@@ -23,7 +23,6 @@
   "scripts": {
     "build:docs": "typedoc",
     "changelog:validate": "../../scripts/validate-changelog.sh @metamask/permission-controller",
-    "prepare-manifest:preview": "../../scripts/prepare-preview-manifest.sh",
     "publish:preview": "yarn npm publish --tag preview",
     "test": "jest",
     "test:watch": "jest --watch"

packages/phishing-controller/package.json

@@ -23,7 +23,6 @@
   "scripts": {
     "build:docs": "typedoc",
     "changelog:validate": "../../scripts/validate-changelog.sh @metamask/phishing-controller",
-    "prepare-manifest:preview": "../../scripts/prepare-preview-manifest.sh",
     "publish:preview": "yarn npm publish --tag preview",
     "test": "jest",
     "test:watch": "jest --watch"

packages/preferences-controller/package.json

@@ -23,7 +23,6 @@
   "scripts": {
     "build:docs": "typedoc",
     "changelog:validate": "../../scripts/validate-changelog.sh @metamask/preferences-controller",
-    "prepare-manifest:preview": "../../scripts/prepare-preview-manifest.sh",
     "publish:preview": "yarn npm publish --tag preview",
     "test": "jest",
     "test:watch": "jest --watch"

packages/rate-limit-controller/package.json

@@ -23,7 +23,6 @@
   "scripts": {
     "build:docs": "typedoc",
     "changelog:validate": "../../scripts/validate-changelog.sh @metamask/rate-limit-controller",
-    "prepare-manifest:preview": "../../scripts/prepare-preview-manifest.sh",
     "publish:preview": "yarn npm publish --tag preview",
     "test": "jest",
     "test:watch": "jest --watch"

packages/signature-controller/package.json

@@ -23,7 +23,6 @@
   "scripts": {
     "build:docs": "typedoc",
     "changelog:validate": "../../scripts/validate-changelog.sh @metamask/signature-controller",
-    "prepare-manifest:preview": "../../scripts/prepare-preview-manifest.sh",
     "publish:preview": "yarn npm publish --tag preview",
     "test": "jest",
     "test:watch": "jest --watch"

packages/transaction-controller/package.json

@@ -23,7 +23,6 @@
   "scripts": {
     "build:docs": "typedoc",
     "changelog:validate": "../../scripts/validate-changelog.sh @metamask/transaction-controller",
-    "prepare-manifest:preview": "../../scripts/prepare-preview-manifest.sh",
     "publish:preview": "yarn npm publish --tag preview",
     "test": "jest",
     "test:watch": "jest --watch"

scripts/prepare-preview-builds.jq

@@ -0,0 +1,52 @@
+# The name is overwritten, causing the package to get published under a
+# different NPM scope than non-preview builds.
+.name |= sub("@metamask/"; "\($npm_scope)/") |
+
+# The prerelease version is overwritten, preserving the non-prerelease portion
+# of the version. Technically we'd want to bump the non-prerelease portion as
+# well if we wanted this to be SemVer-compliant, but it was simpler not to.
+# This is just for testing, it doesn't need to strictly follow SemVer.
+.version |= split("-")[0] + "-preview.\($hash)" |
+
+# The workspace dependencies are updated to point to the exact preview build
+# version, so that Yarn does not try to resolve a different version from the
+# registry.
+
+if has("devDependencies") then
+  .devDependencies |= with_entries(
+    if .value | startswith("workspace:") then
+      .key |= sub("@metamask/"; "@metamask-previews/") |
+      .value |= sub("workspace:\\^"; "workspace:*")
+    else
+      .
+    end
+  )
+else
+  .
+end |
+
+if has("peerDependencies") then
+  .peerDependencies |= with_entries(
+    if .value | startswith("workspace:") then
+      .key |= sub("@metamask/"; "@metamask-previews/") |
+      .value |= sub("workspace:\\^"; "workspace:*")
+    else
+      .
+    end
+  )
+else
+  .
+end |
+
+if has("dependencies") then
+  .dependencies |= with_entries(
+    if .value | startswith("workspace:") then
+      .key |= sub("@metamask/"; "@metamask-previews/") |
+      .value |= sub("workspace:\\^"; "workspace:*")
+    else
+      .
+    end
+  )
+else
+  .
+end