From 472893daa88cae7898e7a9876054033bc10f3a8a Mon Sep 17 00:00:00 2001 From: Toby Dylan Hocking Date: Mon, 27 Nov 2023 23:11:02 -0700 Subject: [PATCH 01/29] first draft GOVERNANCE.md --- GOVERNANCE.md | 126 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 126 insertions(+) create mode 100644 GOVERNANCE.md diff --git a/GOVERNANCE.md b/GOVERNANCE.md new file mode 100644 index 0000000000..669d453a19 --- /dev/null +++ b/GOVERNANCE.md @@ -0,0 +1,126 @@ +Governance for the R data.table project + +# Purpose and scope + +## This document + +The purpose of this document is to define how people related to the project work together, so that the project can expand to handle more contributions, from a larger and more diverse group of contributors. + +## The R package + +The purpose of the project is to maintain the R data.table package, +which is guided by the following principles: + +* Few (if any) dependencies +* Time & memory efficiency +* Concise syntax (minimal redundancy in code) +* Stable code base (easy for users to upgrade to new data.table, and compatible with old R versions) +* Comprehensive and accessible documentation +* Clear error and warning messages + +To prioritize developer time, we define what is in and out of scope. +The current scope of package functionality includes: +* Data manipulation and analysis + * reshaping/pivoting + * aggregation/summarizing + * subsetting rows + * all sorts of joins + * adding/updating/deleting columns +* Reading/writing of data from/to many file formats + +Functionality that is out of scope: +* plotting/graphics (like ggplot2) +* manipulating data stored on disk (or remote SQL DB) rather than in memory (like sqldf / dbplyr) +* machine learning / modeling (like mlr3) +* regular expression builders (like rex and nc packages) + +# Roles + +## Contributor + +* Definition: a user who has written/commented at least one issue, worked to label/triage issues, written a blog post, given a talk, etc. +* How this role is recognized: there is no central list of contributors / no formal recognition for contributors. + +## Project member + +* Definition: some one who has submitted at least one PR that has been merged into master. +* How to obtain this role: any user can become a member by submitting a PR, then having it reviewed and merged into master. Contributors who have written issues should be encouraged to submit their first PR to become a project member. +* How this role is recognized: Members are credited via role="ctb" in DESCRIPTION (so they appear in Author list on CRAN), and they are added to https://github.com/orgs/Rdatatable/teams/project-members so they can create new branches in the Rdatatable/data.table GitHub repo. They also appear on https://github.com/Rdatatable/data.table/graphs/contributors (Contributions to master, excluding merge commits). + +## Reviewer + +* Definition: a member who has volunteered to do code reviews for some features/files. +* How to obtain this role: after one or more significant PRs to a given file, a member should be invited to add their name as a reviewer of that file in CODEOWNERS, and after that is merged into master, then they are considered a reviewer. +* How this role is recognized: same credit in DESCRIPTION as a regular member, role="ctb" (so they appear in Author list on CRAN). +* Note: having your name in CODEOWNERS does not give any special permission, but it does mean that you will be notified whenever there is a new PR with changes to that file. + +## Committer + +* Definition: permission to commit to, and merge PRs into, master branch. +* How to obtain this role: after a reviewer has a consistent history of careful reviews of others' PRs, then a current GitHub maintainer should ask all other current GitHub maintainers if they approve promoting the reviewer to GitHub maintainer, and it should be done if there is general consensus/agreement. +* How this role is recognized: credited via role="aut" in DESCRIPTION (so they appear in Author list on CRAN), and added to https://github.com/orgs/Rdatatable/teams/maintainers which gives permission to merge PRs into master branch. + +## CRAN maintainer + +* Definition: in charge of communication with CRAN. Responsible for submitting releases to CRAN on a regular basis, and in response to requests from CRAN. +* How to obtain this role: (1) merge into master a PR adding role="cre" to DESCRIPTION, and (2) submit updated package to CRAN (previous CRAN maintainer will have to confirm change by email to CRAN). +* How this role is recognized: credited via role="cre" in DESCRIPTION, so they appear as Maintainer on CRAN. + +## Other roles + +Each of the below roles is important, and a wiki page should be +created to explain (1) details of responsibilities, (2) who to contact +to obtain this role. + +* translation manager (to communicate with translators), no special permission. +* triage manager (to decide which issues/PRs to include in next releases), no special permission. +* performance testing manager (to prevent performance regressions), need permission to edit continuous integration scripts. +* reverse dependency manager (to ensure compatibility with other CRAN packages), need access to a computer that can run revdep checks in parallel. +* binary manager (to build binaries of development branches for user testing before release), need permission to edit continuous deployment scripts. + +# Decision-making processes + +## Pull Requests + +A pull request can be merged, as long as there is one approving review +(ideally from a reviewer of the affected files, different person from +author of PR), and no opposing/blocking comments from +Reviewers/Committers. + +## CRAN updates + +* Regular CRAN releases should ideally occur twice per year, and can include new features. +* A hotfix/patch CRAN release should occur when CRAN asks for one, at which time the CRAN maintainer should post an issue on github, and ask others to help fix/prepare the release. +* Both kinds of releases should be discussed in an issue, and the release should happen only if there is general consensus that it is ok to go ahead (no Reviewer/Committer has expressed major blocking concerns). +* It is the responsibility of the CRAN maintainer to ensure quality prior to release. This includes CRAN checks, unit tests, performance tests, etc, and these tasks can be delegated to others. + +## Changing this GOVERNANCE.md document + +There is no special process for changing this document (submit a PR +and ask for review). + +# Code of conduct + +As contributors and maintainers of this project, we pledge to respect all people who contribute through reporting issues, posting feature requests, updating documentation, submitting pull requests or patches, and other activities. + +We are committed to making participation in this project a harassment-free experience for everyone, regardless of level of experience, gender, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, religion, etc. + +Examples of unacceptable behavior by participants include the use of sexual language or imagery, derogatory comments or personal attacks, trolling, public or private harassment, insults, or other unprofessional conduct. + +Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct. A person with special roles who does not follow the Code of Conduct may have their roles revoked. + +Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by opening an issue or emailing one or more of the Committers. + +This Code of Conduct is adapted from Tidyverse code of conduct. + +# Version numbering + +data.table Version line in DESCRIPTION typically has the following meanings + +* x.y.z where x=major, y=minor, z=patch/hotfix. +* x should be incremented only for major backwards-incompatible changes. +* z is even for CRAN releases, odd for GitHub development. +* z=99 for master branch with new features (for example 1.14.99 or 1.15.99), which eventually becomes a regular CRAN release, with incremented y and z=0 (for example 1.15.0 or 1.16.0). +* patch/hotfix development should occur on GitHub as z=odd (1.15.1) and release to CRAN as z=even (1.15.2). + + From cc58300d354a6d4ca97c310843567517a1b450a9 Mon Sep 17 00:00:00 2001 From: Toby Dylan Hocking Date: Thu, 30 Nov 2023 08:48:25 -0700 Subject: [PATCH 02/29] more contributions Co-authored-by: Michael Chirico --- GOVERNANCE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/GOVERNANCE.md b/GOVERNANCE.md index 669d453a19..eb86f89c66 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -4,7 +4,7 @@ Governance for the R data.table project ## This document -The purpose of this document is to define how people related to the project work together, so that the project can expand to handle more contributions, from a larger and more diverse group of contributors. +The purpose of this document is to define how people related to the project work together, so that the project can expand to handle a larger and more diverse group of contributors. ## The R package From 92561572054ace15993e2baae4777c416338f6b2 Mon Sep 17 00:00:00 2001 From: Toby Dylan Hocking Date: Thu, 30 Nov 2023 08:54:23 -0700 Subject: [PATCH 03/29] run-time signals are accessible Co-authored-by: Michael Chirico --- GOVERNANCE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/GOVERNANCE.md b/GOVERNANCE.md index eb86f89c66..297d6b39c5 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -15,7 +15,7 @@ which is guided by the following principles: * Time & memory efficiency * Concise syntax (minimal redundancy in code) * Stable code base (easy for users to upgrade to new data.table, and compatible with old R versions) -* Comprehensive and accessible documentation +* Comprehensive and accessible documentation and run-time signals (errors, warnings) * Clear error and warning messages To prioritize developer time, we define what is in and out of scope. From 8027427ec989f5daff4eaea4cd84eb69a9552474 Mon Sep 17 00:00:00 2001 From: Toby Dylan Hocking Date: Thu, 30 Nov 2023 08:57:31 -0700 Subject: [PATCH 04/29] Update GOVERNANCE.md Co-authored-by: Michael Chirico --- GOVERNANCE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/GOVERNANCE.md b/GOVERNANCE.md index 297d6b39c5..72b9f7bdd5 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -30,7 +30,7 @@ The current scope of package functionality includes: Functionality that is out of scope: * plotting/graphics (like ggplot2) -* manipulating data stored on disk (or remote SQL DB) rather than in memory (like sqldf / dbplyr) +* manipulating out-of-memory data, e.g. data stored on disk or remote SQL DB, (as opposed e.g. to sqldf / dbplyr) * machine learning / modeling (like mlr3) * regular expression builders (like rex and nc packages) From a812be4354f4f72aeb462aadb5f3d59434e88a67 Mon Sep 17 00:00:00 2001 From: Toby Dylan Hocking Date: Thu, 30 Nov 2023 09:02:43 -0700 Subject: [PATCH 05/29] CRAN maintainer responsible for responding Co-authored-by: Michael Chirico --- GOVERNANCE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/GOVERNANCE.md b/GOVERNANCE.md index 72b9f7bdd5..24a1be3ff6 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -62,7 +62,7 @@ Functionality that is out of scope: ## CRAN maintainer -* Definition: in charge of communication with CRAN. Responsible for submitting releases to CRAN on a regular basis, and in response to requests from CRAN. +* Definition: in charge of communication with CRAN. Responsible for submitting releases to CRAN on a regular basis, and for responding to requests from CRAN. * How to obtain this role: (1) merge into master a PR adding role="cre" to DESCRIPTION, and (2) submit updated package to CRAN (previous CRAN maintainer will have to confirm change by email to CRAN). * How this role is recognized: credited via role="cre" in DESCRIPTION, so they appear as Maintainer on CRAN. From 7eb8c76bc8a8049ab8c3bf65ca580dccc2cad4f6 Mon Sep 17 00:00:00 2001 From: Toby Dylan Hocking Date: Thu, 30 Nov 2023 09:07:39 -0700 Subject: [PATCH 06/29] ok -> OK Co-authored-by: Michael Chirico --- GOVERNANCE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/GOVERNANCE.md b/GOVERNANCE.md index 24a1be3ff6..d85ade16f2 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -91,7 +91,7 @@ Reviewers/Committers. * Regular CRAN releases should ideally occur twice per year, and can include new features. * A hotfix/patch CRAN release should occur when CRAN asks for one, at which time the CRAN maintainer should post an issue on github, and ask others to help fix/prepare the release. -* Both kinds of releases should be discussed in an issue, and the release should happen only if there is general consensus that it is ok to go ahead (no Reviewer/Committer has expressed major blocking concerns). +* Both kinds of releases should be discussed in an issue, and the release should happen only if there is general consensus that it is OK to go ahead (no Reviewer/Committer has expressed major blocking concerns). * It is the responsibility of the CRAN maintainer to ensure quality prior to release. This includes CRAN checks, unit tests, performance tests, etc, and these tasks can be delegated to others. ## Changing this GOVERNANCE.md document From 7070a54ffce578d272ade3a3d7256ce7ed37c6c9 Mon Sep 17 00:00:00 2001 From: Toby Dylan Hocking Date: Thu, 30 Nov 2023 09:10:19 -0700 Subject: [PATCH 07/29] remove comma Co-authored-by: Michael Chirico --- GOVERNANCE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/GOVERNANCE.md b/GOVERNANCE.md index d85ade16f2..086569ffcc 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -82,7 +82,7 @@ to obtain this role. ## Pull Requests -A pull request can be merged, as long as there is one approving review +A pull request can be merged as long as there is one approving review (ideally from a reviewer of the affected files, different person from author of PR), and no opposing/blocking comments from Reviewers/Committers. From 45202ee3450a17143a763b293c4d874119c563a7 Mon Sep 17 00:00:00 2001 From: Toby Dylan Hocking Date: Thu, 30 Nov 2023 09:45:49 -0700 Subject: [PATCH 08/29] various updates based on comments from Jan and Michael --- GOVERNANCE.md | 42 +++++++++++++++++++++--------------------- 1 file changed, 21 insertions(+), 21 deletions(-) diff --git a/GOVERNANCE.md b/GOVERNANCE.md index 086569ffcc..7a441c9f9d 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -11,14 +11,15 @@ The purpose of this document is to define how people related to the project work The purpose of the project is to maintain the R data.table package, which is guided by the following principles: -* Few (if any) dependencies +* No external Imports/LinkingTo/Depends dependencies (external meaning those not maintained by the project) +* Few (if any) Suggests/Enhances dependencies * Time & memory efficiency * Concise syntax (minimal redundancy in code) * Stable code base (easy for users to upgrade to new data.table, and compatible with old R versions) * Comprehensive and accessible documentation and run-time signals (errors, warnings) * Clear error and warning messages -To prioritize developer time, we define what is in and out of scope. +To prioritize developer time, we define what is in and out of current scope. The current scope of package functionality includes: * Data manipulation and analysis * reshaping/pivoting @@ -26,13 +27,13 @@ The current scope of package functionality includes: * subsetting rows * all sorts of joins * adding/updating/deleting columns -* Reading/writing of data from/to many file formats +* Sorting (forder) +* Reading/writing of data from/to flat (plain text) files like CSV -Functionality that is out of scope: +Functionality that is out of current scope: * plotting/graphics (like ggplot2) * manipulating out-of-memory data, e.g. data stored on disk or remote SQL DB, (as opposed e.g. to sqldf / dbplyr) * machine learning / modeling (like mlr3) -* regular expression builders (like rex and nc packages) # Roles @@ -43,7 +44,7 @@ Functionality that is out of scope: ## Project member -* Definition: some one who has submitted at least one PR that has been merged into master. +* Definition: some one who has submitted at least one PR with code contributions, that has been merged into master. (spelling fixes in docs do not count) * How to obtain this role: any user can become a member by submitting a PR, then having it reviewed and merged into master. Contributors who have written issues should be encouraged to submit their first PR to become a project member. * How this role is recognized: Members are credited via role="ctb" in DESCRIPTION (so they appear in Author list on CRAN), and they are added to https://github.com/orgs/Rdatatable/teams/project-members so they can create new branches in the Rdatatable/data.table GitHub repo. They also appear on https://github.com/Rdatatable/data.table/graphs/contributors (Contributions to master, excluding merge commits). @@ -57,7 +58,7 @@ Functionality that is out of scope: ## Committer * Definition: permission to commit to, and merge PRs into, master branch. -* How to obtain this role: after a reviewer has a consistent history of careful reviews of others' PRs, then a current GitHub maintainer should ask all other current GitHub maintainers if they approve promoting the reviewer to GitHub maintainer, and it should be done if there is general consensus/agreement. +* How to obtain this role: after a reviewer has a consistent history of careful reviews of others' PRs, then a current Committer should ask all other current Committers if they approve promoting the Reviewer to Committer, and it should be done if there is Consensus among active Committers. * How this role is recognized: credited via role="aut" in DESCRIPTION (so they appear in Author list on CRAN), and added to https://github.com/orgs/Rdatatable/teams/maintainers which gives permission to merge PRs into master branch. ## CRAN maintainer @@ -70,28 +71,29 @@ Functionality that is out of scope: Each of the below roles is important, and a wiki page should be created to explain (1) details of responsibilities, (2) who to contact -to obtain this role. +to obtain this role. -* translation manager (to communicate with translators), no special permission. -* triage manager (to decide which issues/PRs to include in next releases), no special permission. -* performance testing manager (to prevent performance regressions), need permission to edit continuous integration scripts. -* reverse dependency manager (to ensure compatibility with other CRAN packages), need access to a computer that can run revdep checks in parallel. -* binary manager (to build binaries of development branches for user testing before release), need permission to edit continuous deployment scripts. +* Triage manager (to decide which issues/PRs to include in next releases), no special permissions. +* Translation manager (to communicate with translators, and source code changes related to i18n, for example making sure messages are translation-ready, updating the .pot template), no special permissions (updates should happen in a pull request as usual). +* Performance testing manager (to prevent performance regressions), no special permissions (updates should happen in a pull request as usual). +* Continuous integration/deployment manager (to maintain script which run tests and build binaries of development branches for user testing before release), no special permissions (updates should happen in a pull request as usual). +* Reverse dependency manager (to ensure compatibility with other CRAN packages), need access to a computer that can run revdep checks in parallel. # Decision-making processes +## Definition of Consensus + +Most decisions in the project happen by Consensus, which means that no active people (typically Reviewers and/or Committers) have expressed major blocking concerns, in a public discussion (typically in a GitHub issue or pull request). In Consensus, non-response by inactive members indicates tacit agreement. + ## Pull Requests -A pull request can be merged as long as there is one approving review -(ideally from a reviewer of the affected files, different person from -author of PR), and no opposing/blocking comments from -Reviewers/Committers. +A pull request can be merged as long as there is one approving review (ideally from a reviewer of the affected files, different person from author of PR), and Consensus from active Reviewers and Committers. ## CRAN updates * Regular CRAN releases should ideally occur twice per year, and can include new features. * A hotfix/patch CRAN release should occur when CRAN asks for one, at which time the CRAN maintainer should post an issue on github, and ask others to help fix/prepare the release. -* Both kinds of releases should be discussed in an issue, and the release should happen only if there is general consensus that it is OK to go ahead (no Reviewer/Committer has expressed major blocking concerns). +* Both kinds of releases should be discussed in an issue, and the release should happen only if there is Consensus among active Reviewers and Committers. * It is the responsibility of the CRAN maintainer to ensure quality prior to release. This includes CRAN checks, unit tests, performance tests, etc, and these tasks can be delegated to others. ## Changing this GOVERNANCE.md document @@ -107,7 +109,7 @@ We are committed to making participation in this project a harassment-free exper Examples of unacceptable behavior by participants include the use of sexual language or imagery, derogatory comments or personal attacks, trolling, public or private harassment, insults, or other unprofessional conduct. -Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct. A person with special roles who does not follow the Code of Conduct may have their roles revoked. +Committers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct. A person with special roles who does not follow the Code of Conduct may have their roles revoked. Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by opening an issue or emailing one or more of the Committers. @@ -122,5 +124,3 @@ data.table Version line in DESCRIPTION typically has the following meanings * z is even for CRAN releases, odd for GitHub development. * z=99 for master branch with new features (for example 1.14.99 or 1.15.99), which eventually becomes a regular CRAN release, with incremented y and z=0 (for example 1.15.0 or 1.16.0). * patch/hotfix development should occur on GitHub as z=odd (1.15.1) and release to CRAN as z=even (1.15.2). - - From 0eb3cde2d181578a45d387ed260b4544976069a8 Mon Sep 17 00:00:00 2001 From: Toby Dylan Hocking Date: Thu, 30 Nov 2023 09:52:58 -0700 Subject: [PATCH 09/29] process for changing scope --- GOVERNANCE.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/GOVERNANCE.md b/GOVERNANCE.md index 7a441c9f9d..6427d406a1 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -8,8 +8,7 @@ The purpose of this document is to define how people related to the project work ## The R package -The purpose of the project is to maintain the R data.table package, -which is guided by the following principles: +The purpose of the project is to maintain the R data.table package, which is guided by the following principles: * No external Imports/LinkingTo/Depends dependencies (external meaning those not maintained by the project) * Few (if any) Suggests/Enhances dependencies @@ -19,7 +18,8 @@ which is guided by the following principles: * Comprehensive and accessible documentation and run-time signals (errors, warnings) * Clear error and warning messages -To prioritize developer time, we define what is in and out of current scope. +To prioritize developer time, we define what is in and out of current scope. Feature requests in issues and pull requests that are out of current scope should be closed immediately, because they are not the current priority. If someone wants to contribute code that is currently out of scope, they first have to make a pull request that changes the scope as defined below. + The current scope of package functionality includes: * Data manipulation and analysis * reshaping/pivoting From 8c501a0fbecdaa736c947799b92e3e7bffb4a63e Mon Sep 17 00:00:00 2001 From: Toby Dylan Hocking Date: Tue, 5 Dec 2023 14:35:11 -0700 Subject: [PATCH 10/29] user-friendly --- GOVERNANCE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/GOVERNANCE.md b/GOVERNANCE.md index 6427d406a1..77630a75ea 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -14,7 +14,7 @@ The purpose of the project is to maintain the R data.table package, which is gui * Few (if any) Suggests/Enhances dependencies * Time & memory efficiency * Concise syntax (minimal redundancy in code) -* Stable code base (easy for users to upgrade to new data.table, and compatible with old R versions) +* Stable code base (strong preference for user-friendly back-compatibility with data.table itself and with old versions of R) * Comprehensive and accessible documentation and run-time signals (errors, warnings) * Clear error and warning messages From fa7b5b63c6ceeeb95ccc9d60e65a1d0a5fc2dcf2 Mon Sep 17 00:00:00 2001 From: Toby Dylan Hocking Date: Tue, 5 Dec 2023 14:36:54 -0700 Subject: [PATCH 11/29] rbindlist --- GOVERNANCE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/GOVERNANCE.md b/GOVERNANCE.md index 77630a75ea..f31f55470b 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -24,7 +24,7 @@ The current scope of package functionality includes: * Data manipulation and analysis * reshaping/pivoting * aggregation/summarizing - * subsetting rows + * subset/bind rows (rbindlist) * all sorts of joins * adding/updating/deleting columns * Sorting (forder) From 1b3c844ca67f59630832f86100b9baeb79dfac87 Mon Sep 17 00:00:00 2001 From: Toby Dylan Hocking Date: Fri, 8 Dec 2023 16:39:17 +0000 Subject: [PATCH 12/29] add history, other updates as suggested by reviewers --- GOVERNANCE.md | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/GOVERNANCE.md b/GOVERNANCE.md index f31f55470b..48ff20838f 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -16,7 +16,6 @@ The purpose of the project is to maintain the R data.table package, which is gui * Concise syntax (minimal redundancy in code) * Stable code base (strong preference for user-friendly back-compatibility with data.table itself and with old versions of R) * Comprehensive and accessible documentation and run-time signals (errors, warnings) -* Clear error and warning messages To prioritize developer time, we define what is in and out of current scope. Feature requests in issues and pull requests that are out of current scope should be closed immediately, because they are not the current priority. If someone wants to contribute code that is currently out of scope, they first have to make a pull request that changes the scope as defined below. @@ -31,9 +30,10 @@ The current scope of package functionality includes: * Reading/writing of data from/to flat (plain text) files like CSV Functionality that is out of current scope: -* plotting/graphics (like ggplot2) -* manipulating out-of-memory data, e.g. data stored on disk or remote SQL DB, (as opposed e.g. to sqldf / dbplyr) -* machine learning / modeling (like mlr3) +* Plotting/graphics (like ggplot2) +* Manipulating out-of-memory data, e.g. data stored on disk or remote SQL DB, (as opposed e.g. to sqldf / dbplyr) +* Machine learning / modeling (like mlr3) +* Reading/writing of data from/to binary files like parquet # Roles @@ -45,7 +45,7 @@ Functionality that is out of current scope: ## Project member * Definition: some one who has submitted at least one PR with code contributions, that has been merged into master. (spelling fixes in docs do not count) -* How to obtain this role: any user can become a member by submitting a PR, then having it reviewed and merged into master. Contributors who have written issues should be encouraged to submit their first PR to become a project member. +* How to obtain this role: any user/contributor can become a member by submitting a PR with code contributions, then having it reviewed and merged into master. Contributors who have written issues should be encouraged to submit their first PR to become a project member. Contributors can look at https://github.com/Rdatatable/data.table/labels/beginner-task for easy issues to work on. * How this role is recognized: Members are credited via role="ctb" in DESCRIPTION (so they appear in Author list on CRAN), and they are added to https://github.com/orgs/Rdatatable/teams/project-members so they can create new branches in the Rdatatable/data.table GitHub repo. They also appear on https://github.com/Rdatatable/data.table/graphs/contributors (Contributions to master, excluding merge commits). ## Reviewer @@ -124,3 +124,8 @@ data.table Version line in DESCRIPTION typically has the following meanings * z is even for CRAN releases, odd for GitHub development. * z=99 for master branch with new features (for example 1.14.99 or 1.15.99), which eventually becomes a regular CRAN release, with incremented y and z=0 (for example 1.15.0 or 1.16.0). * patch/hotfix development should occur on GitHub as z=odd (1.15.1) and release to CRAN as z=even (1.15.2). + +# Governance history + +Nov-Dec 2023: initial version drafted by Toby Dylan Hocking and +reviewed by Tyson Barrett, Jan Gorecki, Michael Chirico. From 9d502acaaf24a614d4fbbe5abf36123f3c7e5d59 Mon Sep 17 00:00:00 2001 From: Toby Dylan Hocking Date: Tue, 12 Dec 2023 09:14:47 -0700 Subject: [PATCH 13/29] sorting -> common functions Co-authored-by: Jan Gorecki --- GOVERNANCE.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/GOVERNANCE.md b/GOVERNANCE.md index 48ff20838f..3b410ce8c7 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -26,7 +26,11 @@ The current scope of package functionality includes: * subset/bind rows (rbindlist) * all sorts of joins * adding/updating/deleting columns -* Sorting (forder) +* high-performance common functions (`frank`, `fcase`, `fifelse`, `transpose`, `chmatch`, `fsort`, `forder`, `uniqueN`, ...) +* common convenience functions (`%like%`, `%notin%`, `timetaken`, `substitute2`, ...) +* ordered data functions (rleid, shift, coalesce, _locf_/_nocb_ nafill, rolling functions) +* date and time related classes and functions (`IDate`, `ITime`) +* technical functions (`address`, `tables`, `update_dev_pkg`) * Reading/writing of data from/to flat (plain text) files like CSV Functionality that is out of current scope: From 708c7ff0dccc97bae60757834687a34a502532e6 Mon Sep 17 00:00:00 2001 From: Toby Dylan Hocking Date: Tue, 12 Dec 2023 09:15:10 -0700 Subject: [PATCH 14/29] summarize by and grouping sets Co-authored-by: Jan Gorecki --- GOVERNANCE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/GOVERNANCE.md b/GOVERNANCE.md index 3b410ce8c7..efd01eb8b1 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -22,7 +22,7 @@ To prioritize developer time, we define what is in and out of current scope. Fea The current scope of package functionality includes: * Data manipulation and analysis * reshaping/pivoting - * aggregation/summarizing + * aggregation/summarizing (via `[,, by=...]` and _grouping sets_) * subset/bind rows (rbindlist) * all sorts of joins * adding/updating/deleting columns From 2ed51f58f8ac3662f4ba0782c58b064122958cf7 Mon Sep 17 00:00:00 2001 From: Toby Dylan Hocking Date: Tue, 12 Dec 2023 09:15:31 -0700 Subject: [PATCH 15/29] add set ops Co-authored-by: Jan Gorecki --- GOVERNANCE.md | 1 + 1 file changed, 1 insertion(+) diff --git a/GOVERNANCE.md b/GOVERNANCE.md index efd01eb8b1..d2f4925d4f 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -26,6 +26,7 @@ The current scope of package functionality includes: * subset/bind rows (rbindlist) * all sorts of joins * adding/updating/deleting columns + * set operations (union/rbind, intersection, difference) * high-performance common functions (`frank`, `fcase`, `fifelse`, `transpose`, `chmatch`, `fsort`, `forder`, `uniqueN`, ...) * common convenience functions (`%like%`, `%notin%`, `timetaken`, `substitute2`, ...) * ordered data functions (rleid, shift, coalesce, _locf_/_nocb_ nafill, rolling functions) From 505c34cda927b93f40da44c373ef3b3091f61826 Mon Sep 17 00:00:00 2001 From: Toby Dylan Hocking Date: Tue, 12 Dec 2023 09:15:54 -0700 Subject: [PATCH 16/29] subset -> filtering Co-authored-by: Jan Gorecki --- GOVERNANCE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/GOVERNANCE.md b/GOVERNANCE.md index d2f4925d4f..2aa1100d40 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -23,7 +23,7 @@ The current scope of package functionality includes: * Data manipulation and analysis * reshaping/pivoting * aggregation/summarizing (via `[,, by=...]` and _grouping sets_) - * subset/bind rows (rbindlist) + * filtering rows * all sorts of joins * adding/updating/deleting columns * set operations (union/rbind, intersection, difference) From 232eb057efc8b50daf240913d67a0d872646edc7 Mon Sep 17 00:00:00 2001 From: Toby Dylan Hocking Date: Tue, 12 Dec 2023 20:29:37 +0000 Subject: [PATCH 17/29] code -> substantial --- GOVERNANCE.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/GOVERNANCE.md b/GOVERNANCE.md index 2aa1100d40..690cbcc93e 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -49,8 +49,8 @@ Functionality that is out of current scope: ## Project member -* Definition: some one who has submitted at least one PR with code contributions, that has been merged into master. (spelling fixes in docs do not count) -* How to obtain this role: any user/contributor can become a member by submitting a PR with code contributions, then having it reviewed and merged into master. Contributors who have written issues should be encouraged to submit their first PR to become a project member. Contributors can look at https://github.com/Rdatatable/data.table/labels/beginner-task for easy issues to work on. +* Definition: some one who has submitted at least one PR with substantial contributions, that has been merged into master. PRs improving documentation are welcome, and substantial contributions to the docs should count toward membership, but minor contributions such as spelling fixes do not count toward membership. +* How to obtain this role: any user/contributor can become a member by submitting a PR with substantial contributions, then having it reviewed and merged into master. Contributors who have written issues should be encouraged to submit their first PR to become a project member. Contributors can look at https://github.com/Rdatatable/data.table/labels/beginner-task for easy issues to work on. * How this role is recognized: Members are credited via role="ctb" in DESCRIPTION (so they appear in Author list on CRAN), and they are added to https://github.com/orgs/Rdatatable/teams/project-members so they can create new branches in the Rdatatable/data.table GitHub repo. They also appear on https://github.com/Rdatatable/data.table/graphs/contributors (Contributions to master, excluding merge commits). ## Reviewer From 83ef82c6f1a5cec282fd3960647fa89735cafde8 Mon Sep 17 00:00:00 2001 From: Toby Dylan Hocking Date: Tue, 12 Dec 2023 13:45:01 -0700 Subject: [PATCH 18/29] backquotes Co-authored-by: Benjamin Schwendinger <52290390+ben-schwen@users.noreply.github.com> --- GOVERNANCE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/GOVERNANCE.md b/GOVERNANCE.md index 690cbcc93e..b7c7a28967 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -29,7 +29,7 @@ The current scope of package functionality includes: * set operations (union/rbind, intersection, difference) * high-performance common functions (`frank`, `fcase`, `fifelse`, `transpose`, `chmatch`, `fsort`, `forder`, `uniqueN`, ...) * common convenience functions (`%like%`, `%notin%`, `timetaken`, `substitute2`, ...) -* ordered data functions (rleid, shift, coalesce, _locf_/_nocb_ nafill, rolling functions) +* ordered data functions (`rleid`, `shift`, `fcoalesce`, _locf_/_nocb_ `nafill`, rolling functions) * date and time related classes and functions (`IDate`, `ITime`) * technical functions (`address`, `tables`, `update_dev_pkg`) * Reading/writing of data from/to flat (plain text) files like CSV From 5f9f3da5e719bf0e7660600873fb9819546953bd Mon Sep 17 00:00:00 2001 From: Toby Dylan Hocking Date: Tue, 12 Dec 2023 13:45:40 -0700 Subject: [PATCH 19/29] z version includes devel Co-authored-by: Benjamin Schwendinger <52290390+ben-schwen@users.noreply.github.com> --- GOVERNANCE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/GOVERNANCE.md b/GOVERNANCE.md index b7c7a28967..d41e515feb 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -124,7 +124,7 @@ This Code of Conduct is adapted from Tidyverse code of conduct. data.table Version line in DESCRIPTION typically has the following meanings -* x.y.z where x=major, y=minor, z=patch/hotfix. +* x.y.z where x=major, y=minor, z=patch/hotfix/devel. * x should be incremented only for major backwards-incompatible changes. * z is even for CRAN releases, odd for GitHub development. * z=99 for master branch with new features (for example 1.14.99 or 1.15.99), which eventually becomes a regular CRAN release, with incremented y and z=0 (for example 1.15.0 or 1.16.0). From 279e6ed9fe88a041c8afe988e2d7aa48873dc2ed Mon Sep 17 00:00:00 2001 From: Toby Dylan Hocking Date: Tue, 12 Dec 2023 20:46:00 +0000 Subject: [PATCH 20/29] PR 3 bullets --- GOVERNANCE.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/GOVERNANCE.md b/GOVERNANCE.md index 690cbcc93e..e8ac82a28a 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -92,7 +92,10 @@ Most decisions in the project happen by Consensus, which means that no active pe ## Pull Requests -A pull request can be merged as long as there is one approving review (ideally from a reviewer of the affected files, different person from author of PR), and Consensus from active Reviewers and Committers. +A pull request can be merged by any committer, if there is one approving review, and Consensus from active Reviewers and Committers. +* approving review must come from different person from author of PR. +* approving review ideally comes from a reviewer of the affected files. +* approving review can be same as committer who merges the PR. ## CRAN updates @@ -108,7 +111,7 @@ and ask for review). # Code of conduct -As contributors and maintainers of this project, we pledge to respect all people who contribute through reporting issues, posting feature requests, updating documentation, submitting pull requests or patches, and other activities. +As contributors of this project, we pledge to respect all people who contribute through reporting issues, posting feature requests, updating documentation, submitting pull requests or patches, and other activities. We are committed to making participation in this project a harassment-free experience for everyone, regardless of level of experience, gender, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, religion, etc. From eb084ac8829a91a8567e5e3d32bb7168d2f66248 Mon Sep 17 00:00:00 2001 From: Toby Dylan Hocking Date: Tue, 12 Dec 2023 22:41:16 +0000 Subject: [PATCH 21/29] ben reviewed --- GOVERNANCE.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/GOVERNANCE.md b/GOVERNANCE.md index 9511b49da5..e8a8301db0 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -74,8 +74,7 @@ Functionality that is out of current scope: ## Other roles -Each of the below roles is important, and a wiki page should be -created to explain (1) details of responsibilities, (2) who to contact +Each of the below roles is important, , (2) who to contact to obtain this role. * Triage manager (to decide which issues/PRs to include in next releases), no special permissions. @@ -136,4 +135,5 @@ data.table Version line in DESCRIPTION typically has the following meanings # Governance history Nov-Dec 2023: initial version drafted by Toby Dylan Hocking and -reviewed by Tyson Barrett, Jan Gorecki, Michael Chirico. +reviewed by Tyson Barrett, Jan Gorecki, Michael Chirico, Benjamin +Schwendinger. From 7ff67f4b5174d703837257f68c6ababb1f94d038 Mon Sep 17 00:00:00 2001 From: Toby Dylan Hocking Date: Wed, 13 Dec 2023 12:29:34 -0700 Subject: [PATCH 22/29] rm modeling Co-authored-by: Jan Gorecki --- GOVERNANCE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/GOVERNANCE.md b/GOVERNANCE.md index e8a8301db0..7180d15c2f 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -37,7 +37,7 @@ The current scope of package functionality includes: Functionality that is out of current scope: * Plotting/graphics (like ggplot2) * Manipulating out-of-memory data, e.g. data stored on disk or remote SQL DB, (as opposed e.g. to sqldf / dbplyr) -* Machine learning / modeling (like mlr3) +* Machine learning (like mlr3) * Reading/writing of data from/to binary files like parquet # Roles From 05e16e750e43bff319ce96ed90b20e6388322066 Mon Sep 17 00:00:00 2001 From: Toby Dylan Hocking Date: Thu, 14 Dec 2023 09:03:00 -0700 Subject: [PATCH 23/29] efficiency first Co-authored-by: Michael Chirico --- GOVERNANCE.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/GOVERNANCE.md b/GOVERNANCE.md index 7180d15c2f..06d9836ecb 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -10,8 +10,13 @@ The purpose of this document is to define how people related to the project work The purpose of the project is to maintain the R data.table package, which is guided by the following principles: +* Time & memory efficiency +* Concise syntax (minimal redundancy in code) * No external Imports/LinkingTo/Depends dependencies (external meaning those not maintained by the project) * Few (if any) Suggests/Enhances dependencies +* Stable code base (strong preference for user-friendly back-compatibility with data.table itself and with old versions of R) +* Comprehensive and accessible documentation and run-time signals (errors, warnings) +* Few (if any) Suggests/Enhances dependencies * Time & memory efficiency * Concise syntax (minimal redundancy in code) * Stable code base (strong preference for user-friendly back-compatibility with data.table itself and with old versions of R) From 85fb6dc98e4269193d84f895d97c869572c4df17 Mon Sep 17 00:00:00 2001 From: Toby Dylan Hocking Date: Thu, 14 Dec 2023 09:03:19 -0700 Subject: [PATCH 24/29] Data->data Co-authored-by: Michael Chirico --- GOVERNANCE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/GOVERNANCE.md b/GOVERNANCE.md index 06d9836ecb..9a502ff5d3 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -25,7 +25,7 @@ The purpose of the project is to maintain the R data.table package, which is gui To prioritize developer time, we define what is in and out of current scope. Feature requests in issues and pull requests that are out of current scope should be closed immediately, because they are not the current priority. If someone wants to contribute code that is currently out of scope, they first have to make a pull request that changes the scope as defined below. The current scope of package functionality includes: -* Data manipulation and analysis +* data manipulation and analysis * reshaping/pivoting * aggregation/summarizing (via `[,, by=...]` and _grouping sets_) * filtering rows From 57ccd738dfab0939d3ac8e132d6aedf144e19c9b Mon Sep 17 00:00:00 2001 From: Toby Dylan Hocking Date: Thu, 14 Dec 2023 09:05:36 -0700 Subject: [PATCH 25/29] remove other roles --- GOVERNANCE.md | 11 ----------- 1 file changed, 11 deletions(-) diff --git a/GOVERNANCE.md b/GOVERNANCE.md index 9a502ff5d3..3aec18a853 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -77,17 +77,6 @@ Functionality that is out of current scope: * How to obtain this role: (1) merge into master a PR adding role="cre" to DESCRIPTION, and (2) submit updated package to CRAN (previous CRAN maintainer will have to confirm change by email to CRAN). * How this role is recognized: credited via role="cre" in DESCRIPTION, so they appear as Maintainer on CRAN. -## Other roles - -Each of the below roles is important, , (2) who to contact -to obtain this role. - -* Triage manager (to decide which issues/PRs to include in next releases), no special permissions. -* Translation manager (to communicate with translators, and source code changes related to i18n, for example making sure messages are translation-ready, updating the .pot template), no special permissions (updates should happen in a pull request as usual). -* Performance testing manager (to prevent performance regressions), no special permissions (updates should happen in a pull request as usual). -* Continuous integration/deployment manager (to maintain script which run tests and build binaries of development branches for user testing before release), no special permissions (updates should happen in a pull request as usual). -* Reverse dependency manager (to ensure compatibility with other CRAN packages), need access to a computer that can run revdep checks in parallel. - # Decision-making processes ## Definition of Consensus From d14db6bf86a89db3d1b59e95e320039ee5ec5b1a Mon Sep 17 00:00:00 2001 From: Toby Dylan Hocking Date: Thu, 14 Dec 2023 09:06:16 -0700 Subject: [PATCH 26/29] different person -> someone other Co-authored-by: Michael Chirico --- GOVERNANCE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/GOVERNANCE.md b/GOVERNANCE.md index 3aec18a853..92408e6173 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -86,7 +86,7 @@ Most decisions in the project happen by Consensus, which means that no active pe ## Pull Requests A pull request can be merged by any committer, if there is one approving review, and Consensus from active Reviewers and Committers. -* approving review must come from different person from author of PR. +* approving review must come from someone other than the author of the PR. * approving review ideally comes from a reviewer of the affected files. * approving review can be same as committer who merges the PR. From c7ebe5a2f6cf21e706443510e9c836b8a05cb2f5 Mon Sep 17 00:00:00 2001 From: Toby Dylan Hocking Date: Thu, 14 Dec 2023 09:06:53 -0700 Subject: [PATCH 27/29] same as -> often will be Co-authored-by: Michael Chirico --- GOVERNANCE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/GOVERNANCE.md b/GOVERNANCE.md index 92408e6173..66feef24c7 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -88,7 +88,7 @@ Most decisions in the project happen by Consensus, which means that no active pe A pull request can be merged by any committer, if there is one approving review, and Consensus from active Reviewers and Committers. * approving review must come from someone other than the author of the PR. * approving review ideally comes from a reviewer of the affected files. -* approving review can be same as committer who merges the PR. +* approving review can and often will be by the committer who merges the PR. ## CRAN updates From d9a9c30e9049a3b395ed33bc856c82d7c27c5856 Mon Sep 17 00:00:00 2001 From: Toby Dylan Hocking Date: Thu, 14 Dec 2023 09:07:35 -0700 Subject: [PATCH 28/29] hotfix/patch should not include new features Co-authored-by: Michael Chirico --- GOVERNANCE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/GOVERNANCE.md b/GOVERNANCE.md index 66feef24c7..1852cac51d 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -93,7 +93,7 @@ A pull request can be merged by any committer, if there is one approving review, ## CRAN updates * Regular CRAN releases should ideally occur twice per year, and can include new features. -* A hotfix/patch CRAN release should occur when CRAN asks for one, at which time the CRAN maintainer should post an issue on github, and ask others to help fix/prepare the release. +* A hotfix/patch CRAN release should occur when CRAN asks for one, at which time the CRAN maintainer should post an issue on github, and ask others to help fix/prepare the release. It should not include new features. * Both kinds of releases should be discussed in an issue, and the release should happen only if there is Consensus among active Reviewers and Committers. * It is the responsibility of the CRAN maintainer to ensure quality prior to release. This includes CRAN checks, unit tests, performance tests, etc, and these tasks can be delegated to others. From c6a282ee2d7d84e702840098f56bb422d86744df Mon Sep 17 00:00:00 2001 From: Toby Dylan Hocking Date: Thu, 14 Dec 2023 09:49:12 -0700 Subject: [PATCH 29/29] rm repetition --- GOVERNANCE.md | 5 ----- 1 file changed, 5 deletions(-) diff --git a/GOVERNANCE.md b/GOVERNANCE.md index 3aec18a853..c582cea05f 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -16,11 +16,6 @@ The purpose of the project is to maintain the R data.table package, which is gui * Few (if any) Suggests/Enhances dependencies * Stable code base (strong preference for user-friendly back-compatibility with data.table itself and with old versions of R) * Comprehensive and accessible documentation and run-time signals (errors, warnings) -* Few (if any) Suggests/Enhances dependencies -* Time & memory efficiency -* Concise syntax (minimal redundancy in code) -* Stable code base (strong preference for user-friendly back-compatibility with data.table itself and with old versions of R) -* Comprehensive and accessible documentation and run-time signals (errors, warnings) To prioritize developer time, we define what is in and out of current scope. Feature requests in issues and pull requests that are out of current scope should be closed immediately, because they are not the current priority. If someone wants to contribute code that is currently out of scope, they first have to make a pull request that changes the scope as defined below.