-
Notifications
You must be signed in to change notification settings - Fork 48
feat(DOC-1970): add missing rpk connect reference pages #1608
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
mfernest
wants to merge
9
commits into
main
Choose a base branch
from
fix/doc-1970-rpk-connect-reference
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
8804181
feat(DOC-1970): add missing rpk connect reference pages
mfernest ade95c3
review fixes: chroot flags, dry-run usage, plugin-init example (#1616)
mmatczuk c185e88
style(DOC-1970): rename plugin-init example from ./my-plugin to examp…
mfernest 7624950
fix(DOC-1970): address kbatuigas review feedback on rpk connect refer…
mfernest c2be6b3
Merge remote-tracking branch 'origin/main' into fix/doc-1970-rpk-conn…
mfernest 161cc8e
fix(DOC-1970): use empty xref brackets for auto-pulled titles
mfernest 1bb1847
Merge remote-tracking branch 'origin/main' into fix/doc-1970-rpk-conn…
mfernest b66e845
fix(DOC-1970): address review comments for rpk connect reference pages
JakeSCahill 2f3f9de
fix(DOC-1970): apply doc team standards to rpk connect reference pages
JakeSCahill File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
41 changes: 41 additions & 0 deletions
41
modules/reference/pages/rpk/rpk-connect/rpk-connect-agent-init.adoc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| = rpk connect agent init | ||
| :description: Initialize a Redpanda Connect agent. | ||
| :page-topic-type: reference | ||
| :learning-objective-1: Find the syntax for initializing an agent template | ||
| :learning-objective-2: Identify available initialization options | ||
|
|
||
| [IMPORTANT] | ||
| ==== | ||
| This command is experimental and subject to change. | ||
| ==== | ||
|
|
||
| Initialize a template for building a Redpanda Connect agent. | ||
|
|
||
| Use this reference to: | ||
|
|
||
| * [ ] {learning-objective-1} | ||
| * [ ] {learning-objective-2} | ||
|
|
||
| == Usage | ||
|
|
||
| rpk connect agent init [OPTIONS] | ||
|
|
||
| == Example | ||
|
|
||
| [,bash] | ||
| ---- | ||
| rpk connect agent init ./repo | ||
| ---- | ||
|
|
||
| == Flags | ||
|
|
||
| [cols="1m,2a"] | ||
| |=== | ||
| | Option | Description | ||
|
|
||
| | --name | ||
| | The name of the agent. | ||
|
|
||
| | --help, -h | ||
| | Show help for the command. | ||
| |=== | ||
59 changes: 59 additions & 0 deletions
59
modules/reference/pages/rpk/rpk-connect/rpk-connect-agent-run.adoc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,59 @@ | ||
| = rpk connect agent run | ||
| :description: Execute a Redpanda Connect agent as part of a pipeline with MCP tool access. | ||
| :page-topic-type: reference | ||
| :learning-objective-1: Find the syntax for executing agents | ||
| :learning-objective-2: Identify secret management options | ||
| :learning-objective-3: Look up security-related flags | ||
|
|
||
| [IMPORTANT] | ||
| ==== | ||
| This command is experimental and subject to change. | ||
| ==== | ||
|
|
||
| Execute a Redpanda Connect agent as part of a pipeline that has access to tools via the Model Context Protocol (MCP). The command reads resource definitions from the `mcp` subdirectory and exposes them as tools. It then runs the agent defined in `redpanda_agents.yaml` along with any Python agent modules. | ||
|
|
||
| Use this reference to: | ||
|
|
||
| * [ ] {learning-objective-1} | ||
| * [ ] {learning-objective-2} | ||
| * [ ] {learning-objective-3} | ||
|
|
||
| == Usage | ||
|
|
||
| rpk connect agent run [OPTIONS] | ||
|
|
||
| == Example | ||
|
|
||
| [,bash] | ||
| ---- | ||
| rpk connect agent run ./repo | ||
| ---- | ||
|
|
||
| To disable all secret lookups: | ||
|
|
||
| [,bash] | ||
| ---- | ||
| rpk connect agent run --secrets none: ./repo | ||
| ---- | ||
|
|
||
| == Flags | ||
|
|
||
| [cols="1m,2a"] | ||
| |=== | ||
| | Option | Description | ||
|
|
||
| | --secrets | ||
| | Attempt to load secrets from a provided URN. If more than one entry is specified, they are attempted in order until a value is found. Environment variable lookups are specified with the URN `env:`, which by default is the only entry. To disable all secret lookups, specify a single entry of `none:` (default: `env:`). | ||
|
|
||
| | --redpanda-license | ||
| | Provide an explicit Redpanda license, which enables enterprise functionality. By default, licenses found at the path `/etc/redpanda/redpanda.license` are applied. | ||
|
|
||
| | --help, -h | ||
| | Show help for the command. | ||
|
|
||
| | --chroot | ||
| | (Linux only) Chroot into the provided directory after parsing configuration. Common `/etc/` files are copied to the chroot directory, and the directory is made read-only. | ||
|
|
||
| | --chroot-passthrough | ||
| | (Linux only) Specify additional files to be copied into the chroot directory. Can be specified multiple times. Only valid when `--chroot` is used. | ||
| |=== |
37 changes: 37 additions & 0 deletions
37
modules/reference/pages/rpk/rpk-connect/rpk-connect-agent.adoc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| = rpk connect agent | ||
| :description: Redpanda Connect agent commands. | ||
| :page-topic-type: reference | ||
| :learning-objective-1: Look up available agent subcommands | ||
| :learning-objective-2: Identify agent command options | ||
|
|
||
| [IMPORTANT] | ||
| ==== | ||
| This command is experimental and subject to change. | ||
| ==== | ||
|
|
||
| Run and initialize Redpanda Connect agents. Agents are pipelines that have access to tools via the Model Context Protocol (MCP) and can interact with AI models. | ||
|
|
||
| Use this reference to: | ||
|
|
||
| * [ ] {learning-objective-1} | ||
| * [ ] {learning-objective-2} | ||
|
|
||
| == Usage | ||
|
|
||
| rpk connect agent [command] | ||
|
|
||
| == Subcommands | ||
|
|
||
| * xref:reference:rpk/rpk-connect/rpk-connect-agent-init.adoc[init]: Initialize a Redpanda Connect agent. | ||
| * xref:reference:rpk/rpk-connect/rpk-connect-agent-run.adoc[run]: Execute a Redpanda Connect agent as part of a pipeline with MCP tool access. | ||
| * help, h: Show a list of commands or help for one command. | ||
|
|
||
| == Flags | ||
|
|
||
| [cols="1m,2a"] | ||
| |=== | ||
| | Option | Description | ||
|
|
||
| | --help, -h | ||
| | Show help for the command. | ||
| |=== |
52 changes: 52 additions & 0 deletions
52
modules/reference/pages/rpk/rpk-connect/rpk-connect-dry-run.adoc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| = rpk connect dry-run | ||
| :description: Parse Redpanda Connect configs and test the connections of each plugin. | ||
| :page-topic-type: reference | ||
| :learning-objective-1: Find the syntax for testing pipeline configurations | ||
| :learning-objective-2: Look up connection testing options | ||
|
|
||
| Parse Redpanda Connect configs and test the connections of each plugin. Exits with a status code of `1` if the command detects any connection errors. | ||
|
|
||
| Use this reference to: | ||
|
|
||
| * [ ] {learning-objective-1} | ||
| * [ ] {learning-objective-2} | ||
|
|
||
| == Usage | ||
|
|
||
| rpk connect dry-run [OPTIONS] [files...] | ||
|
|
||
| == Example | ||
|
|
||
| [,bash] | ||
| ---- | ||
| rpk connect dry-run ./pipeline.yaml | ||
| ---- | ||
|
|
||
| To disable all secret lookups: | ||
|
|
||
| [,bash] | ||
| ---- | ||
| rpk connect dry-run --secrets none: ./pipeline.yaml | ||
| ---- | ||
|
|
||
| == Flags | ||
|
|
||
| [cols="1m,2a"] | ||
| |=== | ||
| | Option | Description | ||
|
|
||
| | --verbose | ||
| | Print the lint result for each target file (default: false). | ||
|
|
||
| | --secrets | ||
| | Attempt to load secrets from a provided URN. If more than one entry is specified, they are attempted in order until a value is found. Environment variable lookups are specified with the URN `env:`, which by default is the only entry. To disable all secret lookups, specify a single entry of `none:` (default: `env:`). | ||
|
|
||
| | --env-file, -e | ||
| | Import environment variables from a dotenv file. | ||
|
|
||
| | --redpanda-license | ||
| | Provide an explicit Redpanda license, which enables enterprise functionality. By default, licenses found at the path `/etc/redpanda/redpanda.license` are applied. | ||
|
|
||
| | --help, -h | ||
| | Show help for the command. | ||
| |=== |
44 changes: 44 additions & 0 deletions
44
modules/reference/pages/rpk/rpk-connect/rpk-connect-plugin-init.adoc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| = rpk connect plugin init | ||
| :description: Create the boilerplate for a custom Redpanda Connect plugin. | ||
| :page-topic-type: reference | ||
| :learning-objective-1: Find the syntax for creating custom plugins | ||
| :learning-objective-2: Identify supported languages and component types | ||
|
|
||
| [IMPORTANT] | ||
| ==== | ||
| This command is experimental and subject to change. | ||
| ==== | ||
|
|
||
| Generate a project on the local filesystem that can be used as a starting point for building a custom component for Redpanda Connect. The command overwrites existing files in the specified directory. | ||
|
|
||
| Use this reference to: | ||
|
|
||
| * [ ] {learning-objective-1} | ||
| * [ ] {learning-objective-2} | ||
|
|
||
| == Usage | ||
|
|
||
| rpk connect plugin init [OPTIONS] | ||
|
|
||
| == Example | ||
|
|
||
| [,bash] | ||
| ---- | ||
| rpk connect plugin init example-plugin | ||
| ---- | ||
|
|
||
| == Flags | ||
|
|
||
| [cols="1m,2a"] | ||
| |=== | ||
| | Option | Description | ||
|
|
||
| | --language, --lang | ||
| | The programming language for the plugin. Supported languages are `golang` and `python` (default: `python`). | ||
|
|
||
| | --component | ||
| | The type of component to generate. Supported components are `input`, `output`, and `processor` (default: `processor`). | ||
|
|
||
| | --help, -h | ||
| | Show help for the command. | ||
| |=== |
36 changes: 36 additions & 0 deletions
36
modules/reference/pages/rpk/rpk-connect/rpk-connect-plugin.adoc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| = rpk connect plugin | ||
| :description: Plugin management commands for Redpanda Connect. | ||
| :page-topic-type: reference | ||
| :learning-objective-1: Look up available plugin management commands | ||
| :learning-objective-2: Identify plugin command options | ||
|
|
||
| [IMPORTANT] | ||
| ==== | ||
| This command is experimental and subject to change. | ||
| ==== | ||
|
|
||
| Manage custom Redpanda Connect plugins. Use these commands to scaffold and build custom components. | ||
|
|
||
| Use this reference to: | ||
|
|
||
| * [ ] {learning-objective-1} | ||
| * [ ] {learning-objective-2} | ||
|
|
||
| == Usage | ||
|
|
||
| rpk connect plugin [command] | ||
|
|
||
| == Subcommands | ||
|
|
||
| * xref:reference:rpk/rpk-connect/rpk-connect-plugin-init.adoc[init]: Create the boilerplate for a custom component plugin. | ||
| * help, h: Show a list of commands or help for one command. | ||
|
|
||
| == Flags | ||
|
|
||
| [cols="1m,2a"] | ||
| |=== | ||
| | Option | Description | ||
|
|
||
| | --help, -h | ||
| | Show help for the command. | ||
| |=== |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.