Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -474,6 +474,15 @@
***** xref:reference:rpk/rpk-security/rpk-security-user-delete.adoc[]
***** xref:reference:rpk/rpk-security/rpk-security-user-update.adoc[]
***** xref:reference:rpk/rpk-security/rpk-security-user-list.adoc[]
*** xref:reference:rpk/rpk-shadow/rpk-shadow.adoc[]
**** xref:reference:rpk/rpk-shadow/rpk-shadow-config-generate.adoc[]
**** xref:reference:rpk/rpk-shadow/rpk-shadow-create.adoc[]
**** xref:reference:rpk/rpk-shadow/rpk-shadow-delete.adoc[]
**** xref:reference:rpk/rpk-shadow/rpk-shadow-describe.adoc[]
**** xref:reference:rpk/rpk-shadow/rpk-shadow-failover.adoc[]
**** xref:reference:rpk/rpk-shadow/rpk-shadow-list.adoc[]
**** xref:reference:rpk/rpk-shadow/rpk-shadow-status.adoc[]
**** xref:reference:rpk/rpk-shadow/rpk-shadow-update.adoc[]
*** xref:reference:rpk/rpk-topic/rpk-topic.adoc[]
**** xref:reference:rpk/rpk-topic/rpk-topic-add-partitions.adoc[]
**** xref:reference:rpk/rpk-topic/rpk-topic-alter-config.adoc[]
Expand Down
20 changes: 17 additions & 3 deletions modules/get-started/pages/release-notes/redpanda.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ Redpanda v25.3 introduces xref:deploy:redpanda/manual/disaster-recovery/shadowin

The shadow cluster operates in read-only mode while continuously receiving updates from the source cluster. During a disaster, you can failover individual topics or an entire shadow link to make resources fully writable for production traffic. See xref:deploy:redpanda/manual/disaster-recovery/shadowing/failover-runbook.adoc[] for emergency procedures.


== Connected client monitoring

You can view details about Kafka client connections using `rpk` or the Admin API ListKafkaConnections endpoint. This allows you to view detailed information about active client connections on a cluster, and identify and troubleshoot problematic clients. For more information, see the xref:manage:cluster-maintenance/manage-throughput.adoc#view-connected-client-details[connected client details] example in the Manage Throughput guide.
Expand Down Expand Up @@ -49,10 +50,23 @@ You can now generate a security report for your Redpanda cluster using the link:

Redpanda v25.3 implements topic identifiers using 16 byte UUIDs as proposed in https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers[KIP-516^].

== New rpk commands
== New commands

Redpanda v25.3 introduces new xref:reference:rpk/rpk-shadow/rpk-shadow.adoc[`rpk shadow`] commands for managing Redpanda shadow links:

* xref:reference:rpk/rpk-shadow/rpk-shadow-config-generate.adoc[`rpk shadow config generate`] - Generate configuration files for shadow links
* xref:reference:rpk/rpk-shadow/rpk-shadow-create.adoc[`rpk shadow create`] - Create new shadow links
* xref:reference:rpk/rpk-shadow/rpk-shadow-update.adoc[`rpk shadow update`] - Update existing shadow link configurations
* xref:reference:rpk/rpk-shadow/rpk-shadow-list.adoc[`rpk shadow list`] - List all shadow links
* xref:reference:rpk/rpk-shadow/rpk-shadow-describe.adoc[`rpk shadow describe`] - View shadow link configuration details
* xref:reference:rpk/rpk-shadow/rpk-shadow-status.adoc[`rpk shadow status`] - Monitor shadow link replication status
* xref:reference:rpk/rpk-shadow/rpk-shadow-failover.adoc[`rpk shadow failover`] - Perform emergency failover operations
* xref:reference:rpk/rpk-shadow/rpk-shadow-delete.adoc[`rpk shadow delete`] - Delete shadow links

In addition, the following commands have been added:

* **xref:reference:rpk/rpk-cluster/rpk-cluster-connections.adoc[rpk cluster connections]** - Monitor cluster connections and client statistics.
* **xref:reference:rpk/rpk-redpanda/rpk-redpanda-config-print.adoc[rpk redpanda config print]** - Display node configuration.
* xref:reference:rpk/rpk-cluster/rpk-cluster-connections.adoc[`rpk cluster connections`] - Monitor cluster connections and client statistics.
* xref:reference:rpk/rpk-redpanda/rpk-redpanda-config-print.adoc[`rpk redpanda config print`] - Display node configuration.

== Deprecations

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,8 @@ rpk shadow describe <shadow-link-name>
rpk shadow status <shadow-link-name>
----

For detailed command options, see xref:reference:rpk/rpk-shadow/rpk-shadow-list.adoc[`rpk shadow list`], xref:reference:rpk/rpk-shadow/rpk-shadow-describe.adoc[`rpk shadow describe`], and xref:reference:rpk/rpk-shadow/rpk-shadow-status.adoc[`rpk shadow status`].

Verify that the following conditions exist before proceeding with failover:

* Shadow link state should be `ACTIVE`.
Expand All @@ -86,7 +88,7 @@ Verify that the following conditions exist before proceeding with failover:

**Understanding replication lag:**

Use `rpk shadow status <shadow-link-name>` to check lag, which shows the message count difference between source and shadow partitions:
Use xref:reference:rpk/rpk-shadow/rpk-shadow-status.adoc[`rpk shadow status`] to check lag, which shows the message count difference between source and shadow partitions:

* **Acceptable lag examples**: 0-1000 messages for low-throughput topics, 0-10000 messages for high-throughput topics
* **Concerning lag examples**: Growing lag over 50,000 messages, or lag that continuously increases without recovering
Expand All @@ -106,7 +108,7 @@ rpk shadow status <shadow-link-name> > failover-status-$(date +%Y%m%d-%H%M%S).lo
// TODO: Verify this output format by running actual rpk shadow status command
Example output showing healthy replication before failover:
----
Shadow Link: <shadow-link-name>
shadow link: <shadow-link-name>

Overview:
NAME <shadow-link-name>
Expand Down Expand Up @@ -139,6 +141,8 @@ A complete cluster failover is appropriate If you observe that the source cluste
rpk shadow failover <shadow-link-name> --all
----

For detailed command options, see xref:reference:rpk/rpk-shadow/rpk-shadow-failover.adoc[`rpk shadow failover`].

For selective topic failover (when only specific services are affected):

[,bash]
Expand All @@ -165,7 +169,7 @@ rpk shadow status <shadow-link-name> --print-topic
// TODO: Verify this output format by running actual rpk shadow status command during failover
Example output during successful failover:
----
Shadow Link: <shadow-link-name>
shadow link: <shadow-link-name>

Overview:
NAME <shadow-link-name>
Expand Down Expand Up @@ -217,6 +221,8 @@ After all applications are running normally:
rpk shadow delete <shadow-link-name>
----

For detailed command options, see xref:reference:rpk/rpk-shadow/rpk-shadow-delete.adoc[`rpk shadow delete`].

Document the time of failover initiation and completion, applications affected and recovery times, data loss estimates based on replication lag, and issues encountered during failover.

== Troubleshoot common issues
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ To fail over a specific shadow topic while leaving other topics in the shadow li
rpk shadow failover <shadow-link-name> --topic <topic-name>
----

Use this approach when you need to selectively failover specific workloads or when testing failover procedures.
Use this approach when you need to selectively failover specific workloads or when testing failover procedures. For detailed command options, see xref:reference:rpk/rpk-shadow/rpk-shadow-failover.adoc[`rpk shadow failover`].

=== Complete shadow link failover (cluster failover)

Expand Down Expand Up @@ -88,7 +88,7 @@ Monitor failover progress using the status command:
rpk shadow status <shadow-link-name>
----

The output shows individual topic states and any issues encountered during the failover process.
The output shows individual topic states and any issues encountered during the failover process. For detailed command options, see xref:reference:rpk/rpk-shadow/rpk-shadow-status.adoc[`rpk shadow status`].

**Task states during monitoring:**

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ View shadow link configuration details:
rpk shadow describe <my-disaster-recovery-link>
----

For detailed command options, see xref:reference:rpk/rpk-shadow/rpk-shadow-list.adoc[`rpk shadow list`] and xref:reference:rpk/rpk-shadow/rpk-shadow-describe.adoc[`rpk shadow describe`].

This command shows the complete configuration of the shadow link, including connection settings, filters, and synchronization options.

Check your shadow link status to ensure proper operation:
Expand All @@ -44,7 +46,7 @@ rpk shadow status <shadow-link-name>
rpk shadow status <shadow-link-name>
----

For troubleshooting specific issues, you can use command options to show individual status sections. See the rpk reference for available status options.
For troubleshooting specific issues, you can use command options to show individual status sections. See xref:reference:rpk/rpk-shadow/rpk-shadow-status.adoc[`rpk shadow status`] for available status options.

The status output includes:

Expand Down
6 changes: 5 additions & 1 deletion modules/manage/pages/disaster-recovery/shadowing/setup.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -276,7 +276,7 @@ Use `rpk` to generate a sample configuration file with common filter patterns:
rpk shadow config generate --output shadow-config.yaml
----

This creates a template configuration file that you can customize for your environment.
This creates a template configuration file that you can customize for your environment. For detailed command options, see xref:reference:rpk/rpk-shadow/rpk-shadow-config-generate.adoc[`rpk shadow config generate`].

=== Networking

Expand Down Expand Up @@ -374,6 +374,8 @@ You can establish a shadow link using either xref:get-started:rpk/index.adoc[`rp
rpk shadow create --config-file shadow-config.yaml
----

For detailed command options, see xref:reference:rpk/rpk-shadow/rpk-shadow-create.adoc[`rpk shadow create`].

=== Update an existing shadow link

If you need to modify a shadow link configuration after creation, use the update command:
Expand All @@ -383,6 +385,8 @@ If you need to modify a shadow link configuration after creation, use the update
rpk shadow update <shadow-link-name>
----

For detailed command options, see xref:reference:rpk/rpk-shadow/rpk-shadow-update.adoc[`rpk shadow update`].

This opens your default editor to modify the shadow link configuration. Only changed fields are updated on the server. The shadow link name cannot be changed - you must delete and recreate the link to rename it.

[TIP]
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
= rpk shadow config generate
// tag::single-source[]
Comment on lines +1 to +2
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
= rpk shadow config generate
// tag::single-source[]
// tag::rpk_shadow_config_generate[]
= rpk shadow config generate


Generate a configuration file for creating a shadow link.

By default, this command creates a sample configuration file with placeholder values that you customize for your environment.

Use the `--print-template` flag to generate a configuration template with detailed field documentations.

By default, this command prints the configuration to standard output. Use the `--output` flag to save the configuration to a file.

After you generate the configuration file, update the placeholder values with your actual connection details and settings. Then use xref:reference:rpk/rpk-shadow/rpk-shadow-create.adoc[`rpk shadow create`] to create the shadow link.

== Usage

[,bash]
----
rpk shadow config generate [flags]
----

== Examples

Generate a sample configuration and print it to standard output:
[,bash]
----
rpk shadow config generate
----

Generate a configuration template with all the field documentation:

[,bash]
----
rpk shadow config generate --print-template
----

Save the sample configuration to a file:

[,bash]
----
rpk shadow config generate -o shadow-link.yaml
----

Save the template with documentation to a file:

[,bash]
----
rpk shadow config generate --print-template -o shadow-link.yaml
----

== Flags

[cols="1m,1a,2a"]
|===
|*Value* |*Type* |*Description*

|-o, --output |string |File path identifying where to save the generated configuration file. If not specified, prints to standard output.

|--print-template |- |Generate a configuration template with field documentation instead of a sample configuration.

|-h, --help |- |Help for generate.

|--config |string |Redpanda or `rpk` config file; default search paths are `/var/lib/redpanda/.config/rpk/rpk.yaml`, `$PWD/redpanda.yaml`, and `/etc/redpanda/redpanda.yaml`.

|-X, --config-opt |stringArray |Override `rpk` configuration settings. See xref:reference:rpk/rpk-x-options.adoc[`rpk -X`] or execute `rpk -X help` for inline detail or `rpk -X list` for terser detail.

|--profile |string |Profile to use. See xref:reference:rpk/rpk-profile.adoc[`rpk profile`] for more details.

|-v, --verbose |- |Enable verbose logging.
|===

// end::single-source[]
56 changes: 56 additions & 0 deletions modules/reference/pages/rpk/rpk-shadow/rpk-shadow-create.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
= rpk shadow create
// tag::single-source[]

Creates a Redpanda shadow link.

This command creates a shadow link using a configuration file that defines the connection details and synchronization settings.

Before you create a shadow link, generate a configuration file with xref:reference:rpk/rpk-shadow/rpk-shadow-config-generate.adoc[`rpk shadow config generate`] and update it with your source cluster details. The command prompts you to confirm the creation. Use the `--no-confirm` flag to skip the confirmation prompt.

After you create the shadow link, use xref:reference:rpk/rpk-shadow/rpk-shadow-status.adoc[`rpk shadow status`] to monitor the replication progress.

== Usage

[,bash]
----
rpk shadow create [flags]
----

== Examples

Create a shadow link using a configuration file:

[,bash]
----
rpk shadow create --config-file shadow-link.yaml
----

Create a shadow link without a confirmation prompt:

[,bash]
----
rpk shadow create -c shadow-link.yaml --no-confirm
----

== Flags

[cols="1m,1a,2a"]
|===
|*Value* |*Type* |*Description*

|-c, --config-file |string |Path to configuration file to use for the shadow link; use `--help` for details.

|--no-confirm |- |Disable confirmation prompt.

|-h, --help |- |Help for create.

|--config |string |Redpanda or `rpk` config file; default search paths are `/var/lib/redpanda/.config/rpk/rpk.yaml`, `$PWD/redpanda.yaml`, and `/etc/redpanda/redpanda.yaml`.

|-X, --config-opt |stringArray |Override `rpk` configuration settings. See xref:reference:rpk/rpk-x-options.adoc[`rpk -X`] or execute `rpk -X help` for inline detail or `rpk -X list` for terser detail.

|--profile |string |Profile to use. See xref:reference:rpk/rpk-profile.adoc[`rpk profile`] for more details.

|-v, --verbose |- |Enable verbose logging.
|===

// end::single-source[]
63 changes: 63 additions & 0 deletions modules/reference/pages/rpk/rpk-shadow/rpk-shadow-delete.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
= rpk shadow delete
// tag::single-source[]

Delete a Redpanda shadow link.

This command deletes a shadow link by name. By default, you cannot delete a shadow link that has active shadow topics. Use xref:reference:rpk/rpk-shadow/rpk-shadow-failover.adoc[`rpk shadow failover`] first to deactivate topics before deletion, or use the `--force` flag to delete the shadow link and failover all its active shadow topics.

The command prompts you to confirm the deletion. Use the `--no-confirm` flag to skip the confirmation prompt. The `--force` flag automatically disables the confirmation prompt.

WARNING: Deleting a shadow link with `--force` permanently removes all shadow topics and stops replication. This operation cannot be undone.

== Usage

[,bash]
----
rpk shadow delete [LINK_NAME] [flags]
----

== Examples

Delete a shadow link:

[,bash]
----
rpk shadow delete my-shadow-link
----

Delete a shadow link without confirmation:

[,bash]
----
rpk shadow delete my-shadow-link --no-confirm
----

Force delete a shadow link with active shadow topics:

[,bash]
----
rpk shadow delete my-shadow-link --force
----

== Flags

[cols="1m,1a,2a"]
|===
|*Value* |*Type* |*Description*

|-f, --force |- |If set, forces a delete while there are active shadow topics; disables confirmation prompts as well.

|--no-confirm |- |Disable confirmation prompt.

|-h, --help |- |Help for delete.

|--config |string |Redpanda or `rpk` config file; default search paths are `/var/lib/redpanda/.config/rpk/rpk.yaml`, `$PWD/redpanda.yaml`, and `/etc/redpanda/redpanda.yaml`.

|-X, --config-opt |stringArray |Override `rpk` configuration settings. See xref:reference:rpk/rpk-x-options.adoc[`rpk -X`] or execute `rpk -X help` for inline detail or `rpk -X list` for terser detail.

|--profile |string |Profile to use. See xref:reference:rpk/rpk-profile.adoc[`rpk profile`] for more details.

|-v, --verbose |- |Enable verbose logging.
|===

// end::single-source[]
Loading