Skip to content

HDDS-11560. show containers replication metrics in DN decommissioning status command output #7500

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 148 commits into
base: master
Choose a base branch
from

Conversation

vtutrinov
Copy link
Contributor

@vtutrinov vtutrinov commented Nov 28, 2024

What changes were proposed in this pull request?

Show containers replication metrics in the output of the datanode decommission status command

bash-5.1$ ozone admin datanode status decommission

Decommission Status: DECOMMISSIONING - 1 node(s)

Datanode: a05eedd4-fc28-4046-a8a4-92ce6f269ea9 (/default-rack/172.18.0.8/ozone-ha-datanode2-1.ozone-ha_default)
Decommission Started At : 28/11/2024 09:10:40 UTC
No. of Unclosed Pipelines: 0
No. of UnderReplicated Containers: 5.0
No. of Unclosed Containers: 0.0
{UnderReplicated=[#5, #1, #2, #3, #4], UnClosed=[]}
ContainersReplicationMetrics:
numTimeoutReplications: 0
numRequestedReplications: 8
numQueuedReplications: 0
failureTime: 0
numInFlightReplications: 5
queueTime: 0
failureBytes: 0
maxReplicationStreams: 20
successTime: 51349
numSkippedReplications: 0
numFailedReplications: 0
transferredBytes: 807620608
numSuccessReplications: 3
closedContainersCount: 8

UPD: update replication metrics are

bash-5.1$ ozone admin datanode status decommission

Decommission Status: DECOMMISSIONING - 1 node(s)

Datanode: 8d83e10e-bee5-4aaf-8a0d-cdad4cea5a7c (/default-rack/172.21.0.5/ozone-ha-datanode-4.ozone-ha_default)
Decommission Started At : 13/04/2025 12:16:54 UTC
No. of Unclosed Pipelines: 4
No. of UnderReplicated Containers: 0.0
No. of Unclosed Containers: 0.0
{}
Containers Replication Metrics:
Containers for replication: 23
Time spent on failed replication attempts: 0
Bytes transferred for failed replication attempts: 0
Time spent on successful replication tasks: 201206
Failed replications: 0
Bytes transferred for successful replication tasks: 3393880064
Successfully replicated containers: 23

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-11560

How was this patch tested?

A new unit test for the CLI command was created, and manual testing of the CLI command was performed too

@errose28 errose28 self-assigned this Dec 2, 2024
@errose28 errose28 self-requested a review December 2, 2024 16:45
@errose28 errose28 removed their assignment Dec 2, 2024
@adoroszlai
Copy link
Contributor

Thanks @vtutrinov for working on this. This PR came up in the community sync last week. The general sentiment was that it's a useful change, but:

  • shouldn't dump all metrics, just a few select ones,
  • should provide a better description for those.

Maybe others in the reviewers list can provide more details on which items to include/exclude and how to improve descriptions.

@adoroszlai adoroszlai marked this pull request as draft January 15, 2025 10:09
@adoroszlai
Copy link
Contributor

@vtutrinov do you plan to continue work on this?

@vtutrinov
Copy link
Contributor Author

@adoroszlai Yes, I do. I will send the required changes, a cropped list of metrics with descriptions, by the weekend.

@vtutrinov vtutrinov force-pushed the HDDS-11560-dn-decommission-metrics branch 3 times, most recently from 5887fc3 to 03bd678 Compare April 14, 2025 06:48
@vtutrinov vtutrinov force-pushed the HDDS-11560-dn-decommission-metrics branch from 03bd678 to 1440035 Compare April 14, 2025 08:44
@vtutrinov vtutrinov marked this pull request as ready for review April 14, 2025 14:02
@siddhantsangwan siddhantsangwan requested a review from Tejaskriya May 5, 2025 10:47
@Tejaskriya
Copy link
Contributor

Got to this a bit late, my bad!
@vtutrinov, there are come merge conflicts could you please resolve them. I am reviewing the patch, will post the comments soon

szetszwo and others added 27 commits May 13, 2025 14:39
@Tejaskriya
Copy link
Contributor

@vtutrinov looks like all master commits got pulled into this (instead of getting rebased or merged as 1 commit). Could you fix it? (You might need to fix the commit history locally and then force push it)

@vtutrinov vtutrinov force-pushed the HDDS-11560-dn-decommission-metrics branch from 86bfad4 to 3503220 Compare May 14, 2025 08:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.