Skip to content

HDDS-11753. Deprecate file per chunk layout from datanode code. #7654

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jan 7, 2025

Conversation

jojochuang
Copy link
Contributor

What changes were proposed in this pull request?

HDDS-11753. Deprecate file per chunk layout from datanode code.

Please describe your PR in detail:

  • Mark FILE_PER_CHUNK as deprecated.
  • Fallback to FILE_PER_BLOCK during DataNode startup if FILE_PER_CHUNK is configured.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-11753

How was this patch tested?

Unit test; one test case was added.

Change-Id: I2f1b168a5884abd5ad3e392dff3adcbc42edc6ee
Change-Id: I5cce02dd3dded77ca3bbeba12aac335043c38c7c
Change-Id: If5ead80f998d8e016df515cd43a203416db78bc2
Change-Id: If8090f6dbdce3bb3e19ab6fa433b4ed011e6b951
@jojochuang
Copy link
Contributor Author

This one supersedes #7585

Comment on lines 198 to 199
OzoneConfiguration.of(conf).set(ScmConfigKeys.OZONE_SCM_CONTAINER_LAYOUT_KEY,
"FILE_PER_BLOCK");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we are setting OZONE_SCM_CONTAINER_LAYOUT_KEY to FILE_PER_BLOCK in the code, we are actually removing support for FILE_PER_CHUNK not deprecating it.

With this change, there is no way to use FILE_PER_CHUNK anymore. Can we at least log a clear WARN message stating that the value of OZONE_SCM_CONTAINER_LAYOUT_KEY is getting overwritten in the code?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR meant to disallow creating new FILE_PER_CHUNK containers, but it would still permit reading existing FILE_PER_CHUNK containers.

I'd love to get rid of FILE_PER_CHUNK code altogether but I'm not sure if anyone still have FILE_PER_CHUNK containers.

jojochuang and others added 2 commits January 6, 2025 22:28
…ozone/container/keyvalue/KeyValueHandler.java

Co-authored-by: Nandakumar Vadivelu <[email protected]>
Change-Id: I56ed5d2decb1354b24ffdf09e70ba5373df2e5ed
Copy link
Contributor

@nandakumar131 nandakumar131 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jojochuang jojochuang merged commit 3dfd241 into apache:master Jan 7, 2025
42 checks passed
@jojochuang
Copy link
Contributor Author

Merged. Thanks @nandakumar131 for the review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants