HDDS-8436. Support setSafeMode(), isFileClosed() FileSystem API #4825

jojochuang · 2023-06-03T01:03:07Z

What changes were proposed in this pull request?

Implement new Hadoop file system APIs to support HBase.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-8436 Support setSafeMode(), isFileClosed() FileSystem API

How was this patch tested?

Unit tests

hadoop-ozone/interface-client/src/main/proto/OmClientProtocol.proto

szetszwo

@jojochuang , thanks a lot for working on this! The change looks good. Please see the comments inlined.

szetszwo · 2023-06-28T15:00:40Z

hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java

+    case LEAVE:
+    case FORCE_EXIT:


Currently, we only support FORCE_EXIT. We should support LEAVE later.

For LEAVE, it should exit safe mode only if preCheck is true. We should make the change separately.

//SCMSafeModeManager @@ -237,20 +237,25 @@ public void completePreCheck(EventPublisher eventQueue) { * 2. Emits START_REPLICATION for ReplicationManager. * 3. Cleanup resources. * 4. Emit safe mode status. - * @param eventQueue */ @VisibleForTesting - public void exitSafeMode(EventPublisher eventQueue) { - LOG.info("SCM exiting safe mode."); - // If safemode is exiting, then pre check must also have passed so - // set it to true. - setPreCheckComplete(true); + public boolean exitSafeMode(boolean force) { + LOG.info("SCM exiting safe mode, force? {}", force); + if (force) { + // If force exiting, set preCheck to true. + setPreCheckComplete(true); + } else { + if (!getPreCheckComplete()) { + return false; + } + } setInSafeMode(false); // TODO: Remove handler registration as there is no need to listen to // register events anymore. emitSafeModeStatus(); + return true; }

szetszwo · 2023-06-28T15:40:17Z

...one/ozonefs-common/src/main/java/org/apache/hadoop/fs/ozone/BasicOzoneClientAdapterImpl.java

+    if (!ofsPath.isKey()) {
+      throw new IOException("not a file");
+    }
+    OzoneFileStatus status = bucket.getFileStatus(pathStr);


We should check status.isFile().

szetszwo · 2023-06-28T15:43:46Z

...onefs-common/src/main/java/org/apache/hadoop/fs/ozone/BasicRootedOzoneClientAdapterImpl.java

+      if (ofsPath.isSnapshotPath()) {
+        throw new IOException("file is in a snapshot.");
+      } else {
+        OzoneFileStatus status = bucket.getFileStatus(key);


Check status.isFile().

szetszwo · 2023-06-28T15:46:48Z

hadoop-ozone/ozonefs-hadoop3/src/main/java/org/apache/hadoop/fs/ozone/OzoneFileSystem.java

+  @Override
+  public boolean setSafeMode(SafeModeAction action, boolean isChecked)
+      throws IOException {
+    statistics.incrementWriteOps(1);


Check if the action is not GET before incrementWriteOps.

szetszwo · 2023-06-28T15:47:52Z

hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/OzoneFileSystem.java

+  @Override
+  public boolean setSafeMode(SafeModeAction action, boolean isChecked)
+      throws IOException {
+    statistics.incrementWriteOps(1);


Check if the action is not GET before incrementWriteOps.

szetszwo · 2023-06-28T15:49:10Z

hadoop-ozone/ozonefs/src/main/java/org/apache/hadoop/fs/ozone/RootedOzoneFileSystem.java

+  @Override
+  public boolean setSafeMode(SafeModeAction action, boolean isChecked)
+      throws IOException {
+    statistics.incrementWriteOps(1);


Check if the action is not GET before incrementWriteOps.

szetszwo

@jojochuang , thanks for the update!

We need to check GET before incrementWriteOp in two of RootedOzoneFileSystems. BTW, is it possible to move the setSafeMode method to Basic RootedOzoneFileSystem?

szetszwo · 2023-06-29T07:38:03Z

...op-ozone/ozonefs-hadoop3/src/main/java/org/apache/hadoop/fs/ozone/RootedOzoneFileSystem.java

+  @Override
+  public boolean setSafeMode(SafeModeAction action, boolean isChecked)
+      throws IOException {
+    statistics.incrementWriteOps(1);


Check if the action is not GET before incrementWriteOps.

jojochuang · 2023-06-29T21:40:03Z

BTW, is it possible to move the setSafeMode method to Basic RootedOzoneFileSystem?

We can't, because Hadoop 2 does not have SafeMode interface (introduced in Hadoop 3.3.6). So I can only put it where Ozone is not run with Hadoop 2 runtime.

jojochuang · 2023-06-29T21:43:28Z

But as it turns out, we test MR jobs with Hadoop 3.1 3.2 and 3.3 runtime, and the MR job tests fail with Hadoop 3.1 and 3.2 (due to the lacking of SafeMode and LeaseRecoverable interface). So either we find a workaround or remove the Hadoop 3.1 3.2 tests.

@OverRide

Change-Id: Iba9c50fac4da3f90acce8168758f4284e47825cb Compiles Change-Id: I53b7c4dbe33735ca3b39e4502be983134b05c0fe Compiles Change-Id: I699803f225460713f8f47371a3c45b4e10cf9cdf Completed safe mode implementation. Change-Id: I57ba72625c98bf7faa5c6e54526a7d61d7731fee Checkstyle. Change-Id: I1ffd5f1b2756fad7b03864cc3725e9d41cd72b10 Add missing @OverRide annotations, increment statistics properly. Change-Id: Idb45f85b8a8f72776f393a8a0f22eb23e72f815b

Change-Id: I1d0260c14c3dac2ed28f6ea86730183c0630174f

Change-Id: I4ebc4360dde010c9906b292e0e32008146759aa3

Change-Id: Ia0e9e0c429ae6718f18a890258388b61cbcafabd

Change-Id: I5fec0e5e7653a1070b38be012e17d476fd865a7d

Change-Id: I48f884edac8092af910397536754cfda809c399a

szetszwo

+1 the change looks good.

* master: (36 commits) HDDS-8990. Intermittent timeout waiting on datanode4 9856 to become available (apache#5039) Revert "HDDS-7750. Incorrect WRITE ACL check. (apache#4992)" HDDS-7750. Incorrect WRITE ACL check. (apache#4992) HDDS-8985. Intermittent timeout exiting safe mode in HA secure tests (apache#5033) HDDS-8593. Add RootCARotationPoller to CertClient (apache#5030) HDDS-7645. Kubernetes check should fail fast if cluster cannot start (apache#5028) HDDS-8981. TestRootedOzoneFileSystem runs out of disk space (apache#5029) HDDS-8592. Fetch and save all root certificates during service's certificate rotation. (apache#5025) HDDS-8981. Disable TestRootedOzoneFileSystem#testSafeMode HDDS-8591. Create scheduler to check for new root ca certificates (apache#4961) HDDS-8979. error validating kustomization.yaml (apache#5024) HDDS-8973. Ozone SCM HA should not allocates duplicate IDs when transferring leadership (apache#5018) HDDS-8970. Snapshot Diff should return path relative to bucket root (apache#5015) HDDS-8975. Clarify SCM HA auto-bootstrap doc (apache#5021) HDDS-8689. Rotate Root CA and Sub CA in SCM. (apache#4943) HDDS-8436. Support setSafeMode(), isFileClosed() FileSystem API (apache#4825) HDDS-8880. Intermittent fork timeout in TestOMRatisSnapshots (apache#5022) HDDS-8962. Ensure docker env is stopped (apache#5011) HDDS-7794. [snapshot] SnapshotDiff should throw better error messages for exception handling (apache#5007) HDDS-7922. [FSO] S3G folder support fso layout filestatus s3A compatibility (apache#4448) ...

jojochuang changed the title ~~Hdds 8436~~ HDDS-8436. Support setSafeMode(), isFileClosed() FileSystem API Jun 3, 2023

szetszwo reviewed Jun 14, 2023

View reviewed changes

hadoop-ozone/interface-client/src/main/proto/OmClientProtocol.proto Outdated Show resolved Hide resolved

jojochuang force-pushed the HDDS-8436 branch from 8574d12 to 99dc31d Compare June 15, 2023 17:51

jojochuang force-pushed the HDDS-8436 branch 3 times, most recently from be98bf6 to 613acce Compare June 27, 2023 16:46

szetszwo reviewed Jun 28, 2023

View reviewed changes

jojochuang force-pushed the HDDS-8436 branch from 613acce to 9f9444a Compare June 29, 2023 01:52

szetszwo reviewed Jun 29, 2023

View reviewed changes

jojochuang force-pushed the HDDS-8436 branch from 84bdfbe to 1dc7d57 Compare June 30, 2023 15:40

jojochuang marked this pull request as ready for review June 30, 2023 15:40

jojochuang added 5 commits July 1, 2023 18:23

Review comments.

a0c901f

Change-Id: I1d0260c14c3dac2ed28f6ea86730183c0630174f

Update setSafeMode()

7a07923

Change-Id: I4ebc4360dde010c9906b292e0e32008146759aa3

Add hack

70d3744

Change-Id: Ia0e9e0c429ae6718f18a890258388b61cbcafabd

Checkstyle

8c906e1

Change-Id: I5fec0e5e7653a1070b38be012e17d476fd865a7d

jojochuang force-pushed the HDDS-8436 branch from 1dc7d57 to 8c906e1 Compare July 1, 2023 09:23

Update after HDDS-8966

de924e6

Change-Id: I48f884edac8092af910397536754cfda809c399a

szetszwo approved these changes Jul 5, 2023

View reviewed changes

szetszwo merged commit b97d943 into apache:master Jul 5, 2023

jojochuang added the hbase HBase on Ozone support label Jan 24, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HDDS-8436. Support setSafeMode(), isFileClosed() FileSystem API #4825

HDDS-8436. Support setSafeMode(), isFileClosed() FileSystem API #4825

Uh oh!

jojochuang commented Jun 3, 2023

Uh oh!

Uh oh!

szetszwo left a comment

Uh oh!

szetszwo Jun 28, 2023

Uh oh!

szetszwo Jun 28, 2023

Uh oh!

szetszwo Jun 28, 2023

Uh oh!

szetszwo Jun 28, 2023

Uh oh!

szetszwo Jun 28, 2023

Uh oh!

szetszwo Jun 28, 2023

Uh oh!

szetszwo Jun 28, 2023

Uh oh!

szetszwo left a comment

Uh oh!

szetszwo Jun 29, 2023

Uh oh!

jojochuang commented Jun 29, 2023

Uh oh!

jojochuang commented Jun 29, 2023

Uh oh!

szetszwo left a comment

Uh oh!

Uh oh!

HDDS-8436. Support setSafeMode(), isFileClosed() FileSystem API #4825

HDDS-8436. Support setSafeMode(), isFileClosed() FileSystem API #4825

Uh oh!

Conversation

jojochuang commented Jun 3, 2023

What changes were proposed in this pull request?

What is the link to the Apache JIRA

How was this patch tested?

Uh oh!

Uh oh!

szetszwo left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

szetszwo left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jojochuang commented Jun 29, 2023

Uh oh!

jojochuang commented Jun 29, 2023

Uh oh!

szetszwo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!