HDDS-12825. ReconIncrementalContainerReportHandler is not synchronized on datanode. #8272
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@devmadhuu , thanks for reviewing this!
I guess you are talking about the Logger changes. Reverted them. |
ashishkumar50
approved these changes
Apr 14, 2025
|
@devmadhuu , @ashishkumar50 , thanks a lot for reviewing this! |
aswinshakil
added a commit
to aswinshakil/ozone
that referenced
this pull request
Apr 25, 2025
…239-container-reconciliation Commits: 318 68452d0 HDDS-12893. addendum: cp: warning: behavior of -n is non-portable and may change in future; use --update=none instead (apache#8334) 6852e3e HDDS-12861. Add AvoidStringBufferField PMD rule (apache#8312) c21ec5b HDDS-12880. Move field declarations to start of class in hdds-server-framework (apache#8310) 56b5b02 HDDS-12707. Recon - In-memory extraction of om tarball from network input stream (apache#8212) 64dbf44 HDDS-12893. cp: warning: behavior of -n is non-portable and may change in future; use --update=none instead (apache#8328) 250d3c1 HDDS-12891. OMKeyAclRequestWithFSO is incorrectly setting full path as key name. (apache#8326) f6597d1 HDDS-12888. Add negative test cases for FS operations on OBS buckets (apache#8323) cdab4eb HDDS-12173. Follow RocksDB basic tuning guide (apache#8206) 6f25b43 HDDS-12347. Fix spotbugs warnings in integration tests (apache#8298) e83569f HDDS-12816. Ozone debug replicas chunk-info has incorrect or None in FileLocations (apache#8289) 621f018 HDDS-12624. Fix pipeline limit check to prevent incorrect pipeline creation (apache#8216) dd8950b HDDS-12340. Remove unnecessary findbugs exclusions (apache#8309) 0d88da0 HDDS-12723. Handle Volume Db failure in volume scanner (apache#8256) 434a5d5 HDDS-12823. SnapshotDiffReportOzone#fromProtobuf empty token handling (apache#8280) 7c23cbd HDDS-12504. Replace calls to deprecated RandomStringUtils methods (apache#8306) 3ee57e6 HDDS-12813. Replace calls to deprecated RandomUtils methods (apache#8302) 84b2ff6 HDDS-12876. Bump awssdk to 2.31.25 (apache#8304) db9076e HDDS-12867. Replace hard-coded namespace URL with constant S3_XML_NAMESPACE (apache#8299) 97b35ee HDDS-12874. Bump picocli to 4.7.7 (apache#8303) 3eed6a6 HDDS-12847. Use DatanodeID instead of DatanodeDetails.getUuidString (apache#8293) 205cb2c HDDS-12846. Log DatanodeDetails instead of DatanodeDetails.getUuidString (apache#8295) 213c2fe HDDS-11734. Bump maven-compiler-plugin to 3.14.0 (apache#8301) 889f3b7 HDDS-10284. Move GenericTestUtils#getTempPath to MiniOzoneCluster (apache#8300) e613598 HDDS-12113. Move HAProxy test to HA environment (apache#8271) d8a3915 HDDS-12413. Move field declarations to start of class in hdds-container-service (apache#7968) 5c91b44 HDDS-8802. Added pagination support for ListSnapshotDiff jobs (apache#8124) e9e1493 HDDS-12704. Add missing audit logs for SCM client operations (apache#8251) e8af186 HDDS-12152. Stop testing with Hadoop 3.1.2 (apache#7773) f6a4a48 HDDS-12840. Avoid long containerId in KeyValueContainer (apache#8291) 3c9e146 HDDS-12350. Reduce duplication between OmBucketReadWrite tests (apache#8241) 93613c1 HDDS-12734. Enable native lib in CI checks (apache#8190) 3d4b5fd HDDS-6631. Fix typos in output/exception messages (apache#8294) 978dd71 HDDS-12806. Replace commons-logging with jcl-over-slf4j (apache#8265) ef8e0da HDDS-12524. Reuse TestDataUtil.createKey in more tests (apache#8270) b91e8e7 HDDS-12518. Auto-compact tables which can tend to be large in size at intervals (apache#8260) 55f6924 HDDS-12845. Reuse TestDataUtil.createKey in OzoneRpcClientTests (apache#8274) 8a4deeb HDDS-12827. Move out NodeStateMap.Entry and ContainerMap.Entry (apache#8273) 62df306 HDDS-12493. Move container upgrade under repair (apache#8205) b33ed23 HDDS-12412. Make hdds-common compliant with FieldDeclarationsShouldBeAtStartOfClass (apache#7967) a95d338 HDDS-12740. Use DatanodeID in HddsTestUtils (apache#8281) f26e2f0 HDDS-12771. xcompat fails if run in itself due to undefined OZONE_CURRENT_VERSION (apache#8236) c7117dc HDDS-11974. Split Container Safemode Rule into Ratis & EC Container Safemode Rules (apache#7951) bb2c63c HDDS-12803. OmKeyInfo#isKeyInfoSame should handle object tags (apache#8259) a27c1cf HDDS-12837. Bump vite to 4.5.13 (apache#8283) 9b48b6e HDDS-12838. Bump awssdk to 2.31.21 (apache#8267) 3d8644e HDDS-12830. Add RocksDatabaseException. (apache#8277) 9e4da5b HDDS-12821. Update Build from Source user doc. (apache#8262) 5292ac5 HDDS-12836. Bump junit to 5.12.2 (apache#8269) e6b9d5a HDDS-12825. ReconIncrementalContainerReportHandler is not synchronized on datanode. (apache#8272) a92fe59 HDDS-12581. Multi-threaded Log File Parsing with Batch Updates to DB (apache#8254) eed5924 HDDS-12770. Use ContainerID instead of Long in CONTAINER_IDS_TABLE. (apache#8247) 7dfd8c1 HDDS-12060. Replace System.currentTimeMillis() with Time.monotonicNow() for duration calculation (apache#8096) a21e362 HDDS-12547. Container creation and import use the same VolumeChoosingPolicy (apache#8090) 22734a9 HDDS-12461. Bump Ranger to 2.6.0 (apache#8120) 0d6231c HDDS-12801. SCM should remove pipeline before DN. (apache#8261) fdebdac HDDS-12761. Add missing network configuration properties in ozone-default.xml (apache#8257) 7062685 HDDS-12805. Use slf4j for HTTP request log (apache#8255) 7ff8ad7 HDDS-12755. Redundant declaration in TestHadoopNestedDirGenerator#spanCheck() (apache#8218) fbb8706 HDDS-12145. Remove unused config hdds.container.ratis.replication.level (apache#8252) 6913835 HDDS-12519. Generate auto-complete script for Ozone commands (apache#8030) 25dda2d HDDS-12408. Create mixin for ContainerID list parameters (apache#7970) f22f32e HDDS-12764. NodeDecommissionManager#monitorInterval should get with ms unit from config (apache#8250) dc9952e HDDS-12746. Reduce visibility of Logger members (apache#8210) 102ae3f HDDS-8660. Notify ReplicationManager when nodes go dead or out of service (apache#7997) a1345f9 HDDS-12368. Seek to correct start key in KeyManagerImpl#getTableEntries (apache#7925) 1ebf265 HDDS-12580. Set up Temporary DB for Storing Container Log Information (apache#8072) 189fbdb HDDS-12463. Add perf metrics in SCM for allocateBlock and openPipeline (apache#8111) fa8afa3 HDDS-12741. Recon UI should show full path from ContainerKeys API response (apache#8200) a0387af HDDS-12800. Bump aspectj-maven-plugin to 1.14.1 (apache#8230) 5733c55 HDDS-11799. Remove config hdds.scm.safemode.pipeline-availability.check (apache#8095) c962b98 HDDS-12404. Grafana dashboard for snapshot metrics (apache#7954) bf20540 HDDS-12458. Refactor DataNodeSafeModeRule to use NodeManager (apache#7998) 25b7102 HDDS-12775. flaky-test-check builds the workflow branch (apache#8237) 2a4de14 HDDS-12772. Configure initial heartbeat and first election time for quicker MiniOzoneCluster startup (apache#8235) 0f5b590 HDDS-12691. Calculation of committed space in Datanode seems incorrect (apache#8228) 6d0a830 HDDS-12721. Log more details about volumes (apache#8181) 1bee369 HDDS-12525. Replace some TestHelper#createKey usage with TestDataUtil#createKey (apache#8233) 5834fcf HDDS-10091. TestOzoneDebugShell fails with FILE_SYSTEM_OPTIMIZED bucket layout (apache#8227) 3822943 HDDS-12768. Bump vite to 4.5.12 (apache#8234) b6dac4a HDDS-12766. Bump awssdk to 2.31.16 (apache#8232) 0ef1f02 HDDS-12767. Bump jacoco to 0.8.13 (apache#8231) 2bbeaaf HDDS-12700. Upgrade test overwrites previous result (apache#8229) 0a558c1 HDDS-12758. Error in OmUtils.normalizeKey for key name starting with `//` (apache#8225) fee8817 HDDS-12759. Bump vitest to 1.6.1 (apache#7828) 1f716ea HDDS-12699. Bump vite to 4.5.11 (apache#8224) abc1e02 HDDS-12757. Duplicated declaration of dnsInterface in HddsUtils (apache#8222) 3fb57c1 HDDS-12037. Removing unit from quota namespace (apache#8148) ebcece4 HDDS-11038. Add Helm Chart to the Ozone on Kubernetes doc (apache#8220) bba8a67 HDDS-12750. Move StorageTypeProto from ScmServerDatanodeHeartbeatProtocol.proto to hdds.proto (apache#8208) bc19a4c HDDS-12711. Limit number of excluded SST files logged at info level (apache#8186) 6bcebe8 HDDS-4517. Remove leftover references to RaftServerImpl (apache#8223) 868a237 HDDS-12756. Speed up TestReconfigShell and TestOzoneDebugShell (apache#8219) 40d02b9 HDDS-12639. Add info for TimeoutException (apache#8113) 5f74da3 HDDS-12760. Mark TestContainerReplication#testImportedContainerIsClosed as flaky ed01115 HDDS-9241. Document S3 Gateway REST Secret Revoke command (apache#8221) 479f6d9 HDDS-12690. Remove om.fs.snapshot.max.limit from RDBStore and OmMetadataManagerImpl (apache#8195) 0f52a34 HDDS-12559. Implement Bulk Ozone Locks for taking locks on multiple snapshots (apache#8052) de73c00 HDDS-12738. Refactor AbstractContainerReportHandler and its subclasses. (apache#8207) 30e4aa4 HDDS-12748. Remove unused config ozone.manager.db.checkpoint.transfer.bandwidthPerSec (apache#8213) 693548b HDDS-12676. Prefer minFreeSpace if minFreeSpacePercent is also defined (apache#8180) 092fe40 HDDS-12751. native build fails with CMake 4 (apache#8215) 1433d0c HDDS-1480. Prefer resolved datanode ip address over persisted ip address (apache#7495) af5301e HDDS-12233. Atomically import a container (apache#7934) 2475949 HDDS-11107. Remove unnecessary run_test_scripts call in upgrade tests (apache#8201) 20a13da HDDS-12615. Failure of any OM task during bootstrapping of Recon needs to be handled (apache#8098) 7fb92b0 HDDS-12717. Combine nodeMap and nodeToContainer in NodeStateMap. (apache#8199) fc6776d HDDS-12739. Replace the getNodeByUuid methods in NodeManager. (apache#8202) 0d40030 HDDS-12698. Unused FailureService in MiniOzoneChaosCluster (apache#8197) 400835f HDDS-12736. Bump hadoop-thirdparty to 1.4.0 (apache#8193) a540684 HDDS-12720. Use DatanodeID in SimpleMockNodeManager (apache#8198) 604576a HDDS-12735. Unused rocksDBConfiguration variable in `OmMetadataManagerImpl#start` (apache#8196) 64318f7 HDDS-12498. Allow limiting flaky-test-check to specific submodule (apache#8194) 3e7b5eb HDDS-12719. Remove the setContainers(..) method from NodeManager. (apache#8191) 78e2e73 HDDS-12724. hdds-rocks-native build fails if JAVA_HOME not set (apache#8183) fbc696c HDDS-12594. Optimize replica checksum verifier (apache#8151) 4411e35 HDDS-12733. Bump axios to 0.30.0 (apache#8189) 31aabc1 HDDS-12732. Bump awssdk to 2.31.11 (apache#8188) d6e2509 HDDS-12731. Bump restrict-imports-enforcer-rule to 2.6.1 (apache#8187) 4213307 HDDS-12648. Fix checkLeaderStatus in removeOMFromRatisRing and addOMToRatisRing (apache#8142) e4e9547 HDDS-12709. Intermittent failure in Balancer acceptance test (apache#8182) c1d5b4f HDDS-12718. Use NodeStateMap in MockNodeManager instead of test-specific Node2ContainerMap (apache#8179) 65a9d6c HDDS-12587. Detect test class module in flaky-test-check (apache#8162) aac9383 HDDS-12529. Clean up code in AbstractContainerReportHandler. (apache#8033) 6891315 HDDS-12603. Fix ContainerTable keys re-rendering issue (apache#8163) 2a1a6bf HDDS-12670. Improve encapsulation of volume spare space check (apache#8167) 05b6eb3 HDDS-12696. Replace link to Hadoop with Ozone in httpfs site.xml (apache#8177) 7e87a8a HDDS-11879. MiniKMS fails with ClassNotFoundException: com.sun.jersey....ServletContainer (apache#8158) 841d297 HDDS-12592. Remove find missing padding check in favor of of block metadata check (apache#8145) ce003e8 HDDS-12702. Move checknative under ozone debug (apache#8170) 6ee9c2b HDDS-8007. Add more detailed stages for SnapDiff job progress tracking (apache#8010) 3ebe5e7 HDDS-12660. Allow --verbose option of GenericCli at leaf subcommands (apache#8166) c8b3ccb HDDS-12528. Create new module for S3 integration tests (apache#8168) 435dee9 HDDS-12703. Close pipeline command should display error on failure (apache#8169) deb8e7c HDDS-12705. Replace Whitebox with HddsWhiteboxTestUtils (apache#8172) 0d8aecc HDDS-12327. HDDS-12668. Fix HSync upgrade test failure in non-HA upgrade test (apache#8171) bef4bee HDDS-12239. (addendum) Volume should not be marked as unhealthy when disk full - fix compile error e3e47ea HDDS-12235. Reserve space on DN during container import operation. (apache#7981) 4ffae70 HDDS-12239. Volume should not be marked as unhealthy when disk full (apache#7830) 0781cbe HDDS-12694. Disable TestMiniChaosOzoneCluster after fixing init and shutdown (apache#8159) 92aa71d HDDS-12687. Avoid ambiguity in URI descriptions (apache#8155) 5367016 HDDS-12679. Merge VolumeInfo into StorageVolume (apache#8147) 2cf6d59 HDDS-12553. ozone admin container list should output real JSON array (apache#8050) 63fcb27 HDDS-12465. Intermittent failure in TestOzoneFileSystemMetrics (apache#8130) bab26a4 HDDS-12684. Update NOTICE and LICENSE file (apache#8160) 857cb76 HDDS-12327. Restore non-HA (to HA) upgrade test (apache#7880) 587e9ff HDDS-12320. Collect OM performance metrics for FSO key delete (apache#7883) 560d017 HDDS-12500. Do not skip JUnit tests in post-commit runs (apache#8024) 273a627 HDDS-12686. Remove output of OzoneAddress in --verbose mode CLI (apache#8153) 4d3d834 HDDS-12650. Added logs to SnapshotDeletingService to indicate skipped snapshots. (apache#8123) 7524736 Revert "HDDS-12528. Create new module for S3 integration tests (apache#8152)" d44ebf5 HDDS-12528. Create new module for S3 integration tests (apache#8152) 3739b05 HDDS-12604. Reduce duplication in TestContainerStateMachine (apache#8104) a1616ae HDDS-12485. Repair tool should only print user warning for offline commands (apache#8140) de5c0a3 HDDS-12668. HSync upgrade test failure (apache#8137) bee10f0 HDDS-12671. Include .editorconfig and .run in source tarball (apache#8139) 8e7d370 HDDS-12666. Remove -SNAPSHOT from OZONE_CURRENT_VERSION in upgrade test (apache#8136) 4e0a764 HDDS-12486. Warmup KMS encrypted keys when OM starts (apache#8081) 0910997 HDDS-12636. Reduce code duplication for tarball creation (apache#8121) cbafa02 HDDS-12642. ACL test assertions depend on JSON element order (apache#8143) 8691adf HDDS-12622. Refactor minFreeSpace calculation (apache#8119) 1b0e912 HDDS-12661. Standardize Maven module names (apache#8129) ebf5cc6 HDDS-12662. Rename upgrade callback directory 1.5.0 to 2.0.0 (apache#8131) 828b2d1 HDDS-12667. Bump awssdk to 2.31.6 (apache#8134) 2596b3b HDDS-12665. Bump zstd-jni to 1.5.7-2 (apache#8133) 0e07983 HDDS-12664. Bump copy-rename-maven-plugin to 1.0.1 (apache#8132) 3173501 HDDS-12462. Use exclude rules for defining shaded filesystem jar contents (apache#8008) ee6201b HDDS-12341. Share cluster in client tests (apache#8126) bd579b3 HDDS-12646. Improve OM decommission check (apache#8122) 482024e HDDS-12569. Extract MiniOzoneCluster to separate module (apache#8067) 7a3ad16 HDDS-12641. Move Lease to hdds-server-framework (apache#8128) e805c15 HDDS-12310. Online repair command to perform compaction on om.db (apache#7957) 7b9c152 HDDS-12473. Trim duplicate space in proto message definition (apache#8005) 918bb98 HDDS-12626. Move the compare method in NodeStatus to ECPipelineProvider. (apache#8116) bf5a2f2 HDDS-12644. Create factory method for OzoneAcl (apache#8115) a646b85 HDDS-12557. Add progress indicator for checkpoint tarball in leader OM (apache#8085) 9e23d59 HDDS-12627. NodeStateMap may handle opStateExpiryEpochSeconds incorrectly. (apache#8117) d19f3bc HDDS-12640. Move GrpcMetrics to hdds-server-framework (apache#8114) 1c0b445 HDDS-12426. SCM replication should check double of container size. (apache#8080) 157cca4 HDDS-12358. Intermittent failure in compatibility acceptance test (apache#8012) 9b25d02 HDDS-12549. refactor ratis request to common place (apache#8059) 3349248 HDDS-12588. Recon Containers page shows number of blocks, not keys (apache#8074) 693d0f4 HDDS-12589. Fix Incorrect FSO Key Listing for Container-to-Key Mapping. (apache#8109) 17dbc4f HDDS-12585. Recon ContainerHealthTask ConstraintViolationException error handling. (apache#8070) 53c0a32 HDDS-12620. Fix OM Mismatch Deleted Container API (apache#8102) 45c900d HDDS-12619. Optimize Recon OM Container Mismatch API (apache#8101) 57fda0c HDDS-12633. KEY_NOT_FOUND in OzoneRpcClientTests for LEGACY bucket with enableFileSystemPaths=true (apache#8108) 23b0505 Revert "HDDS-12630. Enable GitHub Discussions in asf.yml (apache#8107)" b402b7c HDDS-12621. Change NodeStatus to value-based. (apache#8105) 20aeda5 HDDS-12601. Unknown tarball cleanup for Recon OM DB snapshot. (apache#8084) 93cd5aa HDDS-12630. Enable GitHub Discussions in asf.yml (apache#8107) 61a36f6 HDDS-12373. Use File.getUsableSpace() instead of File.getFreeSpace() to calcuate usedSpace in DedicatedDiskSpaceUsage (apache#7927) 953e718 HDDS-12565. Treat volumeFreeSpaceToSpare as reserved space (apache#8086) 8489cc8 HDDS-11735. Update ozone-default.xml for volume choosing policy (apache#8103) 13c5be8 HDDS-12577. [Ozone 2.0] Update master branch version number. 1a9d9f7 HDDS-12446. Add a Grafana dashboard for low level RocksDB operations. (apache#7992) c9990ca HDDS-12576. [Ozone 2.0] Update proto.lock files (apache#8064) b68b94c HDDS-12617. Use DatanodeID as keys in NodeStateMap. (apache#8100) b71d408 Revert "HDDS-12589. Fix Incorrect FSO Key Listing for Container-to-Key Mapping. (apache#8078)" 68e3842 HDDS-12589. Fix Incorrect FSO Key Listing for Container-to-Key Mapping. (apache#8078) c8e77f8 HDDS-12602. Intermittent failure in TestContainerStateMachine.testWriteFailure (apache#8099) 08ac32d HDDS-12608. Race condition in datanode version file creation (apache#8093) e5ef35d HDDS-12582. TypedTable support using different codec (apache#8073) dc47897 HDDS-12611. Snapshot creation is removing extra keys from AOS's DB (apache#8094) ff3ef51 HDDS-12551. Replace dnsToUuidMap with dnsToDnIdMap in SCMNodeManager. (apache#8087) 28520a7 HDDS-12614. Configurable java version in flaky-test-check with default to 21 (apache#8097) 434d5bd HDDS-12616. Bump junit to 5.12.1 (apache#8088) 699ee88 HDDS-12610. Bump awssdk to 2.31.1 (apache#8089) 0a85f9b HDDS-12602. Mark TestContainerStateMachineLeader/Follower as flaky 03c80f8 HDDS-12591. Include ContainerInfo in ContainerAttribute. (apache#8083) 49a2c85 HDDS-12097. Enhance Container Key Mapper for Faster Processing. (apache#8002) 5b7f96c HDDS-12303. Move ozone.om.user.max.volume into OmConfig (apache#8082) afc4043 HDDS-12566. Handle Over replication of Quasi Closed Stuck containers (apache#8061) eb96ff4 HDDS-12590. Used db name as the threadNamePrefix. (apache#8076) cf5bad7 HDDS-12539. Enable some String-related rules in PMD (apache#8047) d26f711 HDDS-12533. Offline repair command for generic rocksDB compaction (apache#8039) 93563e9 HDDS-12573. Pipeline#toString should separate ReplicaIndex from next node UUID. (apache#8063) 6e40831 HDDS-12572. Remove the ContainerID parameter when it has ContainerReplica. (apache#8075) 786da39 HDDS-12057. Implement command ozone debug replicas verify checksums (apache#7748) 66bc7ea HDDS-12535. Intermittent failure in TestContainerReportHandling (apache#8060) 87a674c HDDS-12555. Combine containerMap and replicaMap in ContainerStateMap. (apache#8057) f3689b6 HDDS-12458. Show safemode rules status irrespective of whether SCM is in safe mode in verbose mode. (apache#8049) 7d8c771 HDDS-12469. Mark statemachine unhealthy for write operation timeout. (apache#8022) 9aa41fd HDDS-12574. Add script to find modules by test classes (apache#8062) 5e28acb HDDS-12537. Selective checks: skip tests for PMD ruleset change (apache#8040) 9a3433b HDDS-12520. Move auditparser under debug (apache#8041) 0335385 HDDS-12420. Move FinalizeUpgradeCommandUtil to hdds-common. (apache#8023) 75bbba3 HDDS-12552. Fix raw use of generic class SCMCommand (apache#8048) d71aadf HDDS-12527. Separate S3 Gateway from MiniOzoneCluster (apache#8058) de8cf16 HDDS-12483. Quasi Closed Stuck should have 2 replicas of each origin (apache#8014) 57a139e HDDS-11576. Create a separate S3 client factory (apache#8051) e8a4668 HDDS-12383. Fix spotbugs warnings in hdds-common and httpfsgateway (apache#8046) 0dab553 HDDS-12541. Change ContainerID to value-based (apache#8044) 1202f6d HDDS-12532. Support only Enum in ContainerAttribute. (apache#8036) a2ad1e3 HDDS-12536. Move InMemoryTestTable to test (apache#8043) 7164c76 HDDS-12488. S3G should handle the signature calculation with trailers (apache#8020) c8c6d0e HDDS-12534. Remove drop_column_family command from ozone debug ldb (apache#8038) 9a8321e HDDS-12535. Mark TestContainerReportHandling as flaky ddd89fb HDDS-12543. Remove duplicate license information (apache#8045) 08e2c0a HDDS-12531. Use AtomicFileOutputStream to write YAML files. (apache#8035) 1bd8d8f HDDS-11813. Reduce duplication in CI workflow (apache#7497) 250bd5f HDDS-12450. Enable SimplifiableTestAssertion check in PMD (apache#8032) 96273ae HDDS-12489. Intermittent timeout in TestSCMContainerManagerMetrics.testReportProcessingMetrics (apache#8021) 2315927 HDDS-12476. Add TestDataUtil#createKey variant with small random content (apache#8028) d95ca4c HDDS-12421. ContainerReportHandler should not make the call to delete replicas (apache#7976) 26c859c HDDS-12204. Improve failover logging (apache#7867) efbf79c HDDS-12236. ContainerStateMachine should not apply or write future transactions in the event of failure (apache#7862) 3f88dbe HDDS-12377. Improve error handling of OM background tasks processing in case of abrupt crash of Recon. (apache#7960) 72da3a6 HDDS-12477. Do not force RATIS/ONE replication in TestDataUtil#createKey (apache#8017) a428b15 HDDS-12496. Use TextFormat#shortDebugString to flatten proto message in SCMDatanodeProtocolServer. (apache#8019) ecd2de0 HDDS-12409. Log an error before increasing the sequence id of a CLOSED container in SCM (apache#7964) 978e4a7 HDDS-12168. Create new Grafana panel to display cluster growth rate (apache#7978) 9ab7c70 HDDS-12456. Avoid FileInputStream and FileOutputStream (apache#8015) 9d41cd7 HDDS-12474. Add latency metrics of deletion services to grafana dashboard (apache#8007) 2b48e8c HDDS-12354. Move Storage and UpgradeFinalizer to hdds-server-framework (apache#7973) ed737b3 HDDS-12428. Avoid force closing OPEN/CLOSING replica of a CLOSED Container (apache#7985) d1e8b90 HDDS-12210. Use correct BootstrapStateHandler.Lock in SnapshotDeletingService (apache#7991) b769a26 HDDS-12295. Allow updating OM default replication config for tests (apache#7974) dd74eee HDDS-12430. Document in ozone-default.xml the config keys moved from DFSConfigKeysLegacy (apache#7987) 34041ca HDDS-12466. Set default commit message to PR title (apache#8013) 1ea0735 HDDS-12467. Enable new asf.yaml parser (apache#8011) e87b8db HDDS-12417. Reduce duplication of createKey variants in TestDataUtil (apache#7999) 86d2027 HDDS-12193. Provide option to disable RDBStoreMetrics for Snapshotted DB (apache#7982) 7c1d201 HDDS-12451. Create factory for MultiTenantAccessController (apache#7996) c2a934c HDDS-12470. Revert workaround added by HDDS-8715 to preserve thread name. (apache#8004) b8c93cc HDDS-12376. (addendum: fix findbugs) Remove scmRatisEnabled from ScmInfo. (apache#7931) 4005a10 HDDS-12376. (addendum: fix pmd) Remove scmRatisEnabled from ScmInfo. (apache#7931) abfa3be HDDS-12442. Add latency metrics for OM deletion services (apache#7986) 14db15c HDDS-12376. Remove scmRatisEnabled from ScmInfo. (apache#7931) 7d31d9e HDDS-12410. Add detailed block info for ALLOCATE_BLOCK audit log (apache#7965) a8f0ff3 HDDS-12460. (addendum) Move hdds-test-utils code to src/test 4c28c7f HDDS-12460. Move hdds-test-utils code to src/test (apache#8000) bb16f66 HDDS-12448. Avoid using Jackson1 (apache#7994) be0e1e6 HDDS-12445. Remove unused code from ContainerStateMap. (apache#7990) a3c9c0e HDDS-12443. Intermittent failure in TestContainerBalancerSubCommand (apache#7989) 83fd8d7 HDDS-12449. Enable UseCollectionIsEmpty check in PMD (apache#7995) d18da13 HDDS-12452. Bump slf4j to 2.0.17 (apache#7993) 6a9e8b1 HDDS-12348. Reuse `TestDataUtil.createKey` method (apache#7971) 384d774 HDDS-12416. Enable UnusedPrivateField check in PMD (apache#7975) 39d7da3 HDDS-12424. Allow config key to include config group prefix (apache#7979) ba0939b HDDS-12156. Add container health task metrics in Recon. (apache#7786) d34aee4 HDDS-12172. Rename Java constants of DFSConfigKeysLegacy keys (apache#7922) 59aaa5c HDDS-12418. Remove healthyReplicaCountAdapter from RatisContainerReplicaCount (apache#7972) 232e780 HDDS-12150. Abnormal container states should not crash the SCM ContainerReportHandler thread (apache#7882) a708ea4 HDDS-12351. Move SCMHAUtils and ServerUtils to hdds-server-framework (apache#7961) 052bd2d HDDS-12198. Exclude Recon generated code in coverage (apache#7962) ba44e12 HDDS-12382. Fix other spotbugs warnings (apache#7969) e2bf599 HDDS-12345. Share cluster in filesystem tests (apache#7959) f9f1c80 HDDS-12411. Make hdds-client compliant with FieldDeclarationsShouldBeAtStartOfClass (apache#7966) 46f4986 HDDS-11512. Create Grafana dashboard for tracking system wide deletion (apache#7813) f7e9aed HDDS-12396. Enable UnusedPrivateMethod check in PMD (apache#7956) 619f524 HDDS-1234. Fix spotbugs warnings in ozone-manager (apache#7963) 1d64b37 HDDS-12380. Fix spotbugs warnings in hdds-container-service (apache#7958) 5c2b8f6 HDDS-12065. Checkpoint directory should be cleared on startup (apache#7681) a31755a HDDS-12315. Speed up some Freon integration tests (apache#7870) d8f3149 HDDS-12399. Enable PMD.ForLoopCanBeForeach rule (apache#7952) 393211a HDDS-12062. Recon - Error handling in NSSummaryTask to avoid data inconsistencies. (apache#7723) ecc2403 HDDS-9792. Add tests for Pipelines page (apache#7859) e4de75e HDDS-12388. Key rewrite tests should be skipped if feature is disabled (apache#7953) 7b0fe61 HDDS-12288. (addendum) fix checkstyle d5df7e4 HDDS-12288. Improve bootstrap logging to indicate progress of snapshot download. (apache#7861) 58a0453 HDDS-12381. Fix spotbugs warnings in TestHddsUtils (apache#7955) 1a0c223 HDDS-11768. Extract SCM failover proxy provider logic (apache#7950) b45de0f HDDS-12403. Bump zstd-jni to 1.5.7-1 (apache#7949) a57809e HDDS-12402. Bump sqlite-jdbc to 3.49.1.0 (apache#7948) 221d53d HDDS-12400. Bump junit to 5.12.0 (apache#7947) f17abae HDDS-12398. Enable PMD checks for tests (apache#7946) 540f67b HDDS-12353. Move SpaceUsage implementations to hdds-server-framework (apache#7926) 70b93dc HDDS-10764.Tarball creation failing on leader OM node. (apache#7941) 6b20afc HDDS-12387. Cleanup TestContainerOperations (apache#7940) 417ae7c HDDS-12365. Provide editor settings and IDEA run config in standard location (apache#7924) a2c5c8e HDDS-12293. Make ozone.om.server.list.max.size reconfigurable (apache#7938) e0bd2cc HDDS-12185. Enhance FileSizeCountTask for Faster Processing. (apache#7796) 589eeef HDDS-12371. Duplicated key scanning on multipartInfo table when listing multipart uploads (apache#7937) 6e766bf HDDS-12367. Change ignorePipeline log level to DEBUG in OmKeyInfo (apache#7939) c87caa5 HDDS-12226. TestSecureOzoneRpcClient tests not run due to UnknownHostException (apache#7827) 1513c34 HDDS-11883. SCM HA: Move proxy object creation code to SCMRatisServer (apache#7914) 8c42c02 HDDS-12349. Speed up some HDDS integration tests (apache#7932) a00787c HDDS-11532. Sort multipart uploads on ListMultipartUploads response (apache#7929) Conflicts: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/ContainerDataYaml.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/HddsDispatcher.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/interfaces/Container.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/interfaces/Handler.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/DatanodeConfiguration.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainer.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainerCheck.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueHandler.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/ozoneimpl/OzoneContainer.java hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/ContainerTestUtils.java hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/TestSchemaTwoBackwardsCompatibility.java hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/impl/TestHddsDispatcher.java hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/interfaces/TestHandler.java hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/TestKeyValueHandler.java hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/replication/TestReplicationSupervisor.java hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMClientProtocolServer.java hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/ozone/audit/SCMAction.java hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/TestContainerReportHandler.java hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/TestIncrementalContainerReportHandler.java hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/container/ContainerCommands.java hadoop-ozone/dist/src/main/smoketest/admincli/container.robot hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestECKeyOutputStream.java hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/metrics/TestContainerMetrics.java hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/ozoneimpl/TestOzoneContainerWithTLS.java hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/server/TestContainerServer.java hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/server/TestSecureContainerServer.java hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/fsck/ContainerHealthTask.java hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/fsck/TestContainerHealthTask.java
aswinshakil
added a commit
to aswinshakil/ozone
that referenced
this pull request
Apr 25, 2025
…239-container-reconciliation Commits: 318 68452d0 HDDS-12893. addendum: cp: warning: behavior of -n is non-portable and may change in future; use --update=none instead (apache#8334) 6852e3e HDDS-12861. Add AvoidStringBufferField PMD rule (apache#8312) c21ec5b HDDS-12880. Move field declarations to start of class in hdds-server-framework (apache#8310) 56b5b02 HDDS-12707. Recon - In-memory extraction of om tarball from network input stream (apache#8212) 64dbf44 HDDS-12893. cp: warning: behavior of -n is non-portable and may change in future; use --update=none instead (apache#8328) 250d3c1 HDDS-12891. OMKeyAclRequestWithFSO is incorrectly setting full path as key name. (apache#8326) f6597d1 HDDS-12888. Add negative test cases for FS operations on OBS buckets (apache#8323) cdab4eb HDDS-12173. Follow RocksDB basic tuning guide (apache#8206) 6f25b43 HDDS-12347. Fix spotbugs warnings in integration tests (apache#8298) e83569f HDDS-12816. Ozone debug replicas chunk-info has incorrect or None in FileLocations (apache#8289) 621f018 HDDS-12624. Fix pipeline limit check to prevent incorrect pipeline creation (apache#8216) dd8950b HDDS-12340. Remove unnecessary findbugs exclusions (apache#8309) 0d88da0 HDDS-12723. Handle Volume Db failure in volume scanner (apache#8256) 434a5d5 HDDS-12823. SnapshotDiffReportOzone#fromProtobuf empty token handling (apache#8280) 7c23cbd HDDS-12504. Replace calls to deprecated RandomStringUtils methods (apache#8306) 3ee57e6 HDDS-12813. Replace calls to deprecated RandomUtils methods (apache#8302) 84b2ff6 HDDS-12876. Bump awssdk to 2.31.25 (apache#8304) db9076e HDDS-12867. Replace hard-coded namespace URL with constant S3_XML_NAMESPACE (apache#8299) 97b35ee HDDS-12874. Bump picocli to 4.7.7 (apache#8303) 3eed6a6 HDDS-12847. Use DatanodeID instead of DatanodeDetails.getUuidString (apache#8293) 205cb2c HDDS-12846. Log DatanodeDetails instead of DatanodeDetails.getUuidString (apache#8295) 213c2fe HDDS-11734. Bump maven-compiler-plugin to 3.14.0 (apache#8301) 889f3b7 HDDS-10284. Move GenericTestUtils#getTempPath to MiniOzoneCluster (apache#8300) e613598 HDDS-12113. Move HAProxy test to HA environment (apache#8271) d8a3915 HDDS-12413. Move field declarations to start of class in hdds-container-service (apache#7968) 5c91b44 HDDS-8802. Added pagination support for ListSnapshotDiff jobs (apache#8124) e9e1493 HDDS-12704. Add missing audit logs for SCM client operations (apache#8251) e8af186 HDDS-12152. Stop testing with Hadoop 3.1.2 (apache#7773) f6a4a48 HDDS-12840. Avoid long containerId in KeyValueContainer (apache#8291) 3c9e146 HDDS-12350. Reduce duplication between OmBucketReadWrite tests (apache#8241) 93613c1 HDDS-12734. Enable native lib in CI checks (apache#8190) 3d4b5fd HDDS-6631. Fix typos in output/exception messages (apache#8294) 978dd71 HDDS-12806. Replace commons-logging with jcl-over-slf4j (apache#8265) ef8e0da HDDS-12524. Reuse TestDataUtil.createKey in more tests (apache#8270) b91e8e7 HDDS-12518. Auto-compact tables which can tend to be large in size at intervals (apache#8260) 55f6924 HDDS-12845. Reuse TestDataUtil.createKey in OzoneRpcClientTests (apache#8274) 8a4deeb HDDS-12827. Move out NodeStateMap.Entry and ContainerMap.Entry (apache#8273) 62df306 HDDS-12493. Move container upgrade under repair (apache#8205) b33ed23 HDDS-12412. Make hdds-common compliant with FieldDeclarationsShouldBeAtStartOfClass (apache#7967) a95d338 HDDS-12740. Use DatanodeID in HddsTestUtils (apache#8281) f26e2f0 HDDS-12771. xcompat fails if run in itself due to undefined OZONE_CURRENT_VERSION (apache#8236) c7117dc HDDS-11974. Split Container Safemode Rule into Ratis & EC Container Safemode Rules (apache#7951) bb2c63c HDDS-12803. OmKeyInfo#isKeyInfoSame should handle object tags (apache#8259) a27c1cf HDDS-12837. Bump vite to 4.5.13 (apache#8283) 9b48b6e HDDS-12838. Bump awssdk to 2.31.21 (apache#8267) 3d8644e HDDS-12830. Add RocksDatabaseException. (apache#8277) 9e4da5b HDDS-12821. Update Build from Source user doc. (apache#8262) 5292ac5 HDDS-12836. Bump junit to 5.12.2 (apache#8269) e6b9d5a HDDS-12825. ReconIncrementalContainerReportHandler is not synchronized on datanode. (apache#8272) a92fe59 HDDS-12581. Multi-threaded Log File Parsing with Batch Updates to DB (apache#8254) eed5924 HDDS-12770. Use ContainerID instead of Long in CONTAINER_IDS_TABLE. (apache#8247) 7dfd8c1 HDDS-12060. Replace System.currentTimeMillis() with Time.monotonicNow() for duration calculation (apache#8096) a21e362 HDDS-12547. Container creation and import use the same VolumeChoosingPolicy (apache#8090) 22734a9 HDDS-12461. Bump Ranger to 2.6.0 (apache#8120) 0d6231c HDDS-12801. SCM should remove pipeline before DN. (apache#8261) fdebdac HDDS-12761. Add missing network configuration properties in ozone-default.xml (apache#8257) 7062685 HDDS-12805. Use slf4j for HTTP request log (apache#8255) 7ff8ad7 HDDS-12755. Redundant declaration in TestHadoopNestedDirGenerator#spanCheck() (apache#8218) fbb8706 HDDS-12145. Remove unused config hdds.container.ratis.replication.level (apache#8252) 6913835 HDDS-12519. Generate auto-complete script for Ozone commands (apache#8030) 25dda2d HDDS-12408. Create mixin for ContainerID list parameters (apache#7970) f22f32e HDDS-12764. NodeDecommissionManager#monitorInterval should get with ms unit from config (apache#8250) dc9952e HDDS-12746. Reduce visibility of Logger members (apache#8210) 102ae3f HDDS-8660. Notify ReplicationManager when nodes go dead or out of service (apache#7997) a1345f9 HDDS-12368. Seek to correct start key in KeyManagerImpl#getTableEntries (apache#7925) 1ebf265 HDDS-12580. Set up Temporary DB for Storing Container Log Information (apache#8072) 189fbdb HDDS-12463. Add perf metrics in SCM for allocateBlock and openPipeline (apache#8111) fa8afa3 HDDS-12741. Recon UI should show full path from ContainerKeys API response (apache#8200) a0387af HDDS-12800. Bump aspectj-maven-plugin to 1.14.1 (apache#8230) 5733c55 HDDS-11799. Remove config hdds.scm.safemode.pipeline-availability.check (apache#8095) c962b98 HDDS-12404. Grafana dashboard for snapshot metrics (apache#7954) bf20540 HDDS-12458. Refactor DataNodeSafeModeRule to use NodeManager (apache#7998) 25b7102 HDDS-12775. flaky-test-check builds the workflow branch (apache#8237) 2a4de14 HDDS-12772. Configure initial heartbeat and first election time for quicker MiniOzoneCluster startup (apache#8235) 0f5b590 HDDS-12691. Calculation of committed space in Datanode seems incorrect (apache#8228) 6d0a830 HDDS-12721. Log more details about volumes (apache#8181) 1bee369 HDDS-12525. Replace some TestHelper#createKey usage with TestDataUtil#createKey (apache#8233) 5834fcf HDDS-10091. TestOzoneDebugShell fails with FILE_SYSTEM_OPTIMIZED bucket layout (apache#8227) 3822943 HDDS-12768. Bump vite to 4.5.12 (apache#8234) b6dac4a HDDS-12766. Bump awssdk to 2.31.16 (apache#8232) 0ef1f02 HDDS-12767. Bump jacoco to 0.8.13 (apache#8231) 2bbeaaf HDDS-12700. Upgrade test overwrites previous result (apache#8229) 0a558c1 HDDS-12758. Error in OmUtils.normalizeKey for key name starting with `//` (apache#8225) fee8817 HDDS-12759. Bump vitest to 1.6.1 (apache#7828) 1f716ea HDDS-12699. Bump vite to 4.5.11 (apache#8224) abc1e02 HDDS-12757. Duplicated declaration of dnsInterface in HddsUtils (apache#8222) 3fb57c1 HDDS-12037. Removing unit from quota namespace (apache#8148) ebcece4 HDDS-11038. Add Helm Chart to the Ozone on Kubernetes doc (apache#8220) bba8a67 HDDS-12750. Move StorageTypeProto from ScmServerDatanodeHeartbeatProtocol.proto to hdds.proto (apache#8208) bc19a4c HDDS-12711. Limit number of excluded SST files logged at info level (apache#8186) 6bcebe8 HDDS-4517. Remove leftover references to RaftServerImpl (apache#8223) 868a237 HDDS-12756. Speed up TestReconfigShell and TestOzoneDebugShell (apache#8219) 40d02b9 HDDS-12639. Add info for TimeoutException (apache#8113) 5f74da3 HDDS-12760. Mark TestContainerReplication#testImportedContainerIsClosed as flaky ed01115 HDDS-9241. Document S3 Gateway REST Secret Revoke command (apache#8221) 479f6d9 HDDS-12690. Remove om.fs.snapshot.max.limit from RDBStore and OmMetadataManagerImpl (apache#8195) 0f52a34 HDDS-12559. Implement Bulk Ozone Locks for taking locks on multiple snapshots (apache#8052) de73c00 HDDS-12738. Refactor AbstractContainerReportHandler and its subclasses. (apache#8207) 30e4aa4 HDDS-12748. Remove unused config ozone.manager.db.checkpoint.transfer.bandwidthPerSec (apache#8213) 693548b HDDS-12676. Prefer minFreeSpace if minFreeSpacePercent is also defined (apache#8180) 092fe40 HDDS-12751. native build fails with CMake 4 (apache#8215) 1433d0c HDDS-1480. Prefer resolved datanode ip address over persisted ip address (apache#7495) af5301e HDDS-12233. Atomically import a container (apache#7934) 2475949 HDDS-11107. Remove unnecessary run_test_scripts call in upgrade tests (apache#8201) 20a13da HDDS-12615. Failure of any OM task during bootstrapping of Recon needs to be handled (apache#8098) 7fb92b0 HDDS-12717. Combine nodeMap and nodeToContainer in NodeStateMap. (apache#8199) fc6776d HDDS-12739. Replace the getNodeByUuid methods in NodeManager. (apache#8202) 0d40030 HDDS-12698. Unused FailureService in MiniOzoneChaosCluster (apache#8197) 400835f HDDS-12736. Bump hadoop-thirdparty to 1.4.0 (apache#8193) a540684 HDDS-12720. Use DatanodeID in SimpleMockNodeManager (apache#8198) 604576a HDDS-12735. Unused rocksDBConfiguration variable in `OmMetadataManagerImpl#start` (apache#8196) 64318f7 HDDS-12498. Allow limiting flaky-test-check to specific submodule (apache#8194) 3e7b5eb HDDS-12719. Remove the setContainers(..) method from NodeManager. (apache#8191) 78e2e73 HDDS-12724. hdds-rocks-native build fails if JAVA_HOME not set (apache#8183) fbc696c HDDS-12594. Optimize replica checksum verifier (apache#8151) 4411e35 HDDS-12733. Bump axios to 0.30.0 (apache#8189) 31aabc1 HDDS-12732. Bump awssdk to 2.31.11 (apache#8188) d6e2509 HDDS-12731. Bump restrict-imports-enforcer-rule to 2.6.1 (apache#8187) 4213307 HDDS-12648. Fix checkLeaderStatus in removeOMFromRatisRing and addOMToRatisRing (apache#8142) e4e9547 HDDS-12709. Intermittent failure in Balancer acceptance test (apache#8182) c1d5b4f HDDS-12718. Use NodeStateMap in MockNodeManager instead of test-specific Node2ContainerMap (apache#8179) 65a9d6c HDDS-12587. Detect test class module in flaky-test-check (apache#8162) aac9383 HDDS-12529. Clean up code in AbstractContainerReportHandler. (apache#8033) 6891315 HDDS-12603. Fix ContainerTable keys re-rendering issue (apache#8163) 2a1a6bf HDDS-12670. Improve encapsulation of volume spare space check (apache#8167) 05b6eb3 HDDS-12696. Replace link to Hadoop with Ozone in httpfs site.xml (apache#8177) 7e87a8a HDDS-11879. MiniKMS fails with ClassNotFoundException: com.sun.jersey....ServletContainer (apache#8158) 841d297 HDDS-12592. Remove find missing padding check in favor of of block metadata check (apache#8145) ce003e8 HDDS-12702. Move checknative under ozone debug (apache#8170) 6ee9c2b HDDS-8007. Add more detailed stages for SnapDiff job progress tracking (apache#8010) 3ebe5e7 HDDS-12660. Allow --verbose option of GenericCli at leaf subcommands (apache#8166) c8b3ccb HDDS-12528. Create new module for S3 integration tests (apache#8168) 435dee9 HDDS-12703. Close pipeline command should display error on failure (apache#8169) deb8e7c HDDS-12705. Replace Whitebox with HddsWhiteboxTestUtils (apache#8172) 0d8aecc HDDS-12327. HDDS-12668. Fix HSync upgrade test failure in non-HA upgrade test (apache#8171) bef4bee HDDS-12239. (addendum) Volume should not be marked as unhealthy when disk full - fix compile error e3e47ea HDDS-12235. Reserve space on DN during container import operation. (apache#7981) 4ffae70 HDDS-12239. Volume should not be marked as unhealthy when disk full (apache#7830) 0781cbe HDDS-12694. Disable TestMiniChaosOzoneCluster after fixing init and shutdown (apache#8159) 92aa71d HDDS-12687. Avoid ambiguity in URI descriptions (apache#8155) 5367016 HDDS-12679. Merge VolumeInfo into StorageVolume (apache#8147) 2cf6d59 HDDS-12553. ozone admin container list should output real JSON array (apache#8050) 63fcb27 HDDS-12465. Intermittent failure in TestOzoneFileSystemMetrics (apache#8130) bab26a4 HDDS-12684. Update NOTICE and LICENSE file (apache#8160) 857cb76 HDDS-12327. Restore non-HA (to HA) upgrade test (apache#7880) 587e9ff HDDS-12320. Collect OM performance metrics for FSO key delete (apache#7883) 560d017 HDDS-12500. Do not skip JUnit tests in post-commit runs (apache#8024) 273a627 HDDS-12686. Remove output of OzoneAddress in --verbose mode CLI (apache#8153) 4d3d834 HDDS-12650. Added logs to SnapshotDeletingService to indicate skipped snapshots. (apache#8123) 7524736 Revert "HDDS-12528. Create new module for S3 integration tests (apache#8152)" d44ebf5 HDDS-12528. Create new module for S3 integration tests (apache#8152) 3739b05 HDDS-12604. Reduce duplication in TestContainerStateMachine (apache#8104) a1616ae HDDS-12485. Repair tool should only print user warning for offline commands (apache#8140) de5c0a3 HDDS-12668. HSync upgrade test failure (apache#8137) bee10f0 HDDS-12671. Include .editorconfig and .run in source tarball (apache#8139) 8e7d370 HDDS-12666. Remove -SNAPSHOT from OZONE_CURRENT_VERSION in upgrade test (apache#8136) 4e0a764 HDDS-12486. Warmup KMS encrypted keys when OM starts (apache#8081) 0910997 HDDS-12636. Reduce code duplication for tarball creation (apache#8121) cbafa02 HDDS-12642. ACL test assertions depend on JSON element order (apache#8143) 8691adf HDDS-12622. Refactor minFreeSpace calculation (apache#8119) 1b0e912 HDDS-12661. Standardize Maven module names (apache#8129) ebf5cc6 HDDS-12662. Rename upgrade callback directory 1.5.0 to 2.0.0 (apache#8131) 828b2d1 HDDS-12667. Bump awssdk to 2.31.6 (apache#8134) 2596b3b HDDS-12665. Bump zstd-jni to 1.5.7-2 (apache#8133) 0e07983 HDDS-12664. Bump copy-rename-maven-plugin to 1.0.1 (apache#8132) 3173501 HDDS-12462. Use exclude rules for defining shaded filesystem jar contents (apache#8008) ee6201b HDDS-12341. Share cluster in client tests (apache#8126) bd579b3 HDDS-12646. Improve OM decommission check (apache#8122) 482024e HDDS-12569. Extract MiniOzoneCluster to separate module (apache#8067) 7a3ad16 HDDS-12641. Move Lease to hdds-server-framework (apache#8128) e805c15 HDDS-12310. Online repair command to perform compaction on om.db (apache#7957) 7b9c152 HDDS-12473. Trim duplicate space in proto message definition (apache#8005) 918bb98 HDDS-12626. Move the compare method in NodeStatus to ECPipelineProvider. (apache#8116) bf5a2f2 HDDS-12644. Create factory method for OzoneAcl (apache#8115) a646b85 HDDS-12557. Add progress indicator for checkpoint tarball in leader OM (apache#8085) 9e23d59 HDDS-12627. NodeStateMap may handle opStateExpiryEpochSeconds incorrectly. (apache#8117) d19f3bc HDDS-12640. Move GrpcMetrics to hdds-server-framework (apache#8114) 1c0b445 HDDS-12426. SCM replication should check double of container size. (apache#8080) 157cca4 HDDS-12358. Intermittent failure in compatibility acceptance test (apache#8012) 9b25d02 HDDS-12549. refactor ratis request to common place (apache#8059) 3349248 HDDS-12588. Recon Containers page shows number of blocks, not keys (apache#8074) 693d0f4 HDDS-12589. Fix Incorrect FSO Key Listing for Container-to-Key Mapping. (apache#8109) 17dbc4f HDDS-12585. Recon ContainerHealthTask ConstraintViolationException error handling. (apache#8070) 53c0a32 HDDS-12620. Fix OM Mismatch Deleted Container API (apache#8102) 45c900d HDDS-12619. Optimize Recon OM Container Mismatch API (apache#8101) 57fda0c HDDS-12633. KEY_NOT_FOUND in OzoneRpcClientTests for LEGACY bucket with enableFileSystemPaths=true (apache#8108) 23b0505 Revert "HDDS-12630. Enable GitHub Discussions in asf.yml (apache#8107)" b402b7c HDDS-12621. Change NodeStatus to value-based. (apache#8105) 20aeda5 HDDS-12601. Unknown tarball cleanup for Recon OM DB snapshot. (apache#8084) 93cd5aa HDDS-12630. Enable GitHub Discussions in asf.yml (apache#8107) 61a36f6 HDDS-12373. Use File.getUsableSpace() instead of File.getFreeSpace() to calcuate usedSpace in DedicatedDiskSpaceUsage (apache#7927) 953e718 HDDS-12565. Treat volumeFreeSpaceToSpare as reserved space (apache#8086) 8489cc8 HDDS-11735. Update ozone-default.xml for volume choosing policy (apache#8103) 13c5be8 HDDS-12577. [Ozone 2.0] Update master branch version number. 1a9d9f7 HDDS-12446. Add a Grafana dashboard for low level RocksDB operations. (apache#7992) c9990ca HDDS-12576. [Ozone 2.0] Update proto.lock files (apache#8064) b68b94c HDDS-12617. Use DatanodeID as keys in NodeStateMap. (apache#8100) b71d408 Revert "HDDS-12589. Fix Incorrect FSO Key Listing for Container-to-Key Mapping. (apache#8078)" 68e3842 HDDS-12589. Fix Incorrect FSO Key Listing for Container-to-Key Mapping. (apache#8078) c8e77f8 HDDS-12602. Intermittent failure in TestContainerStateMachine.testWriteFailure (apache#8099) 08ac32d HDDS-12608. Race condition in datanode version file creation (apache#8093) e5ef35d HDDS-12582. TypedTable support using different codec (apache#8073) dc47897 HDDS-12611. Snapshot creation is removing extra keys from AOS's DB (apache#8094) ff3ef51 HDDS-12551. Replace dnsToUuidMap with dnsToDnIdMap in SCMNodeManager. (apache#8087) 28520a7 HDDS-12614. Configurable java version in flaky-test-check with default to 21 (apache#8097) 434d5bd HDDS-12616. Bump junit to 5.12.1 (apache#8088) 699ee88 HDDS-12610. Bump awssdk to 2.31.1 (apache#8089) 0a85f9b HDDS-12602. Mark TestContainerStateMachineLeader/Follower as flaky 03c80f8 HDDS-12591. Include ContainerInfo in ContainerAttribute. (apache#8083) 49a2c85 HDDS-12097. Enhance Container Key Mapper for Faster Processing. (apache#8002) 5b7f96c HDDS-12303. Move ozone.om.user.max.volume into OmConfig (apache#8082) afc4043 HDDS-12566. Handle Over replication of Quasi Closed Stuck containers (apache#8061) eb96ff4 HDDS-12590. Used db name as the threadNamePrefix. (apache#8076) cf5bad7 HDDS-12539. Enable some String-related rules in PMD (apache#8047) d26f711 HDDS-12533. Offline repair command for generic rocksDB compaction (apache#8039) 93563e9 HDDS-12573. Pipeline#toString should separate ReplicaIndex from next node UUID. (apache#8063) 6e40831 HDDS-12572. Remove the ContainerID parameter when it has ContainerReplica. (apache#8075) 786da39 HDDS-12057. Implement command ozone debug replicas verify checksums (apache#7748) 66bc7ea HDDS-12535. Intermittent failure in TestContainerReportHandling (apache#8060) 87a674c HDDS-12555. Combine containerMap and replicaMap in ContainerStateMap. (apache#8057) f3689b6 HDDS-12458. Show safemode rules status irrespective of whether SCM is in safe mode in verbose mode. (apache#8049) 7d8c771 HDDS-12469. Mark statemachine unhealthy for write operation timeout. (apache#8022) 9aa41fd HDDS-12574. Add script to find modules by test classes (apache#8062) 5e28acb HDDS-12537. Selective checks: skip tests for PMD ruleset change (apache#8040) 9a3433b HDDS-12520. Move auditparser under debug (apache#8041) 0335385 HDDS-12420. Move FinalizeUpgradeCommandUtil to hdds-common. (apache#8023) 75bbba3 HDDS-12552. Fix raw use of generic class SCMCommand (apache#8048) d71aadf HDDS-12527. Separate S3 Gateway from MiniOzoneCluster (apache#8058) de8cf16 HDDS-12483. Quasi Closed Stuck should have 2 replicas of each origin (apache#8014) 57a139e HDDS-11576. Create a separate S3 client factory (apache#8051) e8a4668 HDDS-12383. Fix spotbugs warnings in hdds-common and httpfsgateway (apache#8046) 0dab553 HDDS-12541. Change ContainerID to value-based (apache#8044) 1202f6d HDDS-12532. Support only Enum in ContainerAttribute. (apache#8036) a2ad1e3 HDDS-12536. Move InMemoryTestTable to test (apache#8043) 7164c76 HDDS-12488. S3G should handle the signature calculation with trailers (apache#8020) c8c6d0e HDDS-12534. Remove drop_column_family command from ozone debug ldb (apache#8038) 9a8321e HDDS-12535. Mark TestContainerReportHandling as flaky ddd89fb HDDS-12543. Remove duplicate license information (apache#8045) 08e2c0a HDDS-12531. Use AtomicFileOutputStream to write YAML files. (apache#8035) 1bd8d8f HDDS-11813. Reduce duplication in CI workflow (apache#7497) 250bd5f HDDS-12450. Enable SimplifiableTestAssertion check in PMD (apache#8032) 96273ae HDDS-12489. Intermittent timeout in TestSCMContainerManagerMetrics.testReportProcessingMetrics (apache#8021) 2315927 HDDS-12476. Add TestDataUtil#createKey variant with small random content (apache#8028) d95ca4c HDDS-12421. ContainerReportHandler should not make the call to delete replicas (apache#7976) 26c859c HDDS-12204. Improve failover logging (apache#7867) efbf79c HDDS-12236. ContainerStateMachine should not apply or write future transactions in the event of failure (apache#7862) 3f88dbe HDDS-12377. Improve error handling of OM background tasks processing in case of abrupt crash of Recon. (apache#7960) 72da3a6 HDDS-12477. Do not force RATIS/ONE replication in TestDataUtil#createKey (apache#8017) a428b15 HDDS-12496. Use TextFormat#shortDebugString to flatten proto message in SCMDatanodeProtocolServer. (apache#8019) ecd2de0 HDDS-12409. Log an error before increasing the sequence id of a CLOSED container in SCM (apache#7964) 978e4a7 HDDS-12168. Create new Grafana panel to display cluster growth rate (apache#7978) 9ab7c70 HDDS-12456. Avoid FileInputStream and FileOutputStream (apache#8015) 9d41cd7 HDDS-12474. Add latency metrics of deletion services to grafana dashboard (apache#8007) 2b48e8c HDDS-12354. Move Storage and UpgradeFinalizer to hdds-server-framework (apache#7973) ed737b3 HDDS-12428. Avoid force closing OPEN/CLOSING replica of a CLOSED Container (apache#7985) d1e8b90 HDDS-12210. Use correct BootstrapStateHandler.Lock in SnapshotDeletingService (apache#7991) b769a26 HDDS-12295. Allow updating OM default replication config for tests (apache#7974) dd74eee HDDS-12430. Document in ozone-default.xml the config keys moved from DFSConfigKeysLegacy (apache#7987) 34041ca HDDS-12466. Set default commit message to PR title (apache#8013) 1ea0735 HDDS-12467. Enable new asf.yaml parser (apache#8011) e87b8db HDDS-12417. Reduce duplication of createKey variants in TestDataUtil (apache#7999) 86d2027 HDDS-12193. Provide option to disable RDBStoreMetrics for Snapshotted DB (apache#7982) 7c1d201 HDDS-12451. Create factory for MultiTenantAccessController (apache#7996) c2a934c HDDS-12470. Revert workaround added by HDDS-8715 to preserve thread name. (apache#8004) b8c93cc HDDS-12376. (addendum: fix findbugs) Remove scmRatisEnabled from ScmInfo. (apache#7931) 4005a10 HDDS-12376. (addendum: fix pmd) Remove scmRatisEnabled from ScmInfo. (apache#7931) abfa3be HDDS-12442. Add latency metrics for OM deletion services (apache#7986) 14db15c HDDS-12376. Remove scmRatisEnabled from ScmInfo. (apache#7931) 7d31d9e HDDS-12410. Add detailed block info for ALLOCATE_BLOCK audit log (apache#7965) a8f0ff3 HDDS-12460. (addendum) Move hdds-test-utils code to src/test 4c28c7f HDDS-12460. Move hdds-test-utils code to src/test (apache#8000) bb16f66 HDDS-12448. Avoid using Jackson1 (apache#7994) be0e1e6 HDDS-12445. Remove unused code from ContainerStateMap. (apache#7990) a3c9c0e HDDS-12443. Intermittent failure in TestContainerBalancerSubCommand (apache#7989) 83fd8d7 HDDS-12449. Enable UseCollectionIsEmpty check in PMD (apache#7995) d18da13 HDDS-12452. Bump slf4j to 2.0.17 (apache#7993) 6a9e8b1 HDDS-12348. Reuse `TestDataUtil.createKey` method (apache#7971) 384d774 HDDS-12416. Enable UnusedPrivateField check in PMD (apache#7975) 39d7da3 HDDS-12424. Allow config key to include config group prefix (apache#7979) ba0939b HDDS-12156. Add container health task metrics in Recon. (apache#7786) d34aee4 HDDS-12172. Rename Java constants of DFSConfigKeysLegacy keys (apache#7922) 59aaa5c HDDS-12418. Remove healthyReplicaCountAdapter from RatisContainerReplicaCount (apache#7972) 232e780 HDDS-12150. Abnormal container states should not crash the SCM ContainerReportHandler thread (apache#7882) a708ea4 HDDS-12351. Move SCMHAUtils and ServerUtils to hdds-server-framework (apache#7961) 052bd2d HDDS-12198. Exclude Recon generated code in coverage (apache#7962) ba44e12 HDDS-12382. Fix other spotbugs warnings (apache#7969) e2bf599 HDDS-12345. Share cluster in filesystem tests (apache#7959) f9f1c80 HDDS-12411. Make hdds-client compliant with FieldDeclarationsShouldBeAtStartOfClass (apache#7966) 46f4986 HDDS-11512. Create Grafana dashboard for tracking system wide deletion (apache#7813) f7e9aed HDDS-12396. Enable UnusedPrivateMethod check in PMD (apache#7956) 619f524 HDDS-1234. Fix spotbugs warnings in ozone-manager (apache#7963) 1d64b37 HDDS-12380. Fix spotbugs warnings in hdds-container-service (apache#7958) 5c2b8f6 HDDS-12065. Checkpoint directory should be cleared on startup (apache#7681) a31755a HDDS-12315. Speed up some Freon integration tests (apache#7870) d8f3149 HDDS-12399. Enable PMD.ForLoopCanBeForeach rule (apache#7952) 393211a HDDS-12062. Recon - Error handling in NSSummaryTask to avoid data inconsistencies. (apache#7723) ecc2403 HDDS-9792. Add tests for Pipelines page (apache#7859) e4de75e HDDS-12388. Key rewrite tests should be skipped if feature is disabled (apache#7953) 7b0fe61 HDDS-12288. (addendum) fix checkstyle d5df7e4 HDDS-12288. Improve bootstrap logging to indicate progress of snapshot download. (apache#7861) 58a0453 HDDS-12381. Fix spotbugs warnings in TestHddsUtils (apache#7955) 1a0c223 HDDS-11768. Extract SCM failover proxy provider logic (apache#7950) b45de0f HDDS-12403. Bump zstd-jni to 1.5.7-1 (apache#7949) a57809e HDDS-12402. Bump sqlite-jdbc to 3.49.1.0 (apache#7948) 221d53d HDDS-12400. Bump junit to 5.12.0 (apache#7947) f17abae HDDS-12398. Enable PMD checks for tests (apache#7946) 540f67b HDDS-12353. Move SpaceUsage implementations to hdds-server-framework (apache#7926) 70b93dc HDDS-10764.Tarball creation failing on leader OM node. (apache#7941) 6b20afc HDDS-12387. Cleanup TestContainerOperations (apache#7940) 417ae7c HDDS-12365. Provide editor settings and IDEA run config in standard location (apache#7924) a2c5c8e HDDS-12293. Make ozone.om.server.list.max.size reconfigurable (apache#7938) e0bd2cc HDDS-12185. Enhance FileSizeCountTask for Faster Processing. (apache#7796) 589eeef HDDS-12371. Duplicated key scanning on multipartInfo table when listing multipart uploads (apache#7937) 6e766bf HDDS-12367. Change ignorePipeline log level to DEBUG in OmKeyInfo (apache#7939) c87caa5 HDDS-12226. TestSecureOzoneRpcClient tests not run due to UnknownHostException (apache#7827) 1513c34 HDDS-11883. SCM HA: Move proxy object creation code to SCMRatisServer (apache#7914) 8c42c02 HDDS-12349. Speed up some HDDS integration tests (apache#7932) a00787c HDDS-11532. Sort multipart uploads on ListMultipartUploads response (apache#7929) Conflicts: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/ContainerDataYaml.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/HddsDispatcher.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/interfaces/Container.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/interfaces/Handler.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/DatanodeConfiguration.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainer.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainerCheck.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueHandler.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/ozoneimpl/OzoneContainer.java hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/ContainerTestUtils.java hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/TestSchemaTwoBackwardsCompatibility.java hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/impl/TestHddsDispatcher.java hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/interfaces/TestHandler.java hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/TestKeyValueHandler.java hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/replication/TestReplicationSupervisor.java hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMClientProtocolServer.java hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/ozone/audit/SCMAction.java hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/TestContainerReportHandler.java hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/TestIncrementalContainerReportHandler.java hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/container/ContainerCommands.java hadoop-ozone/dist/src/main/smoketest/admincli/container.robot hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestECKeyOutputStream.java hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/metrics/TestContainerMetrics.java hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/ozoneimpl/TestOzoneContainerWithTLS.java hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/server/TestContainerServer.java hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/server/TestSecureContainerServer.java hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/fsck/ContainerHealthTask.java hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/fsck/TestContainerHealthTask.java
aswinshakil
added a commit
to aswinshakil/ozone
that referenced
this pull request
Apr 25, 2025
…239-container-reconciliation Commits: 320 a689ec8 Revert "HDDS-12173. Follow RocksDB basic tuning guide (apache#8206)" 5c5db8e HDDS-12897. Enable EmptyLineSeparator checkstyle rule (apache#8329) 68452d0 HDDS-12893. addendum: cp: warning: behavior of -n is non-portable and may change in future; use --update=none instead (apache#8334) 6852e3e HDDS-12861. Add AvoidStringBufferField PMD rule (apache#8312) c21ec5b HDDS-12880. Move field declarations to start of class in hdds-server-framework (apache#8310) 56b5b02 HDDS-12707. Recon - In-memory extraction of om tarball from network input stream (apache#8212) 64dbf44 HDDS-12893. cp: warning: behavior of -n is non-portable and may change in future; use --update=none instead (apache#8328) 250d3c1 HDDS-12891. OMKeyAclRequestWithFSO is incorrectly setting full path as key name. (apache#8326) f6597d1 HDDS-12888. Add negative test cases for FS operations on OBS buckets (apache#8323) cdab4eb HDDS-12173. Follow RocksDB basic tuning guide (apache#8206) 6f25b43 HDDS-12347. Fix spotbugs warnings in integration tests (apache#8298) e83569f HDDS-12816. Ozone debug replicas chunk-info has incorrect or None in FileLocations (apache#8289) 621f018 HDDS-12624. Fix pipeline limit check to prevent incorrect pipeline creation (apache#8216) dd8950b HDDS-12340. Remove unnecessary findbugs exclusions (apache#8309) 0d88da0 HDDS-12723. Handle Volume Db failure in volume scanner (apache#8256) 434a5d5 HDDS-12823. SnapshotDiffReportOzone#fromProtobuf empty token handling (apache#8280) 7c23cbd HDDS-12504. Replace calls to deprecated RandomStringUtils methods (apache#8306) 3ee57e6 HDDS-12813. Replace calls to deprecated RandomUtils methods (apache#8302) 84b2ff6 HDDS-12876. Bump awssdk to 2.31.25 (apache#8304) db9076e HDDS-12867. Replace hard-coded namespace URL with constant S3_XML_NAMESPACE (apache#8299) 97b35ee HDDS-12874. Bump picocli to 4.7.7 (apache#8303) 3eed6a6 HDDS-12847. Use DatanodeID instead of DatanodeDetails.getUuidString (apache#8293) 205cb2c HDDS-12846. Log DatanodeDetails instead of DatanodeDetails.getUuidString (apache#8295) 213c2fe HDDS-11734. Bump maven-compiler-plugin to 3.14.0 (apache#8301) 889f3b7 HDDS-10284. Move GenericTestUtils#getTempPath to MiniOzoneCluster (apache#8300) e613598 HDDS-12113. Move HAProxy test to HA environment (apache#8271) d8a3915 HDDS-12413. Move field declarations to start of class in hdds-container-service (apache#7968) 5c91b44 HDDS-8802. Added pagination support for ListSnapshotDiff jobs (apache#8124) e9e1493 HDDS-12704. Add missing audit logs for SCM client operations (apache#8251) e8af186 HDDS-12152. Stop testing with Hadoop 3.1.2 (apache#7773) f6a4a48 HDDS-12840. Avoid long containerId in KeyValueContainer (apache#8291) 3c9e146 HDDS-12350. Reduce duplication between OmBucketReadWrite tests (apache#8241) 93613c1 HDDS-12734. Enable native lib in CI checks (apache#8190) 3d4b5fd HDDS-6631. Fix typos in output/exception messages (apache#8294) 978dd71 HDDS-12806. Replace commons-logging with jcl-over-slf4j (apache#8265) ef8e0da HDDS-12524. Reuse TestDataUtil.createKey in more tests (apache#8270) b91e8e7 HDDS-12518. Auto-compact tables which can tend to be large in size at intervals (apache#8260) 55f6924 HDDS-12845. Reuse TestDataUtil.createKey in OzoneRpcClientTests (apache#8274) 8a4deeb HDDS-12827. Move out NodeStateMap.Entry and ContainerMap.Entry (apache#8273) 62df306 HDDS-12493. Move container upgrade under repair (apache#8205) b33ed23 HDDS-12412. Make hdds-common compliant with FieldDeclarationsShouldBeAtStartOfClass (apache#7967) a95d338 HDDS-12740. Use DatanodeID in HddsTestUtils (apache#8281) f26e2f0 HDDS-12771. xcompat fails if run in itself due to undefined OZONE_CURRENT_VERSION (apache#8236) c7117dc HDDS-11974. Split Container Safemode Rule into Ratis & EC Container Safemode Rules (apache#7951) bb2c63c HDDS-12803. OmKeyInfo#isKeyInfoSame should handle object tags (apache#8259) a27c1cf HDDS-12837. Bump vite to 4.5.13 (apache#8283) 9b48b6e HDDS-12838. Bump awssdk to 2.31.21 (apache#8267) 3d8644e HDDS-12830. Add RocksDatabaseException. (apache#8277) 9e4da5b HDDS-12821. Update Build from Source user doc. (apache#8262) 5292ac5 HDDS-12836. Bump junit to 5.12.2 (apache#8269) e6b9d5a HDDS-12825. ReconIncrementalContainerReportHandler is not synchronized on datanode. (apache#8272) a92fe59 HDDS-12581. Multi-threaded Log File Parsing with Batch Updates to DB (apache#8254) eed5924 HDDS-12770. Use ContainerID instead of Long in CONTAINER_IDS_TABLE. (apache#8247) 7dfd8c1 HDDS-12060. Replace System.currentTimeMillis() with Time.monotonicNow() for duration calculation (apache#8096) a21e362 HDDS-12547. Container creation and import use the same VolumeChoosingPolicy (apache#8090) 22734a9 HDDS-12461. Bump Ranger to 2.6.0 (apache#8120) 0d6231c HDDS-12801. SCM should remove pipeline before DN. (apache#8261) fdebdac HDDS-12761. Add missing network configuration properties in ozone-default.xml (apache#8257) 7062685 HDDS-12805. Use slf4j for HTTP request log (apache#8255) 7ff8ad7 HDDS-12755. Redundant declaration in TestHadoopNestedDirGenerator#spanCheck() (apache#8218) fbb8706 HDDS-12145. Remove unused config hdds.container.ratis.replication.level (apache#8252) 6913835 HDDS-12519. Generate auto-complete script for Ozone commands (apache#8030) 25dda2d HDDS-12408. Create mixin for ContainerID list parameters (apache#7970) f22f32e HDDS-12764. NodeDecommissionManager#monitorInterval should get with ms unit from config (apache#8250) dc9952e HDDS-12746. Reduce visibility of Logger members (apache#8210) 102ae3f HDDS-8660. Notify ReplicationManager when nodes go dead or out of service (apache#7997) a1345f9 HDDS-12368. Seek to correct start key in KeyManagerImpl#getTableEntries (apache#7925) 1ebf265 HDDS-12580. Set up Temporary DB for Storing Container Log Information (apache#8072) 189fbdb HDDS-12463. Add perf metrics in SCM for allocateBlock and openPipeline (apache#8111) fa8afa3 HDDS-12741. Recon UI should show full path from ContainerKeys API response (apache#8200) a0387af HDDS-12800. Bump aspectj-maven-plugin to 1.14.1 (apache#8230) 5733c55 HDDS-11799. Remove config hdds.scm.safemode.pipeline-availability.check (apache#8095) c962b98 HDDS-12404. Grafana dashboard for snapshot metrics (apache#7954) bf20540 HDDS-12458. Refactor DataNodeSafeModeRule to use NodeManager (apache#7998) 25b7102 HDDS-12775. flaky-test-check builds the workflow branch (apache#8237) 2a4de14 HDDS-12772. Configure initial heartbeat and first election time for quicker MiniOzoneCluster startup (apache#8235) 0f5b590 HDDS-12691. Calculation of committed space in Datanode seems incorrect (apache#8228) 6d0a830 HDDS-12721. Log more details about volumes (apache#8181) 1bee369 HDDS-12525. Replace some TestHelper#createKey usage with TestDataUtil#createKey (apache#8233) 5834fcf HDDS-10091. TestOzoneDebugShell fails with FILE_SYSTEM_OPTIMIZED bucket layout (apache#8227) 3822943 HDDS-12768. Bump vite to 4.5.12 (apache#8234) b6dac4a HDDS-12766. Bump awssdk to 2.31.16 (apache#8232) 0ef1f02 HDDS-12767. Bump jacoco to 0.8.13 (apache#8231) 2bbeaaf HDDS-12700. Upgrade test overwrites previous result (apache#8229) 0a558c1 HDDS-12758. Error in OmUtils.normalizeKey for key name starting with `//` (apache#8225) fee8817 HDDS-12759. Bump vitest to 1.6.1 (apache#7828) 1f716ea HDDS-12699. Bump vite to 4.5.11 (apache#8224) abc1e02 HDDS-12757. Duplicated declaration of dnsInterface in HddsUtils (apache#8222) 3fb57c1 HDDS-12037. Removing unit from quota namespace (apache#8148) ebcece4 HDDS-11038. Add Helm Chart to the Ozone on Kubernetes doc (apache#8220) bba8a67 HDDS-12750. Move StorageTypeProto from ScmServerDatanodeHeartbeatProtocol.proto to hdds.proto (apache#8208) bc19a4c HDDS-12711. Limit number of excluded SST files logged at info level (apache#8186) 6bcebe8 HDDS-4517. Remove leftover references to RaftServerImpl (apache#8223) 868a237 HDDS-12756. Speed up TestReconfigShell and TestOzoneDebugShell (apache#8219) 40d02b9 HDDS-12639. Add info for TimeoutException (apache#8113) 5f74da3 HDDS-12760. Mark TestContainerReplication#testImportedContainerIsClosed as flaky ed01115 HDDS-9241. Document S3 Gateway REST Secret Revoke command (apache#8221) 479f6d9 HDDS-12690. Remove om.fs.snapshot.max.limit from RDBStore and OmMetadataManagerImpl (apache#8195) 0f52a34 HDDS-12559. Implement Bulk Ozone Locks for taking locks on multiple snapshots (apache#8052) de73c00 HDDS-12738. Refactor AbstractContainerReportHandler and its subclasses. (apache#8207) 30e4aa4 HDDS-12748. Remove unused config ozone.manager.db.checkpoint.transfer.bandwidthPerSec (apache#8213) 693548b HDDS-12676. Prefer minFreeSpace if minFreeSpacePercent is also defined (apache#8180) 092fe40 HDDS-12751. native build fails with CMake 4 (apache#8215) 1433d0c HDDS-1480. Prefer resolved datanode ip address over persisted ip address (apache#7495) af5301e HDDS-12233. Atomically import a container (apache#7934) 2475949 HDDS-11107. Remove unnecessary run_test_scripts call in upgrade tests (apache#8201) 20a13da HDDS-12615. Failure of any OM task during bootstrapping of Recon needs to be handled (apache#8098) 7fb92b0 HDDS-12717. Combine nodeMap and nodeToContainer in NodeStateMap. (apache#8199) fc6776d HDDS-12739. Replace the getNodeByUuid methods in NodeManager. (apache#8202) 0d40030 HDDS-12698. Unused FailureService in MiniOzoneChaosCluster (apache#8197) 400835f HDDS-12736. Bump hadoop-thirdparty to 1.4.0 (apache#8193) a540684 HDDS-12720. Use DatanodeID in SimpleMockNodeManager (apache#8198) 604576a HDDS-12735. Unused rocksDBConfiguration variable in `OmMetadataManagerImpl#start` (apache#8196) 64318f7 HDDS-12498. Allow limiting flaky-test-check to specific submodule (apache#8194) 3e7b5eb HDDS-12719. Remove the setContainers(..) method from NodeManager. (apache#8191) 78e2e73 HDDS-12724. hdds-rocks-native build fails if JAVA_HOME not set (apache#8183) fbc696c HDDS-12594. Optimize replica checksum verifier (apache#8151) 4411e35 HDDS-12733. Bump axios to 0.30.0 (apache#8189) 31aabc1 HDDS-12732. Bump awssdk to 2.31.11 (apache#8188) d6e2509 HDDS-12731. Bump restrict-imports-enforcer-rule to 2.6.1 (apache#8187) 4213307 HDDS-12648. Fix checkLeaderStatus in removeOMFromRatisRing and addOMToRatisRing (apache#8142) e4e9547 HDDS-12709. Intermittent failure in Balancer acceptance test (apache#8182) c1d5b4f HDDS-12718. Use NodeStateMap in MockNodeManager instead of test-specific Node2ContainerMap (apache#8179) 65a9d6c HDDS-12587. Detect test class module in flaky-test-check (apache#8162) aac9383 HDDS-12529. Clean up code in AbstractContainerReportHandler. (apache#8033) 6891315 HDDS-12603. Fix ContainerTable keys re-rendering issue (apache#8163) 2a1a6bf HDDS-12670. Improve encapsulation of volume spare space check (apache#8167) 05b6eb3 HDDS-12696. Replace link to Hadoop with Ozone in httpfs site.xml (apache#8177) 7e87a8a HDDS-11879. MiniKMS fails with ClassNotFoundException: com.sun.jersey....ServletContainer (apache#8158) 841d297 HDDS-12592. Remove find missing padding check in favor of of block metadata check (apache#8145) ce003e8 HDDS-12702. Move checknative under ozone debug (apache#8170) 6ee9c2b HDDS-8007. Add more detailed stages for SnapDiff job progress tracking (apache#8010) 3ebe5e7 HDDS-12660. Allow --verbose option of GenericCli at leaf subcommands (apache#8166) c8b3ccb HDDS-12528. Create new module for S3 integration tests (apache#8168) 435dee9 HDDS-12703. Close pipeline command should display error on failure (apache#8169) deb8e7c HDDS-12705. Replace Whitebox with HddsWhiteboxTestUtils (apache#8172) 0d8aecc HDDS-12327. HDDS-12668. Fix HSync upgrade test failure in non-HA upgrade test (apache#8171) bef4bee HDDS-12239. (addendum) Volume should not be marked as unhealthy when disk full - fix compile error e3e47ea HDDS-12235. Reserve space on DN during container import operation. (apache#7981) 4ffae70 HDDS-12239. Volume should not be marked as unhealthy when disk full (apache#7830) 0781cbe HDDS-12694. Disable TestMiniChaosOzoneCluster after fixing init and shutdown (apache#8159) 92aa71d HDDS-12687. Avoid ambiguity in URI descriptions (apache#8155) 5367016 HDDS-12679. Merge VolumeInfo into StorageVolume (apache#8147) 2cf6d59 HDDS-12553. ozone admin container list should output real JSON array (apache#8050) 63fcb27 HDDS-12465. Intermittent failure in TestOzoneFileSystemMetrics (apache#8130) bab26a4 HDDS-12684. Update NOTICE and LICENSE file (apache#8160) 857cb76 HDDS-12327. Restore non-HA (to HA) upgrade test (apache#7880) 587e9ff HDDS-12320. Collect OM performance metrics for FSO key delete (apache#7883) 560d017 HDDS-12500. Do not skip JUnit tests in post-commit runs (apache#8024) 273a627 HDDS-12686. Remove output of OzoneAddress in --verbose mode CLI (apache#8153) 4d3d834 HDDS-12650. Added logs to SnapshotDeletingService to indicate skipped snapshots. (apache#8123) 7524736 Revert "HDDS-12528. Create new module for S3 integration tests (apache#8152)" d44ebf5 HDDS-12528. Create new module for S3 integration tests (apache#8152) 3739b05 HDDS-12604. Reduce duplication in TestContainerStateMachine (apache#8104) a1616ae HDDS-12485. Repair tool should only print user warning for offline commands (apache#8140) de5c0a3 HDDS-12668. HSync upgrade test failure (apache#8137) bee10f0 HDDS-12671. Include .editorconfig and .run in source tarball (apache#8139) 8e7d370 HDDS-12666. Remove -SNAPSHOT from OZONE_CURRENT_VERSION in upgrade test (apache#8136) 4e0a764 HDDS-12486. Warmup KMS encrypted keys when OM starts (apache#8081) 0910997 HDDS-12636. Reduce code duplication for tarball creation (apache#8121) cbafa02 HDDS-12642. ACL test assertions depend on JSON element order (apache#8143) 8691adf HDDS-12622. Refactor minFreeSpace calculation (apache#8119) 1b0e912 HDDS-12661. Standardize Maven module names (apache#8129) ebf5cc6 HDDS-12662. Rename upgrade callback directory 1.5.0 to 2.0.0 (apache#8131) 828b2d1 HDDS-12667. Bump awssdk to 2.31.6 (apache#8134) 2596b3b HDDS-12665. Bump zstd-jni to 1.5.7-2 (apache#8133) 0e07983 HDDS-12664. Bump copy-rename-maven-plugin to 1.0.1 (apache#8132) 3173501 HDDS-12462. Use exclude rules for defining shaded filesystem jar contents (apache#8008) ee6201b HDDS-12341. Share cluster in client tests (apache#8126) bd579b3 HDDS-12646. Improve OM decommission check (apache#8122) 482024e HDDS-12569. Extract MiniOzoneCluster to separate module (apache#8067) 7a3ad16 HDDS-12641. Move Lease to hdds-server-framework (apache#8128) e805c15 HDDS-12310. Online repair command to perform compaction on om.db (apache#7957) 7b9c152 HDDS-12473. Trim duplicate space in proto message definition (apache#8005) 918bb98 HDDS-12626. Move the compare method in NodeStatus to ECPipelineProvider. (apache#8116) bf5a2f2 HDDS-12644. Create factory method for OzoneAcl (apache#8115) a646b85 HDDS-12557. Add progress indicator for checkpoint tarball in leader OM (apache#8085) 9e23d59 HDDS-12627. NodeStateMap may handle opStateExpiryEpochSeconds incorrectly. (apache#8117) d19f3bc HDDS-12640. Move GrpcMetrics to hdds-server-framework (apache#8114) 1c0b445 HDDS-12426. SCM replication should check double of container size. (apache#8080) 157cca4 HDDS-12358. Intermittent failure in compatibility acceptance test (apache#8012) 9b25d02 HDDS-12549. refactor ratis request to common place (apache#8059) 3349248 HDDS-12588. Recon Containers page shows number of blocks, not keys (apache#8074) 693d0f4 HDDS-12589. Fix Incorrect FSO Key Listing for Container-to-Key Mapping. (apache#8109) 17dbc4f HDDS-12585. Recon ContainerHealthTask ConstraintViolationException error handling. (apache#8070) 53c0a32 HDDS-12620. Fix OM Mismatch Deleted Container API (apache#8102) 45c900d HDDS-12619. Optimize Recon OM Container Mismatch API (apache#8101) 57fda0c HDDS-12633. KEY_NOT_FOUND in OzoneRpcClientTests for LEGACY bucket with enableFileSystemPaths=true (apache#8108) 23b0505 Revert "HDDS-12630. Enable GitHub Discussions in asf.yml (apache#8107)" b402b7c HDDS-12621. Change NodeStatus to value-based. (apache#8105) 20aeda5 HDDS-12601. Unknown tarball cleanup for Recon OM DB snapshot. (apache#8084) 93cd5aa HDDS-12630. Enable GitHub Discussions in asf.yml (apache#8107) 61a36f6 HDDS-12373. Use File.getUsableSpace() instead of File.getFreeSpace() to calcuate usedSpace in DedicatedDiskSpaceUsage (apache#7927) 953e718 HDDS-12565. Treat volumeFreeSpaceToSpare as reserved space (apache#8086) 8489cc8 HDDS-11735. Update ozone-default.xml for volume choosing policy (apache#8103) 13c5be8 HDDS-12577. [Ozone 2.0] Update master branch version number. 1a9d9f7 HDDS-12446. Add a Grafana dashboard for low level RocksDB operations. (apache#7992) c9990ca HDDS-12576. [Ozone 2.0] Update proto.lock files (apache#8064) b68b94c HDDS-12617. Use DatanodeID as keys in NodeStateMap. (apache#8100) b71d408 Revert "HDDS-12589. Fix Incorrect FSO Key Listing for Container-to-Key Mapping. (apache#8078)" 68e3842 HDDS-12589. Fix Incorrect FSO Key Listing for Container-to-Key Mapping. (apache#8078) c8e77f8 HDDS-12602. Intermittent failure in TestContainerStateMachine.testWriteFailure (apache#8099) 08ac32d HDDS-12608. Race condition in datanode version file creation (apache#8093) e5ef35d HDDS-12582. TypedTable support using different codec (apache#8073) dc47897 HDDS-12611. Snapshot creation is removing extra keys from AOS's DB (apache#8094) ff3ef51 HDDS-12551. Replace dnsToUuidMap with dnsToDnIdMap in SCMNodeManager. (apache#8087) 28520a7 HDDS-12614. Configurable java version in flaky-test-check with default to 21 (apache#8097) 434d5bd HDDS-12616. Bump junit to 5.12.1 (apache#8088) 699ee88 HDDS-12610. Bump awssdk to 2.31.1 (apache#8089) 0a85f9b HDDS-12602. Mark TestContainerStateMachineLeader/Follower as flaky 03c80f8 HDDS-12591. Include ContainerInfo in ContainerAttribute. (apache#8083) 49a2c85 HDDS-12097. Enhance Container Key Mapper for Faster Processing. (apache#8002) 5b7f96c HDDS-12303. Move ozone.om.user.max.volume into OmConfig (apache#8082) afc4043 HDDS-12566. Handle Over replication of Quasi Closed Stuck containers (apache#8061) eb96ff4 HDDS-12590. Used db name as the threadNamePrefix. (apache#8076) cf5bad7 HDDS-12539. Enable some String-related rules in PMD (apache#8047) d26f711 HDDS-12533. Offline repair command for generic rocksDB compaction (apache#8039) 93563e9 HDDS-12573. Pipeline#toString should separate ReplicaIndex from next node UUID. (apache#8063) 6e40831 HDDS-12572. Remove the ContainerID parameter when it has ContainerReplica. (apache#8075) 786da39 HDDS-12057. Implement command ozone debug replicas verify checksums (apache#7748) 66bc7ea HDDS-12535. Intermittent failure in TestContainerReportHandling (apache#8060) 87a674c HDDS-12555. Combine containerMap and replicaMap in ContainerStateMap. (apache#8057) f3689b6 HDDS-12458. Show safemode rules status irrespective of whether SCM is in safe mode in verbose mode. (apache#8049) 7d8c771 HDDS-12469. Mark statemachine unhealthy for write operation timeout. (apache#8022) 9aa41fd HDDS-12574. Add script to find modules by test classes (apache#8062) 5e28acb HDDS-12537. Selective checks: skip tests for PMD ruleset change (apache#8040) 9a3433b HDDS-12520. Move auditparser under debug (apache#8041) 0335385 HDDS-12420. Move FinalizeUpgradeCommandUtil to hdds-common. (apache#8023) 75bbba3 HDDS-12552. Fix raw use of generic class SCMCommand (apache#8048) d71aadf HDDS-12527. Separate S3 Gateway from MiniOzoneCluster (apache#8058) de8cf16 HDDS-12483. Quasi Closed Stuck should have 2 replicas of each origin (apache#8014) 57a139e HDDS-11576. Create a separate S3 client factory (apache#8051) e8a4668 HDDS-12383. Fix spotbugs warnings in hdds-common and httpfsgateway (apache#8046) 0dab553 HDDS-12541. Change ContainerID to value-based (apache#8044) 1202f6d HDDS-12532. Support only Enum in ContainerAttribute. (apache#8036) a2ad1e3 HDDS-12536. Move InMemoryTestTable to test (apache#8043) 7164c76 HDDS-12488. S3G should handle the signature calculation with trailers (apache#8020) c8c6d0e HDDS-12534. Remove drop_column_family command from ozone debug ldb (apache#8038) 9a8321e HDDS-12535. Mark TestContainerReportHandling as flaky ddd89fb HDDS-12543. Remove duplicate license information (apache#8045) 08e2c0a HDDS-12531. Use AtomicFileOutputStream to write YAML files. (apache#8035) 1bd8d8f HDDS-11813. Reduce duplication in CI workflow (apache#7497) 250bd5f HDDS-12450. Enable SimplifiableTestAssertion check in PMD (apache#8032) 96273ae HDDS-12489. Intermittent timeout in TestSCMContainerManagerMetrics.testReportProcessingMetrics (apache#8021) 2315927 HDDS-12476. Add TestDataUtil#createKey variant with small random content (apache#8028) d95ca4c HDDS-12421. ContainerReportHandler should not make the call to delete replicas (apache#7976) 26c859c HDDS-12204. Improve failover logging (apache#7867) efbf79c HDDS-12236. ContainerStateMachine should not apply or write future transactions in the event of failure (apache#7862) 3f88dbe HDDS-12377. Improve error handling of OM background tasks processing in case of abrupt crash of Recon. (apache#7960) 72da3a6 HDDS-12477. Do not force RATIS/ONE replication in TestDataUtil#createKey (apache#8017) a428b15 HDDS-12496. Use TextFormat#shortDebugString to flatten proto message in SCMDatanodeProtocolServer. (apache#8019) ecd2de0 HDDS-12409. Log an error before increasing the sequence id of a CLOSED container in SCM (apache#7964) 978e4a7 HDDS-12168. Create new Grafana panel to display cluster growth rate (apache#7978) 9ab7c70 HDDS-12456. Avoid FileInputStream and FileOutputStream (apache#8015) 9d41cd7 HDDS-12474. Add latency metrics of deletion services to grafana dashboard (apache#8007) 2b48e8c HDDS-12354. Move Storage and UpgradeFinalizer to hdds-server-framework (apache#7973) ed737b3 HDDS-12428. Avoid force closing OPEN/CLOSING replica of a CLOSED Container (apache#7985) d1e8b90 HDDS-12210. Use correct BootstrapStateHandler.Lock in SnapshotDeletingService (apache#7991) b769a26 HDDS-12295. Allow updating OM default replication config for tests (apache#7974) dd74eee HDDS-12430. Document in ozone-default.xml the config keys moved from DFSConfigKeysLegacy (apache#7987) 34041ca HDDS-12466. Set default commit message to PR title (apache#8013) 1ea0735 HDDS-12467. Enable new asf.yaml parser (apache#8011) e87b8db HDDS-12417. Reduce duplication of createKey variants in TestDataUtil (apache#7999) 86d2027 HDDS-12193. Provide option to disable RDBStoreMetrics for Snapshotted DB (apache#7982) 7c1d201 HDDS-12451. Create factory for MultiTenantAccessController (apache#7996) c2a934c HDDS-12470. Revert workaround added by HDDS-8715 to preserve thread name. (apache#8004) b8c93cc HDDS-12376. (addendum: fix findbugs) Remove scmRatisEnabled from ScmInfo. (apache#7931) 4005a10 HDDS-12376. (addendum: fix pmd) Remove scmRatisEnabled from ScmInfo. (apache#7931) abfa3be HDDS-12442. Add latency metrics for OM deletion services (apache#7986) 14db15c HDDS-12376. Remove scmRatisEnabled from ScmInfo. (apache#7931) 7d31d9e HDDS-12410. Add detailed block info for ALLOCATE_BLOCK audit log (apache#7965) a8f0ff3 HDDS-12460. (addendum) Move hdds-test-utils code to src/test 4c28c7f HDDS-12460. Move hdds-test-utils code to src/test (apache#8000) bb16f66 HDDS-12448. Avoid using Jackson1 (apache#7994) be0e1e6 HDDS-12445. Remove unused code from ContainerStateMap. (apache#7990) a3c9c0e HDDS-12443. Intermittent failure in TestContainerBalancerSubCommand (apache#7989) 83fd8d7 HDDS-12449. Enable UseCollectionIsEmpty check in PMD (apache#7995) d18da13 HDDS-12452. Bump slf4j to 2.0.17 (apache#7993) 6a9e8b1 HDDS-12348. Reuse `TestDataUtil.createKey` method (apache#7971) 384d774 HDDS-12416. Enable UnusedPrivateField check in PMD (apache#7975) 39d7da3 HDDS-12424. Allow config key to include config group prefix (apache#7979) ba0939b HDDS-12156. Add container health task metrics in Recon. (apache#7786) d34aee4 HDDS-12172. Rename Java constants of DFSConfigKeysLegacy keys (apache#7922) 59aaa5c HDDS-12418. Remove healthyReplicaCountAdapter from RatisContainerReplicaCount (apache#7972) 232e780 HDDS-12150. Abnormal container states should not crash the SCM ContainerReportHandler thread (apache#7882) a708ea4 HDDS-12351. Move SCMHAUtils and ServerUtils to hdds-server-framework (apache#7961) 052bd2d HDDS-12198. Exclude Recon generated code in coverage (apache#7962) ba44e12 HDDS-12382. Fix other spotbugs warnings (apache#7969) e2bf599 HDDS-12345. Share cluster in filesystem tests (apache#7959) f9f1c80 HDDS-12411. Make hdds-client compliant with FieldDeclarationsShouldBeAtStartOfClass (apache#7966) 46f4986 HDDS-11512. Create Grafana dashboard for tracking system wide deletion (apache#7813) f7e9aed HDDS-12396. Enable UnusedPrivateMethod check in PMD (apache#7956) 619f524 HDDS-1234. Fix spotbugs warnings in ozone-manager (apache#7963) 1d64b37 HDDS-12380. Fix spotbugs warnings in hdds-container-service (apache#7958) 5c2b8f6 HDDS-12065. Checkpoint directory should be cleared on startup (apache#7681) a31755a HDDS-12315. Speed up some Freon integration tests (apache#7870) d8f3149 HDDS-12399. Enable PMD.ForLoopCanBeForeach rule (apache#7952) 393211a HDDS-12062. Recon - Error handling in NSSummaryTask to avoid data inconsistencies. (apache#7723) ecc2403 HDDS-9792. Add tests for Pipelines page (apache#7859) e4de75e HDDS-12388. Key rewrite tests should be skipped if feature is disabled (apache#7953) 7b0fe61 HDDS-12288. (addendum) fix checkstyle d5df7e4 HDDS-12288. Improve bootstrap logging to indicate progress of snapshot download. (apache#7861) 58a0453 HDDS-12381. Fix spotbugs warnings in TestHddsUtils (apache#7955) 1a0c223 HDDS-11768. Extract SCM failover proxy provider logic (apache#7950) b45de0f HDDS-12403. Bump zstd-jni to 1.5.7-1 (apache#7949) a57809e HDDS-12402. Bump sqlite-jdbc to 3.49.1.0 (apache#7948) 221d53d HDDS-12400. Bump junit to 5.12.0 (apache#7947) f17abae HDDS-12398. Enable PMD checks for tests (apache#7946) 540f67b HDDS-12353. Move SpaceUsage implementations to hdds-server-framework (apache#7926) 70b93dc HDDS-10764.Tarball creation failing on leader OM node. (apache#7941) 6b20afc HDDS-12387. Cleanup TestContainerOperations (apache#7940) 417ae7c HDDS-12365. Provide editor settings and IDEA run config in standard location (apache#7924) a2c5c8e HDDS-12293. Make ozone.om.server.list.max.size reconfigurable (apache#7938) e0bd2cc HDDS-12185. Enhance FileSizeCountTask for Faster Processing. (apache#7796) 589eeef HDDS-12371. Duplicated key scanning on multipartInfo table when listing multipart uploads (apache#7937) 6e766bf HDDS-12367. Change ignorePipeline log level to DEBUG in OmKeyInfo (apache#7939) c87caa5 HDDS-12226. TestSecureOzoneRpcClient tests not run due to UnknownHostException (apache#7827) 1513c34 HDDS-11883. SCM HA: Move proxy object creation code to SCMRatisServer (apache#7914) 8c42c02 HDDS-12349. Speed up some HDDS integration tests (apache#7932) a00787c HDDS-11532. Sort multipart uploads on ListMultipartUploads response (apache#7929) Conflicts: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/ContainerDataYaml.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/HddsDispatcher.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/interfaces/Container.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/interfaces/Handler.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/DatanodeConfiguration.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainer.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainerCheck.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueHandler.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/ozoneimpl/OzoneContainer.java hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/ContainerTestUtils.java hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/TestSchemaTwoBackwardsCompatibility.java hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/impl/TestHddsDispatcher.java hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/interfaces/TestHandler.java hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/TestKeyValueHandler.java hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/replication/TestReplicationSupervisor.java hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMClientProtocolServer.java hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/ozone/audit/SCMAction.java hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/TestContainerReportHandler.java hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/TestIncrementalContainerReportHandler.java hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/container/ContainerCommands.java hadoop-ozone/dist/src/main/smoketest/admincli/container.robot hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestECKeyOutputStream.java hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/metrics/TestContainerMetrics.java hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/ozoneimpl/TestOzoneContainerWithTLS.java hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/server/TestContainerServer.java hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/server/TestSecureContainerServer.java hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/fsck/ContainerHealthTask.java hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/fsck/TestContainerHealthTask.java
aswinshakil
added a commit
to aswinshakil/ozone
that referenced
this pull request
Apr 28, 2025
…239-container-reconciliation Commits: 320 a689ec8 Revert "HDDS-12173. Follow RocksDB basic tuning guide (apache#8206)" 5c5db8e HDDS-12897. Enable EmptyLineSeparator checkstyle rule (apache#8329) 68452d0 HDDS-12893. addendum: cp: warning: behavior of -n is non-portable and may change in future; use --update=none instead (apache#8334) 6852e3e HDDS-12861. Add AvoidStringBufferField PMD rule (apache#8312) c21ec5b HDDS-12880. Move field declarations to start of class in hdds-server-framework (apache#8310) 56b5b02 HDDS-12707. Recon - In-memory extraction of om tarball from network input stream (apache#8212) 64dbf44 HDDS-12893. cp: warning: behavior of -n is non-portable and may change in future; use --update=none instead (apache#8328) 250d3c1 HDDS-12891. OMKeyAclRequestWithFSO is incorrectly setting full path as key name. (apache#8326) f6597d1 HDDS-12888. Add negative test cases for FS operations on OBS buckets (apache#8323) cdab4eb HDDS-12173. Follow RocksDB basic tuning guide (apache#8206) 6f25b43 HDDS-12347. Fix spotbugs warnings in integration tests (apache#8298) e83569f HDDS-12816. Ozone debug replicas chunk-info has incorrect or None in FileLocations (apache#8289) 621f018 HDDS-12624. Fix pipeline limit check to prevent incorrect pipeline creation (apache#8216) dd8950b HDDS-12340. Remove unnecessary findbugs exclusions (apache#8309) 0d88da0 HDDS-12723. Handle Volume Db failure in volume scanner (apache#8256) 434a5d5 HDDS-12823. SnapshotDiffReportOzone#fromProtobuf empty token handling (apache#8280) 7c23cbd HDDS-12504. Replace calls to deprecated RandomStringUtils methods (apache#8306) 3ee57e6 HDDS-12813. Replace calls to deprecated RandomUtils methods (apache#8302) 84b2ff6 HDDS-12876. Bump awssdk to 2.31.25 (apache#8304) db9076e HDDS-12867. Replace hard-coded namespace URL with constant S3_XML_NAMESPACE (apache#8299) 97b35ee HDDS-12874. Bump picocli to 4.7.7 (apache#8303) 3eed6a6 HDDS-12847. Use DatanodeID instead of DatanodeDetails.getUuidString (apache#8293) 205cb2c HDDS-12846. Log DatanodeDetails instead of DatanodeDetails.getUuidString (apache#8295) 213c2fe HDDS-11734. Bump maven-compiler-plugin to 3.14.0 (apache#8301) 889f3b7 HDDS-10284. Move GenericTestUtils#getTempPath to MiniOzoneCluster (apache#8300) e613598 HDDS-12113. Move HAProxy test to HA environment (apache#8271) d8a3915 HDDS-12413. Move field declarations to start of class in hdds-container-service (apache#7968) 5c91b44 HDDS-8802. Added pagination support for ListSnapshotDiff jobs (apache#8124) e9e1493 HDDS-12704. Add missing audit logs for SCM client operations (apache#8251) e8af186 HDDS-12152. Stop testing with Hadoop 3.1.2 (apache#7773) f6a4a48 HDDS-12840. Avoid long containerId in KeyValueContainer (apache#8291) 3c9e146 HDDS-12350. Reduce duplication between OmBucketReadWrite tests (apache#8241) 93613c1 HDDS-12734. Enable native lib in CI checks (apache#8190) 3d4b5fd HDDS-6631. Fix typos in output/exception messages (apache#8294) 978dd71 HDDS-12806. Replace commons-logging with jcl-over-slf4j (apache#8265) ef8e0da HDDS-12524. Reuse TestDataUtil.createKey in more tests (apache#8270) b91e8e7 HDDS-12518. Auto-compact tables which can tend to be large in size at intervals (apache#8260) 55f6924 HDDS-12845. Reuse TestDataUtil.createKey in OzoneRpcClientTests (apache#8274) 8a4deeb HDDS-12827. Move out NodeStateMap.Entry and ContainerMap.Entry (apache#8273) 62df306 HDDS-12493. Move container upgrade under repair (apache#8205) b33ed23 HDDS-12412. Make hdds-common compliant with FieldDeclarationsShouldBeAtStartOfClass (apache#7967) a95d338 HDDS-12740. Use DatanodeID in HddsTestUtils (apache#8281) f26e2f0 HDDS-12771. xcompat fails if run in itself due to undefined OZONE_CURRENT_VERSION (apache#8236) c7117dc HDDS-11974. Split Container Safemode Rule into Ratis & EC Container Safemode Rules (apache#7951) bb2c63c HDDS-12803. OmKeyInfo#isKeyInfoSame should handle object tags (apache#8259) a27c1cf HDDS-12837. Bump vite to 4.5.13 (apache#8283) 9b48b6e HDDS-12838. Bump awssdk to 2.31.21 (apache#8267) 3d8644e HDDS-12830. Add RocksDatabaseException. (apache#8277) 9e4da5b HDDS-12821. Update Build from Source user doc. (apache#8262) 5292ac5 HDDS-12836. Bump junit to 5.12.2 (apache#8269) e6b9d5a HDDS-12825. ReconIncrementalContainerReportHandler is not synchronized on datanode. (apache#8272) a92fe59 HDDS-12581. Multi-threaded Log File Parsing with Batch Updates to DB (apache#8254) eed5924 HDDS-12770. Use ContainerID instead of Long in CONTAINER_IDS_TABLE. (apache#8247) 7dfd8c1 HDDS-12060. Replace System.currentTimeMillis() with Time.monotonicNow() for duration calculation (apache#8096) a21e362 HDDS-12547. Container creation and import use the same VolumeChoosingPolicy (apache#8090) 22734a9 HDDS-12461. Bump Ranger to 2.6.0 (apache#8120) 0d6231c HDDS-12801. SCM should remove pipeline before DN. (apache#8261) fdebdac HDDS-12761. Add missing network configuration properties in ozone-default.xml (apache#8257) 7062685 HDDS-12805. Use slf4j for HTTP request log (apache#8255) 7ff8ad7 HDDS-12755. Redundant declaration in TestHadoopNestedDirGenerator#spanCheck() (apache#8218) fbb8706 HDDS-12145. Remove unused config hdds.container.ratis.replication.level (apache#8252) 6913835 HDDS-12519. Generate auto-complete script for Ozone commands (apache#8030) 25dda2d HDDS-12408. Create mixin for ContainerID list parameters (apache#7970) f22f32e HDDS-12764. NodeDecommissionManager#monitorInterval should get with ms unit from config (apache#8250) dc9952e HDDS-12746. Reduce visibility of Logger members (apache#8210) 102ae3f HDDS-8660. Notify ReplicationManager when nodes go dead or out of service (apache#7997) a1345f9 HDDS-12368. Seek to correct start key in KeyManagerImpl#getTableEntries (apache#7925) 1ebf265 HDDS-12580. Set up Temporary DB for Storing Container Log Information (apache#8072) 189fbdb HDDS-12463. Add perf metrics in SCM for allocateBlock and openPipeline (apache#8111) fa8afa3 HDDS-12741. Recon UI should show full path from ContainerKeys API response (apache#8200) a0387af HDDS-12800. Bump aspectj-maven-plugin to 1.14.1 (apache#8230) 5733c55 HDDS-11799. Remove config hdds.scm.safemode.pipeline-availability.check (apache#8095) c962b98 HDDS-12404. Grafana dashboard for snapshot metrics (apache#7954) bf20540 HDDS-12458. Refactor DataNodeSafeModeRule to use NodeManager (apache#7998) 25b7102 HDDS-12775. flaky-test-check builds the workflow branch (apache#8237) 2a4de14 HDDS-12772. Configure initial heartbeat and first election time for quicker MiniOzoneCluster startup (apache#8235) 0f5b590 HDDS-12691. Calculation of committed space in Datanode seems incorrect (apache#8228) 6d0a830 HDDS-12721. Log more details about volumes (apache#8181) 1bee369 HDDS-12525. Replace some TestHelper#createKey usage with TestDataUtil#createKey (apache#8233) 5834fcf HDDS-10091. TestOzoneDebugShell fails with FILE_SYSTEM_OPTIMIZED bucket layout (apache#8227) 3822943 HDDS-12768. Bump vite to 4.5.12 (apache#8234) b6dac4a HDDS-12766. Bump awssdk to 2.31.16 (apache#8232) 0ef1f02 HDDS-12767. Bump jacoco to 0.8.13 (apache#8231) 2bbeaaf HDDS-12700. Upgrade test overwrites previous result (apache#8229) 0a558c1 HDDS-12758. Error in OmUtils.normalizeKey for key name starting with `//` (apache#8225) fee8817 HDDS-12759. Bump vitest to 1.6.1 (apache#7828) 1f716ea HDDS-12699. Bump vite to 4.5.11 (apache#8224) abc1e02 HDDS-12757. Duplicated declaration of dnsInterface in HddsUtils (apache#8222) 3fb57c1 HDDS-12037. Removing unit from quota namespace (apache#8148) ebcece4 HDDS-11038. Add Helm Chart to the Ozone on Kubernetes doc (apache#8220) bba8a67 HDDS-12750. Move StorageTypeProto from ScmServerDatanodeHeartbeatProtocol.proto to hdds.proto (apache#8208) bc19a4c HDDS-12711. Limit number of excluded SST files logged at info level (apache#8186) 6bcebe8 HDDS-4517. Remove leftover references to RaftServerImpl (apache#8223) 868a237 HDDS-12756. Speed up TestReconfigShell and TestOzoneDebugShell (apache#8219) 40d02b9 HDDS-12639. Add info for TimeoutException (apache#8113) 5f74da3 HDDS-12760. Mark TestContainerReplication#testImportedContainerIsClosed as flaky ed01115 HDDS-9241. Document S3 Gateway REST Secret Revoke command (apache#8221) 479f6d9 HDDS-12690. Remove om.fs.snapshot.max.limit from RDBStore and OmMetadataManagerImpl (apache#8195) 0f52a34 HDDS-12559. Implement Bulk Ozone Locks for taking locks on multiple snapshots (apache#8052) de73c00 HDDS-12738. Refactor AbstractContainerReportHandler and its subclasses. (apache#8207) 30e4aa4 HDDS-12748. Remove unused config ozone.manager.db.checkpoint.transfer.bandwidthPerSec (apache#8213) 693548b HDDS-12676. Prefer minFreeSpace if minFreeSpacePercent is also defined (apache#8180) 092fe40 HDDS-12751. native build fails with CMake 4 (apache#8215) 1433d0c HDDS-1480. Prefer resolved datanode ip address over persisted ip address (apache#7495) af5301e HDDS-12233. Atomically import a container (apache#7934) 2475949 HDDS-11107. Remove unnecessary run_test_scripts call in upgrade tests (apache#8201) 20a13da HDDS-12615. Failure of any OM task during bootstrapping of Recon needs to be handled (apache#8098) 7fb92b0 HDDS-12717. Combine nodeMap and nodeToContainer in NodeStateMap. (apache#8199) fc6776d HDDS-12739. Replace the getNodeByUuid methods in NodeManager. (apache#8202) 0d40030 HDDS-12698. Unused FailureService in MiniOzoneChaosCluster (apache#8197) 400835f HDDS-12736. Bump hadoop-thirdparty to 1.4.0 (apache#8193) a540684 HDDS-12720. Use DatanodeID in SimpleMockNodeManager (apache#8198) 604576a HDDS-12735. Unused rocksDBConfiguration variable in `OmMetadataManagerImpl#start` (apache#8196) 64318f7 HDDS-12498. Allow limiting flaky-test-check to specific submodule (apache#8194) 3e7b5eb HDDS-12719. Remove the setContainers(..) method from NodeManager. (apache#8191) 78e2e73 HDDS-12724. hdds-rocks-native build fails if JAVA_HOME not set (apache#8183) fbc696c HDDS-12594. Optimize replica checksum verifier (apache#8151) 4411e35 HDDS-12733. Bump axios to 0.30.0 (apache#8189) 31aabc1 HDDS-12732. Bump awssdk to 2.31.11 (apache#8188) d6e2509 HDDS-12731. Bump restrict-imports-enforcer-rule to 2.6.1 (apache#8187) 4213307 HDDS-12648. Fix checkLeaderStatus in removeOMFromRatisRing and addOMToRatisRing (apache#8142) e4e9547 HDDS-12709. Intermittent failure in Balancer acceptance test (apache#8182) c1d5b4f HDDS-12718. Use NodeStateMap in MockNodeManager instead of test-specific Node2ContainerMap (apache#8179) 65a9d6c HDDS-12587. Detect test class module in flaky-test-check (apache#8162) aac9383 HDDS-12529. Clean up code in AbstractContainerReportHandler. (apache#8033) 6891315 HDDS-12603. Fix ContainerTable keys re-rendering issue (apache#8163) 2a1a6bf HDDS-12670. Improve encapsulation of volume spare space check (apache#8167) 05b6eb3 HDDS-12696. Replace link to Hadoop with Ozone in httpfs site.xml (apache#8177) 7e87a8a HDDS-11879. MiniKMS fails with ClassNotFoundException: com.sun.jersey....ServletContainer (apache#8158) 841d297 HDDS-12592. Remove find missing padding check in favor of of block metadata check (apache#8145) ce003e8 HDDS-12702. Move checknative under ozone debug (apache#8170) 6ee9c2b HDDS-8007. Add more detailed stages for SnapDiff job progress tracking (apache#8010) 3ebe5e7 HDDS-12660. Allow --verbose option of GenericCli at leaf subcommands (apache#8166) c8b3ccb HDDS-12528. Create new module for S3 integration tests (apache#8168) 435dee9 HDDS-12703. Close pipeline command should display error on failure (apache#8169) deb8e7c HDDS-12705. Replace Whitebox with HddsWhiteboxTestUtils (apache#8172) 0d8aecc HDDS-12327. HDDS-12668. Fix HSync upgrade test failure in non-HA upgrade test (apache#8171) bef4bee HDDS-12239. (addendum) Volume should not be marked as unhealthy when disk full - fix compile error e3e47ea HDDS-12235. Reserve space on DN during container import operation. (apache#7981) 4ffae70 HDDS-12239. Volume should not be marked as unhealthy when disk full (apache#7830) 0781cbe HDDS-12694. Disable TestMiniChaosOzoneCluster after fixing init and shutdown (apache#8159) 92aa71d HDDS-12687. Avoid ambiguity in URI descriptions (apache#8155) 5367016 HDDS-12679. Merge VolumeInfo into StorageVolume (apache#8147) 2cf6d59 HDDS-12553. ozone admin container list should output real JSON array (apache#8050) 63fcb27 HDDS-12465. Intermittent failure in TestOzoneFileSystemMetrics (apache#8130) bab26a4 HDDS-12684. Update NOTICE and LICENSE file (apache#8160) 857cb76 HDDS-12327. Restore non-HA (to HA) upgrade test (apache#7880) 587e9ff HDDS-12320. Collect OM performance metrics for FSO key delete (apache#7883) 560d017 HDDS-12500. Do not skip JUnit tests in post-commit runs (apache#8024) 273a627 HDDS-12686. Remove output of OzoneAddress in --verbose mode CLI (apache#8153) 4d3d834 HDDS-12650. Added logs to SnapshotDeletingService to indicate skipped snapshots. (apache#8123) 7524736 Revert "HDDS-12528. Create new module for S3 integration tests (apache#8152)" d44ebf5 HDDS-12528. Create new module for S3 integration tests (apache#8152) 3739b05 HDDS-12604. Reduce duplication in TestContainerStateMachine (apache#8104) a1616ae HDDS-12485. Repair tool should only print user warning for offline commands (apache#8140) de5c0a3 HDDS-12668. HSync upgrade test failure (apache#8137) bee10f0 HDDS-12671. Include .editorconfig and .run in source tarball (apache#8139) 8e7d370 HDDS-12666. Remove -SNAPSHOT from OZONE_CURRENT_VERSION in upgrade test (apache#8136) 4e0a764 HDDS-12486. Warmup KMS encrypted keys when OM starts (apache#8081) 0910997 HDDS-12636. Reduce code duplication for tarball creation (apache#8121) cbafa02 HDDS-12642. ACL test assertions depend on JSON element order (apache#8143) 8691adf HDDS-12622. Refactor minFreeSpace calculation (apache#8119) 1b0e912 HDDS-12661. Standardize Maven module names (apache#8129) ebf5cc6 HDDS-12662. Rename upgrade callback directory 1.5.0 to 2.0.0 (apache#8131) 828b2d1 HDDS-12667. Bump awssdk to 2.31.6 (apache#8134) 2596b3b HDDS-12665. Bump zstd-jni to 1.5.7-2 (apache#8133) 0e07983 HDDS-12664. Bump copy-rename-maven-plugin to 1.0.1 (apache#8132) 3173501 HDDS-12462. Use exclude rules for defining shaded filesystem jar contents (apache#8008) ee6201b HDDS-12341. Share cluster in client tests (apache#8126) bd579b3 HDDS-12646. Improve OM decommission check (apache#8122) 482024e HDDS-12569. Extract MiniOzoneCluster to separate module (apache#8067) 7a3ad16 HDDS-12641. Move Lease to hdds-server-framework (apache#8128) e805c15 HDDS-12310. Online repair command to perform compaction on om.db (apache#7957) 7b9c152 HDDS-12473. Trim duplicate space in proto message definition (apache#8005) 918bb98 HDDS-12626. Move the compare method in NodeStatus to ECPipelineProvider. (apache#8116) bf5a2f2 HDDS-12644. Create factory method for OzoneAcl (apache#8115) a646b85 HDDS-12557. Add progress indicator for checkpoint tarball in leader OM (apache#8085) 9e23d59 HDDS-12627. NodeStateMap may handle opStateExpiryEpochSeconds incorrectly. (apache#8117) d19f3bc HDDS-12640. Move GrpcMetrics to hdds-server-framework (apache#8114) 1c0b445 HDDS-12426. SCM replication should check double of container size. (apache#8080) 157cca4 HDDS-12358. Intermittent failure in compatibility acceptance test (apache#8012) 9b25d02 HDDS-12549. refactor ratis request to common place (apache#8059) 3349248 HDDS-12588. Recon Containers page shows number of blocks, not keys (apache#8074) 693d0f4 HDDS-12589. Fix Incorrect FSO Key Listing for Container-to-Key Mapping. (apache#8109) 17dbc4f HDDS-12585. Recon ContainerHealthTask ConstraintViolationException error handling. (apache#8070) 53c0a32 HDDS-12620. Fix OM Mismatch Deleted Container API (apache#8102) 45c900d HDDS-12619. Optimize Recon OM Container Mismatch API (apache#8101) 57fda0c HDDS-12633. KEY_NOT_FOUND in OzoneRpcClientTests for LEGACY bucket with enableFileSystemPaths=true (apache#8108) 23b0505 Revert "HDDS-12630. Enable GitHub Discussions in asf.yml (apache#8107)" b402b7c HDDS-12621. Change NodeStatus to value-based. (apache#8105) 20aeda5 HDDS-12601. Unknown tarball cleanup for Recon OM DB snapshot. (apache#8084) 93cd5aa HDDS-12630. Enable GitHub Discussions in asf.yml (apache#8107) 61a36f6 HDDS-12373. Use File.getUsableSpace() instead of File.getFreeSpace() to calcuate usedSpace in DedicatedDiskSpaceUsage (apache#7927) 953e718 HDDS-12565. Treat volumeFreeSpaceToSpare as reserved space (apache#8086) 8489cc8 HDDS-11735. Update ozone-default.xml for volume choosing policy (apache#8103) 13c5be8 HDDS-12577. [Ozone 2.0] Update master branch version number. 1a9d9f7 HDDS-12446. Add a Grafana dashboard for low level RocksDB operations. (apache#7992) c9990ca HDDS-12576. [Ozone 2.0] Update proto.lock files (apache#8064) b68b94c HDDS-12617. Use DatanodeID as keys in NodeStateMap. (apache#8100) b71d408 Revert "HDDS-12589. Fix Incorrect FSO Key Listing for Container-to-Key Mapping. (apache#8078)" 68e3842 HDDS-12589. Fix Incorrect FSO Key Listing for Container-to-Key Mapping. (apache#8078) c8e77f8 HDDS-12602. Intermittent failure in TestContainerStateMachine.testWriteFailure (apache#8099) 08ac32d HDDS-12608. Race condition in datanode version file creation (apache#8093) e5ef35d HDDS-12582. TypedTable support using different codec (apache#8073) dc47897 HDDS-12611. Snapshot creation is removing extra keys from AOS's DB (apache#8094) ff3ef51 HDDS-12551. Replace dnsToUuidMap with dnsToDnIdMap in SCMNodeManager. (apache#8087) 28520a7 HDDS-12614. Configurable java version in flaky-test-check with default to 21 (apache#8097) 434d5bd HDDS-12616. Bump junit to 5.12.1 (apache#8088) 699ee88 HDDS-12610. Bump awssdk to 2.31.1 (apache#8089) 0a85f9b HDDS-12602. Mark TestContainerStateMachineLeader/Follower as flaky 03c80f8 HDDS-12591. Include ContainerInfo in ContainerAttribute. (apache#8083) 49a2c85 HDDS-12097. Enhance Container Key Mapper for Faster Processing. (apache#8002) 5b7f96c HDDS-12303. Move ozone.om.user.max.volume into OmConfig (apache#8082) afc4043 HDDS-12566. Handle Over replication of Quasi Closed Stuck containers (apache#8061) eb96ff4 HDDS-12590. Used db name as the threadNamePrefix. (apache#8076) cf5bad7 HDDS-12539. Enable some String-related rules in PMD (apache#8047) d26f711 HDDS-12533. Offline repair command for generic rocksDB compaction (apache#8039) 93563e9 HDDS-12573. Pipeline#toString should separate ReplicaIndex from next node UUID. (apache#8063) 6e40831 HDDS-12572. Remove the ContainerID parameter when it has ContainerReplica. (apache#8075) 786da39 HDDS-12057. Implement command ozone debug replicas verify checksums (apache#7748) 66bc7ea HDDS-12535. Intermittent failure in TestContainerReportHandling (apache#8060) 87a674c HDDS-12555. Combine containerMap and replicaMap in ContainerStateMap. (apache#8057) f3689b6 HDDS-12458. Show safemode rules status irrespective of whether SCM is in safe mode in verbose mode. (apache#8049) 7d8c771 HDDS-12469. Mark statemachine unhealthy for write operation timeout. (apache#8022) 9aa41fd HDDS-12574. Add script to find modules by test classes (apache#8062) 5e28acb HDDS-12537. Selective checks: skip tests for PMD ruleset change (apache#8040) 9a3433b HDDS-12520. Move auditparser under debug (apache#8041) 0335385 HDDS-12420. Move FinalizeUpgradeCommandUtil to hdds-common. (apache#8023) 75bbba3 HDDS-12552. Fix raw use of generic class SCMCommand (apache#8048) d71aadf HDDS-12527. Separate S3 Gateway from MiniOzoneCluster (apache#8058) de8cf16 HDDS-12483. Quasi Closed Stuck should have 2 replicas of each origin (apache#8014) 57a139e HDDS-11576. Create a separate S3 client factory (apache#8051) e8a4668 HDDS-12383. Fix spotbugs warnings in hdds-common and httpfsgateway (apache#8046) 0dab553 HDDS-12541. Change ContainerID to value-based (apache#8044) 1202f6d HDDS-12532. Support only Enum in ContainerAttribute. (apache#8036) a2ad1e3 HDDS-12536. Move InMemoryTestTable to test (apache#8043) 7164c76 HDDS-12488. S3G should handle the signature calculation with trailers (apache#8020) c8c6d0e HDDS-12534. Remove drop_column_family command from ozone debug ldb (apache#8038) 9a8321e HDDS-12535. Mark TestContainerReportHandling as flaky ddd89fb HDDS-12543. Remove duplicate license information (apache#8045) 08e2c0a HDDS-12531. Use AtomicFileOutputStream to write YAML files. (apache#8035) 1bd8d8f HDDS-11813. Reduce duplication in CI workflow (apache#7497) 250bd5f HDDS-12450. Enable SimplifiableTestAssertion check in PMD (apache#8032) 96273ae HDDS-12489. Intermittent timeout in TestSCMContainerManagerMetrics.testReportProcessingMetrics (apache#8021) 2315927 HDDS-12476. Add TestDataUtil#createKey variant with small random content (apache#8028) d95ca4c HDDS-12421. ContainerReportHandler should not make the call to delete replicas (apache#7976) 26c859c HDDS-12204. Improve failover logging (apache#7867) efbf79c HDDS-12236. ContainerStateMachine should not apply or write future transactions in the event of failure (apache#7862) 3f88dbe HDDS-12377. Improve error handling of OM background tasks processing in case of abrupt crash of Recon. (apache#7960) 72da3a6 HDDS-12477. Do not force RATIS/ONE replication in TestDataUtil#createKey (apache#8017) a428b15 HDDS-12496. Use TextFormat#shortDebugString to flatten proto message in SCMDatanodeProtocolServer. (apache#8019) ecd2de0 HDDS-12409. Log an error before increasing the sequence id of a CLOSED container in SCM (apache#7964) 978e4a7 HDDS-12168. Create new Grafana panel to display cluster growth rate (apache#7978) 9ab7c70 HDDS-12456. Avoid FileInputStream and FileOutputStream (apache#8015) 9d41cd7 HDDS-12474. Add latency metrics of deletion services to grafana dashboard (apache#8007) 2b48e8c HDDS-12354. Move Storage and UpgradeFinalizer to hdds-server-framework (apache#7973) ed737b3 HDDS-12428. Avoid force closing OPEN/CLOSING replica of a CLOSED Container (apache#7985) d1e8b90 HDDS-12210. Use correct BootstrapStateHandler.Lock in SnapshotDeletingService (apache#7991) b769a26 HDDS-12295. Allow updating OM default replication config for tests (apache#7974) dd74eee HDDS-12430. Document in ozone-default.xml the config keys moved from DFSConfigKeysLegacy (apache#7987) 34041ca HDDS-12466. Set default commit message to PR title (apache#8013) 1ea0735 HDDS-12467. Enable new asf.yaml parser (apache#8011) e87b8db HDDS-12417. Reduce duplication of createKey variants in TestDataUtil (apache#7999) 86d2027 HDDS-12193. Provide option to disable RDBStoreMetrics for Snapshotted DB (apache#7982) 7c1d201 HDDS-12451. Create factory for MultiTenantAccessController (apache#7996) c2a934c HDDS-12470. Revert workaround added by HDDS-8715 to preserve thread name. (apache#8004) b8c93cc HDDS-12376. (addendum: fix findbugs) Remove scmRatisEnabled from ScmInfo. (apache#7931) 4005a10 HDDS-12376. (addendum: fix pmd) Remove scmRatisEnabled from ScmInfo. (apache#7931) abfa3be HDDS-12442. Add latency metrics for OM deletion services (apache#7986) 14db15c HDDS-12376. Remove scmRatisEnabled from ScmInfo. (apache#7931) 7d31d9e HDDS-12410. Add detailed block info for ALLOCATE_BLOCK audit log (apache#7965) a8f0ff3 HDDS-12460. (addendum) Move hdds-test-utils code to src/test 4c28c7f HDDS-12460. Move hdds-test-utils code to src/test (apache#8000) bb16f66 HDDS-12448. Avoid using Jackson1 (apache#7994) be0e1e6 HDDS-12445. Remove unused code from ContainerStateMap. (apache#7990) a3c9c0e HDDS-12443. Intermittent failure in TestContainerBalancerSubCommand (apache#7989) 83fd8d7 HDDS-12449. Enable UseCollectionIsEmpty check in PMD (apache#7995) d18da13 HDDS-12452. Bump slf4j to 2.0.17 (apache#7993) 6a9e8b1 HDDS-12348. Reuse `TestDataUtil.createKey` method (apache#7971) 384d774 HDDS-12416. Enable UnusedPrivateField check in PMD (apache#7975) 39d7da3 HDDS-12424. Allow config key to include config group prefix (apache#7979) ba0939b HDDS-12156. Add container health task metrics in Recon. (apache#7786) d34aee4 HDDS-12172. Rename Java constants of DFSConfigKeysLegacy keys (apache#7922) 59aaa5c HDDS-12418. Remove healthyReplicaCountAdapter from RatisContainerReplicaCount (apache#7972) 232e780 HDDS-12150. Abnormal container states should not crash the SCM ContainerReportHandler thread (apache#7882) a708ea4 HDDS-12351. Move SCMHAUtils and ServerUtils to hdds-server-framework (apache#7961) 052bd2d HDDS-12198. Exclude Recon generated code in coverage (apache#7962) ba44e12 HDDS-12382. Fix other spotbugs warnings (apache#7969) e2bf599 HDDS-12345. Share cluster in filesystem tests (apache#7959) f9f1c80 HDDS-12411. Make hdds-client compliant with FieldDeclarationsShouldBeAtStartOfClass (apache#7966) 46f4986 HDDS-11512. Create Grafana dashboard for tracking system wide deletion (apache#7813) f7e9aed HDDS-12396. Enable UnusedPrivateMethod check in PMD (apache#7956) 619f524 HDDS-1234. Fix spotbugs warnings in ozone-manager (apache#7963) 1d64b37 HDDS-12380. Fix spotbugs warnings in hdds-container-service (apache#7958) 5c2b8f6 HDDS-12065. Checkpoint directory should be cleared on startup (apache#7681) a31755a HDDS-12315. Speed up some Freon integration tests (apache#7870) d8f3149 HDDS-12399. Enable PMD.ForLoopCanBeForeach rule (apache#7952) 393211a HDDS-12062. Recon - Error handling in NSSummaryTask to avoid data inconsistencies. (apache#7723) ecc2403 HDDS-9792. Add tests for Pipelines page (apache#7859) e4de75e HDDS-12388. Key rewrite tests should be skipped if feature is disabled (apache#7953) 7b0fe61 HDDS-12288. (addendum) fix checkstyle d5df7e4 HDDS-12288. Improve bootstrap logging to indicate progress of snapshot download. (apache#7861) 58a0453 HDDS-12381. Fix spotbugs warnings in TestHddsUtils (apache#7955) 1a0c223 HDDS-11768. Extract SCM failover proxy provider logic (apache#7950) b45de0f HDDS-12403. Bump zstd-jni to 1.5.7-1 (apache#7949) a57809e HDDS-12402. Bump sqlite-jdbc to 3.49.1.0 (apache#7948) 221d53d HDDS-12400. Bump junit to 5.12.0 (apache#7947) f17abae HDDS-12398. Enable PMD checks for tests (apache#7946) 540f67b HDDS-12353. Move SpaceUsage implementations to hdds-server-framework (apache#7926) 70b93dc HDDS-10764.Tarball creation failing on leader OM node. (apache#7941) 6b20afc HDDS-12387. Cleanup TestContainerOperations (apache#7940) 417ae7c HDDS-12365. Provide editor settings and IDEA run config in standard location (apache#7924) a2c5c8e HDDS-12293. Make ozone.om.server.list.max.size reconfigurable (apache#7938) e0bd2cc HDDS-12185. Enhance FileSizeCountTask for Faster Processing. (apache#7796) 589eeef HDDS-12371. Duplicated key scanning on multipartInfo table when listing multipart uploads (apache#7937) 6e766bf HDDS-12367. Change ignorePipeline log level to DEBUG in OmKeyInfo (apache#7939) c87caa5 HDDS-12226. TestSecureOzoneRpcClient tests not run due to UnknownHostException (apache#7827) 1513c34 HDDS-11883. SCM HA: Move proxy object creation code to SCMRatisServer (apache#7914) 8c42c02 HDDS-12349. Speed up some HDDS integration tests (apache#7932) a00787c HDDS-11532. Sort multipart uploads on ListMultipartUploads response (apache#7929) Conflicts: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/ContainerDataYaml.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/impl/HddsDispatcher.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/interfaces/Container.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/interfaces/Handler.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/DatanodeConfiguration.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainer.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueContainerCheck.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/keyvalue/KeyValueHandler.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/ozoneimpl/OzoneContainer.java hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/ContainerTestUtils.java hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/TestSchemaTwoBackwardsCompatibility.java hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/impl/TestHddsDispatcher.java hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/interfaces/TestHandler.java hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/TestKeyValueHandler.java hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/replication/TestReplicationSupervisor.java hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMClientProtocolServer.java hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/ozone/audit/SCMAction.java hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/TestContainerReportHandler.java hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/TestIncrementalContainerReportHandler.java hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/container/ContainerCommands.java hadoop-ozone/dist/src/main/smoketest/admincli/container.robot hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestECKeyOutputStream.java hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/metrics/TestContainerMetrics.java hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/ozoneimpl/TestOzoneContainerWithTLS.java hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/server/TestContainerServer.java hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/server/TestSecureContainerServer.java hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/fsck/ContainerHealthTask.java hadoop-ozone/recon/src/test/java/org/apache/hadoop/ozone/recon/fsck/TestContainerHealthTask.java
vtutrinov
pushed a commit
to vtutrinov/ozone
that referenced
this pull request
May 13, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
HDDS-5249 has added synchronized (datanodeDetails) to ContainerReportHandler and IncrementalContainerReportHandler. Since ReconContainerReportHandler uses the super method in ContainerReportHandler, it is also synchronized.
However, ReconIncrementalContainerReportHandler is not synchronized.
What is the link to the Apache JIRA
HDDS-12825
How was this patch tested?
By existing tests