2024 June APAC Community Sync

[ChenSammi]

June 7th
Attenders: Hongbin, Pony, Minyu, Conway, Hualong, Yuanben, Sammi

Sammi

• CoC EU is hosted this week. There are several Ozone topics, including Recon, Ratis, Performance, etc.
• Trouble shooting guides of critical features and functions.

Hongbin/Conway

• Found issue that OM is sending duplicate blocks to SCM for deletion. Suggest to apply the patch https://issues.apache.org/jira/browse/HDDS-10680, and add more log information to debug this issue.
• G1 and ZGC are scheduled to try on the production cluster soon later.
• Found one EC reconstruction issue, same as Minyu raised.

Pony

• Ratis 3.1.0 RC is under voting now.
• Prefer to continue to release 1.4.1.

Minyu

• Found a EC reconstruction failure issue, filed JIRA https://issues.apache.org/jira/browse/HDDS-10985

[ChenSammi]

June 21th
Attenders: Hongbin, Guohao, Pony, Minyu, Shilun, Hualong, Yuanben, Chungen, Sammi

Sammi

• HDDS-10656 Atomic Key Overwrite and Key Replacement, will build the foundation for auto file converting(3 factor <-> EX)
• HDDS-10239 Storage Container Reconciliation, enabling the possibility of block level recovery.

Pony

• RATIS 3.1.0 RC2 is under vote.

Guohao

• HDDS-10985 EC Reconstruction failed because the size of currentChunks was not equal to checksumBlockDataChunks. Shilun explained that the Reconstruction failure is because one current check is rigid. It will reject a common data failure case to be recovered. Since SCM will retry the same container reconstruction until it succeeds, the disk workload becomes very high when this problem happens with quite a few containers.

Hongbin/Shilun

• EC performance tuning. Suggest to tune property PIPELINE_PER_VOLUME_FACTOR_KEY by @sodonnel .
• Piloted the JDK 17 on production cluster. Met one performance issue caused by JDK bug JDK-8188055. Detail information can be found here https://docs.google.com/document/d/1g1h-63fvA-be-clvyVRAHLWehoadCnjNRWyX8Bp-UIU/edit. The problem is resolved by upgrading to a higher JDK 17 version. After the JDK issue is fixed, will continue pilot 17 and new GC algorithm.