GET _migration/deprecations doesn't show deprecations for hidden indices

[valeriy42]

Elasticsearch Version

8.18

Installed Plugins

No response

Java Version

bundled

OS Version

Linux

Problem Description

The GET _migration/deprecations API in 8.18 no longer lists deprecations for hidden indices like .ml-* by default:

• if you run GET _migration/deprecations the deprecation warnings are not displayed
• if you run GET .ml*/_migration/deprecations they are displayed

This is a change from the previous behavior in 7.17, where GET _migration/deprecations would list the deprecation for .ml* indices in the index_settings part and the deprecation for ML model snapshots in the ml_settings part.

This change in the API behavior breaks the expected workflow in Kibana Update Assistant in 8.18 since now no ES deprecation warnings for .ml-* indices are shown before the major upgrade.

Steps to Reproduce

1. Set up a cluster in 7.17
2. Create an anomaly detection job, e.g. using kibana flights data
3. Upgrade the cluster to 8.18
4. Run GET _migration/deprecations in Dev Console

Logs (if relevant)

No response

[elasticsearchmachine]

Pinging @elastic/es-data-management (Team:Data Management)

[sophiec20]

This is blocking being able to test e2e migrations cc @dakrone

[dakrone]

I have been able to reproduce this, when I change

elasticsearch/x-pack/plugin/deprecation/src/main/java/org/elasticsearch/xpack/deprecation/DeprecationInfoAction.java

Line 369 in ec27e20

private static final IndicesOptions INDICES_OPTIONS = IndicesOptions.fromOptions(false, true, true, true);

to include hidden indices, then it sees the hidden index. Hopefully it will be a simple fix in that case.