Skip to content

[ML] Check if the anomaly results index has been rolled over #125404

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Mar 27, 2025

Conversation

davidkyle
Copy link
Member

The ML upgrade code automatically rolls over legacy v7 indices however it should not be rolling over indices that have already been rolled. This PR adds a check against that case and makes the rollover command more robust by handling ResourceAlreadyExists exceptions (index has already been rolled over) as not an error.

This PR is opened against the 8.18 branch so the upgrade tests will start with a 7.last cluster.

# Conflicts:
#	x-pack/plugin/ml/src/main/java/org/elasticsearch/xpack/ml/MlAnomaliesIndexUpdate.java
@davidkyle davidkyle added >upgrade :ml Machine learning test-full-bwc Trigger full BWC version matrix tests v8.18.1 v8.19.0 v9.0.1 v9.1.0 labels Mar 21, 2025
@elasticsearchmachine elasticsearchmachine added the Team:ML Meta label for the ML team label Mar 21, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/ml-core (Team:ML)

@elasticsearchmachine
Copy link
Collaborator

Hi @davidkyle, I've created a changelog YAML for you.

@davidkyle davidkyle requested a review from valeriy42 March 21, 2025 13:47
Copy link
Contributor

@valeriy42 valeriy42 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. I have primarily cosmetic comments.

Comment on lines +258 to +260
* Strip any suffix from the index name and find any other indices
* that match the base name. Then return the latest index from the
* matching ones.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this doesn't have to be in the Javadoc. The name explains well enough what behavior is expected from the function. You can put this comment inside the function implementation.

Comment on lines +185 to +187
metadata.put(createSharedResultsIndex(".ml-anomalies-custom-foo", IndexVersion.current(), List.of("job1")));
metadata.put(createSharedResultsIndex(".ml-anomalies-custom-bar", IndexVersion.current(), List.of("job2")));
metadata.put(createSharedResultsIndex(".ml-anomalies-custom-bax", IndexVersion.current(), List.of("job3")));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe you can refactor and extract the method here to reduce code duplication.

@davidkyle davidkyle enabled auto-merge (squash) March 27, 2025 13:52
@davidkyle davidkyle added the auto-backport Automatically create backport pull requests when merged label Mar 27, 2025
@davidkyle davidkyle merged commit 592e681 into elastic:8.18 Mar 27, 2025
14 of 15 checks passed
davidkyle added a commit to davidkyle/elasticsearch that referenced this pull request Mar 27, 2025
…#125404)

If the v7 index has already been rolled over don't try again
davidkyle added a commit to davidkyle/elasticsearch that referenced this pull request Mar 27, 2025
…#125404)

If the v7 index has already been rolled over don't try again
@elasticsearchmachine
Copy link
Collaborator

💚 Backport successful

Status Branch Result
8.x
9.0
main

davidkyle added a commit to davidkyle/elasticsearch that referenced this pull request Mar 27, 2025
…#125404)

If the v7 index has already been rolled over don't try again
elasticsearchmachine pushed a commit that referenced this pull request Mar 27, 2025
#125786)

If the v7 index has already been rolled over don't try again
elasticsearchmachine pushed a commit that referenced this pull request Mar 27, 2025
#125785)

If the v7 index has already been rolled over don't try again
omricohenn pushed a commit to omricohenn/elasticsearch that referenced this pull request Mar 28, 2025
…#125404) (elastic#125786)

If the v7 index has already been rolled over don't try again
elasticsearchmachine pushed a commit that referenced this pull request Apr 3, 2025
#125784)

If the v7 index has already been rolled over don't try again
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-backport Automatically create backport pull requests when merged :ml Machine learning Team:ML Meta label for the ML team test-full-bwc Trigger full BWC version matrix tests >upgrade v8.18.1 v8.19.0 v9.0.1 v9.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants