Skip to content

[Transform] Create upgrade mode #117858

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Dec 30, 2024
Merged

Conversation

prwhelan
Copy link
Member

@prwhelan prwhelan commented Dec 2, 2024

Prevent writes to the Transform system index while it is being reindexed during upgrade.

  • Add a new REST API, _transform/set_upgrade_mode, similar to _ml, which will enable via ?enabled or disable (when omitted).
  • Add a new Transport action, enabling or disabling upgrade mode. When enabled, all Transforms will abort and move into the waiting state, preventing Transforms from running and writing to the system index.
  • Hook into the System Index upgrade action, calling the new Transform action to set and unset upgrade mode.
  • Stop all API calls that modify the Transform state in the system index

Prevent writes to the Transform system index while it is being reindexed
during upgrade.

- Add a new REST API, `_transform/set_upgrade_mode`, similar to `_ml`,
  which will enable via `?enabled` or disable (when omitted).
- Add a new Transport action, enabling or disabling upgrade mode.  When
  enabled, all Transforms will abort and move into the `waiting` state,
  preventing Transforms from running and writing to the system index.
- Hook into the System Index upgrade action, calling the new Transform
  action to set and unset upgrade mode.
@prwhelan prwhelan added >enhancement :ml/Transform Transform Team:ML Meta label for the ML team auto-backport Automatically create backport pull requests when merged v9.0.0 v8.18.0 labels Dec 2, 2024
@elasticsearchmachine
Copy link
Collaborator

Hi @prwhelan, I've created a changelog YAML for you.

@prwhelan prwhelan marked this pull request as ready for review December 3, 2024 00:44
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/ml-core (Team:ML)

Copy link
Member

@davidkyle davidkyle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

) {
var wasAlreadyInUpgradeMode = (boolean) preUpgradeMetadata.getOrDefault("already_in_upgrade_mode", false);
if (wasAlreadyInUpgradeMode) {
// Transform was already in upgrade mode before system indices upgrade started - we shouldn't disable it
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 nice

@prwhelan
Copy link
Member Author

@elasticmachine update branch

@prwhelan
Copy link
Member Author

@elasticmachine update branch

@prwhelan prwhelan merged commit e1a954a into elastic:main Dec 30, 2024
16 checks passed
@elasticsearchmachine
Copy link
Collaborator

💚 Backport successful

Status Branch Result
8.x

prwhelan added a commit to prwhelan/elasticsearch that referenced this pull request Dec 30, 2024
Prevent writes to the Transform system index while it is being reindexed
during upgrade.

- Add a new REST API, `_transform/set_upgrade_mode`, similar to `_ml`,
  which will enable via `?enabled` or disable (when omitted).
- Add a new Transport action, enabling or disabling upgrade mode.  When
  enabled, all Transforms will abort and move into the `waiting` state,
  preventing Transforms from running and writing to the system index.
- Hook into the System Index upgrade action, calling the new Transform
  action to set and unset upgrade mode.

Co-authored-by: David Kyle <[email protected]>
Co-authored-by: Elastic Machine <[email protected]>
elasticsearchmachine pushed a commit that referenced this pull request Dec 30, 2024
Prevent writes to the Transform system index while it is being reindexed
during upgrade.

- Add a new REST API, `_transform/set_upgrade_mode`, similar to `_ml`,
  which will enable via `?enabled` or disable (when omitted).
- Add a new Transport action, enabling or disabling upgrade mode.  When
  enabled, all Transforms will abort and move into the `waiting` state,
  preventing Transforms from running and writing to the system index.
- Hook into the System Index upgrade action, calling the new Transform
  action to set and unset upgrade mode.

Co-authored-by: David Kyle <[email protected]>
Co-authored-by: Elastic Machine <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-backport Automatically create backport pull requests when merged >enhancement :ml/Transform Transform Team:ML Meta label for the ML team v8.18.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants