Test/107515 RestoreTemplateWithMatchOnlyTextMapperIT

[drempapis]

Closes #107515

I observed that when comparing the cluster states, local and master were equal, but the order was different in some cases. Converting to a byte array and getting the length can introduce junk data. Alternatively, the comparison can be made by checking if the returned state strings are anagrams, they contain the same characters.

[elasticsearchmachine]

Hi @drempapis, I've created a changelog YAML for you.

[elasticsearchmachine]

Pinging @elastic/es-search-foundations (Team:Search Foundations)

[drempapis]

@elastic/es-core-infra, can you please have a look to the provided solution?

[thecoop]

Can you give an example of how the sizes could be different but the content is the same? (it's unclear from the linked issue exactly what the difference is)

[drempapis]

I found it debugging the failing test case. The string representation of the cluster state was the following for both masterClusterState and localClusterStatebut in a different order. The calculated byte sizes were different.

"index_template":{
   "index_template":{
      "t1":{
         "index_patterns":[
            "test-index-*"
         ],
         "template":{
            "mappings":{
               "properties":{
                  "@timestamp":{
                     "format":"date_optional_time||epoch_millis",
                     "type":"date"
                  },
                  "flag":{
                     "type":"boolean"
                  },
                  "message":{
                     "type":"match_only_text"
                  }
               }
            }
         },
         "composed_of":[
            
         ],
         "data_stream":{
            "hidden":false,
            "allow_custom_routing":false
         }
      }
   }
}"index-graveyard":"IndexGraveyard"[
   [
      [
         "index="[
            test-idx-no-ds/xNBvevzTQ5CzuUFxnzjj-g
         ],
         "deleteDate=2025-01-17T13":"44":31.746Z
      ]
   ]
]"persistent_tasks":{
   "last_allocation_id":3,
   "tasks":[[89, 83, 85, 73, 84, 69, 45, 84, 69, 83, 84, 95, 87, 79, 82, 75, 69, 82, 95, 86, 77, 61, 91, 51, 48, 93, 45, 67, 76, 85, 83, 84, 69, 82, 95, 83, 69, 69, 68, 61, 91, 45, 51, 51, 54, 55, 53, 53, 52, 57, 53, 51, 52, 57, 52, 56, 51, 55, 54, 54, 57, 93, 45, 72, 65, 83, 72, 61, 91, 49, 51, 69, 69, 68, 51, 56, 49, 69, 56, 52, 55, 93, 45, 99, 108, 117, 115, 116, 101, 114, 0, 0, 0, 0, 0, 0, 0, 28, 22, 99, +4,464 more]
      {
         "id":"health-node",
         "task":{
            "health-node":{
               "params":{
                  
               }
            }
         },
         "allocation_id":3,
         "assignment":{
            "executor_node":"XcbfjiTHRWqhLHWvaWJOvA",
            "explanation":""
         },
         "allocation_id_on_last_status_update":0
      }
   ]
}"repositories":{
   "repo":{
      "type":"fs",
      "uuid":"N3yt0VE5S1aVe7VQgx5tkw",
      "settings":{
         "location":"/Users/dimitrisrempapis/workspace/elasticsearch/modules/data-streams/build/testrun/internalClusterTest/temp/org.elasticsearch.datastreams.RestoreTemplateWithMatchOnlyTextMapperIT_460BA435414BE7D1-001/tempDir-002/repos/IcxztPjmUe"
      },
      "generation":0,
      "pending_generation":0
   }
}"data_stream":{
   "data_stream":{
      
   },
   "data_stream_aliases":{
      
   }
}

[thecoop]

So the main way to check that two xcontent things are the same is to convert them to maps, and check the map equality. That way it doesnt consider order. There's other methods on XContentTestUtils that help with equality checks too

[drempapis]

Thank you, @thecoop, for the feedback. Do you want me to update this pr using the 'XContentTestUtils` or to assign it to core-infra for a second look?

[thecoop]

Could you update it to use XContentTestUtils (you could copy the equality checks used by ensureClusterStateCanBeReadByNodeTool for example)? If it still doesnt pass, then assign it to us to take a closer look.

[drempapis]

@thecoop, I updated the code using the XContentTestUtils to compare the stateMaps. Please review the proposed solution

[thecoop]

LGTM

[elasticsearchmachine]

💔 Backport failed

Status	Branch	Result
❌	8.x	Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 120392