Create an ELSER inference endpoint | Elasticsearch API documentation

Create an ELSER inference endpoint Deprecated Generally available; Added in 8.11.0

PUT /_inference/{task_type}/{elser_inference_id}

Create an inference endpoint to perform an inference task with the elser service. You can also deploy ELSER by using the Elasticsearch inference integration.

Your Elasticsearch deployment contains a preconfigured ELSER inference endpoint, you only need to create the enpoint using the API if you want to customize the settings.

The API request will automatically download and deploy the ELSER model if it isn't already downloaded.

You might see a 502 bad gateway error in the response when using the Kibana Console. This error usually just reflects a timeout, while the model downloads in the background. You can check the download progress in the Machine Learning UI. If using the Python client, you can set the timeout parameter to a higher value.

After creating the endpoint, wait for the model deployment to complete before using it. To verify the deployment status, use the get trained model statistics API. Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count". Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.

Required authorization

Cluster privileges: manage_inference

Path parameters

task_type string

The type of the inference task that the model will perform.

Value is sparse_embedding.
elser_inference_id string Required

The unique identifier of the inference endpoint.

Query parameters

timeout string

Specifies the amount of time to wait for the inference endpoint to be created.

Values are -1 or 0.

application/json

Body

chunking_settings object

Chunking configuration object
Hide chunking_settings attributes Show chunking_settings attributes object
- max_chunk_size number
  
  The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).
  
  Default value is 250.
- overlap number
  
  The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.
  
  Default value is 100.
- sentence_overlap number
  
  The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.
  
  Default value is 1.
- separator_group string Required
  
  This parameter is only applicable when using the recursive chunking strategy.
  
  Sets a predefined list of separators in the saved chunking settings based on the selected text type. Values can be markdown or plaintext.
  
  Using this parameter is an alternative to manually specifying a custom separators list.
- separators array[string] Required
  
  A list of strings used as possible split points when chunking text with the recursive strategy.
  
  Each string can be a plain string or a regular expression (regex) pattern. The system tries each separator in order to split the text, starting from the first item in the list.
  
  After splitting, it attempts to recombine smaller pieces into larger chunks that stay within the max_chunk_size limit, to reduce the total number of chunks generated.
- strategy string
  The chunking strategy: sentence, word, none or recursive.
  
  If strategy is set to recursive, you must also specify:
  
  max_chunk_size
  
  either separators orseparator_group
  
  Learn more about different chunking strategies in the linked documentation.
  Default value is sentence.
  
  External documentation
service string Required

Value is elser.
service_settings object Required
Hide service_settings attributes Show service_settings attributes object
- adaptive_allocations object
  Hide adaptive_allocations attributes Show adaptive_allocations attributes object
  
  enabled boolean
  
  Turn on adaptive_allocations.
  
  Default value is false.
  
  max_number_of_allocations number
  
  The maximum number of allocations to scale to. If set, it must be greater than or equal to min_number_of_allocations.
  
  min_number_of_allocations number
  
  The minimum number of allocations to scale to. If set, it must be greater than or equal to 0. If not defined, the deployment scales to 0.
- num_allocations number Required
  
  The total number of allocations this model is assigned across machine learning nodes. Increasing this value generally increases the throughput. If adaptive allocations is enabled, do not set this value because it's automatically set.
- num_threads number Required
  
  The number of threads used by each model allocation during inference. Increasing this value generally increases the speed per inference request. The inference process is a compute-bound process; threads_per_allocations must not exceed the number of available allocated processors per node. The value must be a power of 2. The maximum value is 32.
  
  If you want to optimize your ELSER endpoint for ingest, set the number of threads to 1. If you want to optimize your ELSER endpoint for search, set the number of threads to greater than 1.

Responses

200 application/json
Hide response attributes Show response attributes object
- chunking_settings object
  
  Chunking configuration object
  
  Hide chunking_settings attributes Show chunking_settings attributes object
  
  max_chunk_size number
  
  The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).
  
  Default value is 250.
  
  overlap number
  
  The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.
  
  Default value is 100.
  
  sentence_overlap number
  
  The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.
  
  Default value is 1.
  
  separator_group string Required
  
  This parameter is only applicable when using the recursive chunking strategy.
  
  Sets a predefined list of separators in the saved chunking settings based on the selected text type. Values can be markdown or plaintext.
  
  Using this parameter is an alternative to manually specifying a custom separators list.
  
  separators array[string] Required
  
  A list of strings used as possible split points when chunking text with the recursive strategy.
  
  Each string can be a plain string or a regular expression (regex) pattern. The system tries each separator in order to split the text, starting from the first item in the list.
  
  After splitting, it attempts to recombine smaller pieces into larger chunks that stay within the max_chunk_size limit, to reduce the total number of chunks generated.
  
  strategy string
  
  The chunking strategy: sentence, word, none or recursive.
  
  If strategy is set to recursive, you must also specify:
  
  max_chunk_size
  
  either separators orseparator_group
  
  Learn more about different chunking strategies in the linked documentation.
  
  Default value is sentence.
  
  External documentation
- service string Required
  
  The service type
- service_settings object Required
- task_settings object
- inference_id string Required
  
  The inference Id
- task_type string Required
  
  Value is sparse_embedding.

PUT /_inference/{task_type}/{elser_inference_id}

PUT _inference/sparse_embedding/my-elser-model
{
    "service": "elser",
    "service_settings": {
        "num_allocations": 1,
        "num_threads": 1
    }
}

resp = client.inference.put(
    task_type="sparse_embedding",
    inference_id="my-elser-model",
    inference_config={
        "service": "elser",
        "service_settings": {
            "num_allocations": 1,
            "num_threads": 1
        }
    },
)

const response = await client.inference.put({
  task_type: "sparse_embedding",
  inference_id: "my-elser-model",
  inference_config: {
    service: "elser",
    service_settings: {
      num_allocations: 1,
      num_threads: 1,
    },
  },
});

response = client.inference.put(
  task_type: "sparse_embedding",
  inference_id: "my-elser-model",
  body: {
    "service": "elser",
    "service_settings": {
      "num_allocations": 1,
      "num_threads": 1
    }
  }
)

$resp = $client->inference()->put([
    "task_type" => "sparse_embedding",
    "inference_id" => "my-elser-model",
    "body" => [
        "service" => "elser",
        "service_settings" => [
            "num_allocations" => 1,
            "num_threads" => 1,
        ],
    ],
]);

curl -X PUT -H "Authorization: ApiKey $ELASTIC_API_KEY" -H "Content-Type: application/json" -d '{"service":"elser","service_settings":{"num_allocations":1,"num_threads":1}}' "$ELASTICSEARCH_URL/_inference/sparse_embedding/my-elser-model"

client.inference().put(p -> p
    .inferenceId("my-elser-model")
    .taskType(TaskType.SparseEmbedding)
    .inferenceConfig(i -> i
        .service("elser")
        .serviceSettings(JsonData.fromJson("{\"num_allocations\":1,\"num_threads\":1}"))
    )
);

Request examples

Run `PUT _inference/sparse_embedding/my-elser-model` to create an inference endpoint that performs a `sparse_embedding` task. The request will automatically download the ELSER model if it isn't already downloaded and then deploy the model.

{
    "service": "elser",
    "service_settings": {
        "num_allocations": 1,
        "num_threads": 1
    }
}

Run `PUT _inference/sparse_embedding/my-elser-model` to create an inference endpoint that performs a `sparse_embedding` task with adaptive allocations. When adaptive allocations are enabled, the number of allocations of the model is set automatically based on the current load.

{
    "service": "elser",
    "service_settings": {
        "adaptive_allocations": {
            "enabled": true,
            "min_number_of_allocations": 3,
            "max_number_of_allocations": 10
        },
        "num_threads": 1
    }
}

Response examples (200)

A successful response when creating an ELSER inference endpoint.

{
  "inference_id": "my-elser-model",
  "task_type": "sparse_embedding",
  "service": "elser",
  "service_settings": {
    "num_allocations": 1,
    "num_threads": 1
  },
  "task_settings": {}
}