[elasticsearch] Disable transform validation in system tests #12840
Conversation
mrodm
changed the title
🚀 Benchmarks reportTo see the full report comment with |
This reverts commit f9bc53b.
| # Currently, when trying to run the preview of the transform logs-elasticsearch.index_pivot-default-0.1.0, fails with this error: | ||
| # [400 Bad Request] {"error":{"root_cause":[{"type":"status_exception","reason":"Source indices have been deleted or closed."}],"type":"status_exception","reason":"Source indices have been deleted or closed."},"status":400} | ||
| skip_transform_validation: true |
There was a problem hiding this comment.
When testing with a newer stack version (tested with 8.17.1, 8.17.2 and 8.18.0-SNAPSHOT), there are no errors.
However, testing with the current stack version set in the manifest (8.10.1), this is failing with the error mentioned in the comment
- Maybe is this caused due to the destination index is not created?
failed to preview transform "logs-elasticsearch.index_pivot-default-0.2.0": [400 Bad Request] {"error":{"root_cause":[{"type":"status_exception","reason":"Source indices have been deleted or closed."}],"type":"status_exception","reason":"Source indices have been deleted or closed."},"status":400}
Using other Elastic stack versions 8.14.0, 8.16.0 and 8.17.0, the error is different:
- Maybe the filters/query in the transform cause that there are no documents to be processed?
failed to preview transform "logs-elasticsearch.index_pivot-default-0.2.0": [400 Bad Request] {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"must specify at least one document in [docs]"}],"type":"illegal_argument_exception","reason":"must specify at least one document in [docs]"},"status":400}
There was a problem hiding this comment.
Just found in the documentation, that it looks like it is expected that this transform just works in 8.17.1 and onward
There was a problem hiding this comment.
What would it be better here ? @jsoriano
I was thinking to rephrase the comment set here (and keep disabled the transform validation):
# Current `index_pivot` transform is expected to work just on stack versions >=8.17.1
# remove this setting once the kibana constraint in the package manifest is >=8.17.1
skip_transform_validation: true
There was a problem hiding this comment.
Yeah, lets keep this validation disabled by now if it is going to be problematic.
💚 Build Succeeded
History
cc @mrodm |
|
mrodm
added
Integration:elasticsearch
Proposed commit message
This PR adds a new system test configuration file to disable validation of transforms in
elasticsearchpackage.In the following version of
elastic-packageit is added support to read source indices in transforms defined as a comma-separated list of strings elastic/elastic-package#2388. This will cause that the followingelastic-packageversion detects that the transform should be validated because it matches some of the datastream (index, index_recovery and index_summary data streams).However, the validation fail with this error:
If they are not skip, system tests for
index,index_recoveryandindex_summarydata streams would fail as in this Buildkite build: https://buildkite.com/elastic/integrations/builds/22461Checklist
changelog.ymlfile.Author's Checklist
elastic-packagecommit: https://buildkite.com/elastic/integrations/builds/22503How to test this PR locally
Related issues