DLQ-ing events that trigger an conditional evaluation error.

[andsel]

Release notes

When a conditional evaluation encounter an error in the expression the event that triggered the issue is sent to pipeline's DLQ, if enabled for the executing pipeline.

What does this PR do?

This PR engage with the work done in #16322, the ConditionalEvaluationListener that is receives notifications about if-statements evaluation failure, is improved to also send the event to DLQ (if enabled in the pipeline) and not just logging it.

Why is it important/What is the impact to the user?

Let the user that encounter errors in the evaluation of conditional statements to enqueue in DLQ for a subsequent processing.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• [ ] I have made corresponding change to the default configuration files (and/or docker env variables)
• I have added tests that prove my fix is effective or that my feature works

Author's Checklist

• [ ]

How to test this PR locally

Use a pipeline that would trigger an error in conditional evaluation, like:

input {
    generator {
      message => '{"path":{"to":{"value": 101}}}'
      codec => json
      count => 1
    }
}

filter {
  if [path][to][value] > "100" { mutate { add_tag => "hit" } }
}

output {
    stdout { codec => rubydebug }
}

And enable DLQ, so in config/logstash.yml set

dead_letter_queue.enable: true

Related issues

• Closes Send to DLQ events that trigger a fail on a condition evaluation #16422
• Relates Implements safe evaluation of conditional expressions, logging the error without killing the pipeline #16322

Use cases

Screenshots

Logs

[jsvd]

We should change the log entry to mention that the event was sent to the DLQ instead of "Event was dropped, enable debug logging to see the event's payload."

[jsvd]

Any reason for one to end on a period (.) and the other not?

[andsel]

No, was a typo

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: cda51bb

History

• 💚 Build #1490 succeeded f890bf0
• 💚 Build #1434 succeeded f537a1e
• 💚 Build #1432 succeeded b421e31
• 💚 Build #1431 succeeded 86ebc05

cc @andsel