Backport PR #16881 to 8.17: Avoid flow metrics initialization if pipeline metric.collect is disabled.

[github-actions]

Backport PR #16881 to 8.17 branch, original message:

---

Release notes

Avoid pipeline level flow metrics initialization if pipeline level metric collection is disabled.

What does this PR do?

LS has node and pipeline level metric collection control (metric.collect: true/false) signal. Regardless of disabling metric collection, now flow metrics are initialized and when client requests flow stats, it faces an error. This PR adds a change which considers metric.collect config when initializing pipeline flow metric.

Why is it important/What is the impact to the user?

Fixes the error when users disable pipeline metric collection.

Checklist

• My code follows the style guidelines of this project
• [ ] I have commented my code, particularly in hard-to-understand areas
• [ ] I have made corresponding changes to the documentation
• [ ] I have made corresponding change to the default configuration files (and/or docker env variables)
• [ ] I have added tests that prove my fix is effective or that my feature works

Author's Checklist

• discuss node level metric collection behaviour and create an issue to follow up

How to test this PR locally

• pull current change and build LS with ./gradlew clean bootstrap assemble installDefaultGems
• setup a simple PQ pipeline, disable collecting metrics:

# pipelines.yml
 - pipeline.id: logstash
   pipeline.workers: 1
   queue.type: persisted
   queue.max_bytes: 2048mb
   metric.collect: false
   config.string: "input { generator {} } filter { sleep { time => 0.1 } } output { stdout { codec => dots } }"

• run LS and hit the stats (localhost:9600/_node/stats) API endpoint

Related issues

• Closes Disabling pipeline level metric collection (metric.collect: false) is broken #16864

Use cases

Screenshots

Logs

Build LS from main branch;

• before the change, we see following logs on console

[2025-01-08T15:01:26,948][WARN ][org.logstash.instrument.metrics.LazyInstantiatedFlowMetric] Underlying metrics for `queue_persisted_growth_events` not yet instantiated, could not capture their rates
[2025-01-08T15:01:26,948][WARN ][org.logstash.instrument.metrics.LazyInstantiatedFlowMetric] Underlying metrics for `queue_persisted_growth_bytes` not yet instantiated, could not capture their rates
[2025-01-08T15:01:31,956][WARN ][org.logstash.instrument.metrics.LazyInstantiatedFlowMetric] Underlying metrics for `queue_persisted_growth_events` not yet instantiated, could not capture their rates
[2025-01-08T15:01:31,956][WARN ][org.logstash.instrument.metrics.LazyInstantiatedFlowMetric] Underlying metrics for `queue_persisted_growth_bytes` not yet instantiated, could not capture their rates
[2025-01-08T15:01:36,970][WARN ][org.logstash.instrument.metrics.LazyInstantiatedFlowMetric] Underlying metrics for `queue_persisted_growth_events` not yet instantiated, could not capture their rates
[2025-01-08T15:01:36,970][WARN ][org.logstash.instrument.metrics.LazyInstantiatedFlowMetric] Underlying metrics for `queue_persisted_growth_bytes` not yet instantiated, could not capture their rates

and when we hit localhost:9600/_node/stats, error below:

{
    "status": 500,
    "request_method": "GET",
    "path_info": "/_node/stats",
    "query_string": "",
    "http_version": "HTTP/1.1",
    "http_accept": "*/*",
    "error": "Unexpected Internal Error",
    "class": "NoMethodError",
    "message": "undefined method `value' for nil:NilClass",
    "backtrace": [
        "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/api/commands/stats.rb:35:in `block in queue'",
        "org/jruby/RubyArray.java:1981:in `each'",
        "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/api/commands/stats.rb:33:in `queue'",
        "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/api/modules/node_stats.rb:53:in `queue'",
        "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/api/modules/node_stats.rb:34:in `block in NodeStats'",
        "org/jruby/RubyMethod.java:119:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1807:in `block in compile!'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1074:in `block in route!'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1092:in `route_eval'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1074:in `block in route!'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1123:in `block in process_route'",
        "org/jruby/RubyKernel.java:1426:in `catch'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1121:in `process_route'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1072:in `block in route!'",
        "org/jruby/RubyArray.java:1981:in `each'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1069:in `route!'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1193:in `block in dispatch!'",
        "org/jruby/RubyKernel.java:1426:in `catch'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1164:in `invoke'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1188:in `dispatch!'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1004:in `block in call!'",
        "org/jruby/RubyKernel.java:1426:in `catch'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1164:in `invoke'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1004:in `call!'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:993:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-protection-4.1.1/lib/rack/protection/base.rb:53:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-protection-4.1.1/lib/rack/protection/xss_header.rb:20:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-protection-4.1.1/lib/rack/protection/path_traversal.rb:18:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-protection-4.1.1/lib/rack/protection/json_csrf.rb:28:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-protection-4.1.1/lib/rack/protection/base.rb:53:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-protection-4.1.1/lib/rack/protection/base.rb:53:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-protection-4.1.1/lib/rack/protection/frame_options.rb:33:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/middleware/logger.rb:17:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-3.1.8/lib/rack/head.rb:15:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:227:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:2138:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-3.1.8/lib/rack/urlmap.rb:76:in `block in call'",
        "org/jruby/RubyArray.java:1981:in `each'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-3.1.8/lib/rack/urlmap.rb:60:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/api/rack_app.rb:75:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/api/rack_app.rb:49:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-3.1.8/lib/rack/builder.rb:277:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/puma-6.5.0-java/lib/puma/request.rb:99:in `block in handle_request'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/puma-6.5.0-java/lib/puma/thread_pool.rb:389:in `with_force_shutdown'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/puma-6.5.0-java/lib/puma/request.rb:98:in `handle_request'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/puma-6.5.0-java/lib/puma/server.rb:468:in `process_client'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/puma-6.5.0-java/lib/puma/server.rb:249:in `block in run'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/puma-6.5.0-java/lib/puma/thread_pool.rb:166:in `block in spawn_thread'"
    ]
}

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: f04e9f8

[mashhurs]

lgtm~!