Sidekiq instrumentation: evaluate covering workflows other than perform_async

[fallwith]

Our APM Sidekiq instrumentation has coverage for Sidekiq clients and Sidekiq servers.

For the Sidekiq client instrumentation piece, we really only set a DT header in the job metadata, so the Sidekiq server instrumentation piece needs to be engaged before we can glean proper job metrics.

Because of this setup, any job performed synchronously runs the risk of not being instrumented properly given that the server instrumentation is not engaged.

For example:

# engages client + server, works great
DeadEndJob.perform_async(args)

# engages client, instrumentation lacking
DeadEndJob.perform_inline(args)

# engages neither, instrumentation lacking
DeadEndJob.new.perform(args)

We should evaluate the level of effort required to cover the use cases that are currently unsupported.

[workato-integration]

https://new-relic.atlassian.net/browse/NR-159398

[kaylareopelle]

Re: #2379, add perform_now to the list of methods to investigate

[fallwith]

Using perform_now works just fine. I'm not sure what was causing it not to earlier, as the v6.4.2 version of Sidekiq that was around back then offers a perform_now method that engages with all registered client middleware and server middleware, just like the current version of the method does. It is the engagement of our server middleware that produces a segment. I will add a unit test to demonstrate perform_now working well.

Using MyJobClass.new.perform can't possibly work because that perform method involves 100% Sidekiq user authored code and does not by default engage any Sidekiq middleware. Apart from some minimum requirements about the arguments that the perform method needs to accept, the body of the method can truly do anything the Sidekiq user wishes it to. I currently think that invoking #perform on a job class instance directly is for testing or one-off debugging scenarios and is it a bit of a production context anti-pattern. Users who wish to directly fire off a job without queuing it up first should leverage perform_inline which the agent instruments just fine.

As for perform_now and perform_later, those are ActiveJob-isms. Our Sidekiq suite currently runs outside of Rails and our Rails tests currently don't leverage Sidekiq, so we'd need to set something up that mixes the two.

[fallwith]

Pushing the ActiveJob related testing of perform_now and perform_later off to #2835.

[fallwith]

• perform_inline works great. See Add unit test coverage for Sidekiq job perform_inline #2836
• MyJob.new.perform is not intended to be covered at this time
  • Should any customers bring forth feature requests, we'll consider them
  • Ultimately if #perform needs to be covered, we should reach out to Mike Perham to evaluate an instrumentation agnostic approach that assists with routing #perform calls through Sidekiq middleware
• The perform_now and perform_later ActiveJob-isms will be tracked separately with Add unit test coverage for Sidekiq job perform_inline #2836