-
Notifications
You must be signed in to change notification settings - Fork 267
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Document enhanced tracing in Studio via OTel #5967
base: dev
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
@@ -72,6 +72,70 @@ telemetry: | |||||||||||||||||||||||
field_level_instrumentation_sampler: always_off | ||||||||||||||||||||||||
``` | ||||||||||||||||||||||||
|
||||||||||||||||||||||||
<MinVersion version="1.49.0"> | ||||||||||||||||||||||||
|
||||||||||||||||||||||||
### Enhanced tracing in Studio via OpenTelemetry | ||||||||||||||||||||||||
|
||||||||||||||||||||||||
</MinVersion> | ||||||||||||||||||||||||
|
||||||||||||||||||||||||
<ExperimentalFeature /> | ||||||||||||||||||||||||
|
||||||||||||||||||||||||
Beginning in v1.49.0, the router supports sending traces to Studio via the more detailed OTel (OpenTelemetry) protocol. | ||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Changing references to "OTel protocol" to "OTLP" to be consistent with the option's name and the well-known OTLP acronym
Suggested change
|
||||||||||||||||||||||||
Support for OTel traces has historically only been available for 3rd party APM tools. With this option, | ||||||||||||||||||||||||
Studio can now provide a much more granular view of Router internals than the legacy Apollo tracing protocol. | ||||||||||||||||||||||||
|
||||||||||||||||||||||||
Benefits include: | ||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||||||||||||
|
||||||||||||||||||||||||
- A comprehensive way to visualize the Router execution path in Studio. | ||||||||||||||||||||||||
- Additional spans that were previously not included in Studio traces, such as query parsing, planning, execution, and more. | ||||||||||||||||||||||||
- Additional attributes including HTTP request details, REST connector details, and more. | ||||||||||||||||||||||||
Comment on lines
+89
to
+91
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||||||||||||
|
||||||||||||||||||||||||
It is expected that this will become the default in a future version of Router. | ||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Removing, generally we don't document roadmap in docs
Suggested change
|
||||||||||||||||||||||||
|
||||||||||||||||||||||||
#### Configuration | ||||||||||||||||||||||||
|
||||||||||||||||||||||||
This change adds a new configuration option `telemetry.apollo.experimental_otlp_tracing_sampler`. Use this option to send | ||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||||||||||||
a percentage of traces to Studio via OTLP instead of the native Apollo Usage Reporting protocol. Supported values: | ||||||||||||||||||||||||
|
||||||||||||||||||||||||
- `always_off` (default): send all traces via the legacy Apollo Usage Reporting protocol. | ||||||||||||||||||||||||
- `always_on`: send all traces via OTLP. | ||||||||||||||||||||||||
- `0.0 - 1.0` (used for testing): the ratio of traces to send via OTLP (0.5 = 50 / 50). | ||||||||||||||||||||||||
Comment on lines
+100
to
+102
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||||||||||||
|
||||||||||||||||||||||||
Note that this sampler is only applied _after_ the common tracing sampler, for example: | ||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||||||||||||
|
||||||||||||||||||||||||
#### Sample 1% of traces, send all traces via OTLP: | ||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||||||||||||
|
||||||||||||||||||||||||
```yaml | ||||||||||||||||||||||||
telemetry: | ||||||||||||||||||||||||
apollo: | ||||||||||||||||||||||||
# Send all traces via OTLP | ||||||||||||||||||||||||
experimental_otlp_tracing_sampler: always_on | ||||||||||||||||||||||||
|
||||||||||||||||||||||||
exporters: | ||||||||||||||||||||||||
tracing: | ||||||||||||||||||||||||
common: | ||||||||||||||||||||||||
# Sample traces at 1% of all traffic | ||||||||||||||||||||||||
sampler: 0.01 | ||||||||||||||||||||||||
``` | ||||||||||||||||||||||||
|
||||||||||||||||||||||||
OTel traces sent to Studio will not necessarily be identical to the ones sent to 3rd Party APM tools via OTLP: | ||||||||||||||||||||||||
|
||||||||||||||||||||||||
- Only specific OTLP attributes will be included for parity with what is provided in legacy traces today. This ensures that data privacy | ||||||||||||||||||||||||
is maintained in an equivalent manner. The existing Router configuration options for Apollo telemetry will continue to function | ||||||||||||||||||||||||
with OTLP traces, such as forwarding of GraphQL errors, headers, and variables. | ||||||||||||||||||||||||
- Some features of OTLP traces may only be available in Studio and not in 3rd Party APM tools (e.g. resolver-level timing information from | ||||||||||||||||||||||||
[Federated Tracing](../../federation/metrics/#enabling-federated-tracing)). | ||||||||||||||||||||||||
Comment on lines
+121
to
+127
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||||||||||||
|
||||||||||||||||||||||||
<Note> | ||||||||||||||||||||||||
|
||||||||||||||||||||||||
This change results in using a new wire protocol for traces, and some users may experience an increase in tracing traffic | ||||||||||||||||||||||||
to GraphOS Studio due to the additional detail being captured. In exceptional situations it may be necessary to send fewer traces. | ||||||||||||||||||||||||
This can be achieved via sending fewer traces (`telemetry.exporters.tracing.common.sampler`) or as a last resort, falling back | ||||||||||||||||||||||||
to the old protocol via `telemetry.apollo.otlp_tracing_sampler` to send fewer OTLP traces or fully disable them. | ||||||||||||||||||||||||
Any performance regressions due to the new tracing protocol should also be reported to the Apollo support team. | ||||||||||||||||||||||||
Comment on lines
+131
to
+135
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||||||||||||
|
||||||||||||||||||||||||
</Note> | ||||||||||||||||||||||||
|
||||||||||||||||||||||||
### Experimental local field metrics | ||||||||||||||||||||||||
|
||||||||||||||||||||||||
Apollo Router can send field-level metrics to GraphOS without using FTV1 tracing. This feature is experimental and is not yet displayable in GraphOS Studio. | ||||||||||||||||||||||||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.