Add missing kubernetes fields; update istio doc to use kubernetes.pod.ip instead of host #37578

tetianakravchenko · 2024-01-08T16:33:18Z

Proposed commit message

WHAT: Extend documentation to add missing kubernetes fields; update istio documentation to use explicitly kubernetes.pod.ip instead of host
WHY:
- include all kubernetes.* fields;
- host variable is confusing as it is not clear to which IP it is referring

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

[ ]

How to test this PR locally

Related issues

Relates Align the kubernetes provider variables documentation #37601

Use cases

Screenshots

Logs

….ip instead of host Signed-off-by: Tetiana Kravchenko <[email protected]>

mergify · 2024-01-08T16:34:29Z

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @tetianakravchenko? 🙏.
For such, you'll need to label your PR with:

The upcoming major version of the Elastic Stack
The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

elasticmachine · 2024-01-08T16:42:41Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2024-01-08T16:33:37.988+0000
Duration: 9 min 57 sec

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

gsantoro · 2024-01-09T16:13:21Z

libbeat/docs/shared-autodiscover.asciidoc

@@ -221,18 +221,22 @@ These are the fields available within config templating. The `kubernetes.*` fiel
  * kubernetes.container.image
  * kubernetes.container.name
  * kubernetes.namespace
+  * kubernetes.namespace_uid


Suggested change

* kubernetes.namespace_uid

* kubernetes.namespace.uid

The field name is indeed kubernetes.namespace_uid.
@tetianakravchenko maybe we should find a way in this document to add also kubernetes.labels, kubernetes.annotations, kubernetes.namespace_labels and kubernetes.namespace_annotations. Those metadata fields are also part of the metadata fields that can be used in templating when the right parameters are set.
There is this section in the doc that states

If the `include_annotations` config is added to the provider config, then the list of annotations present in the config are added to the event.

which is ok as an information but we don't list the field. Maybe we could list all fields above and leave those comments there as is.
Also we could add some comments about the namespace_labels and namespace_annotations based on the add_resource_metadata configuration.

@gsantoro there is no such field as kubernetes.namespace.uid :

I see that we have such field documented for elastic-agent - https://www.elastic.co/guide/en/fleet/current/kubernetes-provider.html, I believe it is a mistake in the elastic-agent documentation

@MichaelKatsoulis I am trying to wrap my head around the docs and the implementation.

I can found two references kubernetes dynamic variables at https://www.elastic.co/guide/en/fleet/current/kubernetes-provider.html and https://www.elastic.co/guide/en/beats/metricbeat/current/configuration-autodiscover.html. For example kubernetes.namespace.uid is in the first one and not in the second doc.

Should those two docs have the same variables and both of them have kubernetes.namespace_uid?

what about this code from elastic-agent-autodiscover where the docs point at namespace.uid and the test at namespace_uid? at https://github.com/elastic/elastic-agent-autodiscover/blob/4f4898db82e5a5b37eab7e0e093435b9fb739461/kubernetes/metadata/namespace_test.go#L69

I could use some clarification about where in the code those variables are created.

@gsantoro kubernetes.namespace is unfortunately a keyword and not an object. There was a try in 8.0 to do this breaking change and put all namespace related fields under the kubernetes.namespace object but it was reverted before it reached production. This was done mainly because of the possible problems it would create to existing customers and the not easy migration procedure.

The documentation in https://www.elastic.co/guide/en/fleet/current/kubernetes-provider.html is wrong. The part of the code where those values are added is here https://github.com/elastic/elastic-agent-autodiscover/blob/b42be0df903aba6580b80761af6df86fa80c845d/kubernetes/metadata/namespace.go#L117

…ata.deployment and add_resource_metadata.cronjob as the default value is false Signed-off-by: Tetiana Kravchenko <[email protected]>

tetianakravchenko · 2024-01-10T17:17:54Z

Hey @MichaelKatsoulis I've extended the list of fields - 00cb450, similar to what we have in https://www.elastic.co/guide/en/fleet/current/kubernetes-provider.html, but with correct namespace fields

which is ok as an information but we don't list the field. Maybe we could list all fields above and leave those comments there as is.

does the commit above address this your comment?

elasticmachine · 2024-01-10T17:22:24Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Duration: 8 min 24 sec

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

libbeat/docs/shared-autodiscover.asciidoc

belimawr · 2024-01-15T08:17:21Z

libbeat/docs/shared-autodiscover.asciidoc

+
+|`kubernetes.annotations.*`
+|`object`
+|Object of labels of the Node. Annotations are not available by default


There seems to be a typo here:

Suggested change

|Object of labels of the Node. Annotations are not available by default

|Object of annotations of the Node. Annotations are not available by default

@belimawr done in this commit 2fbf545

Signed-off-by: Tetiana Kravchenko <[email protected]>

elasticmachine · 2024-01-15T18:14:04Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2024-01-15T18:05:50.040+0000
Duration: 8 min 10 sec

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

elasticmachine · 2024-01-15T18:26:38Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Duration: 8 min 3 sec

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

libbeat/docs/shared-autodiscover.asciidoc

Co-authored-by: Andrew Gizas <[email protected]>

elasticmachine · 2024-01-18T12:10:24Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2024-01-18T12:01:16.792+0000
Duration: 9 min 54 sec

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

….ip instead of host (elastic#37578) * add missing kubernetes fields; update istio doc to use kubernetes.pod.ip instead of host Signed-off-by: Tetiana Kravchenko <[email protected]> * extend documentation with full list of fields; fix add_resource_metadata.deployment and add_resource_metadata.cronjob as the default value is false Signed-off-by: Tetiana Kravchenko <[email protected]> * test all variables and adjust documentation accordingly Signed-off-by: Tetiana Kravchenko <[email protected]> * Update libbeat/docs/shared-autodiscover.asciidoc Co-authored-by: Andrew Gizas <[email protected]> * Update libbeat/docs/shared-autodiscover.asciidoc Co-authored-by: Andrew Gizas <[email protected]> --------- Signed-off-by: Tetiana Kravchenko <[email protected]> Co-authored-by: Andrew Gizas <[email protected]>

add missing kubernetes fields; update istio doc to use kubernetes.pod…

a93c80d

….ip instead of host Signed-off-by: Tetiana Kravchenko <[email protected]>

tetianakravchenko requested review from a team as code owners January 8, 2024 16:33

tetianakravchenko requested review from gizas, constanca-m, belimawr and rdner January 8, 2024 16:33

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jan 8, 2024

mergify bot assigned tetianakravchenko Jan 8, 2024

pierrehilbert added the Team:Elastic-Agent Label for the Agent team label Jan 8, 2024

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jan 8, 2024

gsantoro approved these changes Jan 9, 2024

View reviewed changes

MichaelKatsoulis approved these changes Jan 9, 2024

View reviewed changes

gsantoro reviewed Jan 9, 2024

View reviewed changes

extend documentation with full list of fields; fix add_resource_metad…

00cb450

…ata.deployment and add_resource_metadata.cronjob as the default value is false Signed-off-by: Tetiana Kravchenko <[email protected]>

cmacknz approved these changes Jan 10, 2024

View reviewed changes

MichaelKatsoulis requested changes Jan 11, 2024

View reviewed changes

libbeat/docs/shared-autodiscover.asciidoc Outdated Show resolved Hide resolved

belimawr requested changes Jan 15, 2024

View reviewed changes

test all variables and adjust documentation accordingly

2fbf545

Signed-off-by: Tetiana Kravchenko <[email protected]>

Merge branch 'main' into autodiscover-documentation

576106a

MichaelKatsoulis approved these changes Jan 16, 2024

View reviewed changes

gizas reviewed Jan 18, 2024

View reviewed changes

libbeat/docs/shared-autodiscover.asciidoc Outdated Show resolved Hide resolved

gizas reviewed Jan 18, 2024

View reviewed changes

libbeat/docs/shared-autodiscover.asciidoc Outdated Show resolved Hide resolved

gizas approved these changes Jan 18, 2024

View reviewed changes

tetianakravchenko and others added 2 commits January 18, 2024 13:00

Update libbeat/docs/shared-autodiscover.asciidoc

6041d3f

Co-authored-by: Andrew Gizas <[email protected]>

Update libbeat/docs/shared-autodiscover.asciidoc

e9371aa

Co-authored-by: Andrew Gizas <[email protected]>

tetianakravchenko requested a review from belimawr January 18, 2024 17:10

tetianakravchenko mentioned this pull request Jan 22, 2024

Align the kubernetes provider variables documentation #37601

Closed

belimawr approved these changes Jan 22, 2024

View reviewed changes

tetianakravchenko merged commit ca03640 into elastic:main Jan 22, 2024
22 checks passed

tetianakravchenko deleted the autodiscover-documentation branch January 22, 2024 10:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add missing kubernetes fields; update istio doc to use kubernetes.pod.ip instead of host #37578

Add missing kubernetes fields; update istio doc to use kubernetes.pod.ip instead of host #37578

tetianakravchenko commented Jan 8, 2024 •

edited

Loading

mergify bot commented Jan 8, 2024

elasticmachine commented Jan 8, 2024

Build stats

gsantoro Jan 9, 2024

MichaelKatsoulis Jan 10, 2024

tetianakravchenko Jan 10, 2024

gsantoro Jan 10, 2024 •

edited

Loading

MichaelKatsoulis Jan 16, 2024

tetianakravchenko commented Jan 10, 2024 •

edited

Loading

elasticmachine commented Jan 10, 2024

Build stats

belimawr Jan 15, 2024

tetianakravchenko Jan 15, 2024 •

edited

Loading

elasticmachine commented Jan 15, 2024

Build stats

elasticmachine commented Jan 15, 2024

Build stats

elasticmachine commented Jan 18, 2024

Build stats

	\|Object of labels of the Node. Annotations are not available by default
	\|Object of annotations of the Node. Annotations are not available by default

Add missing kubernetes fields; update istio doc to use kubernetes.pod.ip instead of host #37578

Add missing kubernetes fields; update istio doc to use kubernetes.pod.ip instead of host #37578

Conversation

tetianakravchenko commented Jan 8, 2024 • edited Loading

Proposed commit message

Checklist

Author's Checklist

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

mergify bot commented Jan 8, 2024

elasticmachine commented Jan 8, 2024

💚 Build Succeeded

Build stats

❕ Flaky test report

🤖 GitHub comments

gsantoro Jan 9, 2024

Choose a reason for hiding this comment

MichaelKatsoulis Jan 10, 2024

Choose a reason for hiding this comment

tetianakravchenko Jan 10, 2024

Choose a reason for hiding this comment

gsantoro Jan 10, 2024 • edited Loading

Choose a reason for hiding this comment

MichaelKatsoulis Jan 16, 2024

Choose a reason for hiding this comment

tetianakravchenko commented Jan 10, 2024 • edited Loading

elasticmachine commented Jan 10, 2024

💚 Build Succeeded

Build stats

❕ Flaky test report

🤖 GitHub comments

belimawr Jan 15, 2024

Choose a reason for hiding this comment

tetianakravchenko Jan 15, 2024 • edited Loading

Choose a reason for hiding this comment

elasticmachine commented Jan 15, 2024

💚 Build Succeeded

Build stats

❕ Flaky test report

🤖 GitHub comments

elasticmachine commented Jan 15, 2024

💚 Build Succeeded

Build stats

❕ Flaky test report

🤖 GitHub comments

elasticmachine commented Jan 18, 2024

💚 Build Succeeded

Build stats

❕ Flaky test report

🤖 GitHub comments

tetianakravchenko commented Jan 8, 2024 •

edited

Loading

gsantoro Jan 10, 2024 •

edited

Loading

tetianakravchenko commented Jan 10, 2024 •

edited

Loading

tetianakravchenko Jan 15, 2024 •

edited

Loading