[8.12] Regex support of add_resource_metadata (backport #834) (#835)

* Regex support of add_resource_metadata (#834)

* adding page for adavnce agent with k8s

* adding kubernetes provider example

* fxing typos and indentation

* fxing typos and indentation

* Update docs/en/ingest-management/elastic-agent/advanced-kubernetes-managed-by-fleet.asciidoc

Co-authored-by: David Kilfoyle <[email protected]>

* Update docs/en/ingest-management/elastic-agent/advanced-kubernetes-managed-by-fleet.asciidoc

Co-authored-by: David Kilfoyle <[email protected]>

* Update docs/en/ingest-management/elastic-agent/advanced-kubernetes-managed-by-fleet.asciidoc

Co-authored-by: David Kilfoyle <[email protected]>

* Update docs/en/ingest-management/elastic-agent/advanced-kubernetes-managed-by-fleet.asciidoc

Co-authored-by: David Kilfoyle <[email protected]>

* Update docs/en/ingest-management/elastic-agent/advanced-kubernetes-managed-by-fleet.asciidoc

Co-authored-by: David Kilfoyle <[email protected]>

* Update docs/en/ingest-management/elastic-agent/advanced-kubernetes-managed-by-fleet.asciidoc

Co-authored-by: David Kilfoyle <[email protected]>

* Update docs/en/ingest-management/elastic-agent/advanced-kubernetes-managed-by-fleet.asciidoc

Co-authored-by: David Kilfoyle <[email protected]>

* udpating code yaml box and adding parameter of manifest

* updating number in list

* updating number in list and iamge for kibana UI

* adding initial doc for metadata enrichment

* adding initial doc for metadata enrichment

* adding initial doc for metadata enrichment

* adding examples

* Update docs/en/ingest-management/elastic-agent/configuration/providers/kubernetes-provider.asciidoc

Co-authored-by: David Kilfoyle <[email protected]>

* Update docs/en/ingest-management/processors/processor-add_kubernetes_metadata.asciidoc

Co-authored-by: David Kilfoyle <[email protected]>

* fixing conflicts

* fixing part of add_resource

* Fix kubernetes provider documentation (#817)

* fix namespace fields; add information about the deployment and cronjob settings in add_resource_metadata

Signed-off-by: Tetiana Kravchenko <[email protected]>

* Update docs/en/ingest-management/elastic-agent/configuration/providers/kubernetes-provider.asciidoc

Co-authored-by: David Kilfoyle <[email protected]>

* Update docs/en/ingest-management/elastic-agent/configuration/providers/kubernetes-provider.asciidoc

Co-authored-by: Andrew Gizas <[email protected]>

* Update docs/en/ingest-management/elastic-agent/configuration/providers/kubernetes-provider.asciidoc

Co-authored-by: Andrew Gizas <[email protected]>

---------

Signed-off-by: Tetiana Kravchenko <[email protected]>
Co-authored-by: David Kilfoyle <[email protected]>
Co-authored-by: Andrew Gizas <[email protected]>

* squashing

* fixing newline

---------

Signed-off-by: Tetiana Kravchenko <[email protected]>
Co-authored-by: David Kilfoyle <[email protected]>
Co-authored-by: Tetiana Kravchenko <[email protected]>
(cherry picked from commit 64bad76)

# Conflicts:
#	docs/en/ingest-management/elastic-agent/configuration/providers/kubernetes-provider.asciidoc

* Fix merge conflict

---------

Co-authored-by: Andrew Gizas <[email protected]>
Co-authored-by: David Kilfoyle <[email protected]>

docs/en/ingest-management/elastic-agent/configuration/providers/kubernetes-provider.asciidoc

@@ -68,21 +68,33 @@ will be excluded from the event.
 By default it is `true`.
 `annotations.dedot`:: (Optional) If set to be `true` in the provider config, then `.` in annotations will be replaced
 with `_`. By default it is `true`.
-`add_resource_metadata`:: (Optional) Specify labels and annotations filters for the extra metadata coming from node and namespace.
- `add_resource_metadata` can be done for `node` or `namespace`. By default all labels are included
-while annotations are not. These settings are useful when storing labels and annotations
-that require special handling to avoid overloading the storage output. The enrichment of `node` or `namespace` metadata
-can be individually disabled by setting `enabled: false`.
-Example:
+`add_resource_metadata`:: (Optional) Specify filters and configration for the extra metadata, that will be added to the event.
+Configuration parameters:
+ * `node` or `namespace`: Specify labels and annotations filters for the extra metadata coming from node and namespace. By default
+ all labels are included while annotations are not. To change the default behaviour `include_labels`, `exclude_labels` and `include_annotations`
+ can be defined. These settings are useful when storing labels and annotations that require special handling to avoid overloading the storage output.
+ The enrichment of `node` or `namespace` metadata can be individually disabled by setting `enabled: false`.
+ Wildcards are supported in these settings by using `use_regex_include: true` in combination with `include_labels`, and respectively by setting `use_regex_exclude: true` in combination with `exclude_labels`.
+ * `deployment`: If resource is `pod` and it is created from a `deployment`, by default the deployment name isn't added, this can be enabled by setting `deployment: true`.
+ * `cronjob`: If resource is `pod` and it is created from a `cronjob`, by default the cronjob name isn't added, this can be enabled by setting `cronjob: true`.
+ Example:
 
 ["source","yaml",subs="attributes"]
 -------------------------------------------------------------------------------------
       add_resource_metadata:
         namespace:
+          #use_regex_include: false
           include_labels: ["namespacelabel1"]
+          #use_regex_exclude: false
+          #exclude_labels: ["namespacelabel2"]
         node:
+          #use_regex_include: false
           include_labels: ["nodelabel2"]
           include_annotations: ["nodeannotation1"]
+          #use_regex_exclude: false
+          #exclude_labels: ["nodelabel3"]
+        #deployment: false
+        #cronjob: false
 -------------------------------------------------------------------------------------
 
 

docs/en/ingest-management/processors/processor-add_kubernetes_metadata.asciidoc

@@ -137,7 +137,7 @@ a| Filters and configuration for adding extra metadata to the event. This settin
 By default all labels are included, but annotations are not.
 To change the default behavior, you can set `include_labels`, `exclude_labels`, and `include_annotations`.
 These settings are useful when storing labels and annotations that require special handling to avoid overloading the storage output.
-Note that wildcards are not supported in these settings.
+Wildcards are supported in these settings by using `use_regex_include: true` in combination with `include_labels`, and respectively by setting `use_regex_exclude: true` in combination with `exclude_labels`.
 To turn off enrichment of `node` or `namespace` metadata individually, set `enabled: false`.
 * `deployment`: If the resource is `pod` and it is created from a `deployment`, the deployment name is not added by default. To enable this behavior, set `deployment: true`.
 * `cronjob`: If the resource is `pod` and it is created from a `cronjob`, the cronjob name is not added by default. To enable this behavior, set `cronjob: true`.
@@ -151,11 +151,17 @@ To turn off enrichment of `node` or `namespace` metadata individually, set `enab
       add_resource_metadata:
         namespace:
           include_labels: ["namespacelabel1"]
+          # use_regex_include: false
+          # use_regex_exclude: false
+          # exclude_labels: ["namespacelabel2"]
           #labels.dedot: true
           #annotations.dedot: true
         node:
+          # use_regex_include: false
           include_labels: ["nodelabel2"]
           include_annotations: ["nodeannotation1"]
+          # use_regex_exclude: false
+          # exclude_annotations: ["nodeannotation2"]
           #labels.dedot: true
           #annotations.dedot: true
         deployment: true