Running vault v1.14.1 without DBUS_SESSION_BUS_ADDRESS set and no /run/user/<uid>/bus launches a persistent dbus-daemon #22560

brinnjoyce · 2023-08-25T00:48:19Z

Describe the bug
Running vault as a user without DBUS_SESSION_BUS_ADDRESS set and no /run/user/<uid>/bus file causes vault to run dbus-launch which launches a persistent dbus-daemon process like this:
/usr/bin/dbus-daemon --syslog --fork --print-pid 4 --print-address 6 --session.

A new dbus-daemon will be launched every time vault is run.

The dbus-daemon processes use inotify instances, so if vault is executed enough times (configuration management in our instance) the user running vault will run into the fs.inotify.max_user_instances limit or fd limits causing issues for systemd etc.

To Reproduce
Steps to reproduce the behavior:

Run vault --version with no DBUS_SESSION_BUS_ADDRESS set and no /run/user/<uid>/bus file
See that there is now a new dbus-daemon process running as the user that ran vault.

Expected behavior
There should be no extra processes left running.

Environment:

Vault CLI Version (retrieve with vault version): v1.14.1
Server Operating System/Architecture: Ubuntu 18.04/22.04 amd64

Additional context

This is related to the godbus/dbus issue raised here godbus/dbus#372
Testing this as a workaround does avoid the issue:

env DBUS_SESSION_BUS_ADDRESS=$XDG_RUNTIME_DIR/bus vault --version

The text was updated successfully, but these errors were encountered:

biazmoreira · 2023-08-29T12:37:31Z

This is related to the snowflake bug, which I re-opened here snowflakedb/gosnowflake#901

We'll try to work this around with snowflake before trying to fix it directly in Vault.

danielhass · 2023-09-01T07:06:14Z

Thanks @vegaman for reporting this and providing a workaround.

We ran into this issue too with v1.14.1 and setting the env variable DBUS_SESSION_BUS_ADDRESS=$XDG_RUNTIME_DIR/bus seems to be a proper workaround so far.

jkhuang-42 · 2023-09-25T20:21:26Z

Running into this issue on vault version v1.14.3-ent but the workaround did not seem to have any effect, new dbus daemons still seem to be generated each time vault is called.

To be clear, is the workaround just export DBUS_SESSION_BUS_ADDRESS=$XDG_RUNTIME_DIR/bus ?

Also in my environment, XDG_RUNTIME_DIR does not have a value, is that an issue?

Soromeister · 2023-10-09T07:53:27Z

@jkhuang-42, please see this KB article on how to solve this in the meantime. The article also covers the situation with no XDG_RUNTIME_DIR value.

aswen · 2024-06-13T10:50:01Z

I run into this on 2 of the 3 servers that run vault 1.17.0-1 (Debian Bookworm).

It's easy to reproduce:

$ vault --version
WARN[0000]log.go:244 gosnowflake.(*defaultLogger).Warn DBUS_SESSION_BUS_ADDRESS envvar looks to be not set, this can lead to runaway dbus-daemon processes. To avoid this, set envvar DBUS_SESSION_BUS_ADDRESS=$XDG_RUNTIME_DIR/bus (if it exists) or DBUS_SESSION_BUS_ADDRESS=/dev/null. 
Vault v1.17.0 (72850df1bc10581b74ba5f0f7b3736f31c034235), built 2024-06-10T10:11:34Z

and to work around (as mentioned in the opening comment):

$ env DBUS_SESSION_BUS_ADDRESS=$XDG_RUNTIME_DIR/bus vault --version
Vault v1.17.0 (72850df1bc10581b74ba5f0f7b3736f31c034235), built 2024-06-10T10:11:34Z

Without doing anything, Vault runs ok (by the looks of it) on all three servers.

I have a few questions:

What is the risk of running Vault without fixing this? I see no issues on the two vault servers that warn.
Is there anything we should do to fix this permanently?
I see this issue was closed 9 months ago. Did we miss something that fixes this? Should we change something on our servers?
I don't understand that on one of our servers this problem is not manifesting. Even though we don't have DBUS_SESSION_BUS_ADDRESS set on that one.

Thanks for the clarification.

PJ1288 · 2024-06-13T17:36:44Z

I am running into the same issue as @aswen when upgrading to v1.17.0.
level=warning msg="DBUS_SESSION_BUS_ADDRESS envvar looks to be not set, this can lead to runaway dbus-daemon processe>

yaboeing · 2024-06-27T09:53:53Z

We've just upgraded and have the same on

vault --version
WARN[0000]log.go:244 gosnowflake.(*defaultLogger).Warn DBUS_SESSION_BUS_ADDRESS envvar looks to be not set, this can lead to runaway dbus-daemon processes. To avoid this, set envvar DBUS_SESSION_BUS_ADDRESS=$XDG_RUNTIME_DIR/bus (if it exists) or DBUS_SESSION_BUS_ADDRESS=/dev/null.
Vault v1.17.0 (72850df1bc10581b74ba5f0f7b3736f31c034235), built 2024-06-10T10:11:34Z

MMquant · 2024-07-09T06:53:46Z

Just installed vault on RHEL8

$ vault -version
WARN[0000]log.go:244 gosnowflake.(*defaultLogger).Warn DBUS_SESSION_BUS_ADDRESS envvar looks to be not set, this can lead to runaway dbus-daemon processes. To avoid this, set envvar DBUS_SESSION_BUS_ADDRESS=$XDG_RUNTIME_DIR/bus (if it exists) or DBUS_SESSION_BUS_ADDRESS=/dev/null. 
Vault v1.17.1 (b8ab595639e7473eb153571521bbaf522cdafb27), built 2024-06-25T16:33:25Z

sfc-gh-dszmolka · 2024-07-17T06:22:01Z

hi folks, this warning is coming from gosnowflake and tries to detect the dbus-daemon leak which can happen due to the situation described in the Issue description and can lead to real issues, again as described by the original poster.
gosnowflake does absolutely nothing else besides logging that single line.

related issues:

Use the default fallback path for the session bus godbus/dbus#372
Using keyring on Linux leaves dbus-daemons running, perhaps because godbus connection is not closed 99designs/keyring#103
Zombie dbus processes generated by github.com/99designs/keyring snowflakedb/gosnowflake#773

if you found false positives (where the warning is emitted, but there are no actual orphaned dbus-daemon processes generated), we're happy to investigate and fix it if one of you could please raise an issue in gosnowflake with the details if that would be possible. thank you in advance!

thedoc31 · 2024-08-26T21:29:12Z

When will the gosnowflake v1.11.0 with the fix for this warning message be added into the vault executables?

juparker37 · 2024-08-28T00:39:12Z

Edit: Nevermind I see it when using "sudo vault" cli.

sudo vault --help

WARN[0000]log.go:244 gosnowflake.(*defaultLogger).Warn DBUS_SESSION_BUS_ADDRESS envvar looks to be not set, this can lead to runaway dbus-daemon processes. To avoid this, set envvar DBUS_SESSION_BUS_ADDRESS=$XDG_RUNTIME_DIR/bus (if it exists) or DBUS_SESSION_BUS_ADDRESS=/dev/null.
Usage: vault <command> [args]

Common commands:
    read        Read data and retrieves secrets
    write       Write data, configuration, and secrets
    delete      Delete secrets and configuration
    list        List data or secrets
    login       Authenticate locally
    agent       Start a Vault agent
    server      Start a Vault server
    status      Print seal and HA status
    unwrap      Unwrap a wrapped secret

cat /etc/os-release NAME="Red Hat Enterprise Linux" VERSION="8.10 (Ootpa)" ID="rhel" ID_LIKE="fedora" VERSION_ID="8.10" PLATFORM_ID="platform:el8" PRETTY_NAME="Red Hat Enterprise Linux 8.10 (Ootpa)"

sudo yum -y install vault

Updating Subscription Management repositories.
Hashicorp Stable - x86_64                                                      6.9 MB/s | 1.4 MB     00:00
Dependencies resolved.
===============================================================================================================
 Package                 Architecture             Version                    Repository                   Size
===============================================================================================================
Installing:
 vault                   x86_64                   1.17.3-1                   hashicorp                   149 M

Transaction Summary
===============================================================================================================
Install  1 Package

Total download size: 149 M
Installed size: 420 M
Downloading Packages:
vault-1.17.3-1.x86_64.rpm                                                       73 MB/s | 149 MB     00:02
---------------------------------------------------------------------------------------------------------------
Total                                                                           73 MB/s | 149 MB     00:02
Hashicorp Stable - x86_64                                                       61 kB/s | 3.9 kB     00:00
Importing GPG key 0xA621E701:
 Userid     : "HashiCorp Security (HashiCorp Package Signing) <[email protected]>"
 Fingerprint: 798A EC65 4E5C 1542 8C8E 42EE AA16 FCBC A621 E701
 From       : https://rpm.releases.hashicorp.com/gpg
Key imported successfully
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                       1/1
  Running scriptlet: vault-1.17.3-1.x86_64                                                                 1/1
  Installing       : vault-1.17.3-1.x86_64                                                                 1/1
  Running scriptlet: vault-1.17.3-1.x86_64                                                                 1/1
Generating Vault TLS key and self-signed certificate...
Generating a RSA private key
................................................................................................................................................................++++
...............................................................................................++++
writing new private key to 'tls.key'
-----
Vault TLS key and self-signed certificate have been generated in '/opt/vault/tls'.
time="2024-08-27T20:32:54-04:00" level=warning msg="DBUS_SESSION_BUS_ADDRESS envvar looks to be not set, this can lead to runaway dbus-daemon processes. To avoid this, set envvar DBUS_SESSION_BUS_ADDRESS=$XDG_RUNTIME_DIR/bus (if it exists) or DBUS_SESSION_BUS_ADDRESS=/dev/null." func="gosnowflake.(*defaultLogger).Warn" file="log.go:244"

  Verifying        : vault-1.17.3-1.x86_64                                                                 1/1
Installed products updated.

Installed:
  vault-1.17.3-1.x86_64

Complete!

vault --version
Vault v1.17.3 (c91c85442144e1228c02123fc4b19337f7d52700), built 2024-08-06T14:28:45Z

fen9li · 2024-09-01T06:14:47Z

Hey I still get same error even after upgrade to v1.17.3...

[xxxxxx@xxxxxx ~]$ vault -version
WARN[0000]log.go:244 gosnowflake.(*defaultLogger).Warn DBUS_SESSION_BUS_ADDRESS envvar looks to be not set, this can lead to runaway dbus-daemon processes. To avoid this, set envvar DBUS_SESSION_BUS_ADDRESS=$XDG_RUNTIME_DIR/bus (if it exists) or DBUS_SESSION_BUS_ADDRESS=/dev/null.
Vault v1.17.3 (c91c85442144e1228c02123fc4b19337f7d52700), built 2024-08-06T14:28:45Z
[xxxxxx@xxxxxx ~]$

Work around works:

[psplatforms@ip-10-244-145-54 ~]$ export DBUS_SESSION_BUS_ADDRESS=$XDG_RUNTIME_DIR/bus
[psplatforms@ip-10-244-145-54 ~]$ vault --versionVault v1.17.3 (c91c85442144e1228c02123fc4b19337f7d52700), built 2024-08-06T14:28:45Z[psplatforms@ip-10-244-145-54 ~]$

But still want to know how to fix this issue. Thank you.

biazmoreira · 2024-09-05T14:31:49Z

The dependency was updated yesterday and will probably make it to the next release.

joelofftsuny · 2024-09-20T11:13:19Z

The dependency was updated yesterday and will probably make it to the next release.

What version number will include this fix?

brinnjoyce changed the title ~~Running vault v1.14.1 as root launches a forked dbus-daemon~~ Running vault v1.14.1 without DBUS_SESSION_BUS_ADDRESS set and no /run/user/<uid>/bus launches a persistent dbus-daemon Aug 25, 2023

biazmoreira added core Issues and Pull-Requests specific to Vault Core core/dev labels Aug 29, 2023

biazmoreira mentioned this issue Aug 29, 2023

Zombie dbus processes generated by github.com/99designs/keyring snowflakedb/gosnowflake#901

Closed

biazmoreira closed this as completed Aug 29, 2023

sfc-gh-dszmolka mentioned this issue Aug 29, 2023

Zombie dbus processes generated by github.com/99designs/keyring snowflakedb/gosnowflake#773

Closed

Soromeister reopened this Sep 6, 2023

Soromeister closed this as completed Sep 11, 2023

DrDaveD mentioned this issue Jul 23, 2024

SNOW-1552759: Super annoying warning message from gosnowflake every invocation of Hashicorp vault cli under sudo snowflakedb/gosnowflake#1182

Closed

biazmoreira reopened this Sep 5, 2024

biazmoreira closed this as completed Sep 5, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Running vault v1.14.1 without DBUS_SESSION_BUS_ADDRESS set and no /run/user/<uid>/bus launches a persistent dbus-daemon #22560

Running vault v1.14.1 without DBUS_SESSION_BUS_ADDRESS set and no /run/user/<uid>/bus launches a persistent dbus-daemon #22560

brinnjoyce commented Aug 25, 2023 •

edited

Loading

biazmoreira commented Aug 29, 2023

danielhass commented Sep 1, 2023

jkhuang-42 commented Sep 25, 2023

Soromeister commented Oct 9, 2023

aswen commented Jun 13, 2024 •

edited

Loading

PJ1288 commented Jun 13, 2024

yaboeing commented Jun 27, 2024

MMquant commented Jul 9, 2024

sfc-gh-dszmolka commented Jul 17, 2024 •

edited

Loading

thedoc31 commented Aug 26, 2024

juparker37 commented Aug 28, 2024 •

edited

Loading

fen9li commented Sep 1, 2024 •

edited

Loading

biazmoreira commented Sep 5, 2024

joelofftsuny commented Sep 20, 2024

Running vault v1.14.1 without DBUS_SESSION_BUS_ADDRESS set and no /run/user/<uid>/bus launches a persistent dbus-daemon #22560

Running vault v1.14.1 without DBUS_SESSION_BUS_ADDRESS set and no /run/user/<uid>/bus launches a persistent dbus-daemon #22560

Comments

brinnjoyce commented Aug 25, 2023 • edited Loading

biazmoreira commented Aug 29, 2023

danielhass commented Sep 1, 2023

jkhuang-42 commented Sep 25, 2023

Soromeister commented Oct 9, 2023

aswen commented Jun 13, 2024 • edited Loading

PJ1288 commented Jun 13, 2024

yaboeing commented Jun 27, 2024

MMquant commented Jul 9, 2024

sfc-gh-dszmolka commented Jul 17, 2024 • edited Loading

thedoc31 commented Aug 26, 2024

juparker37 commented Aug 28, 2024 • edited Loading

fen9li commented Sep 1, 2024 • edited Loading

biazmoreira commented Sep 5, 2024

joelofftsuny commented Sep 20, 2024

brinnjoyce commented Aug 25, 2023 •

edited

Loading

aswen commented Jun 13, 2024 •

edited

Loading

sfc-gh-dszmolka commented Jul 17, 2024 •

edited

Loading

juparker37 commented Aug 28, 2024 •

edited

Loading

fen9li commented Sep 1, 2024 •

edited

Loading