Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Memory leak? #42

Closed
alexec opened this issue Oct 10, 2023 · 3 comments
Closed

Memory leak? #42

alexec opened this issue Oct 10, 2023 · 3 comments
Labels
bug Something isn't working

Comments

@alexec
Copy link
Collaborator

alexec commented Oct 10, 2023 

All I know is that when I swapped it out, my fans made a lot less noise and my memory usage was considerably lower 😅 .
I did not trace it carefully, but it seemed like there might be a memory leak in the auto-restart feature or something as I noticed the memory just kept increasing when I made changes to the code and not going back down. It could be a memory leak in one of the tasks themselves too though; again, I did not trace it, that was merely an observation and not one of the initial motivators.

There aren't too many places there could be a memory leak. Perhaps it is in the watch?
@alexec alexec added the bug Something isn't working label Oct 10, 2023
@agilgur5
Copy link

For posterity, from this comment: argoproj/argo-workflows#11928 (comment). It was an incidental finding here: argoproj/argo-workflows#11928 (comment).

Perhaps it is in the watch?

In JS-land and Docker/VM-land in years past, watchers were definitely a common area of inefficiencies (partially due to implementation differences across OSes and FSes etc), so I wouldn't be surprised by that, especially as it seems to get worse with code changes.

This was referenced Oct 10, 2023
Draft
Open
@agilgur5
Copy link

agilgur5 commented Oct 10, 2023 •
edited
Loading

So I did some tracing on this incidental finding and it seems like this is more likely a memory leak with Webpack (which would be less surprising too). The only strange part is that it seems to behave slightly differently based on its parent process for some reason 🤔

From the Workflows devcontainer:

ps w/ kit parent of webpack 
vscode ➜ ~/go/src/github.com/argoproj/argo-workflows (fix-build-argo-loglevel) $ ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0   1048     4 ?        Ss   Sep30   2:52 /sbin/docker-init -- /bin/sh -c echo Container started trap "exit 0" 15 /usr/local/share/docker-init.sh exec "$@" while sleep 1 & wait $!; do
root         7  0.1  0.0   2888  1696 ?        S    Sep30  23:17 /bin/sh -c echo Container started trap "exit 0" 15 /usr/local/share/docker-init.sh exec "$@" while sleep 1 & wait $!; do :; done -
root        26  0.2  0.6 1832160 51580 ?       Sl   Sep30  33:24 dockerd
root        65  0.3  0.3 1579052 26536 ?       Ssl  Sep30  49:21 containerd --config /var/run/docker/containerd/containerd.toml --log-level info
root       440  0.4  0.0 1304820 3288 ?        Sl   Sep30  67:45 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 41251 -container-ip 172.20.0.3 -container-port 6443
root       457  0.0  0.0 720500  4508 ?        Sl   Sep30   9:21 /usr/bin/containerd-shim-runc-v2 -namespace moby -id 964c36437af94d0a32d69b266ba3512e1375eaeb01b5ed15f894e45f75653f74 -address /var/run/docke
root       478  0.0  0.0   1048     4 ?        Ss   Sep30   2:44 /sbin/docker-init -- /bin/k3d-entrypoint.sh server --tls-san 0.0.0.0 --tls-san k3d-k3s-default-serverlb
root       491  0.0  0.0 720756  4736 ?        Sl   Sep30   6:44 /usr/bin/containerd-shim-runc-v2 -namespace moby -id 84ad1e920ed6f539994a353bda5062dd58601cc1e16caeda7dd69ad1dfa96a2c -address /var/run/docke
root       517  0.0  0.0   1048     4 ?        Ss   Sep30   2:47 /sbin/docker-init -- /bin/sh -c nginx-proxy
root       600  0.0  0.0    996   564 ?        S    Sep30   0:00 /bin/sh /bin/k3d-entrypoint.sh server --tls-san 0.0.0.0 --tls-san k3d-k3s-default-serverlb
root       619 45.8  6.9 1362748 567768 ?      Sl   Sep30 6946:19 /bin/k3s server
root       627  0.0  0.0   1596   832 ?        S    Sep30   0:00 /bin/sh /usr/bin/nginx-proxy
vscode     750  0.0  0.0   2888   932 ?        Ss   Sep30   0:00 /bin/sh
vscode     904  0.0  0.0   8448  5496 pts/0    Ss   Sep30   0:00 /bin/bash
root       949  8.1  1.7 1346696 145968 ?      Sl   Sep30 1236:01 containerd 
root      1940  0.3  0.1 722576 13424 ?        Sl   Sep30  58:08 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 4d90de7dbd9c13c9b24060555fec10008c9d9a245981f73589646b00f7af2173 -address /run/k3s/contain
root      2013  0.7  0.1 722512 13180 ?        Sl   Sep30 109:59 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 34337ab30bb28be3436243b09b17bdc4a99b361511c8daa372bf276ec8a29432 -address /run/k3s/contain
root      2018  0.3  0.1 722512 12728 ?        Sl   Sep30  57:30 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 9c3e3b24fe347f9d0e9386f45144b3e05c87272ed95aedb0acf31deab7e5f509 -address /run/k3s/contain
65535     2134  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
65535     2135  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
65535     2222  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
root      2378  0.3  0.1 722512 11860 ?        Sl   Sep30  57:13 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 19e78587f14b5b55c4613a6143723eea9610967a84fb243d63a6244041a404a4 -address /run/k3s/contain
root      2416  0.3  0.1 722512 12164 ?        Sl   Sep30  58:14 /bin/containerd-shim-runc-v2 -namespace k8s.io -id f44026865b87983c7032393673434291319544481c58c2e5a4e795987cff5507 -address /run/k3s/contain
root      2421  0.3  0.1 722512 12552 ?        Sl   Sep30  56:37 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 096e1eea26e5b87aa527a6c79c58828d657a08665eb6c001e6219382e84685aa -address /run/k3s/contain
root      2429  1.1  1.6 952512 137688 ?       Ssl  Sep30 168:29 minio server --console-address :9001 /data
65535     2448  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
root      2460  0.3  0.1 722256 13092 ?        Sl   Sep30  57:22 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 7a49faedb9078f91d93cf60e20d881b36e5a06f66afd80a52fb842eb7b925eeb -address /run/k3s/contain
65535     2523  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
65535     2529  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
65535     2556  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
root      2565  0.0  0.0   1616   920 ?        Ss   Sep30   0:00 /bin/sh /usr/bin/entry
root      2625  0.0  0.1 721768 13284 ?        Sl   Sep30   0:05 confd -watch -backend file -file /etc/confd/values.yaml -log-level debug
root      2626  0.0  0.0   5860  3852 ?        S    Sep30   0:00 nginx: master process nginx -g daemon off;
101       2663  0.2  0.0   6512  2004 ?        S    Sep30  37:19 nginx: worker process
101       2667  0.0  0.0   6316  1452 ?        S    Sep30   0:00 nginx: worker process
101       2668  0.0  0.0   6316  1452 ?        S    Sep30   0:00 nginx: worker process
101       2670  0.0  0.0   6316  1452 ?        S    Sep30   0:00 nginx: worker process
101       2671  0.0  0.0   6316  1452 ?        S    Sep30   0:00 nginx: worker process
101       2672  0.0  0.0   6316  1452 ?        S    Sep30   0:00 nginx: worker process
root      2728  1.5  0.4 765168 32880 ?        Ssl  Sep30 240:30 /coredns -conf /etc/coredns/Corefile
vscode    2741  5.2  0.5 760052 42244 ?        Ssl  Sep30 801:16 /metrics-server --cert-dir=/tmp --secure-port=10250 --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --kubelet-use-node-statu
65532     2756  0.3  0.9 840236 80420 ?        Ssl  Sep30  47:38 traefik traefik --global.checknewversion --global.sendanonymoususage --entrypoints.metrics.address=:9100/tcp --entrypoints.traefik.address=:9
root      2758  0.0  0.0   1616   968 ?        Ss   Sep30   0:00 /bin/sh /usr/bin/entry
root      2786  0.3  0.2 734164 21912 ?        Ssl  Sep30  50:38 local-path-provisioner start --config /etc/config/config.json
root      2856  0.0  0.3  54420 24468 ?        Ss   Sep30   8:09 /usr/bin/python3 /usr/local/bin/gunicorn -b 0.0.0.0:80 httpbin:app -k gevent
root      2967  0.3  0.4  77060 35552 ?        S    Sep30  55:12 /usr/bin/python3 /usr/local/bin/gunicorn -b 0.0.0.0:80 httpbin:app -k gevent
vscode    5845  0.0  0.0   8448  7508 pts/1    Ss+  Oct03   0:00 /bin/bash
vscode    7681  0.0  0.0   8448  7576 pts/2    Ss+  Oct03   0:00 /bin/bash
vscode   69657  0.0  0.0   3568  2396 pts/0    S+   15:51   0:00 make start UI=true
vscode   70025  1.7  0.2 721556 19416 pts/0    Sl+  15:51   0:15 kit controller server ui
vscode   71000  0.1  0.5 761368 46360 pts/0    Sl   15:52   0:01 kubectl -n argo port-forward svc/minio 9000:9000
vscode   71013  0.0  0.5 761624 43188 pts/0    Sl   15:52   0:00 kubectl -n argo port-forward svc/minio 9001:9001
vscode   71024  0.9  0.6 1284632 53704 pts/0   Sl   15:52   0:08 ./dist/workflow-controller
vscode   71025  0.1  0.7 1298676 64112 pts/0   Sl   15:52   0:01 ./dist/argo server
vscode   71049  0.1  0.9 1246388 75936 pts/0   Sl   15:52   0:01 node /usr/share/yarn/bin/yarn.js start
vscode   71075  0.0  0.0   2896  1036 pts/0    S    15:52   0:00 /bin/sh -c NODE_OPTIONS='--no-experimental-fetch --openssl-legacy-provider' webpack-dev-server --config ./src/app/webpack.config.js
vscode   71076 24.5 25.5 55683216 2081284 pts/0 Sl  15:52   3:29 webpack
vscode   71445  0.0  0.0   2888   964 ?        Ss   15:55   0:00 /bin/sh
vscode   71598  0.0  0.0   8448  7660 pts/3    Ss   15:55   0:00 /bin/bash
root     73111  0.0  0.0   2788  1076 ?        S    16:06   0:00 sleep 1
vscode   73112  0.0  0.0   7060  1564 pts/3    R+   16:06   0:00 ps aux

Webpack will continuously go up in memory usage and rarely go down. Its CPU can be very spiky on a reload as well (jumping by 100% CPU sometimes). kit remains fairly stable throughout, though at ~1.7% CPU, it is one of the higher utilization processes.

When I run Webpack standalone, outside of kit, here's what it looks like:

ps with Webpack standalone, kit still running on server and controller 
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0   1048     4 ?        Ss   Sep30   2:52 /sbin/docker-init -- /bin/sh -c echo Container started trap "exit 0" 15 /usr/local/share/docker-init.sh exec "$@" while sleep 1 & wait $!; do
root         7  0.1  0.0   2888  1696 ?        S    Sep30  23:17 /bin/sh -c echo Container started trap "exit 0" 15 /usr/local/share/docker-init.sh exec "$@" while sleep 1 & wait $!; do :; done -
root        26  0.2  0.6 1832160 51580 ?       Sl   Sep30  33:25 dockerd
root        65  0.3  0.3 1579052 26536 ?       Ssl  Sep30  49:22 containerd --config /var/run/docker/containerd/containerd.toml --log-level info
root       440  0.4  0.0 1304820 3288 ?        Sl   Sep30  67:46 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 41251 -container-ip 172.20.0.3 -container-port 6443
root       457  0.0  0.0 720500  4476 ?        Sl   Sep30   9:21 /usr/bin/containerd-shim-runc-v2 -namespace moby -id 964c36437af94d0a32d69b266ba3512e1375eaeb01b5ed15f894e45f75653f74 -address /var/run/docke
root       478  0.0  0.0   1048     4 ?        Ss   Sep30   2:44 /sbin/docker-init -- /bin/k3d-entrypoint.sh server --tls-san 0.0.0.0 --tls-san k3d-k3s-default-serverlb
root       491  0.0  0.0 720756  4736 ?        Sl   Sep30   6:44 /usr/bin/containerd-shim-runc-v2 -namespace moby -id 84ad1e920ed6f539994a353bda5062dd58601cc1e16caeda7dd69ad1dfa96a2c -address /var/run/docke
root       517  0.0  0.0   1048     4 ?        Ss   Sep30   2:47 /sbin/docker-init -- /bin/sh -c nginx-proxy
root       600  0.0  0.0    996   564 ?        S    Sep30   0:00 /bin/sh /bin/k3d-entrypoint.sh server --tls-san 0.0.0.0 --tls-san k3d-k3s-default-serverlb
root       619 45.7  6.9 1362184 567208 ?      Sl   Sep30 6948:55 /bin/k3s server
root       627  0.0  0.0   1596   832 ?        S    Sep30   0:00 /bin/sh /usr/bin/nginx-proxy
vscode     750  0.0  0.0   2888   932 ?        Ss   Sep30   0:00 /bin/sh
vscode     904  0.0  0.0   8448  5508 pts/0    Ss   Sep30   0:00 /bin/bash
root       949  8.1  1.8 1346696 146740 ?      Sl   Sep30 1236:19 containerd 
root      1940  0.3  0.1 722576 13672 ?        Sl   Sep30  58:09 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 4d90de7dbd9c13c9b24060555fec10008c9d9a245981f73589646b00f7af2173 -address /run/k3s/contain
root      2013  0.7  0.1 722512 13688 ?        Sl   Sep30 110:01 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 34337ab30bb28be3436243b09b17bdc4a99b361511c8daa372bf276ec8a29432 -address /run/k3s/contain
root      2018  0.3  0.1 722512 12852 ?        Sl   Sep30  57:30 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 9c3e3b24fe347f9d0e9386f45144b3e05c87272ed95aedb0acf31deab7e5f509 -address /run/k3s/contain
65535     2134  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
65535     2135  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
65535     2222  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
root      2378  0.3  0.1 722512 12348 ?        Sl   Sep30  57:14 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 19e78587f14b5b55c4613a6143723eea9610967a84fb243d63a6244041a404a4 -address /run/k3s/contain
root      2416  0.3  0.1 722512 12528 ?        Sl   Sep30  58:15 /bin/containerd-shim-runc-v2 -namespace k8s.io -id f44026865b87983c7032393673434291319544481c58c2e5a4e795987cff5507 -address /run/k3s/contain
root      2421  0.3  0.1 722512 12680 ?        Sl   Sep30  56:38 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 096e1eea26e5b87aa527a6c79c58828d657a08665eb6c001e6219382e84685aa -address /run/k3s/contain
root      2429  1.1  1.6 952512 138020 ?       Ssl  Sep30 168:32 minio server --console-address :9001 /data
65535     2448  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
root      2460  0.3  0.1 722256 13656 ?        Sl   Sep30  57:22 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 7a49faedb9078f91d93cf60e20d881b36e5a06f66afd80a52fb842eb7b925eeb -address /run/k3s/contain
65535     2523  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
65535     2529  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
65535     2556  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
root      2565  0.0  0.0   1616   920 ?        Ss   Sep30   0:00 /bin/sh /usr/bin/entry
root      2625  0.0  0.1 721768 13284 ?        Sl   Sep30   0:05 confd -watch -backend file -file /etc/confd/values.yaml -log-level debug
root      2626  0.0  0.0   5860  3852 ?        S    Sep30   0:00 nginx: master process nginx -g daemon off;
101       2663  0.2  0.0   6512  2012 ?        S    Sep30  37:20 nginx: worker process
101       2667  0.0  0.0   6316  1452 ?        S    Sep30   0:00 nginx: worker process
101       2668  0.0  0.0   6316  1452 ?        S    Sep30   0:00 nginx: worker process
101       2670  0.0  0.0   6316  1452 ?        S    Sep30   0:00 nginx: worker process
101       2671  0.0  0.0   6316  1452 ?        S    Sep30   0:00 nginx: worker process
101       2672  0.0  0.0   6316  1452 ?        S    Sep30   0:00 nginx: worker process
root      2728  1.5  0.4 765168 32756 ?        Ssl  Sep30 240:36 /coredns -conf /etc/coredns/Corefile
vscode    2741  5.2  0.5 760052 42180 ?        Ssl  Sep30 801:29 /metrics-server --cert-dir=/tmp --secure-port=10250 --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --kubelet-use-node-statu
65532     2756  0.3  0.9 840236 80420 ?        Ssl  Sep30  47:39 traefik traefik --global.checknewversion --global.sendanonymoususage --entrypoints.metrics.address=:9100/tcp --entrypoints.traefik.address=:9
root      2758  0.0  0.0   1616   968 ?        Ss   Sep30   0:00 /bin/sh /usr/bin/entry
root      2786  0.3  0.2 734164 21912 ?        Ssl  Sep30  50:40 local-path-provisioner start --config /etc/config/config.json
root      2856  0.0  0.3  54420 24468 ?        Ss   Sep30   8:09 /usr/bin/python3 /usr/local/bin/gunicorn -b 0.0.0.0:80 httpbin:app -k gevent
root      2967  0.3  0.4  77060 35552 ?        S    Sep30  55:14 /usr/bin/python3 /usr/local/bin/gunicorn -b 0.0.0.0:80 httpbin:app -k gevent
vscode    5845  0.0  0.0   8448  7508 pts/1    Ss+  Oct03   0:00 /bin/bash
vscode    7681  0.0  0.0   8448  7576 pts/2    Ss+  Oct03   0:00 /bin/bash
vscode   71445  0.0  0.0   2888   964 ?        Ss   15:55   0:00 /bin/sh
vscode   71598  0.0  0.0   8448  7664 pts/3    Ss   15:55   0:00 /bin/bash
vscode   73609  0.6  0.1 721300 15524 pts/0    Sl+  16:09   0:07 kit controller server
vscode   74531  0.1  0.5 761624 47260 pts/0    Sl   16:10   0:01 kubectl -n argo port-forward svc/minio 9000:9000
vscode   74545  0.0  0.5 761368 41488 pts/0    Sl   16:10   0:00 kubectl -n argo port-forward svc/minio 9001:9001
vscode   74571  0.0  0.7 1298932 64204 pts/0   Sl   16:10   0:00 ./dist/argo server
vscode   74572  0.8  0.6 1284632 51296 pts/0   Sl   16:10   0:10 ./dist/workflow-controller
vscode   74626  0.0  0.9 1246896 76628 pts/3   Sl+  16:10   0:00 node /usr/share/yarn/bin/yarn.js start
vscode   74651  0.0  0.0   2896   980 pts/3    S+   16:10   0:00 /bin/sh -c NODE_OPTIONS='--no-experimental-fetch --openssl-legacy-provider' webpack-dev-server --config ./src/app/webpack.config.js
vscode   74652 10.9 27.4 55837072 2233656 pts/3 Sl+ 16:10   2:03 webpack
vscode   74769  0.0  0.0   2888   952 ?        Ss   16:11   0:00 /bin/sh
vscode   74922  0.0  0.0   8448  7664 pts/4    Ss   16:11   0:00 /bin/bash
root     77209  0.0  0.0   2788  1048 ?        S    16:29   0:00 sleep 1
vscode   77210  0.0  0.0   7060  1552 pts/4    R+   16:29   0:00 ps aux

It seemed like the floor of the CPU is lower when it is standalone. kit is also using less than half of its memory and a third of CPU when not the parent of Webpack. (or not the parent of yarn start, which is the parent of Webpack, to be specific)

So it seems like the memory leak is with Webpack, but kit perhaps exacerbates it somehow by also using more resources? and perhaps having some effect on reaping? Not really sure how this interaction works, it's quite strange to see different behavior due to a parent process; I can't make heads or tails of it

@agilgur5
Copy link

agilgur5 commented Oct 10, 2023 •
edited
Loading

Left kit on for longer as a parent and the floor did drop relatively equivalently. The only observation that adds up for why my fans are running louder / more frequently when using kit is b/c kit's own CPU when it is the parent of Webpack can be substantially higher, so it has some (fairly small) bit of duplicative effect on total CPU usage.

kit's CPU did drop quite substantially throughout the run as well. It started at 5.0%, then dropped to 2.0%, then 1.6%, and eventually to 0.7%.

initial ps 
vscode ➜ ~/go/src/github.com/argoproj/argo-workflows (fix-build-argo-loglevel) $ ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0   1048     4 ?        Ss   Sep30   2:52 /sbin/docker-init -- /bin/sh -c echo Container started trap "exit 0" 15 /usr/local/share/docker-init.sh exec "$@" while sleep 1 & wait $!; do
root         7  0.1  0.0   2888  1696 ?        S    Sep30  23:17 /bin/sh -c echo Container started trap "exit 0" 15 /usr/local/share/docker-init.sh exec "$@" while sleep 1 & wait $!; do :; done -
root        26  0.2  0.6 1832160 51580 ?       Sl   Sep30  33:25 dockerd
root        65  0.3  0.3 1579052 26536 ?       Ssl  Sep30  49:22 containerd --config /var/run/docker/containerd/containerd.toml --log-level info
root       440  0.4  0.0 1304820 3288 ?        Sl   Sep30  67:46 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 41251 -container-ip 172.20.0.3 -container-port 6443
root       457  0.0  0.0 720500  4476 ?        Sl   Sep30   9:22 /usr/bin/containerd-shim-runc-v2 -namespace moby -id 964c36437af94d0a32d69b266ba3512e1375eaeb01b5ed15f894e45f75653f74 -address /var/run/docke
root       478  0.0  0.0   1048     4 ?        Ss   Sep30   2:44 /sbin/docker-init -- /bin/k3d-entrypoint.sh server --tls-san 0.0.0.0 --tls-san k3d-k3s-default-serverlb
root       491  0.0  0.0 720756  4736 ?        Sl   Sep30   6:44 /usr/bin/containerd-shim-runc-v2 -namespace moby -id 84ad1e920ed6f539994a353bda5062dd58601cc1e16caeda7dd69ad1dfa96a2c -address /var/run/docke
root       517  0.0  0.0   1048     4 ?        Ss   Sep30   2:47 /sbin/docker-init -- /bin/sh -c nginx-proxy
root       600  0.0  0.0    996   564 ?        S    Sep30   0:00 /bin/sh /bin/k3d-entrypoint.sh server --tls-san 0.0.0.0 --tls-san k3d-k3s-default-serverlb
root       619 45.7  6.9 1362116 567128 ?      Sl   Sep30 6949:21 /bin/k3s server
root       627  0.0  0.0   1596   832 ?        S    Sep30   0:00 /bin/sh /usr/bin/nginx-proxy
vscode     750  0.0  0.0   2888   932 ?        Ss   Sep30   0:00 /bin/sh
vscode     904  0.0  0.0   8448  5508 pts/0    Ss   Sep30   0:00 /bin/bash
root       949  8.1  1.7 1346696 142792 ?      Sl   Sep30 1236:22 containerd 
root      1940  0.3  0.1 722576 13432 ?        Sl   Sep30  58:09 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 4d90de7dbd9c13c9b24060555fec10008c9d9a245981f73589646b00f7af2173 -address /run/k3s/contain
root      2013  0.7  0.1 722512 13572 ?        Sl   Sep30 110:01 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 34337ab30bb28be3436243b09b17bdc4a99b361511c8daa372bf276ec8a29432 -address /run/k3s/contain
root      2018  0.3  0.1 722512 13100 ?        Sl   Sep30  57:30 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 9c3e3b24fe347f9d0e9386f45144b3e05c87272ed95aedb0acf31deab7e5f509 -address /run/k3s/contain
65535     2134  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
65535     2135  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
65535     2222  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
root      2378  0.3  0.1 722512 12148 ?        Sl   Sep30  57:14 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 19e78587f14b5b55c4613a6143723eea9610967a84fb243d63a6244041a404a4 -address /run/k3s/contain
root      2416  0.3  0.1 722512 12104 ?        Sl   Sep30  58:15 /bin/containerd-shim-runc-v2 -namespace k8s.io -id f44026865b87983c7032393673434291319544481c58c2e5a4e795987cff5507 -address /run/k3s/contain
root      2421  0.3  0.1 722512 12908 ?        Sl   Sep30  56:38 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 096e1eea26e5b87aa527a6c79c58828d657a08665eb6c001e6219382e84685aa -address /run/k3s/contain
root      2429  1.1  1.6 952512 138020 ?       Ssl  Sep30 168:32 minio server --console-address :9001 /data
65535     2448  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
root      2460  0.3  0.1 722256 13168 ?        Sl   Sep30  57:23 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 7a49faedb9078f91d93cf60e20d881b36e5a06f66afd80a52fb842eb7b925eeb -address /run/k3s/contain
65535     2523  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
65535     2529  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
65535     2556  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
root      2565  0.0  0.0   1616   920 ?        Ss   Sep30   0:00 /bin/sh /usr/bin/entry
root      2625  0.0  0.1 721768 13284 ?        Sl   Sep30   0:05 confd -watch -backend file -file /etc/confd/values.yaml -log-level debug
root      2626  0.0  0.0   5860  3852 ?        S    Sep30   0:00 nginx: master process nginx -g daemon off;
101       2663  0.2  0.0   6580  1912 ?        S    Sep30  37:20 nginx: worker process
101       2667  0.0  0.0   6316  1452 ?        S    Sep30   0:00 nginx: worker process
101       2668  0.0  0.0   6316  1452 ?        S    Sep30   0:00 nginx: worker process
101       2670  0.0  0.0   6316  1452 ?        S    Sep30   0:00 nginx: worker process
101       2671  0.0  0.0   6316  1452 ?        S    Sep30   0:00 nginx: worker process
101       2672  0.0  0.0   6316  1452 ?        S    Sep30   0:00 nginx: worker process
root      2728  1.5  0.4 765168 32756 ?        Ssl  Sep30 240:37 /coredns -conf /etc/coredns/Corefile
vscode    2741  5.2  0.5 760052 42472 ?        Ssl  Sep30 801:31 /metrics-server --cert-dir=/tmp --secure-port=10250 --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --kubelet-use-node-statu
65532     2756  0.3  0.9 840236 80420 ?        Ssl  Sep30  47:39 traefik traefik --global.checknewversion --global.sendanonymoususage --entrypoints.metrics.address=:9100/tcp --entrypoints.traefik.address=:9
root      2758  0.0  0.0   1616   968 ?        Ss   Sep30   0:00 /bin/sh /usr/bin/entry
root      2786  0.3  0.2 734164 21912 ?        Ssl  Sep30  50:40 local-path-provisioner start --config /etc/config/config.json
root      2856  0.0  0.3  54420 24468 ?        Ss   Sep30   8:09 /usr/bin/python3 /usr/local/bin/gunicorn -b 0.0.0.0:80 httpbin:app -k gevent
root      2967  0.3  0.4  77060 35552 ?        S    Sep30  55:14 /usr/bin/python3 /usr/local/bin/gunicorn -b 0.0.0.0:80 httpbin:app -k gevent
vscode    5845  0.0  0.0   8448  7508 pts/1    Ss+  Oct03   0:00 /bin/bash
vscode    7681  0.0  0.0   8448  7576 pts/2    Ss+  Oct03   0:00 /bin/bash
vscode   71598  0.0  0.0   8448  7664 pts/3    Ss+  15:55   0:00 /bin/bash
vscode   74769  0.0  0.0   2888   952 ?        Ss   16:11   0:00 /bin/sh
vscode   74922  0.0  0.0   8448  7668 pts/4    Ss   16:11   0:00 /bin/bash
vscode   77663  5.0  0.2 721556 18672 pts/0    Sl+  16:32   0:03 kit controller server ui
vscode   78628  0.3  0.5 761624 44180 pts/0    Sl   16:32   0:00 kubectl -n argo port-forward svc/minio 9000:9000
vscode   78642  0.2  0.5 761624 42884 pts/0    Sl   16:32   0:00 kubectl -n argo port-forward svc/minio 9001:9001
vscode   78656  1.1  0.6 1284376 52816 pts/0   Sl   16:32   0:00 ./dist/workflow-controller
vscode   78657  0.2  0.7 1298676 57328 pts/0   Sl   16:32   0:00 ./dist/argo server
vscode   78687  0.8  0.9 1246128 76008 pts/0   Sl   16:32   0:00 node /usr/share/yarn/bin/yarn.js start
vscode   78712  0.0  0.0   2896   956 pts/0    S    16:32   0:00 /bin/sh -c NODE_OPTIONS='--no-experimental-fetch --openssl-legacy-provider' webpack-dev-server --config ./src/app/webpack.config.js
vscode   78713  144 14.6 54794248 1192724 pts/0 Sl  16:32   0:59 webpack
root     78815  0.0  0.0   2788  1020 ?        S    16:33   0:00 sleep 1
vscode   78816  0.0  0.0   7060  1556 pts/4    R+   16:33   0:00 ps aux
ps shortly after init 
vscode ➜ ~/go/src/github.com/argoproj/argo-workflows (fix-build-argo-loglevel) $ ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0   1048     4 ?        Ss   Sep30   2:52 /sbin/docker-init -- /bin/sh -c echo Container started trap "exit 0" 15 /usr/local/share/docker-init.sh exec "$@" while sleep 1 & wait $!; do
root         7  0.1  0.0   2888  1696 ?        S    Sep30  23:17 /bin/sh -c echo Container started trap "exit 0" 15 /usr/local/share/docker-init.sh exec "$@" while sleep 1 & wait $!; do :; done -
root        26  0.2  0.6 1832160 51580 ?       Sl   Sep30  33:25 dockerd
root        65  0.3  0.3 1579052 26536 ?       Ssl  Sep30  49:22 containerd --config /var/run/docker/containerd/containerd.toml --log-level info
root       440  0.4  0.0 1304820 3288 ?        Sl   Sep30  67:47 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 41251 -container-ip 172.20.0.3 -container-port 6443
root       457  0.0  0.0 720500  4444 ?        Sl   Sep30   9:22 /usr/bin/containerd-shim-runc-v2 -namespace moby -id 964c36437af94d0a32d69b266ba3512e1375eaeb01b5ed15f894e45f75653f74 -address /var/run/docke
root       478  0.0  0.0   1048     4 ?        Ss   Sep30   2:44 /sbin/docker-init -- /bin/k3d-entrypoint.sh server --tls-san 0.0.0.0 --tls-san k3d-k3s-default-serverlb
root       491  0.0  0.0 720756  4672 ?        Sl   Sep30   6:44 /usr/bin/containerd-shim-runc-v2 -namespace moby -id 84ad1e920ed6f539994a353bda5062dd58601cc1e16caeda7dd69ad1dfa96a2c -address /var/run/docke
root       517  0.0  0.0   1048     4 ?        Ss   Sep30   2:47 /sbin/docker-init -- /bin/sh -c nginx-proxy
root       600  0.0  0.0    996   564 ?        S    Sep30   0:00 /bin/sh /bin/k3d-entrypoint.sh server --tls-san 0.0.0.0 --tls-san k3d-k3s-default-serverlb
root       619 45.7  6.9 1364004 569128 ?      Sl   Sep30 6949:32 /bin/k3s server
root       627  0.0  0.0   1596   832 ?        S    Sep30   0:00 /bin/sh /usr/bin/nginx-proxy
vscode     750  0.0  0.0   2888   932 ?        Ss   Sep30   0:00 /bin/sh
vscode     904  0.0  0.0   8448  5508 pts/0    Ss   Sep30   0:00 /bin/bash
root       949  8.1  1.7 1346696 143580 ?      Sl   Sep30 1236:23 containerd 
root      1940  0.3  0.1 722576 13624 ?        Sl   Sep30  58:09 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 4d90de7dbd9c13c9b24060555fec10008c9d9a245981f73589646b00f7af2173 -address /run/k3s/contain
root      2013  0.7  0.1 722512 13444 ?        Sl   Sep30 110:01 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 34337ab30bb28be3436243b09b17bdc4a99b361511c8daa372bf276ec8a29432 -address /run/k3s/contain
root      2018  0.3  0.1 722512 12800 ?        Sl   Sep30  57:30 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 9c3e3b24fe347f9d0e9386f45144b3e05c87272ed95aedb0acf31deab7e5f509 -address /run/k3s/contain
65535     2134  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
65535     2135  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
65535     2222  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
root      2378  0.3  0.1 722512 12312 ?        Sl   Sep30  57:14 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 19e78587f14b5b55c4613a6143723eea9610967a84fb243d63a6244041a404a4 -address /run/k3s/contain
root      2416  0.3  0.1 722512 11884 ?        Sl   Sep30  58:15 /bin/containerd-shim-runc-v2 -namespace k8s.io -id f44026865b87983c7032393673434291319544481c58c2e5a4e795987cff5507 -address /run/k3s/contain
root      2421  0.3  0.1 722512 12528 ?        Sl   Sep30  56:38 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 096e1eea26e5b87aa527a6c79c58828d657a08665eb6c001e6219382e84685aa -address /run/k3s/contain
root      2429  1.1  1.6 952512 138020 ?       Ssl  Sep30 168:33 minio server --console-address :9001 /data
65535     2448  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
root      2460  0.3  0.1 722256 13144 ?        Sl   Sep30  57:23 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 7a49faedb9078f91d93cf60e20d881b36e5a06f66afd80a52fb842eb7b925eeb -address /run/k3s/contain
65535     2523  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
65535     2529  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
65535     2556  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
root      2565  0.0  0.0   1616   920 ?        Ss   Sep30   0:00 /bin/sh /usr/bin/entry
root      2625  0.0  0.1 721768 13284 ?        Sl   Sep30   0:05 confd -watch -backend file -file /etc/confd/values.yaml -log-level debug
root      2626  0.0  0.0   5860  3852 ?        S    Sep30   0:00 nginx: master process nginx -g daemon off;
101       2663  0.2  0.0   6580  1912 ?        S    Sep30  37:20 nginx: worker process
101       2667  0.0  0.0   6316  1452 ?        S    Sep30   0:00 nginx: worker process
101       2668  0.0  0.0   6316  1452 ?        S    Sep30   0:00 nginx: worker process
101       2670  0.0  0.0   6316  1452 ?        S    Sep30   0:00 nginx: worker process
101       2671  0.0  0.0   6316  1452 ?        S    Sep30   0:00 nginx: worker process
101       2672  0.0  0.0   6316  1452 ?        S    Sep30   0:00 nginx: worker process
root      2728  1.5  0.4 765168 32756 ?        Ssl  Sep30 240:37 /coredns -conf /etc/coredns/Corefile
vscode    2741  5.2  0.5 760052 42132 ?        Ssl  Sep30 801:32 /metrics-server --cert-dir=/tmp --secure-port=10250 --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --kubelet-use-node-statu
65532     2756  0.3  0.9 840236 80420 ?        Ssl  Sep30  47:39 traefik traefik --global.checknewversion --global.sendanonymoususage --entrypoints.metrics.address=:9100/tcp --entrypoints.traefik.address=:9
root      2758  0.0  0.0   1616   968 ?        Ss   Sep30   0:00 /bin/sh /usr/bin/entry
root      2786  0.3  0.2 734164 21912 ?        Ssl  Sep30  50:40 local-path-provisioner start --config /etc/config/config.json
root      2856  0.0  0.3  54420 24468 ?        Ss   Sep30   8:09 /usr/bin/python3 /usr/local/bin/gunicorn -b 0.0.0.0:80 httpbin:app -k gevent
root      2967  0.3  0.4  77060 35552 ?        S    Sep30  55:14 /usr/bin/python3 /usr/local/bin/gunicorn -b 0.0.0.0:80 httpbin:app -k gevent
vscode    5845  0.0  0.0   8448  7508 pts/1    Ss+  Oct03   0:00 /bin/bash
vscode    7681  0.0  0.0   8448  7576 pts/2    Ss+  Oct03   0:00 /bin/bash
vscode   71598  0.0  0.0   8448  7664 pts/3    Ss+  15:55   0:00 /bin/bash
vscode   74769  0.0  0.0   2888   952 ?        Ss   16:11   0:00 /bin/sh
vscode   74922  0.0  0.0   8448  7668 pts/4    Ss   16:11   0:00 /bin/bash
vscode   77663  2.0  0.2 721556 19804 pts/0    Sl+  16:32   0:03 kit controller server ui
vscode   78628  0.2  0.5 761624 44440 pts/0    Sl   16:32   0:00 kubectl -n argo port-forward svc/minio 9000:9000
vscode   78642  0.1  0.5 761624 42152 pts/0    Sl   16:32   0:00 kubectl -n argo port-forward svc/minio 9001:9001
vscode   78656  0.8  0.6 1284376 52836 pts/0   Sl   16:32   0:01 ./dist/workflow-controller
vscode   78657  0.0  0.7 1298676 60472 pts/0   Sl   16:32   0:00 ./dist/argo server
vscode   78687  0.2  0.9 1246128 76008 pts/0   Sl   16:32   0:00 node /usr/share/yarn/bin/yarn.js start
vscode   78712  0.0  0.0   2896   956 pts/0    S    16:32   0:00 /bin/sh -c NODE_OPTIONS='--no-experimental-fetch --openssl-legacy-provider' webpack-dev-server --config ./src/app/webpack.config.js
vscode   78713 41.4 11.3 54556688 929044 pts/0 Sl   16:32   1:05 webpack
root     78987  0.0  0.0   2788  1048 ?        S    16:35   0:00 sleep 1
vscode   78988  0.0  0.0   7060  1544 pts/4    R+   16:35   0:00 ps aux
somewhere in the middle ish ps 
vscode ➜ ~/go/src/github.com/argoproj/argo-workflows (fix-build-argo-loglevel) $ ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0   1048     4 ?        Ss   Sep30   2:52 /sbin/docker-init -- /bin/sh -c echo Container started trap "exit 0" 15 /usr/local/share/docker-init.sh exec "$@" while sleep 1 & wait $!; do
root         7  0.1  0.0   2888  1696 ?        S    Sep30  23:17 /bin/sh -c echo Container started trap "exit 0" 15 /usr/local/share/docker-init.sh exec "$@" while sleep 1 & wait $!; do :; done -
root        26  0.2  0.6 1832160 51580 ?       Sl   Sep30  33:25 dockerd
root        65  0.3  0.3 1579052 26536 ?       Ssl  Sep30  49:22 containerd --config /var/run/docker/containerd/containerd.toml --log-level info
root       440  0.4  0.0 1304820 3288 ?        Sl   Sep30  67:47 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 41251 -container-ip 172.20.0.3 -container-port 6443
root       457  0.0  0.0 720500  4412 ?        Sl   Sep30   9:22 /usr/bin/containerd-shim-runc-v2 -namespace moby -id 964c36437af94d0a32d69b266ba3512e1375eaeb01b5ed15f894e45f75653f74 -address /var/run/docke
root       478  0.0  0.0   1048     4 ?        Ss   Sep30   2:44 /sbin/docker-init -- /bin/k3d-entrypoint.sh server --tls-san 0.0.0.0 --tls-san k3d-k3s-default-serverlb
root       491  0.0  0.0 720756  4576 ?        Sl   Sep30   6:44 /usr/bin/containerd-shim-runc-v2 -namespace moby -id 84ad1e920ed6f539994a353bda5062dd58601cc1e16caeda7dd69ad1dfa96a2c -address /var/run/docke
root       517  0.0  0.0   1048     4 ?        Ss   Sep30   2:47 /sbin/docker-init -- /bin/sh -c nginx-proxy
root       600  0.0  0.0    996   564 ?        S    Sep30   0:00 /bin/sh /bin/k3d-entrypoint.sh server --tls-san 0.0.0.0 --tls-san k3d-k3s-default-serverlb
root       619 45.7  6.9 1363236 568244 ?      Sl   Sep30 6949:49 /bin/k3s server
root       627  0.0  0.0   1596   832 ?        S    Sep30   0:00 /bin/sh /usr/bin/nginx-proxy
vscode     750  0.0  0.0   2888   932 ?        Ss   Sep30   0:00 /bin/sh
vscode     904  0.0  0.0   8448  5508 pts/0    Ss   Sep30   0:00 /bin/bash
root       949  8.1  1.7 1346696 143840 ?      Sl   Sep30 1236:25 containerd 
root      1940  0.3  0.1 722576 13636 ?        Sl   Sep30  58:09 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 4d90de7dbd9c13c9b24060555fec10008c9d9a245981f73589646b00f7af2173 -address /run/k3s/contain
root      2013  0.7  0.1 722512 13576 ?        Sl   Sep30 110:01 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 34337ab30bb28be3436243b09b17bdc4a99b361511c8daa372bf276ec8a29432 -address /run/k3s/contain
root      2018  0.3  0.1 722512 12744 ?        Sl   Sep30  57:31 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 9c3e3b24fe347f9d0e9386f45144b3e05c87272ed95aedb0acf31deab7e5f509 -address /run/k3s/contain
65535     2134  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
65535     2135  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
65535     2222  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
root      2378  0.3  0.1 722512 12128 ?        Sl   Sep30  57:14 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 19e78587f14b5b55c4613a6143723eea9610967a84fb243d63a6244041a404a4 -address /run/k3s/contain
root      2416  0.3  0.1 722512 12088 ?        Sl   Sep30  58:15 /bin/containerd-shim-runc-v2 -namespace k8s.io -id f44026865b87983c7032393673434291319544481c58c2e5a4e795987cff5507 -address /run/k3s/contain
root      2421  0.3  0.1 722512 12464 ?        Sl   Sep30  56:38 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 096e1eea26e5b87aa527a6c79c58828d657a08665eb6c001e6219382e84685aa -address /run/k3s/contain
root      2429  1.1  1.6 952512 138020 ?       Ssl  Sep30 168:33 minio server --console-address :9001 /data
65535     2448  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
root      2460  0.3  0.1 722256 12788 ?        Sl   Sep30  57:23 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 7a49faedb9078f91d93cf60e20d881b36e5a06f66afd80a52fb842eb7b925eeb -address /run/k3s/contain
65535     2523  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
65535     2529  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
65535     2556  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
root      2565  0.0  0.0   1616   920 ?        Ss   Sep30   0:00 /bin/sh /usr/bin/entry
root      2625  0.0  0.1 721768 13284 ?        Sl   Sep30   0:05 confd -watch -backend file -file /etc/confd/values.yaml -log-level debug
root      2626  0.0  0.0   5860  3852 ?        S    Sep30   0:00 nginx: master process nginx -g daemon off;
101       2663  0.2  0.0   6580  1912 ?        S    Sep30  37:20 nginx: worker process
101       2667  0.0  0.0   6316  1452 ?        S    Sep30   0:00 nginx: worker process
101       2668  0.0  0.0   6316  1452 ?        S    Sep30   0:00 nginx: worker process
101       2670  0.0  0.0   6316  1452 ?        S    Sep30   0:00 nginx: worker process
101       2671  0.0  0.0   6316  1452 ?        S    Sep30   0:00 nginx: worker process
101       2672  0.0  0.0   6316  1452 ?        S    Sep30   0:00 nginx: worker process
root      2728  1.5  0.4 765168 32756 ?        Ssl  Sep30 240:37 /coredns -conf /etc/coredns/Corefile
vscode    2741  5.2  0.5 760052 42204 ?        Ssl  Sep30 801:33 /metrics-server --cert-dir=/tmp --secure-port=10250 --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --kubelet-use-node-statu
65532     2756  0.3  0.9 840236 80420 ?        Ssl  Sep30  47:39 traefik traefik --global.checknewversion --global.sendanonymoususage --entrypoints.metrics.address=:9100/tcp --entrypoints.traefik.address=:9
root      2758  0.0  0.0   1616   968 ?        Ss   Sep30   0:00 /bin/sh /usr/bin/entry
root      2786  0.3  0.2 734164 21912 ?        Ssl  Sep30  50:40 local-path-provisioner start --config /etc/config/config.json
root      2856  0.0  0.3  54420 24468 ?        Ss   Sep30   8:09 /usr/bin/python3 /usr/local/bin/gunicorn -b 0.0.0.0:80 httpbin:app -k gevent
root      2967  0.3  0.4  77060 35552 ?        S    Sep30  55:14 /usr/bin/python3 /usr/local/bin/gunicorn -b 0.0.0.0:80 httpbin:app -k gevent
vscode    5845  0.0  0.0   8448  7508 pts/1    Ss+  Oct03   0:00 /bin/bash
vscode    7681  0.0  0.0   8448  7576 pts/2    Ss+  Oct03   0:00 /bin/bash
vscode   71598  0.0  0.0   8448  7664 pts/3    Ss+  15:55   0:00 /bin/bash
vscode   74769  0.0  0.0   2888   952 ?        Ss   16:11   0:00 /bin/sh
vscode   74922  0.0  0.0   8448  7668 pts/4    Ss   16:11   0:00 /bin/bash
vscode   77663  1.2  0.2 721556 20324 pts/0    Sl+  16:32   0:04 kit controller server ui
vscode   78628  0.1  0.5 761624 46184 pts/0    Sl   16:32   0:00 kubectl -n argo port-forward svc/minio 9000:9000
vscode   78642  0.0  0.5 761624 42024 pts/0    Sl   16:32   0:00 kubectl -n argo port-forward svc/minio 9001:9001
vscode   78656  0.7  0.6 1284632 52836 pts/0   Sl   16:32   0:02 ./dist/workflow-controller
vscode   78657  0.0  0.7 1298676 61812 pts/0   Sl   16:32   0:00 ./dist/argo server
vscode   78687  0.1  0.9 1246128 76008 pts/0   Sl   16:32   0:00 node /usr/share/yarn/bin/yarn.js start
vscode   78712  0.0  0.0   2896   956 pts/0    S    16:32   0:00 /bin/sh -c NODE_OPTIONS='--no-experimental-fetch --openssl-legacy-provider' webpack-dev-server --config ./src/app/webpack.config.js
vscode   78713 17.7 11.4 54556688 929836 pts/0 Sl   16:32   1:05 webpack
root     79421  0.0  0.0   2788   996 ?        S    16:38   0:00 sleep 1
vscode   79422  0.0  0.0   7060  1584 pts/4    R+   16:38   0:00 ps aux
final ps 
vscode ➜ ~/go/src/github.com/argoproj/argo-workflows (fix-build-argo-loglevel) $ ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0   1048     4 ?        Ss   Sep30   2:52 /sbin/docker-init -- /bin/sh -c echo Container started trap "exit 0" 15 /usr/local/share/docker-init.sh exec "$@" while sleep 1 & wait $!; do
root         7  0.1  0.0   2888  1696 ?        S    Sep30  23:18 /bin/sh -c echo Container started trap "exit 0" 15 /usr/local/share/docker-init.sh exec "$@" while sleep 1 & wait $!; do :; done -
root        26  0.2  0.6 1832160 51580 ?       Sl   Sep30  33:26 dockerd
root        65  0.3  0.3 1579052 26652 ?       Ssl  Sep30  49:24 containerd --config /var/run/docker/containerd/containerd.toml --log-level info
root       440  0.4  0.0 1304820 3288 ?        Sl   Sep30  67:48 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 41251 -container-ip 172.20.0.3 -container-port 6443
root       457  0.0  0.0 720500  4348 ?        Sl   Sep30   9:22 /usr/bin/containerd-shim-runc-v2 -namespace moby -id 964c36437af94d0a32d69b266ba3512e1375eaeb01b5ed15f894e45f75653f74 -address /var/run/docke
root       478  0.0  0.0   1048     4 ?        Ss   Sep30   2:44 /sbin/docker-init -- /bin/k3d-entrypoint.sh server --tls-san 0.0.0.0 --tls-san k3d-k3s-default-serverlb
root       491  0.0  0.0 720756  4640 ?        Sl   Sep30   6:44 /usr/bin/containerd-shim-runc-v2 -namespace moby -id 84ad1e920ed6f539994a353bda5062dd58601cc1e16caeda7dd69ad1dfa96a2c -address /var/run/docke
root       517  0.0  0.0   1048     4 ?        Ss   Sep30   2:48 /sbin/docker-init -- /bin/sh -c nginx-proxy
root       600  0.0  0.0    996   564 ?        S    Sep30   0:00 /bin/sh /bin/k3d-entrypoint.sh server --tls-san 0.0.0.0 --tls-san k3d-k3s-default-serverlb
root       619 45.6  6.9 1364160 569176 ?      Sl   Sep30 6952:34 /bin/k3s server
root       627  0.0  0.0   1596   832 ?        S    Sep30   0:00 /bin/sh /usr/bin/nginx-proxy
vscode     750  0.0  0.0   2888   932 ?        Ss   Sep30   0:00 /bin/sh
vscode     904  0.0  0.0   8448  5508 pts/0    Ss   Sep30   0:00 /bin/bash
root       949  8.1  1.8 1346696 146772 ?      Sl   Sep30 1236:46 containerd 
root      1940  0.3  0.1 722576 13608 ?        Sl   Sep30  58:10 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 4d90de7dbd9c13c9b24060555fec10008c9d9a245981f73589646b00f7af2173 -address /run/k3s/contain
root      2013  0.7  0.1 722512 13244 ?        Sl   Sep30 110:03 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 34337ab30bb28be3436243b09b17bdc4a99b361511c8daa372bf276ec8a29432 -address /run/k3s/contain
root      2018  0.3  0.1 722512 12956 ?        Sl   Sep30  57:31 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 9c3e3b24fe347f9d0e9386f45144b3e05c87272ed95aedb0acf31deab7e5f509 -address /run/k3s/contain
65535     2134  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
65535     2135  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
65535     2222  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
root      2378  0.3  0.1 722512 11956 ?        Sl   Sep30  57:15 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 19e78587f14b5b55c4613a6143723eea9610967a84fb243d63a6244041a404a4 -address /run/k3s/contain
root      2416  0.3  0.1 722512 12468 ?        Sl   Sep30  58:16 /bin/containerd-shim-runc-v2 -namespace k8s.io -id f44026865b87983c7032393673434291319544481c58c2e5a4e795987cff5507 -address /run/k3s/contain
root      2421  0.3  0.1 722512 12800 ?        Sl   Sep30  56:39 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 096e1eea26e5b87aa527a6c79c58828d657a08665eb6c001e6219382e84685aa -address /run/k3s/contain
root      2429  1.1  1.6 952512 138020 ?       Ssl  Sep30 168:37 minio server --console-address :9001 /data
65535     2448  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
root      2460  0.3  0.1 722256 13344 ?        Sl   Sep30  57:24 /bin/containerd-shim-runc-v2 -namespace k8s.io -id 7a49faedb9078f91d93cf60e20d881b36e5a06f66afd80a52fb842eb7b925eeb -address /run/k3s/contain
65535     2523  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
65535     2529  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
65535     2556  0.0  0.0    972     4 ?        Ss   Sep30   0:00 /pause
root      2565  0.0  0.0   1616   920 ?        Ss   Sep30   0:00 /bin/sh /usr/bin/entry
root      2625  0.0  0.1 721768 13284 ?        Sl   Sep30   0:05 confd -watch -backend file -file /etc/confd/values.yaml -log-level debug
root      2626  0.0  0.0   5860  3852 ?        S    Sep30   0:00 nginx: master process nginx -g daemon off;
101       2663  0.2  0.0   6580  1912 ?        S    Sep30  37:21 nginx: worker process
101       2667  0.0  0.0   6316  1452 ?        S    Sep30   0:00 nginx: worker process
101       2668  0.0  0.0   6316  1452 ?        S    Sep30   0:00 nginx: worker process
101       2670  0.0  0.0   6316  1452 ?        S    Sep30   0:00 nginx: worker process
101       2671  0.0  0.0   6316  1452 ?        S    Sep30   0:00 nginx: worker process
101       2672  0.0  0.0   6316  1452 ?        S    Sep30   0:00 nginx: worker process
root      2728  1.5  0.4 765168 32884 ?        Ssl  Sep30 240:43 /coredns -conf /etc/coredns/Corefile
vscode    2741  5.2  0.5 760052 41688 ?        Ssl  Sep30 801:47 /metrics-server --cert-dir=/tmp --secure-port=10250 --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --kubelet-use-node-statu
65532     2756  0.3  0.9 840236 80420 ?        Ssl  Sep30  47:40 traefik traefik --global.checknewversion --global.sendanonymoususage --entrypoints.metrics.address=:9100/tcp --entrypoints.traefik.address=:9
root      2758  0.0  0.0   1616   968 ?        Ss   Sep30   0:00 /bin/sh /usr/bin/entry
root      2786  0.3  0.2 734164 21912 ?        Ssl  Sep30  50:41 local-path-provisioner start --config /etc/config/config.json
root      2856  0.0  0.3  54420 24468 ?        Ss   Sep30   8:09 /usr/bin/python3 /usr/local/bin/gunicorn -b 0.0.0.0:80 httpbin:app -k gevent
root      2967  0.3  0.4  77060 35552 ?        S    Sep30  55:15 /usr/bin/python3 /usr/local/bin/gunicorn -b 0.0.0.0:80 httpbin:app -k gevent
vscode    5845  0.0  0.0   8448  7508 pts/1    Ss+  Oct03   0:00 /bin/bash
vscode    7681  0.0  0.0   8448  7576 pts/2    Ss+  Oct03   0:00 /bin/bash
vscode   71598  0.0  0.0   8448  7664 pts/3    Ss+  15:55   0:00 /bin/bash
vscode   74769  0.0  0.0   2888   952 ?        Ss   16:11   0:00 /bin/sh
vscode   74922  0.0  0.0   8448  7668 pts/4    Ss   16:11   0:00 /bin/bash
vscode   77663  0.7  0.2 721556 22364 pts/0    Sl+  16:32   0:14 kit controller server ui
vscode   78628  0.1  0.5 761624 47232 pts/0    Sl   16:32   0:03 kubectl -n argo port-forward svc/minio 9000:9000
vscode   78642  0.0  0.5 761624 42900 pts/0    Sl   16:32   0:01 kubectl -n argo port-forward svc/minio 9001:9001
vscode   78656  0.8  0.6 1284632 53420 pts/0   Sl   16:32   0:15 ./dist/workflow-controller
vscode   78657  0.0  0.7 1298676 61812 pts/0   Sl   16:32   0:00 ./dist/argo server
vscode   78687  0.0  0.9 1246128 78400 pts/0   Sl   16:32   0:01 node /usr/share/yarn/bin/yarn.js start
vscode   78712  0.0  0.0   2896   956 pts/0    S    16:32   0:00 /bin/sh -c NODE_OPTIONS='--no-experimental-fetch --openssl-legacy-provider' webpack-dev-server --config ./src/app/webpack.config.js
vscode   78713  6.4 29.7 56028988 2422284 pts/0 Sl  16:32   2:04 webpack
root     82094  0.0  0.0   2788  1004 ?        S    17:04   0:00 sleep 1
vscode   82095  0.0  0.0   7060  1552 pts/4    R+   17:04   0:00 ps aux

(Also yes, a timeseries graph would be great to see of this, but this is all local, not running on a server producing metrics somewhere 😅 need a tool that can run locally, do a long timespan (10min+), and drill down individual processes)

In any case, this is not a memory leak in kit, so this can be closed (I can't close it myself as I didn't open it). There may be some CPU wonkiness to watch out for though, particularly when kit is a parent of other spiky or CPU-heavy processes.

@alexec alexec closed this as completed Oct 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@alexec @agilgur5