Reverse-proxy (Caddy) already in place. How install/configure NC AIO?

[peracchi]

Reverse-proxy (Caddy) already in place. How do I install/configure NC AIO in this scenario?

Caddy is running inside his own container and uses "caddy_net" as his network.

All other containers attach to "caddy_net" so no ports conflict and Caddy can reference each host/service by name.

docker-compose.yml

version: "3.7"
services:
  caddy:
    image: caddy:2.5.0-alpine
    hostname: caddy
    container_name: caddy
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile
      - ./certs:/certs
      - ./config:/config
      - ./data:/data
      - ./sites:/srv

networks:
  default:
    name: caddy_net
    external: true

Caddyfile > /dev/null

{
        #debug
        auto_https disable_certs
}

nextcloud.cites.aop {
        tls /certs/nextcloud.cites.aop/fullchain.pem /certs/nextcloud.cites.aop/privkey.pem
        respond "NEXTCLOUD host for CITES.AOP here!"
        # ??? reverse_proxy nextcloud-aio-mastercontainer:8080 ???
        # ??? reverse_proxy nextcloud-aio-mastercontainer:8443 ???
        # ??? reverse_proxy nextcloud-aio-apache:11000 ???
}

And how about docker-compose.yml for Nextcloud AIO?

docker-compose.yml

version: "3.8"

volumes:
  nextcloud_aio_mastercontainer:
    name: nextcloud_aio_mastercontainer

services:
  nextcloud-aio:
    image: nextcloud/all-in-one:latest
    restart: unless-stopped
    container_name: nextcloud-aio-mastercontainer
    volumes:
      - nextcloud_aio_mastercontainer:/mnt/docker-aio-config
      - /var/run/docker.sock:/var/run/docker.sock:ro
    #ports:
    #  - 80:80
    #  - 8080:8080
    #  - 8443:8443
    environment:
      - APACHE_PORT=11000

networks:
  default:
    name: caddy_net
    external: true

[szaimen]

Hello, you should follow https://github.com/nextcloud/all-in-one/blob/main/reverse-proxy.md

[szaimen]

See also this:

[peracchi]

Already tried, do not worked...

As I use a VPS and have the domain configured, I need to do the initial setup "from home".

Tried using docker-compose and several different configurations on Caddyfile.

Will try again now, using only command line.

Thanks for the fast response and by this amazing piece of software!

[ya-d]

Could you report back if you get this to work, please? I seem to be running the same setup and I couldn't figure out what to do with the instructions in the readme referred to by szaimen. Thanks!

[peracchi]

Tried @szaimen suggestion, do not worked. :(

[peracchi]

See complete infos bellow.

[peracchi]

Before run the command...

 ctop - 18:50:27 -03   6 containers

     NAME                           CID          CPU                MEM                NET RX/TX          IO R/W             PIDS UPTIME

   ⏵ caddy                          6926df144378          0%             41M / 16G     12K / 9K           0B / 0B            11   -1m6s
   ⏵ mailserver                     ac9cbc2d6989          0%             337M / 16G    12K / 8K           0B / 0B            55   -43s
   ⏵ openldap                       007f32e2e0a8          0%             38M / 16G     1K / 520B          0B / 0B            5    -1m6s
   ⏵ phpldapadmin                   e62d2da6cb53          0%             85M / 16G     508B / 0B          0B / 0B            139  -1m6s
   ⏵ roundcube                      55638fa96243          0%             103M / 16G    21K / 14K          0B / 0B            6    -1m5s
 ☼ ⏵ uptime-kuma                    b6aa00ea19dd          0%             94M / 16G     7K / 1K            0B / 0B            12   -1m3s

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:25              0.0.0.0:*               LISTEN      746/docker-proxy
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      329/systemd-resolve
tcp        0      0 127.0.0.54:53           0.0.0.0:*               LISTEN      329/systemd-resolve
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      553/docker-proxy
tcp        0      0 0.0.0.0:143             0.0.0.0:*               LISTEN      733/docker-proxy
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      540/docker-proxy
tcp        0      0 0.0.0.0:465             0.0.0.0:*               LISTEN      710/docker-proxy
tcp        0      0 0.0.0.0:587             0.0.0.0:*               LISTEN      684/docker-proxy
tcp        0      0 0.0.0.0:636             0.0.0.0:*               LISTEN      565/docker-proxy
tcp        0      0 0.0.0.0:993             0.0.0.0:*               LISTEN      667/docker-proxy
tcp        0      0 0.0.0.0:2206            0.0.0.0:*               LISTEN      338/sshd: /usr/bin/
tcp        0      0 0.0.0.0:4190            0.0.0.0:*               LISTEN      644/docker-proxy

NAMES          PORTS
caddy          0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 2019/tcp
uptime-kuma    3001/tcp
roundcube      80/tcp
mailserver     0.0.0.0:25->25/tcp, 0.0.0.0:143->143/tcp, 0.0.0.0:465->465/tcp, 0.0.0.0:587->587/tcp, 110/tcp, 0.0.0.0:993->993/tcp, 0.0.0.0:4190->4190/tcp, 995/tcp
phpldapadmin   80/tcp, 443/tcp
openldap       389/tcp, 0.0.0.0:636->636/tcp

Executed command:

docker run -it \
  --name nextcloud-aio-mastercontainer \
  --restart unless-stopped \
  -e APACHE_PORT=11000 \
  --volume nextcloud_aio_mastercontainer:/mnt/docker-aio-config \
  --volume /var/run/docker.sock:/var/run/docker.sock:ro \
  --network caddy_net \
  nextcloud/all-in-one:latest

Output of above command:

Trying to fix docker.sock permissions internally...
Creating docker group internally with id 973
Generating a RSA private key
..............................................................++++
......................................++++
writing new private key to './ssl.key'
-----
Initial startup of Nextcloud All In One complete!
You should be able to open the Nextcloud AIO Interface now on port 8080 of this server!
E.g. https://internal.ip.of.this.server:8080

If your server has port 80 and 8443 open and you point a domain to your server, you can get a valid certificate automatially by opening the Nextcloud AIO Interface via:
https://your-domain-that-points-to-this-server.tld:8443
2022-05-04 21:54:10,327 CRIT Supervisor is running as root.  Privileges were not dropped because no user is specified in the config file.  If you intend to run as root, you can set user=root in the config file to avoid this message.
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.18.0.8. Set the 'ServerName' directive globally to suppress this message
[Wed May 04 21:54:12.123149 2022] [ssl:warn] [pid 100] AH01906: 172.18.0.8:8080:0 server certificate is a CA certificate (BasicConstraints: CA == TRUE !?)
[Wed May 04 21:54:12.123526 2022] [ssl:warn] [pid 100] AH01909: 172.18.0.8:8080:0 server certificate does NOT include an ID which matches the server name
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.18.0.8. Set the 'ServerName' directive globally to suppress this message
[Wed May 04 21:54:12.278406 2022] [ssl:warn] [pid 100] AH01906: 172.18.0.8:8080:0 server certificate is a CA certificate (BasicConstraints: CA == TRUE !?)
[Wed May 04 21:54:12.278688 2022] [ssl:warn] [pid 100] AH01909: 172.18.0.8:8080:0 server certificate does NOT include an ID which matches the server name
[Wed May 04 21:54:12.288115 2022] [mpm_prefork:notice] [pid 100] AH00163: Apache/2.4.53 (Debian) PHP/8.0.18 OpenSSL/1.1.1n configured -- resuming normal operations
[Wed May 04 21:54:12.288481 2022] [core:notice] [pid 100] AH00094: Command line: 'apache2 -D FOREGROUND'
{"level":"info","ts":1651701252.3048458,"msg":"using provided configuration","config_file":"/Caddyfile","config_adapter":""}
{"level":"warn","ts":1651701252.3090332,"msg":"Caddyfile input is not formatted; run the 'caddy fmt' command to fix inconsistencies","adapter":"caddyfile","file":"/Caddyfile","line":2}
{"level":"info","ts":1651701252.3120644,"logger":"admin","msg":"admin endpoint started","address":"tcp/localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
{"level":"warn","ts":1651701252.313019,"logger":"http","msg":"automatic HTTP->HTTPS redirects are disabled","server_name":"srv0"}
{"level":"warn","ts":1651701252.3137174,"logger":"tls","msg":"YOUR SERVER MAY BE VULNERABLE TO ABUSE: on-demand TLS is enabled, but no protections are in place","docs":"https://caddyserver.com/docs/automatic-https#on-demand-tls"}
{"level":"info","ts":1651701252.3132734,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc0002c9570"}
{"level":"error","ts":1651701252.316073,"msg":"unable to create folder for config autosave","dir":"/var/www/.config/caddy","error":"mkdir /var/www/.config: permission denied"}
{"level":"info","ts":1651701252.3161306,"msg":"serving initial configuration"}
{"level":"info","ts":1651701252.3168404,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/mnt/docker-aio-config/caddy/"}
{"level":"info","ts":1651701252.3169158,"logger":"tls","msg":"finished cleaning storage units"}

New situation:

  ctop - 18:56:46 -03   7 containers

     NAME                           CID          CPU                MEM                NET RX/TX          IO R/W             PIDS UPTIME

   ⏵ caddy                          6926df144378          0%             35M / 16G     18K / 16K          0B / 0B            11   -1m6s
   ⏵ mailserver                     ac9cbc2d6989          0%             265M / 16G    13K / 9K           0B / 0B            55   -43s
   ⏵ nextcloud-aio-mastercontainer  15cc387e8e3d          0%             145M / 16G    0B / 0B            0B / 0B            25   2562047h47m16s
   ⏵ openldap                       007f32e2e0a8          0%             25M / 16G     1K / 520B          0B / 0B            5    -1m6s
   ⏵ phpldapadmin                   e62d2da6cb53          0%             56M / 16G     636B / 0B          0B / 0B            139  -1m6s
   ⏵ roundcube                      55638fa96243          0%             39M / 16G     21K / 14K          0B / 0B            6    -1m5s
 ☼ ⏵ uptime-kuma                    b6aa00ea19dd          0%             79M / 16G     7K / 1K            0B / 0B            12   -1m3s

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:25              0.0.0.0:*               LISTEN      746/docker-proxy
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      329/systemd-resolve
tcp        0      0 127.0.0.54:53           0.0.0.0:*               LISTEN      329/systemd-resolve
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      553/docker-proxy
tcp        0      0 0.0.0.0:143             0.0.0.0:*               LISTEN      733/docker-proxy
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      540/docker-proxy
tcp        0      0 0.0.0.0:465             0.0.0.0:*               LISTEN      710/docker-proxy
tcp        0      0 0.0.0.0:587             0.0.0.0:*               LISTEN      684/docker-proxy
tcp        0      0 0.0.0.0:636             0.0.0.0:*               LISTEN      565/docker-proxy
tcp        0      0 0.0.0.0:993             0.0.0.0:*               LISTEN      667/docker-proxy
tcp        0      0 0.0.0.0:2206            0.0.0.0:*               LISTEN      338/sshd: /usr/bin/
tcp        0      0 0.0.0.0:4190            0.0.0.0:*               LISTEN      644/docker-proxy

NAMES                           PORTS
nextcloud-aio-mastercontainer   80/tcp, 8080/tcp, 8443/tcp
caddy                           0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 2019/tcp
uptime-kuma                     3001/tcp
roundcube                       80/tcp
mailserver                      0.0.0.0:25->25/tcp, 0.0.0.0:143->143/tcp, 0.0.0.0:465->465/tcp, 0.0.0.0:587->587/tcp, 110/tcp, 0.0.0.0:993->993/tcp, 0.0.0.0:4190->4190/tcp, 995/tcp
phpldapadmin                    80/tcp, 443/tcp
openldap                        389/tcp, 0.0.0.0:636->636/tcp

Caddy container can identify "nextcloud-aio-mastercontainer" by name and resolv his IP address:

[admin@vps ~]$ docker exec -it caddy /bin/sh
/srv # ping -c 1 nextcloud-aio-mastercontainer
PING nextcloud-aio-mastercontainer (172.18.0.8): 56 data bytes
64 bytes from 172.18.0.8: seq=0 ttl=64 time=0.173 ms

--- nextcloud-aio-mastercontainer ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.173/0.173/0.173 ms

Now, to Caddyfile!

[admin@vps ~]$ vi ~/docker/caddy/Caddyfile

This...

https://<your-nc-domain>:443 {
    header Strict-Transport-Security max-age=31536000;
    reverse_proxy localhost:11000
}

Turns this:

https://cites.aop:443 {
    header Strict-Transport-Security max-age=31536000;
    reverse_proxy nextcloud-aio-mastercontainer:11000
}

Then, docker-compose restart (on Caddy) and open url https://cites.aop:443 on Firefox...

One eye on Caddy log and other on nextcloud-aio-mastercontainer...

Bang! First attempt gives me "SSL_ERROR_INTERNAL_ERROR_ALERT" on Firefox and nothing.

On second try, added my certificate and Caddyfile now is:

https://nextcloud.cites.aop:443 {
        tls /certs/nextcloud.cites.aop/fullchain.pem /certs/nextcloud.cites.aop/privkey.pem
        header Strict-Transport-Security max-age=31536000;
        reverse_proxy nextcloud-aio-mastercontainer:11000
}

Still do not works.

On Caddy logs (with debug on):

{"level":"debug","ts":1651702699.6701021,"logger":"tls.handshake","msg":"choosing certificate","identifier":"nextcloud.cites.aop","num_choices":1}
{"level":"debug","ts":1651702699.670224,"logger":"tls.handshake","msg":"custom certificate selection results","identifier":"nextcloud.cites.aop","subjects":["nextcloud.cites.aop"],"managed":false,"issuer_key":"","hash":"1f6ba58089fa3d79187b7b8109af83e5c44b8406dcd580746ba4f081e714a9fb"}
{"level":"debug","ts":1651702699.670246,"logger":"tls.handshake","msg":"matched certificate in cache","subjects":["nextcloud.cites.aop"],"managed":false,"expiration":1659281158,"hash":"1f6ba58089fa3d79187b7b8109af83e5c44b8406dcd580746ba4f081e714a9fb"}
{"level":"debug","ts":1651702699.8475943,"logger":"http.handlers.reverse_proxy","msg":"selected upstream","dial":"nextcloud-aio-mastercontainer:11000","total_upstreams":1}
{"level":"debug","ts":1651702699.8505163,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"nextcloud-aio-mastercontainer:11000","duration":0.002820639,"request":{"remote_ip":"201.3.161.244","remote_port":"51410","proto":"HTTP/2.0","method":"GET","host":"nextcloud.cites.aop","uri":"/","headers":{"X-Forwarded-For":["201.3.161.244"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:100.0) Gecko/20100101 Firefox/100.0"],"Accept-Encoding":["gzip, deflate, br"],"Te":["trailers"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8"],"Sec-Fetch-User":["?1"],"X-Forwarded-Proto":["https"],"X-Forwarded-Host":["nextcloud.cites.aop"],"Sec-Fetch-Dest":["document"],"Sec-Fetch-Mode":["navigate"],"Accept-Language":["en-US,en;q=0.8,pt-BR;q=0.5,pt;q=0.3"],"Upgrade-Insecure-Requests":["1"],"Sec-Fetch-Site":["none"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"nextcloud.cites.aop"}},"error":"dial tcp 172.18.0.8:11000: connect: connection refused"}
{"level":"error","ts":1651702699.852183,"logger":"http.log.error","msg":"dial tcp 172.18.0.8:11000: connect: connection refused","request":{"remote_ip":"201.3.161.244","remote_port":"51410","proto":"HTTP/2.0","method":"GET","host":"nextcloud.cites.aop","uri":"/","headers":{"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8"],"Accept-Language":["en-US,en;q=0.8,pt-BR;q=0.5,pt;q=0.3"],"Upgrade-Insecure-Requests":["1"],"Sec-Fetch-Site":["none"],"Sec-Fetch-User":["?1"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:100.0) Gecko/20100101 Firefox/100.0"],"Accept-Encoding":["gzip, deflate, br"],"Sec-Fetch-Dest":["document"],"Sec-Fetch-Mode":["navigate"],"Te":["trailers"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"nextcloud.cites.aop"}},"duration":0.005299071,"status":502,"err_id":"6fxbc2r4n","err_trace":"reverseproxy.statusError (reverseproxy.go:1166)"}
{"level":"debug","ts":1651702700.4236941,"logger":"http.handlers.reverse_proxy","msg":"selected upstream","dial":"nextcloud-aio-mastercontainer:11000","total_upstreams":1}
{"level":"debug","ts":1651702700.4248505,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"nextcloud-aio-mastercontainer:11000","duration":0.001095619,"request":{"remote_ip":"201.3.161.244","remote_port":"51410","proto":"HTTP/2.0","method":"GET","host":"nextcloud.cites.aop","uri":"/favicon.ico","headers":{"Sec-Fetch-Mode":["no-cors"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:100.0) Gecko/20100101 Firefox/100.0"],"Te":["trailers"],"Accept":["image/avif,image/webp,*/*"],"Sec-Fetch-Dest":["image"],"X-Forwarded-Proto":["https"],"Accept-Encoding":["gzip, deflate, br"],"Referer":["https://nextcloud.cites.aop/"],"Sec-Fetch-Site":["same-origin"],"Accept-Language":["en-US,en;q=0.8,pt-BR;q=0.5,pt;q=0.3"],"X-Forwarded-For":["201.3.161.244"],"X-Forwarded-Host":["nextcloud.cites.aop"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"nextcloud.cites.aop"}},"error":"dial tcp 172.18.0.8:11000: connect: connection refused"}
{"level":"error","ts":1651702700.4249268,"logger":"http.log.error","msg":"dial tcp 172.18.0.8:11000: connect: connection refused","request":{"remote_ip":"201.3.161.244","remote_port":"51410","proto":"HTTP/2.0","method":"GET","host":"nextcloud.cites.aop","uri":"/favicon.ico","headers":{"Sec-Fetch-Site":["same-origin"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:100.0) Gecko/20100101 Firefox/100.0"],"Accept-Language":["en-US,en;q=0.8,pt-BR;q=0.5,pt;q=0.3"],"Accept-Encoding":["gzip, deflate, br"],"Referer":["https://nextcloud.cites.aop/"],"Sec-Fetch-Mode":["no-cors"],"Accept":["image/avif,image/webp,*/*"],"Sec-Fetch-Dest":["image"],"Te":["trailers"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"nextcloud.cites.aop"}},"duration":0.001303246,"status":502,"err_id":"ra05u0vhe","err_trace":"reverseproxy.statusError (reverseproxy.go:1166)"}

So, any ideas?

[ya-d]

Thanks for the write up! I’m stuck at to same point.

With caddy-file

<my-nc-url>:443 {
    header Strict-Transport-Security max-age=31536000;
    reverse_proxy nextcloud-aio-mastercontainer:11000
}

<my-nc-url>:8443 {
    reverse_proxy nextcloud-aio-mastercontainer:8080 {
        transport http {
            tls_insecure_skip_verify
        }
    }
}

I’m getting the following error in caddy:

{"level":"error","ts":1651707126.1831357,"logger":"http.log.error","msg":"dial tcp 192.168.32.25:11000: connect: connection refused","request":{"remote_ip":"78.43.40.252","remote_port":"61894","proto":"HTTP/2.0","method":"GET","host":"<my-nc-url>","uri":"/","headers":{"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.3 Safari/605.1.15"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"],"Accept-Language":["de-DE,de;q=0.9"],"Accept-Encoding":["gzip, deflate, br"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"<my-nc-url>"}},"duration":0.001463584,"status":502,"err_id":"dteapczdx","err_trace":"reverseproxy.statusError (reverseproxy.go:1166)"}

although "resume normal operations" doesn’t sound too bad, I’m not able the get though with :443, :8080, :844r or anything.

AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 192.168.32.25. Set the 'ServerName' directive globally to suppress this message

[Thu May 05 00:56:49.461884 2022] [ssl:warn] [pid 96] AH01906: 192.168.32.25:8080:0 server certificate is a CA certificate (BasicConstraints: CA == TRUE !?)

[Thu May 05 00:56:49.462297 2022] [ssl:warn] [pid 96] AH01909: 192.168.32.25:8080:0 server certificate does NOT include an ID which matches the server name

AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 192.168.32.25. Set the 'ServerName' directive globally to suppress this message

[Thu May 05 00:56:49.521393 2022] [ssl:warn] [pid 96] AH01906: 192.168.32.25:8080:0 server certificate is a CA certificate (BasicConstraints: CA == TRUE !?)

[Thu May 05 00:56:49.521792 2022] [ssl:warn] [pid 96] AH01909: 192.168.32.25:8080:0 server certificate does NOT include an ID which matches the server name

[Thu May 05 00:56:49.526492 2022] [mpm_prefork:notice] [pid 96] AH00163: Apache/2.4.53 (Debian) PHP/8.0.18 OpenSSL/1.1.1n configured -- **resuming normal operations**

[Thu May 05 00:56:49.527051 2022] [core:notice] [pid 96] AH00094: Command line: 'apache2 -D FOREGROUND'

[szaimen]

Hello, I see we have to drastically improve the reverse proxy documentation. I already have some ideas but will probably not come to it before this afternoon.

[szaimen]

For anyone reading this, please follow the reworked reverse proxy documentation: https://github.com/nextcloud/all-in-one/blob/main/reverse-proxy.md
Feedback is welcome!

[szaimen]

Either way, thanks for being so stubborn! I think it is now better understandable because of you!

[szaimen]

Just tried on a VPS and it works when using the private ip-address of the host!

[szaimen]

@peracchi did it work for you in the end?

[peracchi]

@szaimen will continue my obstinate journey with nextcloud-aio now...

But do you noticed something? Apparently, nextcloud-aio-mastercontainer is not opening port 11000 anywhere nor other container spawned by him is doing this (opening port 11000).

So, Caddy do not have a place to redirect on port 11000.

[szaimen]

I really don't know how to help xou if you resist to read the whole reverse proxy documentation. Indeed the mastercontainer is not opening the given apache_port. Instead a different container that gets spawned by the mastercontainer will use this port

[peracchi]

I even used netshoot on nextcloud-aio-mastercontainer to confirm open ports, see output bellow:

[admin@vps nextcloud-aio]$ docker run -it --net container:nextcloud-aio-mastercontainer nicolaka/netshoot
                    dP            dP                           dP
                    88            88                           88
88d888b. .d8888b. d8888P .d8888b. 88d888b. .d8888b. .d8888b. d8888P
88'  `88 88ooood8   88   Y8ooooo. 88'  `88 88'  `88 88'  `88   88
88    88 88.  ...   88         88 88    88 88.  .88 88.  .88   88
dP    dP `88888P'   dP   `88888P' dP    dP `88888P' `88888P'   dP

Welcome to Netshoot! (github.com/nicolaka/netshoot)



 f83e00c738ed  ~  netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:8000            0.0.0.0:*               LISTEN      -
tcp        0      0 0.0.0.0:8080            0.0.0.0:*               LISTEN      -
tcp        0      0 127.0.0.11:38075        0.0.0.0:*               LISTEN      -
tcp        0      0 127.0.0.1:2019          0.0.0.0:*               LISTEN      -
tcp        0      0 :::8443                 :::*                    LISTEN      -
udp        0      0 127.0.0.11:60204        0.0.0.0:*

I will shutdown every other container and let only Caddy and Nextcloud-AIO running to see better...

[szaimen]

I really don't know how to help xou if you resist to read the whole reverse proxy documentation. Indeed the mastercontainer is not opening the given apache_port. Instead a different container that gets spawned by the mastercontainer will use this port

[peracchi]

Are you kidding?

"if you resist to read the whole reverse proxy documentation"

Do you really think that I do not read it several times?

"Indeed the mastercontainer is not opening the given apache_port" - really?

You do not saw, looking at a plenty of feedback I provided, that NO OTHER CONTAINER is being spawned by "mastercontainer"?

I will use your own words: "I really don't know how to help you, with feedback, if you resist to read/analyze the whole info I provide".

I was preparing a lot more info to give you but now I realize that is better to wait. Other people will have the same problem. Eventually this will be fixed, changed or "better documentation" will be done.

Anyway, thanks for your effort, we are in the open source/free software field, this is expected to happen.

[szaimen]

I am sorry, I overread the part that it is not spawning another container but anyway, I don't have time for this anymore.

[peracchi]

@szaimen "Stubborn" might not be the right word.

"Determined", "persevering" I think is more appropriate.

Maybe I found the root cause of the problem.

The file docker-compose.yml contains an assumption that is probably what prevents nextcloud-aio-mastercontainer from opening nextcloud-aio-domaincheck, this one will run Apache on port 11000.

- /var/run/docker.sock:/var/run/docker.sock:ro

In my case (and will be the case of others):

[admin@vps ~]$ ls -lh /var/run
lrwxrwxrwx 1 root root 6 Dec  6 23:41 /var/run -> ../run
[admin@vps ~]$
[admin@vps ~]$ ls -lh /run/docker.sock
srw-rw---- 1 root docker 0 May  6 09:31 /run/docker.sock
[admin@vps ~]$

I use Arch, BTW...

After change from
- /var/run/docker.sock:/var/run/docker.sock:ro
to
- /run/docker.sock:/var/run/docker.sock:ro
in docker-compose.yml then nextcloud-aio-mastercontainer could spawn nextcloud-aio-domaincheck

┌───────────────────────────────────────────────────────────────┐
│  id      | c5b7146abeef                                       │
│  name    | nextcloud-aio-domaincheck                          │
│  image   | nextcloud/aio-domaincheck:latest                   │
│  ports   | 0.0.0.0:11000 -> 11000/tcp                         │
│  IPs     | bridge:172.17.0.2                                  │
│          | nextcloud-aio:172.19.0.3                           │
│  state   | running                                            │
│  created | Fri May 6 17:46:29 2022                            │
│  uptime  | 7m46s                                              │
│  health  |                                                    │
└───────────────────────────────────────────────────────────────┘

Now I can proceed with the investigation/configuration of Caddy.

If/when I make it work, I will share my findings/configuration.

[szaimen]

"Stubborn" might not be the right word.

"Determined", "persevering" I think is more appropriate.

You are right! That sounds better! :)

After change from
- /var/run/docker.sock:/var/run/docker.sock:ro
to
- /run/docker.sock:/var/run/docker.sock:ro
in docker-compose.yml then nextcloud-aio-mastercontainer could spawn nextcloud-aio-domaincheck

I see. That indeed sounds like it could solve the issue for you. This is btw also one of the reasons why we recommend to install docker with the script...

Now I can proceed with the investigation/configuration of Caddy.

If/when I make it work, I will share my findings/configuration.

Good luck and I am interested to see if it works for you :)

[szaimen]

Feel free to document your findings and how you made it work in this thread: #588 :)

[szaimen]

In the meantime, I added this as advice to the debug section: b3b8c85

[peracchi]

A little progress but still not there...

I got to initial setup, browser opens a new tab but when I enter the domain to be used by the new AIO instance I got "Domain does not point to this server or reverse proxy not configured correctly.".

At this moment, only two Nextcloud containers running, nextcloud-aio-mastercontainer and nextcloud-aio-domaincheck.

Info from both containers (I used ctop):

┌───────────────────────────────────────────────────────────────┐
│  id      | db362af9feb6                                       │
│  name    | nextcloud-aio-mastercontainer                      │
│  image   | nextcloud/all-in-one:latest                        │
│  ports   | 80/tcp                                             │
│          | 8080/tcp                                           │
│          | 8443/tcp                                           │
│  IPs     | caddy_net:172.18.0.3                               │
│          | nextcloud-aio:172.19.0.2                           │
│  state   | running                                            │
│  created | Fri May 6 20:15:24 2022                            │
│  uptime  | 2m51s                                              │
│  health  |                                                    │
└───────────────────────────────────────────────────────────────┘

┌───────────────────────────────────────────────────────────────┐
│  id      | 24b1bd96acaf                                       │
│  name    | nextcloud-aio-domaincheck                          │
│  image   | nextcloud/aio-domaincheck:latest                   │
│  ports   | 0.0.0.0:11000 -> 11000/tcp                         │
│  IPs     | bridge:172.17.0.2                                  │
│          | nextcloud-aio:172.19.0.3                           │
│  state   | running                                            │
│  created | Fri May 6 20:16:21 2022                            │
│  uptime  | 1m53s                                              │
│  health  |                                                    │
└───────────────────────────────────────────────────────────────┘

Caddy container:

┌───────────────────────────────────────────────────────────────┐
│  id      | 1da6eb8c97b4                                       │
│  name    | caddy                                              │
│  image   | caddy_caddy                                        │
│  ports   | 2019/tcp                                           │
│          | 0.0.0.0:443 -> 443/tcp                             │
│          | 0.0.0.0:80 -> 80/tcp                               │
│          | 0.0.0.0:8443 -> 8443/tcp                           │
│  IPs     | caddy_net:172.18.0.2                               │
│  state   | running                                            │
│  created | Fri May 6 20:15:19 2022                            │
│  uptime  | 2m55s                                              │
│  health  |                                                    │
└───────────────────────────────────────────────────────────────┘

Caddy docker-compose.yml

version: "3.7"
services:
  caddy:
    build: ./tmp
    hostname: caddy
    container_name: caddy
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
    # - "8080:8080"
      - "8443:8443"
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile
      - ./certs:/certs
      - ./config:/config
      - ./data:/data
      - ./sites:/srv

networks:
  default:
    name: caddy_net
    external: true

from Caddyfile, configuration relative to Nextcloud

nextcloud.cites.aop {
    tls /certs/nextcloud.cites.aop/fullchain.pem /certs/nextcloud.cites.aop/privkey.pem
    header Strict-Transport-Security max-age=31536000;
    reverse_proxy nextcloud-aio-domaincheck:11000
}

nextcloud.cites.aop:8443 {
    tls /certs/nextcloud.cites.aop/fullchain.pem /certs/nextcloud.cites.aop/privkey.pem
    reverse_proxy nextcloud-aio-mastercontainer:8080 {
        transport http {
            tls_insecure_skip_verify
        }
    }
}

Nextcloud-AIO docker-compose.yml

version: "3.8"
services:
  nextcloud:
    image: nextcloud/all-in-one:latest
    restart: unless-stopped
    container_name: nextcloud-aio-mastercontainer
    volumes:
      - nextcloud_aio_mastercontainer:/mnt/docker-aio-config
      - /run/docker.sock:/var/run/docker.sock:ro
    #ports:
    #  - 80:80
    #  - 8080:8080
    #  - 8443:8443
    environment:
      - APACHE_PORT=11000

volumes:
  nextcloud_aio_mastercontainer:
    name: nextcloud_aio_mastercontainer

networks:
  default:
    name: caddy_net
    external: true

Log on Caddy (with debug enabled) right after click "Submit" button)

{"level":"debug","ts":1651868279.650207,"logger":"http.handlers.reverse_proxy","msg":"selected upstream","dial":"nextcloud-aio-mastercontainer:8080","total_upstreams":1}
{"level":"debug","ts":1651868279.867384,"logger":"http.stdlib","msg":"http: TLS handshake error from 172.18.0.1:60076: EOF"}
{"level":"debug","ts":1651868279.878611,"logger":"tls.handshake","msg":"choosing certificate","identifier":"nextcloud.cites.aop","num_choices":1}
{"level":"debug","ts":1651868279.8786623,"logger":"tls.handshake","msg":"custom certificate selection results","identifier":"nextcloud.cites.aop","subjects":["nextcloud.cites.aop"],"managed":false,"issuer_key":"","hash":"1f6ba58089fa3d79187b7b8109af83e5c44b8406dcd580746ba4f081e714a9fb"}
{"level":"debug","ts":1651868279.878671,"logger":"tls.handshake","msg":"matched certificate in cache","subjects":["nextcloud.cites.aop"],"managed":false,"expiration":1659281158,"hash":"1f6ba58089fa3d79187b7b8109af83e5c44b8406dcd580746ba4f081e714a9fb"}
{"level":"debug","ts":1651868279.8827035,"logger":"http.handlers.reverse_proxy","msg":"selected upstream","dial":"nextcloud-aio-domaincheck:11000","total_upstreams":1}
{"level":"debug","ts":1651868279.9248693,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"nextcloud-aio-domaincheck:11000","duration":0.042095853,"request":{"remote_ip":"172.18.0.1","remote_port":"60078","proto":"HTTP/2.0","method":"GET","host":"nextcloud.cites.aop","uri":"/","headers":{"User-Agent":[""],"X-Forwarded-For":["172.18.0.1"],"X-Forwarded-Proto":["https"],"X-Forwarded-Host":["nextcloud.cites.aop"],"Accept":["*/*"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"nextcloud.cites.aop"}},"error":"dial tcp: lookup nextcloud-aio-domaincheck on 127.0.0.11:53: no such host"}
{"level":"error","ts":1651868279.956106,"logger":"http.log.error","msg":"dial tcp: lookup nextcloud-aio-domaincheck on 127.0.0.11:53: no such host","request":{"remote_ip":"172.18.0.1","remote_port":"60078","proto":"HTTP/2.0","method":"GET","host":"nextcloud.cites.aop","uri":"/","headers":{"Accept":["*/*"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"nextcloud.cites.aop"}},"duration":0.073472942,"status":502,"err_id":"ecc71zrhx","err_trace":"reverseproxy.statusError (reverseproxy.go:1166)"}
{"level":"debug","ts":1651868279.9591422,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"nextcloud-aio-mastercontainer:8080","duration":0.308828851,"request":{"remote_ip":"201.3.131.122","remote_port":"54069","proto":"HTTP/2.0","method":"POST","host":"nextcloud.cites.aop:8443","uri":"/api/configuration","headers":{"Sec-Fetch-Dest":["empty"],"Accept":["*/*"],"Sec-Fetch-Site":["same-origin"],"X-Forwarded-For":["201.3.161.244"],"Accept-Encoding":["gzip, deflate, br"],"Origin":["https://nextcloud.cites.aop:8443"],"Sec-Fetch-Mode":["cors"],"Cookie":[],"X-Forwarded-Proto":["https"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:100.0) Gecko/20100101 Firefox/100.0"],"Content-Length":["101"],"Te":["trailers"],"X-Forwarded-Host":["nextcloud.cites.aop"],"Accept-Language":["en-US,en;q=0.8,pt-BR;q=0.5,pt;q=0.3"],"Content-Type":["application/x-www-form-urlencoded"],"Referer":["https://nextcloud.cites.aop:8443/containers"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"nextcloud.cites.aop"}},"headers":{"Server":["Apache/2.4.53 (Debian)"],"X-Powered-By":["PHP/8.0.18"],"Content-Type":["text/html; charset=UTF-8"],"Date":["Fri, 06 May 2022 20:17:59 GMT"],"Expires":["Thu, 19 Nov 1981 08:52:00 GMT"],"Cache-Control":["no-store, no-cache, must-revalidate"],"Pragma":["no-cache"],"Content-Length":["79"]},"status":422}

Log on nextcloud-aio-mastercontainer right after click "Submit" button)

172.18.0.3:8080 172.18.0.2 - - [06/May/2022:20:17:59 +0000] "POST /api/configuration HTTP/1.1" 422 3173 "https://nextcloud.cites.aop:8443/containers" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:100.0) Gecko/20100101 Firefox/100.0"
172.18.0.2 - - [06/May/2022:20:17:59 +0000] "POST /api/configuration HTTP/1.1" 422 3173 "https://nextcloud.cites.aop:8443/containers" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:100.0) Gecko/20100101 Firefox/100.0"
172.18.0.3:8000 127.0.0.1 - - [06/May/2022:20:17:59 +0000] "POST /api/configuration HTTP/1.1" 422 437 "https://nextcloud.cites.aop:8443/containers" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:100.0) Gecko/20100101 Firefox/100.0"
127.0.0.1 - - [06/May/2022:20:17:59 +0000] "POST /api/configuration HTTP/1.1" 422 437 "https://nextcloud.cites.aop:8443/containers" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:100.0) Gecko/20100101 Firefox/100.0"

[szaimen]

I guess this happens because nextcloud-aio-domaincheck is not part of the caddy_net network.
You should be able to solve that by changing

nextcloud.cites.aop {
    tls /certs/nextcloud.cites.aop/fullchain.pem /certs/nextcloud.cites.aop/privkey.pem
    header Strict-Transport-Security max-age=31536000;
    reverse_proxy nextcloud-aio-domaincheck:11000
}

to

nextcloud.cites.aop {
    tls /certs/nextcloud.cites.aop/fullchain.pem /certs/nextcloud.cites.aop/privkey.pem
    header Strict-Transport-Security max-age=31536000;
    reverse_proxy <private.ip.address.of.the.host:>11000
}

(of course yyou need to exchange <private.ip.address.of.the.host> to the correct ip-address...)
Using a hostname is here simply not possible...

[szaimen]

...which is btw also noted inside the reverse proxy documentation ;)

[szaimen]

Or is this still not clear enough? :/

[peracchi]

I repeated the test now with more one tab watching the logs of nextcloud-aio-domaincheck

This is the only thing that appeared on the log:
2022-05-06 20:41:25: (server.c.1568) server started (lighttpd/1.4.64)

@szaimen will repeat tests with your suggestion. The strange part is that I do not have a "private" IP on this VPS, only a valid IP address. Private addresses, only the ones automatically created by Docker for his network things.

So, will try with the <external.ip.address.of.the.host> instead of the <private.ip.address.of.the.host>.

I also do a lot of documentation. We need to choose the words used carefully because otherwise we can mislead our "users". Clear enough? :/

[peracchi]

@szaimen do you understand that 209.145.62.xxx is not a PRIVATE IP ADDRESS?

[szaimen]

It is not?

[peracchi]

Obviously you know the difference between private and public IPs...

[szaimen]

so if i understand everything correctly, sudo hstname -I shoulld return the ip-addresses that we can try to test...

[szaimen]

Obviously you know the difference between private and public IPs...

No, I know but I didn't have the exact address rooms in my headd...

[peracchi]

Private IPs are that which cannot be routed through Internet.

[szaimen]

yes, I know. Seems like contabo configures their servers then without any if I understand this correctly

[szaimen]

so can you please try to use that ip-address in the caddyfile and see if it works?

[szaimen]

So I think this one shoud work: 209.145.62.xxx

[szaimen]

@peracchi seems like we are finally getting to a solution now :)

[szaimen]

@peracchi does sudo hostname -I | cut -d ' ' -f 1 indeed return this ip-address? if yes, I guess I'll document this in the reverse proxy as simplest way to find out the ip-address that you need to use here...

[peracchi]

ip route | grep default | awk '{print $9}'

Nope. Return empty string.

[szaimen]

hm... and sudo hostname -I still does not work, I suppose?

[szaimen]

Then what about saying use hostname -I and if that does not work, use curl -s ifconfig.me ?

[peracchi]

Then what about saying use hostname -I and if that does not work, use curl -s ifconfig.me ?

hostname is not installed by default, needs installation of inetutils package, for example.

[admin@vps ~]$ pkgfile hostname
core/gettext
core/inetutils
extra/archboot
extra/archboot-arm
extra/archiso
extra/bash-completion
community/byobu

[szaimen]

okay. Lets continue in #568 (reply in thread)

[peracchi]

Created normal user and installed Docker. Nothing more.

Output of ip a and ifconfig

[admin@vps ~]$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:50:56:41:dc:af brd ff:ff:ff:ff:ff:ff
    inet 209.145.62.xxx/20 brd 209.145.63.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 2605:a140:2054:3442::1/64 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::250:56ff:fe41:dcaf/64 scope link
       valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
    link/ether 02:42:db:79:58:57 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever

[admin@vps ~]$ ifconfig
docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
        ether 02:42:db:79:58:57  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 209.145.62.xxx  netmask 255.255.240.0  broadcast 209.145.63.255
        inet6 2605:a140:2054:3442::1  prefixlen 64  scopeid 0x0<global>
        inet6 fe80::250:56ff:fe41:dcaf  prefixlen 64  scopeid 0x20<link>
        ether 00:50:56:41:dc:af  txqueuelen 1000  (Ethernet)
        RX packets 13360  bytes 122488179 (116.8 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 9634  bytes 961435 (938.9 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 160  bytes 15120 (14.7 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 160  bytes 15120 (14.7 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

And some more info...

                   -`                    [email protected]
                  .o+`                   -------------------
                 `ooo/                   OS: Arch Linux x86_64
                `+oooo:                  Host: KVM/QEMU (Standard PC (i440FX + PIIX, 1996) pc-i440fx-5.2)
               `+oooooo:                 Kernel: 5.15.37-1-lts
               -+oooooo+:                Uptime: 6 mins
             `/:-:++oooo+:               Packages: 242 (pacman)
            `/++++/+++++++:              Shell: bash 5.1.16
           `/++++++++++++++:             Resolution: 1024x768
          `/+++ooooooooooooo/`           Terminal: /dev/pts/0
         ./ooosssso++osssssso+`          CPU: AMD EPYC 7282 (6) @ 2.794GHz
        .oossssso-````/ossssss+`         GPU: 00:02.0 Vendor 1234 Device 1111
       -osssssso.      :ssssssso.        Memory: 174MiB / 16001MiB
      :osssssss/        osssso+++.
     /ossssssss/        +ssssooo/-
   `/ossssso+/:-        -:/+osssso+-
  `+sso+:-`                 `.-/+oso:
 `++:.                           `-/+/
 .`                                 `/

[admin@vps ~]$ docker --version
Docker version 20.10.15, build fd82621d35
[admin@vps ~]$
[admin@vps ~]$ docker-compose --version
Docker Compose version 2.5.0
[admin@vps ~]$
[admin@vps ~]$ docker network ls
NETWORK ID     NAME      DRIVER    SCOPE
b184be561474   bridge    bridge    local
3e3e0f78aef3   host      host      local
16966c40a034   none      null      local

[szaimen]

https://<your-nc-domain>:443 {
    header Strict-Transport-Security max-age=31536000;
    reverse_proxy 209.145.62.xxx:11000
}

Optional:

https://<your-nc-domain>:8443 {
    reverse_proxy https://209.145.62.xxx:8080 {
        transport http {
            tls_insecure_skip_verify
        }
    }
}

[szaimen]

@peracchi what do you think about the network host option for the reverse proxy container?
https://docs.docker.com/network/host/

[peracchi]

@peracchi what do you think about the network host option for the reverse proxy container? https://docs.docker.com/network/host/

Not an option. Breaks several other services. Only viable if the only thing running is Nextcloud. Goes against all benefits of use Docker/Containers.

[szaimen]

This option would of course only used for the reverse proxy container... So is this really so bad?

[szaimen]

I wonder if we can recommend the network host option by default for the reverse proxy container but say that you can also instead of localhost use the ip-address of the host?

[peracchi]

Ok, Caddy running with this docker-compose.yml and Caddyfile (Nextcloud not yet)

[admin@vps docker]$ cat caddy/docker-compose.yml
version: "3.7"
services:
  caddy:
    build: ./tmp
    hostname: caddy
    container_name: caddy
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
     #- "8080:8080"
      - "8443:8443"
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile
      - ./certs:/certs
      - ./config:/config
      - ./data:/data
      - ./sites:/srv

networks:
  default:
    name: caddy_net
    external: true

[admin@vps docker]$ cat caddy/Caddyfile
{
        debug
}

(cloudflare) {
        tls [email protected] {
                dns cloudflare <key>
        }
}

(vips_only) {
        @fuck_off_world {
                not remote_ip x.y.z.w/32 a.b.c.d/32
        }
        respond @fuck_off_world 403
}

https://nextcloud.cites.aop:443 {
        import cloudflare
        import vips_only
        header Strict-Transport-Security max-age=31536000;
        reverse_proxy 209.145.62.xxx:11000
}

https://nextcloud.cites.aop:8443 {
        import cloudflare
        import vips_only
        reverse_proxy https://209.145.62.xxx:8080 {
                transport http {
                        tls_insecure_skip_verify
                }
        }
}

[peracchi]

Will test (before run Nextcloud AIO) try to open the URLs from Firefox (just to confirm Caddy is ok and certificate was issued without problems).

[szaimen]

Okay, please report back if that works. I think I'll go to sleep now. Will come back to this tomorrow.

[peracchi]

Okay, please report back if that works. I think I'll go to sleep now. Will come back to this tomorrow.

Good night, I enjoyed work with you (it was a little stressing, I confess). Take a good rest and another day we continue our "battles".

If I found a good solution, you will be the first to know.

[peracchi]

I enter only nextcloud.cites.aop on URL address bar of Firefox. It will try first as HTTP (default behavior) then Caddy will redirect to HTTPS.

As expected, all ok with certificate (thanks, Caddy!) but nobody is listening on IP address 209.145.62.xxx port 11000 to receive data from Caddy.

Same thing with https://nextcloud.cites.aop:8443 - nobody listening on IP address 209.145.62.xxx port 8080 to receive data from Caddy.

Just this tests made me see that things will not work this way, do you see?

IP 209.145.62.xxx is of the HOST of the Docker's containers...

Caddy itself is a container! And his job is to intermediate communication for other containers.

So, Caddyfile must be contructed to "reverse-proxy" to CONTAINER'S NAMES and this is the "beauty" of Docker (containers) system. You do not need to worry about IP addresses, you just use the container's names and voilá! You have security, you have isolation (between cointainers AND networks), etc.

[szaimen]

But did you start AIO already?

[szaimen]

Night!

[peracchi]

But did you start AIO already?

Not yet, but is logical (says the Spock inside me).

Nextcloud containers will expose their ports on "caddy_net", that is another network from the Docker's point of view

[admin@vps docker]$ docker network inspect caddy_net
[
    {
        "Name": "caddy_net",
        "Id": "12b535e4c447b2cc57cf0d2f4c807be8789d012f5f464b2b7dd87ef824a51baf",
        "Created": "2022-05-06T19:38:25.670040804-03:00",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "172.18.0.0/16",
                    "Gateway": "172.18.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "7dd4e34e98af854c88f71e98cf5ff3062bc46ef34aa5c6569ed1fd70bd0dd9fa": {
                "Name": "caddy",
                "EndpointID": "71d204174e66244aa05ab9595ae15184409c90ebbed17adccadbc3c8bf511f58",
                "MacAddress": "02:42:ac:12:00:02",
                "IPv4Address": "172.18.0.2/16",
                "IPv6Address": ""
            }
        },
        "Options": {},
        "Labels": {}
    }
]

[peracchi]

"172.18.0.0/16" - IP 209.145.62.xxx do not have/see the listenings on "172.18.0.x:11000" and "172.18.0.y:8080" (they are different containers, different IPs)

[szaimen]

Alright but I think you are missing that the domaincheck container gets started with -p 11000:11000 which opens the port on the host...

[peracchi]

The closest I came to a solution was with:

Nextcloud docker-compose.yml

version: "3.8"
services:
  nextcloud-aio:
    image: nextcloud/all-in-one:latest
    restart: unless-stopped
    container_name: nextcloud-aio-mastercontainer
    volumes:
      - nextcloud_aio_mastercontainer:/mnt/docker-aio-config
      - /run/docker.sock:/var/run/docker.sock:ro
    environment:
      - APACHE_PORT=11000

volumes:
  nextcloud_aio_mastercontainer:
    name: nextcloud_aio_mastercontainer

networks:
  default:
    name: caddy_net
    external: true

Caddy docker-compose.yml

version: "3.7"
services:
  caddy:
    image: caddy:2.5.0-alpine
    hostname: caddy
    container_name: caddy
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
      - "8443:8443"
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile
      - ./config:/config
      - ./data:/data
      - ./sites:/srv

networks:
  default:
    name: caddy_net
    external: true

and Caddyfile

{
        debug
}

(cloudflare) {
        tls [email protected] {
                dns cloudflare <api token>
        }
}

(vips_only) {
        @fuck_off_world {
                not remote_ip x.y.z.w/32 a.b.c.d/32
        }
        respond @fuck_off_world 403
}

https://nextcloud.cites.aop:443 {
        import cloudflare
        import vips_only
        header Strict-Transport-Security max-age=31536000;
        reverse_proxy https://nextcloud-aio-domaincheck:11000
}

https://nextcloud.cites.aop:8443 {
        import cloudflare
        import vips_only
        reverse_proxy https://nextcloud-aio-mastercontainer:8080 {
                transport http {
                        tls_insecure_skip_verify
                }
        }
}

Absolutely nothing on nextcloud-aio-domaincheck logs:

2022-05-06 23:37:28: (server.c.1568) server started (lighttpd/1.4.64)

Error on Caddy's log:

{"level":"debug","ts":1651880409.4995413,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"nextcloud-aio-mastercontainer:8080","duration":1.185286938,"request":{"remote_ip":"x.y.z.w","remote_port":"55617","proto":"HTTP/2.0","method":"POST","host":"nextcloud.cites.aop:8443","uri":"/api/configuration","headers":{"X-Forwarded-For":["x.y.z.w"],"Accept":["*/*"],"Content-Type":["application/x-www-form-urlencoded"],"Te":["trailers"],"Accept-Language":["en-US,en;q=0.8,pt-BR;q=0.5,pt;q=0.3"],"Origin":["https://nextcloud.cites.aop:8443"],"X-Forwarded-Host":["nextcloud.cites.aop"],"Sec-Fetch-Site":["same-origin"],"Sec-Fetch-Dest":["empty"],"Accept-Encoding":["gzip, deflate, br"],"Content-Length":["101"],"Sec-Fetch-Mode":["cors"],"Referer":["https://nextcloud.cites.aop:8443/containers"],"X-Forwarded-Proto":["https"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:100.0) Gecko/20100101 Firefox/100.0"],"Cookie":[]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"nextcloud.cites.aop"}},"headers":{"Server":["Apache/2.4.53 (Debian)"],"X-Powered-By":["PHP/8.0.18"],"Expires":["Thu, 19 Nov 1981 08:52:00 GMT"],"Content-Type":["text/html; charset=UTF-8"],"Date":["Fri, 06 May 2022 23:40:08 GMT"],"Cache-Control":["no-store, no-cache, must-revalidate"],"Pragma":["no-cache"],"Content-Length":["79"]},"status":422}

Error code = "status":422

[szaimen]

I see. Can you please try out http://209.145.62.xxx:11000 instead of https://nextcloud-aio-domaincheck:11000 in the caddyfile at least once?

[szaimen]

This should work as the domaincheck container gets started with -p 11000:11000 whivh opens the port on the host.

[peracchi]

For us to think together...

Ideal communication flow: [ Internet ]
                            <-> [ host ]
                              <-> [ exposed ports on host by Caddy ]
                                <-> [ caddy container ] <-- reverse-proxy
                                  <-> [ to choosen port/container on Caddyfile ]

• benefits
  • minimizes attack surface
  • prevents port conflicts between containers
  • HTTPS with automatic valid certificate for services that do not offer it
  • Caddy can use host/service names instead of hard coded IPs (that you may not know in advance or may change later)

What I see now on my VPS while trying to setup Nextcloud AIO:

    [ Internet ]
        |
    [ VPS host ] - 209.145.62.xxx on eth0
        |
        |--- exposed ports on host = 80,443 (caddy) / 22 (sshd)
        |
[ Docker Containers ]
        |
        | (( caddy_net = 172.18.0.0/16 on br-66c670728c83
        |
        |--- caddy - 172.18.0.2/16
        |          - exposed ports on container = 80,443,2019
        |
        |--- nextcloud-aio-mastercontainer - 172.18.0.3/16
        |                                  - exposed ports on container = 80,8080,8443
        |
        |                                                                              ))
        |
        |--- nextcloud-aio-domaincheck - 172.17.0.2/16 (eth0) / exposed ports here = 11000
                                       - 172.19.0.3/16 (eth1) / exposed ports here = 11000

I can start Nextcloud AIO setup if I use only

nextcloud.cites.poa {
        import cloudflare
        import vips_only
        reverse_proxy nextcloud-aio-mastercontainer:8080 {
                transport http {
                        tls_insecure_skip_verify
                }
        }
}

on Caddyfile.

I can login into "Nextcloud AIO Login", nextcloud-aio-domaincheck container is spawned and this container exposes port 11000.

The problem is that this container creates a new network "nextcloud-aio" = 172.19.0.0/16, gets IP 172.19.0.3 there and exposes port 11000 on "nextcloud-aio" and on Docker's default bridge, 172.17.0.0/16 as if it was the host...

I am still thinking about it but it seems a network engineering design flaw somewhere.

Docker/containers are meant to make things easy and "portable". I think if your solution only works if Docker is used/configured a certain way then your solution is not a good Docker solution. My two cents...

If someone else wants to give a try, below some copy/paste commands to quickly test this out. You only need to change domain names, credential and api token (if using Cloudflare DNS).

mkdir -p ~/nc-aio/{config,data,tmp} && cd ~/nc-aio

cat << 'EOF' | tee ./tmp/Dockerfile > /dev/null
FROM caddy:2.5.0-builder-alpine AS builder
RUN xcaddy build --with github.com/caddy-dns/cloudflare
FROM caddy:2.5.0-alpine
COPY --from=builder /usr/bin/caddy /usr/bin/caddy
EOF

cat << 'EOF' | tee ./docker-compose.yml > /dev/null
version: "3.7"
services:
  caddy:
    build: ./tmp
    hostname: caddy
    container_name: caddy
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile
      - ./config:/config
      - ./data:/data
  nextcloud-aio:
    image: nextcloud/all-in-one:latest
    restart: unless-stopped
    container_name: nextcloud-aio-mastercontainer
    volumes:
      - nextcloud_aio_mastercontainer:/mnt/docker-aio-config
      - /run/docker.sock:/var/run/docker.sock:ro
    environment:
      - APACHE_PORT=11000
volumes:
  nextcloud_aio_mastercontainer:
    name: nextcloud_aio_mastercontainer
networks:
  default:
    name: caddy_net
    external: true
EOF

cat << 'EOF' | tee ./Caddyfile > /dev/null
{
        debug
}
(cloudflare) {
        tls [email protected] {
                dns cloudflare <put your API key here>
        }
}
(vips_only) {
        @fuck_off_world {
                # for security, while testing out things I restrict access to my computer and cellphone IPs
                not remote_ip x.y.z.w a.b.c.d/32
        }
        respond @fuck_off_world 403
}
nextcloud.example.com {
        import cloudflare
        import vips_only
        reverse_proxy nextcloud-aio-mastercontainer:8080 {
                transport http {
                        tls_insecure_skip_verify
                }
        }
}
EOF

docker network create caddy_net

clear ; docker-compose up -d

Nextcloud AIO seems to be a great idea/piece of software.

Who do not wants a simple and easy Docker solution to an optimized version of Nextcloud?

I was making my own version of "Docker solution to an optimized version of Nextcloud".

Everything wents very well until add Collabora/Code to the equation.

Then I saw some news about Nextcloud AIO and here I am... :)

[szaimen]

Indeed it does the following with the chosen apache_port:

    [ Internet ]
        |
    [ VPS host ] - 209.145.62.xxx on eth0
        |
        |--- exposed ports on host = 80,443 (caddy) / 22 (sshd)
        |--- exposed port on host = 11000 (nextcloud-aio-domaincheck)
        |
[ Docker Containers ]
        |
        | (( caddy_net = 172.18.0.0/16 on br-66c670728c83
        |
        |--- caddy - 172.18.0.2/16
        |          - exposed ports on container = 80,443,2019
        |
        |--- nextcloud-aio-mastercontainer - 172.18.0.3/16
        |                                  - exposed ports on container = 80,8080,8443

[peracchi]

Indeed it does the following with the chosen apache_port:

    [ Internet ]
        |
    [ VPS host ] - 209.145.62.xxx on eth0
        |
        |--- exposed ports on host = 80,443 (caddy) / 22 (sshd)
        |--- exposed port on host = 11000 (nextcloud-aio-domaincheck)
        |
[ Docker Containers ]
        |
        | (( caddy_net = 172.18.0.0/16 on br-66c670728c83
        |
        |--- caddy - 172.18.0.2/16
        |          - exposed ports on container = 80,443,2019
        |
        |--- nextcloud-aio-mastercontainer - 172.18.0.3/16
        |                                  - exposed ports on container = 80,8080,8443

No, it doesn't.

Wait a minute, just restored VPS snapshot, executing the commands I posted above and will show the output of
sudo netstat -tlpn | sort -t: -k2 -n
on the HOST as soon as I started Nextcloud AIO setup again and reach the part that nextcloud-aio-domaincheck is spawned.

[szaimen]

This allows it to work in every setup also e.g. where the reverse proxy is not installed on the same server. It may not be perfect for your case but the only way (that I see) to make it work for everyone.

[peracchi]

Here:

[admin@vps nc-aio]$ sudo netstat -tlpn | sort -t: -k2 -n
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      329/systemd-resolve
tcp        0      0 127.0.0.54:53           0.0.0.0:*               LISTEN      329/systemd-resolve
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      6546/docker-proxy
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      6529/docker-proxy

See?

No open port 11000 on the host as you described above...

[szaimen]

Can you send me the output of docker inspect nextcloud-aio-domaincheck ?

[peracchi]

No problem @szaimen , just documents that Nextcloud AIO only works using the "network = host" parameter.

And do not post wrong affirmatives.

[szaimen]

Can you send me the output of docker inspect nextcloud-aio-domaincheck ?

[szaimen]

Using the host network for the mastercontainer will not make it work for the domaincheck container. So that is not an option.

[peracchi]

[admin@vps nc-aio]$ docker ps
CONTAINER ID   IMAGE                              COMMAND                  CREATED         STATUS          PORTS                                                NAMES
4d6f4c4f1486   nextcloud/aio-domaincheck:latest   "/start.sh"              7 seconds ago   Up 6 seconds    0.0.0.0:11000->11000/tcp                             nextcloud-aio-domaincheck
b19e73008965   nextcloud/all-in-one:latest        "start.sh /usr/bin/s…"   6 minutes ago   Up 6 minutes    80/tcp, 8080/tcp, 8443/tcp                           nextcloud-aio-mastercontainer
28e2c1ffc26d   nc-aio_caddy                       "caddy run --config …"   6 minutes ago   Up 23 seconds   0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 2019/tcp   caddy
[admin@vps nc-aio]$ docker inspect nextcloud-aio-domaincheck
[
    {
        "Id": "4d6f4c4f1486a7d829fdef5eee8aa27769fa6fd6cb396f457f12897db6393731",
        "Created": "2022-05-07T15:08:26.772618483Z",
        "Path": "/start.sh",
        "Args": [],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 8563,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2022-05-07T15:08:27.176598067Z",
            "FinishedAt": "0001-01-01T00:00:00Z"
        },
        "Image": "sha256:e9abb98b052bab9b6a8a9336222c04059e270caaf7ebcb9d18e531d4d97d7ac3",
        "ResolvConfPath": "/var/lib/docker/containers/4d6f4c4f1486a7d829fdef5eee8aa27769fa6fd6cb396f457f12897db6393731/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/4d6f4c4f1486a7d829fdef5eee8aa27769fa6fd6cb396f457f12897db6393731/hostname",
        "HostsPath": "/var/lib/docker/containers/4d6f4c4f1486a7d829fdef5eee8aa27769fa6fd6cb396f457f12897db6393731/hosts",
        "LogPath": "/var/lib/docker/containers/4d6f4c4f1486a7d829fdef5eee8aa27769fa6fd6cb396f457f12897db6393731/4d6f4c4f1486a7d829fdef5eee8aa27769fa6fd6cb396f457f12897db6393731-json.log",
        "Name": "/nextcloud-aio-domaincheck",
        "RestartCount": 0,
        "Driver": "overlay2",
        "Platform": "linux",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": null,
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "default",
            "PortBindings": {
                "11000/tcp": [
                    {
                        "HostIp": "",
                        "HostPort": "11000"
                    }
                ]
            },
            "RestartPolicy": {
                "Name": "",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": null,
            "CapDrop": null,
            "CgroupnsMode": "private",
            "Dns": null,
            "DnsOptions": null,
            "DnsSearch": null,
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "private",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": null,
            "DeviceCgroupRules": null,
            "DeviceRequests": null,
            "KernelMemory": 0,
            "KernelMemoryTCP": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": null,
            "PidsLimit": null,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "MaskedPaths": [
                "/proc/asound",
                "/proc/acpi",
                "/proc/kcore",
                "/proc/keys",
                "/proc/latency_stats",
                "/proc/timer_list",
                "/proc/timer_stats",
                "/proc/sched_debug",
                "/proc/scsi",
                "/sys/firmware"
            ],
            "ReadonlyPaths": [
                "/proc/bus",
                "/proc/fs",
                "/proc/irq",
                "/proc/sys",
                "/proc/sysrq-trigger"
            ]
        },
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/746b98b5969c91ed3f606a6a6beca3788d6308ed783c48d1b4058fd8e2b30e83-init/diff:/var/lib/docker/overlay2/8be91fa2f09545d11a2aa2d5f1b1c4b34026f70c4a149c089075f52aacad4134/diff:/var/lib/docker/overlay2/2f0ce08fd6f92f6aeed91cded01ec7e462e606fdc9b0f89fb904108cad2f4c7b/diff:/var/lib/docker/overlay2/97cc608539f535f49355244df0f54a1678b8b462c11d08a099b0356c256fec1c/diff:/var/lib/docker/overlay2/1a5e2d1765037c171c62102f1ea97bcfafe43ea016bc2559eacc0e0abfda2dd0/diff:/var/lib/docker/overlay2/76df7f2e73ad8dc826730e3decb0955042df25d1a6e6f8f96e73ba842ce6cc24/diff:/var/lib/docker/overlay2/4b9a597c9e57912889b87e30c1b686d3df2961148545c91466931dbe90ace734/diff:/var/lib/docker/overlay2/4f2ee5e98f6b20bb2e2a2e175cfaf5db46fd199936ca5c7d891835b2a2bcebb8/diff:/var/lib/docker/overlay2/39b3615c396a12df877c9bad9435565c2df7bc100f731d5ed7cb156e5e8387b4/diff:/var/lib/docker/overlay2/b9f3c8013450f2d31465928e184d8b62e6e3bb5ecf264e3f1d7093a7175873d8/diff",
                "MergedDir": "/var/lib/docker/overlay2/746b98b5969c91ed3f606a6a6beca3788d6308ed783c48d1b4058fd8e2b30e83/merged",
                "UpperDir": "/var/lib/docker/overlay2/746b98b5969c91ed3f606a6a6beca3788d6308ed783c48d1b4058fd8e2b30e83/diff",
                "WorkDir": "/var/lib/docker/overlay2/746b98b5969c91ed3f606a6a6beca3788d6308ed783c48d1b4058fd8e2b30e83/work"
            },
            "Name": "overlay2"
        },
        "Mounts": [],
        "Config": {
            "Hostname": "4d6f4c4f1486",
            "Domainname": "",
            "User": "www-data",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "11000/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "INSTANCE_ID=3a1b837430c957e693224b4acfe0cfced119372e8b67595f",
                "APACHE_PORT=11000",
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": null,
            "Image": "nextcloud/aio-domaincheck:latest",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": [
                "/start.sh"
            ],
            "OnBuild": null,
            "Labels": {}
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "03b6dd9d6b7527257c0c15a1dd0a3ce44fa45527aceed0276329dfa72874376c",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {
                "11000/tcp": [
                    {
                        "HostIp": "0.0.0.0",
                        "HostPort": "11000"
                    }
                ]
            },
            "SandboxKey": "/var/run/docker/netns/03b6dd9d6b75",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "78e6400a98bbb570432db848634af2e9a7d67e96c3e5f7152c6198abc5e90863",
            "Gateway": "172.17.0.1",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "172.17.0.2",
            "IPPrefixLen": 16,
            "IPv6Gateway": "",
            "MacAddress": "02:42:ac:11:00:02",
            "Networks": {
                "bridge": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "312308ee4f501d247514815ac61dd3e113b51c00c11dd438a54f5b1675894700",
                    "EndpointID": "78e6400a98bbb570432db848634af2e9a7d67e96c3e5f7152c6198abc5e90863",
                    "Gateway": "172.17.0.1",
                    "IPAddress": "172.17.0.2",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:11:00:02",
                    "DriverOpts": null
                },
                "nextcloud-aio": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": [
                        "4d6f4c4f1486"
                    ],
                    "NetworkID": "de05fb57ccc3105cf5ad68e435c291885cc0c9930a74750efa3965ed1f2231ef",
                    "EndpointID": "b1b1903bbb85b7ad0fcde063959429cd7ee4a1722fe1d2c5528a6ee62a04237a",
                    "Gateway": "172.19.0.1",
                    "IPAddress": "172.19.0.3",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:13:00:03",
                    "DriverOpts": null
                }
            }
        }
    }
]

[peracchi]

Maybe it is like this because you presumably did not install docker the recommended way (did you use the script?) but honestly no idea....

Another joke?

Docker is Docker.

Depending the distro you use: pacman -S docker, apt install docker, etc.

You mean your container solution only works if your "install Docker script" is used?

[szaimen]

Another joke?

No joke

Docker is Docker.

It is not as we've seen e.g. with the docker socket that is mounted at a different place in your case. And also the versions will differ.

Depending the distro you use: pacman -S docker, apt install docker, etc.

You mean your container solution only works if your "install Docker script" is used?

That might indeed be possibke and it is not my install script but the official docker install script.

[peracchi]

Ok, tell me which version of Ubuntu and Docker you want I install/use.

Then, I will show you the EXACT SAME PROBLEM when your solution is behind Caddy (running in a container) and without the use of host networking in Docker.

From Docker documentation: "If you use the host network mode for a container, that container’s network stack is not isolated from the Docker host (the container shares the host’s networking namespace), and the container does not get its own IP-address allocated."

This means that you are not using one of the fundamental reasons to use containers, that is, ISOLATION!

I understand that you do not have a solution for the problem but is almost unbelievable that you still do not undestand WHY the problem occurs...

[szaimen]

Try Ubuntu 22.04 and the docker install script

[peracchi]

Try Ubuntu 22.04 and the docker install script

@szaimen I sincerly apologize, with Ubuntu 22.04 and Docker from the official repo nextcloud-aio-domaincheck was able to open port 11000 directly on the host.

This situation invalidates the concept of using the reverse-proxy as a "gatekeeper" and will likely cause a port conflict, since both the nextcloud-aio-domaincheck container and the caddy container would need to listen/open the same port (11000).

I think we both made the mistake of not "walk in someone else’s shoes". That should be the first step, recreate the other's environment to be assured if works or do not works... We could stay days on this "on my setup it's working", "on my setup it's not working"...

It could be a security issue, either in the OS or in Docker network layer. Who's doing it right? Arch, Ubuntu, Docker? I don't know.

Anyway, we still don't have the complete solution (and documentation) to leave Nextcloud AIO fully "behind the reverse-proxy". I saw that nobody contributed to "docker-compose examples for reverse proxies".

I will continue testing, now with Ubuntu, to see what will happen with the port conflict between Caddy and the nextcloud-aio-domaincheck container.

As I said before, we are in the open source/free software field and this type of situation is expected to happen. So many distros, so many versions of same packages, so many VPSs, so many ...

[peracchi]

Now I give up...

Using Ubuntu 22.04, Docker installed using the script, port 11000 open on localhost, Caddyfile as in documentation.

admin@vps:~/nc-aio$ echo ; sudo netstat -tlpn | sort -t: -k2 -n ; echo

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      438/systemd-resolve
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      482/sshd: /usr/sbin

admin@vps:~/nc-aio$ docker-compose up -d
[+] Running 3/3
 ⠿ Volume "nextcloud_aio_mastercontainer"   Created                                                                    0.0s
 ⠿ Container nextcloud-aio-mastercontainer  Started                                                                    0.7s
 ⠿ Container caddy                          Started                                                                    0.7s
admin@vps:~/nc-aio$ echo ; sudo netstat -tlpn | sort -t: -k2 -n ; echo

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      438/systemd-resolve
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      904/docker-proxy
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      891/docker-proxy
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      482/sshd: /usr/sbin
tcp        0      0 0.0.0.0:8080            0.0.0.0:*               LISTEN      875/docker-proxy

admin@vps:~/nc-aio$ echo "open https://nextcloud.cites.aop:8080 on Firefox, proceed to install and log in on Nextcloud AIO"
open https://nextcloud.cites.aop:8080 on Firefox, proceed to install and log in on Nextcloud AIO
admin@vps:~/nc-aio$ echo ; sudo netstat -tlpn | sort -t: -k2 -n ; echo

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      438/systemd-resolve
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      904/docker-proxy
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      891/docker-proxy
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      482/sshd: /usr/sbin
tcp        0      0 0.0.0.0:8080            0.0.0.0:*               LISTEN      875/docker-proxy
tcp        0      0 0.0.0.0:11000           0.0.0.0:*               LISTEN      1862/docker-proxy

admin@vps:~/nc-aio$ echo "Enter 'nextcloud.cites.aop' and click 'Submit' button"
Enter 'nextcloud.cites.aop' and click 'Submit' button
admin@vps:~/nc-aio$ echo "Still gives 'Domain does not point to this server or reverse proxy not configured correctly.'"
Still gives 'Domain does not point to this server or reverse proxy not configured correctly.'
admin@vps:~/nc-aio$ docker ps
CONTAINER ID   IMAGE                              COMMAND                  CREATED          STATUS          PORTS                                                                        NAMES
ab01cf9c5ab0   nextcloud/aio-domaincheck:latest   "/start.sh"              8 minutes ago    Up 8 minutes    0.0.0.0:11000->11000/tcp                                                     nextcloud-aio-domaincheck
0c7a27dce9c1   nextcloud/all-in-one:latest        "start.sh /usr/bin/s…"   11 minutes ago   Up 11 minutes   80/tcp, 8080/tcp, 8443/tcp                                                   nextcloud-aio-mastercontainer
01c9c6338ff0   nc-aio_caddy                       "caddy run --config …"   11 minutes ago   Up 11 minutes   0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:8080->8080/tcp, 2019/tcp   caddy
admin@vps:~/nc-aio$ cat Caddyfile
{
        debug
}

(cloudflare) {
        tls [email protected] {
                dns cloudflare < Cloudflare api token >
        }
}

(vips_only) {
        @fuck_off_world {
                # for security, while testing out things I restrict access to my computer and cellphone IPs
                not remote_ip x.y.z.w/32 a.b.c.d/32
        }
        respond @fuck_off_world 403
}

https://nextcloud.cites.aop:443 {
        import cloudflare
        import vips_only
        header Strict-Transport-Security max-age=31536000;
        reverse_proxy localhost:11000
}

nextcloud.cites.aop:8080 {
        import cloudflare
        import vips_only
        reverse_proxy nextcloud-aio-mastercontainer:8080 {
                transport http {
                        tls_insecure_skip_verify
                }
        }
}
admin@vps:~/nc-aio$ cat docker-compose.yml
version: "3.7"
services:
  caddy:
    build: ./tmp
    hostname: caddy
    container_name: caddy
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
      - "8080:8080"
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile
      - ./config:/config
      - ./data:/data
  nextcloud-aio:
    image: nextcloud/all-in-one:latest
    restart: unless-stopped
    container_name: nextcloud-aio-mastercontainer
    volumes:
      - nextcloud_aio_mastercontainer:/mnt/docker-aio-config
      - /var/run/docker.sock:/var/run/docker.sock:ro
    environment:
      - APACHE_PORT=11000
volumes:
  nextcloud_aio_mastercontainer:
    name: nextcloud_aio_mastercontainer
networks:
  default:
    name: caddy_net
    external: true

And nothing appears on docker logs -f nextcloud-aio-domaincheck

In Caddy log, the last line, that presents an error, is this:

{"level":"debug","ts":1651951788.6699502,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"nextcloud-aio-mastercontainer:8080","duration":0.065301628,"request":{"remote_ip":"x.y.z.w","remote_port":"62910","proto":"HTTP/2.0","method":"POST","host":"nextcloud.cites.aop:8080","uri":"/api/configuration","headers":{"Content-Length":["101"],"Sec-Fetch-Dest":["empty"],"X-Forwarded-Proto":["https"],"X-Forwarded-For":["x.y.z.w"],"X-Forwarded-Host":["nextcloud.cites.aop"],"Accept-Encoding":["gzip, deflate, br"],"Origin":["https://nextcloud.cites.aop:8080"],"Cookie":[],"Te":["trailers"],"Content-Type":["application/x-www-form-urlencoded"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:100.0) Gecko/20100101 Firefox/100.0"],"Accept-Language":["en-US,en;q=0.8,pt-BR;q=0.5,pt;q=0.3"],"Sec-Fetch-Mode":["cors"],"Accept":["*/*"],"Sec-Fetch-Site":["same-origin"],"Referer":["https://nextcloud.cites.aop:8080/containers"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"nextcloud.cites.aop"}},"headers":{"Date":["Sat, 07 May 2022 19:29:48 GMT"],"Expires":["Thu, 19 Nov 1981 08:52:00 GMT"],"Pragma":["no-cache"],"Content-Type":["text/html; charset=UTF-8"],"Server":["Apache/2.4.53 (Debian)"],"X-Powered-By":["PHP/8.0.18"],"Cache-Control":["no-store, no-cache, must-revalidate"],"Content-Length":["79"]},"status":422}

By the way, the same error presented when I was using Arch.

:/

[szaimen]

Since it somehow does not seem to work for you, I guess it makes sense for you to be able to modify all details yourself. See #557
I'll probably work on that soon.

[szaimen]

This is now implemented.
See https://github.com/nextcloud/all-in-one/tree/main/manual-install