Race condition in start.sh

[dljaime]

How to use GitHub

• Please use the 👍 reaction to show that you are affected by the same issue.
• Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
• Subscribe to receive notifications on status change and new comments.

---

Steps to reproduce

Probably not the easiest way but this is how I got here. I'm running on a raspberry pi with ssd

1. Setup nextcloud docker (I used nginx/maria/fpm)
2. Add HaRP

   appapi-harp:
       container_name: appapi-harp
       image: ghcr.io/nextcloud/nextcloud-appapi-harp:latest
       restart: always
       environment:
         - HP_SHARED_KEY=myspecialtestkey
         - NC_INSTANCE_URL=htp://nexcloud.url
       volumes:
          - /var/run/docker.sock:/var/run/docker.sock
          - ./certs:/certs
        secrets:
          - nextcloud-harp-shared-key
   
   secrets:
     nextcloud-harp-shared-key:
       file: ./secret_harp_shared_key.txt
   

3. Setup daemon in nextcloud
4. Click Check connection

Expected behaviour

Connection works

Actual behaviour

Connection times out

HaRP container logs

INFO: FRP server configuration generated at /frps.toml.
INFO: Detected /var/run/docker.sock, generating /frpc-docker.toml configuration file...
INFO: Starting Python HaProxy Agent on 127.0.0.1:8200 and 127.0.0.1:9600...
INFO: Starting FRP server on 0.0.0.0:8782...
INFO: Starting FRP client for Docker Engine...
INFO: Starting HAProxy...
2025-10-18 18:51:04.507 [I] [sub/root.go:142] start frpc service for config file [/frpc-docker.toml]
2025-10-18 18:51:04.508 [I] [client/service.go:295] try to connect to server...
[NOTICE]   (1) : Initializing new worker (37)
2025-10-18 18:51:04.612 [E] [client/service.go:280] send Login request to plugin error
2025-10-18 18:51:04.613 [W] [client/service.go:298] connect to server error: send Login request to plugin error
2025-10-18 18:51:04.614 [I] [sub/root.go:160] frpc service for config file [/frpc-docker.toml] stopped
login to the server failed: send Login request to plugin error. With loginFailExit enabled, no additional retries will be attempted
[NOTICE]   (1) : haproxy version is 3.1.2-cda631a
[WARNING]  (1) : Process 30 exited with code 1 (Exit)
[NOTICE]   (1) : Loading success.
[WARNING]  (37) : Server agents/agent1 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[ALERT]    (37) : backend 'agents' has no server available!
<129>Oct 18 18:51:04 haproxy[37]: Server agents/agent1 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
<128>Oct 18 18:51:04 haproxy[37]: backend agents has no server available!
[WARNING]  (37) : Server agents/agent1 is UP, reason: Layer7 check passed, code: 0, check duration: 5ms. 1 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.

Server configuration

Web server: Nginx

Database: Maria

PHP version: 8.3

Nextcloud version: 32

Browser

Browser name: Arc
Browser version: 141.0.7390.108

Operating system: Windows

More Notes

After much head scratching I checked out this repo and increased the last sleep to 10s:

HaRP/start.sh

Lines 311 to 319 in 079eef3

	log "INFO: Starting FRP server on ${HP_FRP_ADDRESS}..."
	frps -c /frps.toml &
	sleep 1s
	if [ -e "/var/run/docker.sock" ]; then
	log "INFO: Starting FRP client for Docker Engine..."
	frpc -c /frpc-docker.toml &
	fi

Now the logs looks much better and the test connection from Nextcloud worked:

INFO: FRP server configuration generated at /frps.toml.
INFO: Detected /var/run/docker.sock, generating /frpc-docker.toml configuration file...
INFO: Starting Python HaProxy Agent on 127.0.0.1:8200 and 127.0.0.1:9600...
INFO: Starting FRP server on 0.0.0.0:8782...
INFO: Starting FRP client for Docker Engine...
INFO: Starting HAProxy...
2025-10-18 18:52:39.790 [I] [sub/root.go:142] start frpc service for config file [/frpc-docker.toml]
2025-10-18 18:52:39.791 [I] [client/service.go:295] try to connect to server...
[NOTICE]   (1) : Initializing new worker (68)
2025-10-18 18:52:39.858 [I] [client/service.go:287] [a8a00d2229365789] login to server success, get run id [a8a00d2229365789]
2025-10-18 18:52:39.859 [I] [proxy/proxy_manager.go:173] [a8a00d2229365789] proxy added: [bundled-deploy-daemon]
2025-10-18 18:52:39.865 [I] [client/control.go:168] [a8a00d2229365789] [bundled-deploy-daemon] start proxy success
[NOTICE]   (1) : Loading success.

10s is obviously too long for a standard config, but is there a more reliable way of testing whether the FRP server is up and able to take connections?

[oleksandr-nc]

Thanks for reporting that. Can you please check if the linked PR solves your problem?

[dljaime]

Seems that does not quite solve it. Logs below

INFO: /frps.toml already exists. Skipping FRP server configuration generation...
INFO: /frpc-docker.toml already exists. Skipping generation...
INFO: Starting Python HaProxy Agent on 127.0.0.1:8200 and 127.0.0.1:9600...
INFO: Starting FRP server on 0.0.0.0:8782...
INFO: Waiting for FRP server port 127.0.0.1:8782...
INFO: Starting FRP client for Docker Engine...
INFO: Starting HAProxy...
2025-10-20 09:49:45.864 [I] [sub/root.go:142] start frpc service for config file [/frpc-docker.toml]
2025-10-20 09:49:45.864 [I] [client/service.go:295] try to connect to server...
[NOTICE]   (1) : Initializing new worker (36)
2025-10-20 09:49:45.900 [E] [client/service.go:280] send Login request to plugin error
2025-10-20 09:49:45.903 [W] [client/service.go:298] connect to server error: send Login request to plugin error
2025-10-20 09:49:45.903 [I] [sub/root.go:160] frpc service for config file [/frpc-docker.toml] stopped
login to the server failed: send Login request to plugin error. With loginFailExit enabled, no additional retries will be attempted
[NOTICE]   (1) : haproxy version is 3.1.2-cda631a
[WARNING]  (1) : Process 29 exited with code 1 (Exit)
[NOTICE]   (1) : Loading success.
[WARNING]  (36) : Server agents/agent1 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
<129>Oct 20 09:49:46 haproxy[36]: Server agents/agent1 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[ALERT]    (36) : backend 'agents' has no server available!
<128>Oct 20 09:49:46 haproxy[36]: backend agents has no server available!

[dljaime]

I'm starting to wonder if maybe there is something else going on. I noticed loginFailExit = false was in the frpc config example for external docker engines but it wasn't in the default frpc-docker config. I tried adding that instead of the 10s timeout and I get the login failing multiple times until the server ends up banning the client.

[2025-10-20T10:02:06+0000] [WARNING] Recorded failure for IP 127.0.0.1. Failures in window: 8
2025-10-20 10:02:06.429 [E] [client/service.go:280] send Login request to plugin error
2025-10-20 10:02:06.431 [W] [client/service.go:298] connect to server error: send Login request to plugin error
2025-10-20 10:02:16.432 [I] [client/service.go:295] try to connect to server...
[2025-10-20T10:02:16+0000] [WARNING] Recorded failure for IP 127.0.0.1. Failures in window: 9
2025-10-20 10:02:16.456 [E] [client/service.go:280] send Login request to plugin error
2025-10-20 10:02:16.456 [W] [client/service.go:298] connect to server error: send Login request to plugin error
2025-10-20 10:02:26.458 [I] [client/service.go:295] try to connect to server...
[2025-10-20T10:02:26+0000] [WARNING] Recorded failure for IP 127.0.0.1. Failures in window: 10
2025-10-20 10:02:26.509 [E] [client/service.go:280] send Login request to plugin error
2025-10-20 10:02:26.509 [W] [client/service.go:298] connect to server error: send Login request to plugin error
2025-10-20 10:02:36.510 [I] [client/service.go:295] try to connect to server...
2025-10-20 10:02:36.534 [E] [client/service.go:280] banned
2025-10-20 10:02:36.535 [W] [client/service.go:298] connect to server error: banned

I removed that line from the config and restored the 10s timeout and all is right again. I've also noticed that if I reboot the machine the logout failure comes back. But restarting the container gets it going again.

[oleksandr-nc]

please test with new commit, looks like the problem was in first sleep 1s for python agent to start (still have correct waiting instead of both sleep 1s is good)

[dljaime]

No luck

INFO: /frps.toml already exists. Skipping FRP server configuration generation...
INFO: /frpc-docker.toml already exists. Skipping generation...
INFO: Starting Python HaProxy Agent on 127.0.0.1:8200 and 127.0.0.1:9600...
INFO: Waiting for HaRP Agent HTTP (GET http://127.0.0.1:8200/info) to be ready...
INFO: Waiting for SPOA port 127.0.0.1:9600...
INFO: Starting FRP server on 0.0.0.0:8782...
INFO: Waiting for FRP server port 127.0.0.1:8782...
INFO: Starting FRP client for Docker Engine...
INFO: Starting HAProxy...
2025-10-20 10:17:55.861 [I] [sub/root.go:142] start frpc service for config file [/frpc-docker.toml]
2025-10-20 10:17:55.861 [I] [client/service.go:295] try to connect to server...
[NOTICE]   (1) : Initializing new worker (49)
[2025-10-20T10:17:55+0000] [WARNING] Recorded failure for IP 127.0.0.1. Failures in window: 1
2025-10-20 10:17:55.901 [E] [client/service.go:280] send Login request to plugin error
2025-10-20 10:17:55.902 [W] [client/service.go:298] connect to server error: send Login request to plugin error
2025-10-20 10:17:55.902 [I] [sub/root.go:160] frpc service for config file [/frpc-docker.toml] stopped
login to the server failed: send Login request to plugin error. With loginFailExit enabled, no additional retries will be attempted
[NOTICE]   (1) : haproxy version is 3.1.2-cda631a
[WARNING]  (1) : Process 41 exited with code 1 (Exit)
[NOTICE]   (1) : Loading success.

Edit - it worked the first time but when the container restarted I got the above error

[oleksandr-nc]

Edit - it worked the first time but when the container restarted I got the above error

looks like that is a different bug. but for first time code in the PR works, right?

[dljaime]

Scenario 1. If I start with just the pr code then I get the login error

Scenario 2. If I start with my timeout workaround it works, then if I switch to the pr code it still works, then if the container restarts I sometimes get this error - not every time though

[oleksandr-nc]

Was able to reproduce second bug, opened a new PR regarding fix for it: #56

Can you test it? (it should fix situation when nothing is working after reboot)

[oleksandr-nc]

Scenario 1. If I start with just the pr code then I get the login error

Scenario 2. If I start with my timeout workaround it works

I hope that this is not a third bug.

then if I switch to the pr code it still works

What does that exactly mean?

[dljaime]

Sorry I could have been clearer, I was using docker compose up -d --build nextcloud-harp once with my timeout bodge and that worked, then switched to your first branch and that worked.

I've just retried with your new PR and it is working regardless of container or server rebooting.

Thanks so much for the help. I thought I was taking crazy pills for most of Saturday 🤣

[oleksandr-nc]

Thank you for letting us know and testing the changes quickly. I really appreciate it