cannot reach clickhouse host？

[sunny19930321]

When using agents, what are the causes of the following problems?
Caused by: java.lang.Throwable: [ Id: 160857F337DC563A; User "tmplarge"(1) proxying as "default"(1) to "d085126100.aliyun.com:8123"(6); RemoteAddr: "10.13.56.73:51080"; LocalAddr: "10.85.129.101:9090"; Duration: 825 ?s]: cannot reach d085126100.aliyun.com:8123; query: "select timezone()\nFORMAT TabSeparatedWithNamesAndTypes;"

[VitoLiao]

We are meet same question, our chproxy version is v1.14.0, any body know how to fix it ?

[hagen1778]

Can you verify if this request works without using agent, maybe via curl or any other http client?

[sidanasparsh]

Facing the same issue intermittently? Are there updates on this?

[apetrov88]

+1

[502] [ Id: 161FB3A842CE4135; User "default"(1) proxying as "default"(1) to "CHCluster03:8123"(6); RemoteAddr: "10.0.0.24:51324"; LocalAddr: "10.0.0.12:9090"; Duration: 16798 μs]: cannot reach CHCluster03:8123; query: ..

CHProxy 1.14

[sunny19930321]

Can you verify if this request works without using agent, maybe via curl or any other http client?

@hagen1778 ，Verify that CK service is ok to request

[sunny19930321]

Facing the same issue intermittently? Are there updates on this?

@sidanasparsh Try to adjust the timeout, but it doesn't seem to work

[sunny19930321]

Facing the same issue intermittently? Are there updates on this?

Have you found the cause of this problem？

[sunny19930321]

+1

[502] [ Id: 161FB3A842CE4135; User "default"(1) proxying as "default"(1) to "CHCluster03:8123"(6); RemoteAddr: "10.0.0.24:51324"; LocalAddr: "10.0.0.12:9090"; Duration: 16798 μs]: cannot reach CHCluster03:8123; query: ..

CHProxy 1.14

Have you found the cause of this problem？

[karas2015]

+1, how to fix it?

[hagen1778]

Proxy returns "cannot reach" error when it unable to establish connection to the given address https://github.com/Vertamedia/chproxy/blob/1758e7399fe57c97aeec8e55dd13c6300399969b/proxy.go#L186-L193. I do not know what causes this because a lot of things can be involved to affect reachability between "your_application"<=>"chproxy"<=>"clickhouse" scheme.
Btw, proxy exposes host_health metric (and a plenty of others) to show if configured CH host is reachable. Can you check the state of this metric in the moments when query from the agent fails? If you send queries without agent - does it work?

[JustHarris]

We are experiencing the same kind of connection issues

ERROR: 2021/02/03 06:36:39 proxy.go:192: [ Id: 165F9CDFD12DDA70; User "compass-insert-hits"(1) proxying as "admin"(1) to "myclusternode02.io"(6); RemoteAddr: "100.64.4.2:50028"; LocalAddr: "100.65.205.183:80"; Duration: 60133561 μs]: cannot reach myclusternode02.io:8123; query: "INSERT INTO hits ........."
...
ERROR: 2021/02/03 13:05:11 scope.go:643: error while health-checking "myclusternode01.io:8123" host: cannot send request in 3.00015176s: Get "http://myclusternode01.io.io:8123/?query=SELECT%201": context deadline exceeded
.....
ERROR: 2021/02/04 11:46:26 proxy.go:192: [ Id: 165F9CDFD1355353; User "newsroom-cache"(1) proxying as "default"(1) to "asinglechnode.io:8123"(6); RemoteAddr: "100.65.205.170:47344"; LocalAddr: "100.65.205.183:80"; Duration: 1097 μs]: cannot reach asinglechnode.io:8123; query: "SELECT maxMerge(Hit.event_time) as maxEventTime......"

We have done a full network check simulating the healthchecks with curl and checking for packet loss with mtr with no luck. The network is working flawless with not even a single packet lost or a failed curl.

[akimrx]

We have same problem with three clickhouse nodes.
Also, ClickHouse backends available at this time directly.