Error connecting

Thanks for reaching out @hostaan.

Where there are no blocks at Site Kit level some countries do have restrictions to Google services. If you’d like to share the following I can check your query with the team:

1. What is the server location of your other VPS hosted?
2. Do you have any other different CURL configurations between the server with no connection issues to that where hosted sites run into the same error?
3. On any of the sites that don’t connect, do you see any errors or warnings when checking your Site Health status (Tools > Site Health > Status)?

Let me know if you have any questions with the above.

Hi,

Thank you for your reply. Both locations are in Helsinki.

1. We use Upcloud as our provider, the locations are FI-HEL1 and FI-HEL2 (https://upcloud.com/data-centres), FI-HEL1 is the one that we are having the issues with.

2. The configuration is the same (same curl versions with –disabled flag).

3. No errors or warnings.

I tried it again just now, connecting to the same server 2a00:1450:4026:805::2011 from both locations. The result is the same:

FI-HEL1:

*   Trying 2a00:1450:4026:805::2011...
* Connected to sitekit.withgoogle.com (2a00:1450:4026:805::2011) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=*.appspot.com
*  start date: Sep 26 08:18:31 2022 GMT
*  expire date: Dec 19 08:18:30 2022 GMT
*  subjectAltName: host "sitekit.withgoogle.com" matched cert's "*.withgoogle.com"
*  issuer: C=US; O=Google Trust Services LLC; CN=GTS CA 1C3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x5649fde620f0)
> HEAD / HTTP/2
> Host: sitekit.withgoogle.com
> User-Agent: curl/7.64.0
> Accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 403 
HTTP/2 403 
< alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"

FI-HEL2:

*   Trying 2a00:1450:4026:805::2011...
* Connected to sitekit.withgoogle.com (2a00:1450:4026:805::2011) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=*.appspot.com
*  start date: Sep 26 08:18:31 2022 GMT
*  expire date: Dec 19 08:18:30 2022 GMT
*  subjectAltName: host "sitekit.withgoogle.com" matched cert's "*.withgoogle.com"
*  issuer: C=US; O=Google Trust Services LLC; CN=GTS CA 1C3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x55d14554d0f0)
> HEAD / HTTP/2
> Host: sitekit.withgoogle.com
> User-Agent: curl/7.64.0
> Accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 200 
HTTP/2 200 
< content-type: text/html
content-type: text/html
< vary: Accept-Encoding
vary: Accept-Encoding
< x-cloud-trace-context: fb52d007311065aa90d7ec03f9786962
x-cloud-trace-context: fb52d007311065aa90d7ec03f9786962
< date: Mon, 31 Oct 2022 12:14:41 GMT
date: Mon, 31 Oct 2022 12:14:41 GMT
< server: Google Frontend
server: Google Frontend
< alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"

May thanks for sharing these additional insights. It looks like your site is encountering 403 errors directly with cURL commands when trying to communicate with sitekit.withgoogle.com. As this may be server configuration related please share the following and I can then check this with the team.

1. Your Site Health information for one impacted site. You can use this form to share privately if preferred.
2. Have you checked your firewall configurations for any differences between both servers, or can you attempt Site Kit set up on a site on the problematic server with any firewall temporary disabled?
3. When attempting set up from a Chrome browser incognito window on any impacted site, do you see any browser console errors on the same screen where this notice appears? If so please share any such errors.

Let me know if you have any questions with the above.

Hi,

I actually just received a confirmation from Upcloud that the IPv6 block in FI-HEL1 is currently blocked by Google. Can you try and escalate the issue on your end? At least the range 2a04:3540:1000:310:: /64 seems to be affected.

Hi @hostaan,

Appreciate the update. I’ll check this with the team. As far as I know there are no blocks from sitekit.withgoogle.com. Please allow me some time to check this with the team, I’ll have an update for you at some stage hopefully tomorrow.

Hi @hostaan,

Just to keep you updated on this, I’ve spoken to the team regarding your query and the server. They are going to take a look into this to check for any blocks. Unfortunately I don’t have a timeframe on this.

In the meantime can you share the information requested above? I’ve included it below:

1. Your Site Health information for one impacted site. You can use this form to share privately if preferred.
2. Have you checked your firewall configurations for any differences between both servers, or can you attempt Site Kit set up on a site on the problematic server with any firewall temporary disabled?
3. When attempting set up from a Chrome browser incognito window on any impacted site, do you see any browser console errors on the same screen where this notice appears? If so please share any such errors.`

Let me know if you have any questions with the above.

Hi,

Sorry for taking so long to repond.

1. Done.

2. Yes, the configration is the same on our end and we don’t have general restrictions for outgoing HTTP traffic. I cannot completely disable the firewall, but in any case the traffic seems to reach the remote end just fine as we get the HTTP error reponse from there.

3. No errors on the browser console. When clicking the “Sign in with Google” we get a HTTP reponse 500 and WordPress’ error message saying “The request to the authentication proxy has failed with an error: request_failed Get help.”

Do you have an update on the issue? We’ve been in contact with our VPS provider Upcloud, but unfortunately it sounds like they haven’t been able to make progress.

Many thanks for the update @hostaan. I’ll need to check with the team for any update on this. I’ll have an answer for you on this tomorrow. Note that from discussing your case with then the last time they status they don’t suspect a block at server level.

As a workaround for the moment, given this is occurring only on one server, can you migrate your site to FI-HEL2? It may also be useful if you can share any details regarding the below:

I actually just received a confirmation from Upcloud that the IPv6 block in FI-HEL1 is currently blocked by Google. Can you try and escalate the issue on your end? At least the range 2a04:3540:1000:310:: /64 seems to be affected.

Did they provide any additional insights into this, or share what checks were performed?

> I’ll need to check with the team for any update on this. I’ll have an
> answer for you on this tomorrow.

Do you have any new regarding the issue?

> Note that from discussing your case with then the last time they
> status they don’t suspect a block at server level.

In that case, do you have a theory why the test mentioned on the github issue (curl -I https://sitekit.withgoogle.com) returns 403 for us?

> As a workaround for the moment, given this is occurring only on one
> server, can you migrate your site to FI-HEL2?

I’m afraid that is not possible, all in all the block (or whatever technical issue it is) is affecting about a thousand sites across multile servers.

> Did they provide any additional insights into this, or share what
> checks were performed?

No, but I’ll ask them to comment on this thread.

Hi @hostaan,

I spoke with the team again on this earlier this week and we’re still awaiting a check unfortunately. One of the team stated that it’s possible that the server could be configured to be pointing to a region where Google services were blocked, rather than this being a Google level block. I didn’t forget about you however, and I will hopefully have an update next week.

In response to your other comments see below:

In that case, do you have a theory why the test mentioned on the github issue (curl -I https://sitekit.withgoogle.com) returns 403 for us?

That’s correct, I also encountered a 403 error. This doesn’t necessarily mean the block is from the Google side. I’ve relayed my checks to the team, along with the test site I’ve set up.

> As a workaround for the moment, given this is occurring only on one
> server, can you migrate your site to FI-HEL2?

I’m afraid that is not possible, all in all the block (or whatever technical issue it is) is affecting about a thousand sites across multile servers.

Sorry to hear that. I don’t have any other workarounds in that case. Hopefully there will be an update soon.

> Did they provide any additional insights into this, or share what
> checks were performed?

No, but I’ll ask them to comment on this thread.

This would be very useful, and this was one of the questions put to me this week when following up – “what checks were performed at host level to determine a Google block“.

I understand the frustration and I will be sure to keep you updated. Hopefully over the next few working days I’ll have an update.

@hostaan Just to keep you updated on this we are still investigating. In the meantime please do share if you have any additional findings from discussing this with your hosting provider.

@hostaan Thanks for your patience on this. We spent some time performing checks on the same FI-HEL1 server with the hosting account we set up. In our case we did also encounter some possible blocks in line with the IP v6 blocks similar to your own. What we did find also was that any cURL checks to sitekit.withgoogle.com are resolving to an IP v6 address, as opposed to an IP v4 address. When performing cURL checks with the IP v4 address the connection succeeds.

While we didn’t set up any other hosting accounts using different servers (ie. FI-HEL2) it’s possible that in these instances you’re being connected to the Site Kit IP v4 address.

The next steps unfortunately requires further testing. From our side we may be able to create a mini plugin that ensures you’re accessing the IP v4 address when communicating with Site Kit, which is possible if a workaround that would allow you to proceed. I’ll need to check the possibility of this this further with the team. We’re also awaiting further checks on possible blocks on the Google side.

On your own side can you check with your hosting provider for any misconfigurations on the IPv6 side of the host? We did encounter a similar issue previously, with sites on a particular IP ranges on Linode servers.

When checking with your hosting provider they can include any insights to the related GitHub issue I created for your topic specifically:
https://github.com/google/site-kit-wp/issues/6095

I have same issue with Upcloud servers. It seems that also youtube.com has issues with Upcloud IPv6 addresses. If I connect via IPv4 address everything works. Is there a way to use only IPv6 with sitekit? Correct way to fix this is to unblock upcloud, but for that to happen is there a way to go a round force connections to IPv4.

@granath If you’d like to open a new support topic we’d be happy to assist with your case individually. Thank you!

@hostaan Just to keep you updated on this, we’re still investigating this from our side. I will keep you updated as soon as I have news, or if we require further information for testing.