GSKit Codepage issue

[richardchen]

Dear all,

Greeting. I am working on iSeries V7R4. Recently I need to implement TLS 1.2 over TCP/IP for an Internet URL.

The infra looks like:

iSeries -> SOCKS5 -> Proxy Server -> Internet -> Vendor URL

I implement SOCKS5 protocol with a C program. Then user GSKit to secure the channel.
Handshake failed due to protocol version as checked by vendor.
I am not able to enable the GSKit handshake trace as GSKit is not installed in iSereis, only the service pgm available.

I suspect this is due to code page issue: GSKit sending EBCDIC and the TCP server use ASCII.
I can't find any way to change the codepage for GSKit (i try env vars GSK_ENV, failed).

May I know anyone has similar experience on this error? Thanks.

This is the error when i try to establish the connection:

gsk_secure_soc_init() failed with rc = 410 and errno = 0.
rc of 410 means Peer not recognized or badly formatted message received.

[Scott Klement]

Greeting. I am working on iSeries V7R4. Recently I need to implement TLS 1.2 over TCP/IP for an Internet URL.

The final release for iSeries was V5R4. Perhaps you are running IBM i 7.4?

Handshake failed due to protocol version as checked by vendor.
I am not able to enable the GSKit handshake trace as GSKit is not installed in iSereis, only the service pgm available.

Likely that either the vendor or your code is not allowing the TLS 1.2 protocol. It could also be that there wasn't a compatible cipher suite allowed. Try connecting from the command line with openssh and see what protocols and/or cipher suites the server offers. Then check your IBM i system values to make sure that you have enabled those. (Or override in your code to force it to enable them.)

I suspect this is due to code page issue: GSKit sending EBCDIC and the TCP server use ASCII.
I can't find any way to change the codepage for GSKit (i try env vars GSK_ENV, failed).

No. That's not how SSL/TLS works -- it is a binary protocol, it never uses EBCDIC, and there is no code page. Your guess is completely wrong, here.

This is the error when i try to establish the connection:
gsk_secure_soc_init() failed with rc = 410 and errno = 0.
rc of 410 means Peer not recognized or badly formatted message received.

Yes, it doesn't recognize the TLS version and/or cipher suite that the remote site is using. That's why it says "peer not recognized." It could also mean that the remote site is trying to use plain (unencrypted) data instead of SSL/TLS. The way they are communicating, including the version of TLS, the cipher suites, whether TLS is enabled/disabled, etc must all match.

You need to enable compatible versions and/or ciphers on your IBM i, as I already said above.

[richardchen]

The final release for iSeries was V5R4. Perhaps you are running IBM i 7.4?

Yes.

Yes, it doesn't recognize the TLS version and/or cipher suite that the remote site is using. That's why it says "peer not recognized." It could also mean that the remote site is trying to use plain (unencrypted) data instead of SSL/TLS. The way they are communicating, including the version of TLS, the cipher suites, whether TLS is enabled/disabled, etc must all match.

System admin try to capture the traffic from iSeries, but not able to the detail handshake message. This is the same case when I ask Proxy team to capture the traffic.

From destination server (TCP Server)'s handshake log, I could see that iSeries (GSKit) is sending TLS 1.2 and valid cipher suites in Client Hello message. And server pick up TLS 1.2 and one cipher suit in Server Hello. But during handshake, after Server Hello Done, the client site alert Fatal:70 (Protocol Error).

It could also mean that the remote site is trying to use plain (unencrypted) data instead of SSL/TLS. The way they are communicating, including the version of TLS, the cipher suites, whether TLS is enabled/disabled, etc must all match.

Are u referring to the SSL Record Version? I do not know how to check this.
In Client Hello, the Record Version is TLS 1.2, actually I confirmed with destination server on this. The vendor mention some participent use TLS 1.0 or 1.2, but both could do successful handshake.

Nobody knows why this is happening. I try to use most of the default values in GSKit (just disable TLS 1.3), still no luck.

I try to run `curl` command in iSeries and I could connect to desitnation server, can complete SSL handshake and even can get the message from server. I knew curl will hit the http endpoint but nevermind, I use it for connectivty (log compare) purpose only. NB - the server offer TCP Server only, not http service there (no idea why i can get the msg).

Curl output: SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384. This is the same setting with GSKit.

May I know any other areas I could check? Thanks millions!

[Scott Klement]

From destination server (TCP Server)'s handshake log, I could see that iSeries (GSKit) is sending TLS 1.2 and valid cipher suites in Client Hello message. And server pick up TLS 1.2 and one cipher suit in Server Hello. But during handshake, after Server Hello Done, the client site alert Fatal:70 (Protocol Error).

I'm not familar with the tool you're using or what "70 (Protocol Error)" means. However, I suspect it means the same thing as the 410 code you posted earlier. So this doesn't really tell us anything we didn't know.

Are u referring to the SSL Record Version? I do not know how to check this.
In Client Hello, the Record Version is TLS 1.2, actually I confirmed with destination server on this. The vendor mention some participent use TLS 1.0 or 1.2, but both could do successful handshake.

I'm not familiar with the deep internals of TLS, I'm not familiar with terminology like "record version." You were responding to the part of my message where I said you can get this error (410 bad peer) when you try to negotiate TLS/SSL with a socket that is plain text only... since plain text doesn't use TLS or SSL, there's no way it can have a record version, unless I'm not understanding you. But since you've verified that it's using TLS 1.2, this is clearly not what is happening in this instance.

Nobody knows why this is happening. I try to use most of the default values in GSKit (just disable TLS 1.3), still no luck.

I don't see why you'd want to disable TLS 1.3.

As I mentioned before, the likely cause is that there isn't a matching cipher available. TLS negotiates ciphers... The two programs tell each other which ciphers they support. If there isn't one that's usable by both sides of the connection, you get the 410 error that you posted previously.

I try to run `curl` command in iSeries and I could connect to desitnation server, can complete SSL handshake and even can get the message from server. I knew curl will hit the http endpoint but nevermind, I use it for connectivty (log compare) purpose only. NB - the server offer TCP Server only, not http service there (no idea why i can get the msg).

Presumably, curl isn't using the SOCKS5 code that you mentioned earlier, correct? If you connect directly (without SOCKS5) using a GSKit-based program (such as HTTPAPI) does it work correctly? IF so, that tells you that the problem is in the implementation of the SOCKS5 code.

If it doesn't work, it means that your GSKit configuration is different from the OpenSSL configuration that curl uses. In other words, OpenSSL has been configured with different ciphers, and there's one that matches. Whereas GSKit is not configured with any ciphers that match the server.

Curl output: SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384. This is the same setting with GSKit.

Your GSKit has been configured to use the ECDHE-RSA-AES356-GCM-SHA384 cipher? You've checked this? Why haven't you mentioned this before now? If you have information like this, please don't hold it back... my time is valuable, I don't want to waste it guessing things you've already checked.

If you've already verified that the ECDHE-RSA-AES356-GCM-SHA384 cipher works from other software, and that it is configured to work in your code (and you can see that it's being offered and rejected) then you've reached the limit of my expertise. In that case, you'll need to contact IBM for further support.