SecureCRT 7.2 throwing "Session closed" error with GSSAPI enabled servers

[parodymshifter]

I just updated from SecureCRT 7.1 to 7.2. It appears that all of my sessions I have defined to connect to RHEL servers running Centrify-Express with the Centrify enabled OpenSSH are instantly dying on connect with
"Session closed" or "Session reset"

How can I fix this? I tried creating a new session to one of these servers and I continue to get the same error. I also tried disabling GSSAPI and still got the same error.

I am still able to login with putty with GSSAPI enabled to these servers. I'd rather be using SecureCRT.

Here is a trace from one of them with the target server name anonymized:

[LOCAL] : CAP : Remote can do SFTP version 4
[LOCAL] : CAP : Remote x.509v3 uses ASN.1 encoding for DSA signatures
[LOCAL] : CAP : Remote correctly handles zlib@openssh.com
[LOCAL] : SSPI : Requesting full delegation
[LOCAL] : SSPI : [Kerberos] SPN : host@xxx.yyy.com
[LOCAL] : SSPI : Requesting full delegation
[LOCAL] : SSPI : [Kerberos (Group Exchange)] SPN : host@xxx.yyy.com
[LOCAL] : SEND : KEXINIT
[LOCAL] : RECV : Read kexinit
[LOCAL] : Available Remote Kex Methods = gss-gex-sha1-toWM5Slw5Ew8Mqkay+al2g==,gss-group1-sha1-toWM5Slw5Ew8Mqkay+al2g==,gss-group14-sha1-toWM5Slw5Ew8Mqkay+al2g==,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
[LOCAL] : Selected Kex Method = gss-group1-sha1-toWM5Slw5Ew8Mqkay+al2g==
[LOCAL] : Available Remote Host Key Algos = ssh-rsa,ssh-dss,ssh-dss
[LOCAL] : Selected Host Key Algo = ssh-dss
[LOCAL] : Available Remote Send Ciphers = aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
[LOCAL] : Selected Send Cipher = aes256-ctr
[LOCAL] : Available Remote Recv Ciphers = aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-[LOCAL] : SSH2Core version 7.2.0.415
[LOCAL] : Connecting to xxx:22 ...
[LOCAL] : Changing state from STATE_NOT_CONNECTED to STATE_EXPECT_KEX_INIT
SecureCRT - Version 7.2.0 (x64 build 415)
[LOCAL] : Using protocol SSH2
[LOCAL] : RECV : Remote Identifier = 'SSH-2.0-OpenSSH_6.2'
[LOCAL] : CAP : Remote can re-key
[LOCAL] : CAP : Remote sends language in password change requests
[LOCAL] : CAP : Remote sends algorithm name in PK_OK packets
[LOCAL] : CAP : Remote sends algorithm name in public key packets
[LOCAL] : CAP : Remote sends algorithm name in signatures
[LOCAL] : CAP : Remote sends error text in open failure packets
[LOCAL] : CAP : Remote sends name in service accept packets
[LOCAL] : CAP : Remote includes port number in x11 open packets
[LOCAL] : CAP : Remote uses 160 bit keys for SHA1 MAC
[LOCAL] : CAP : Remote supports new diffie-hellman group exchange messages
[LOCAL] : CAP : Remote correctly handles unknown SFTP extensions
[LOCAL] : CAP : Remote correctly encodes OID for gssapi
[LOCAL] : CAP : Remote correctly uses connected addresses in forwarded-tcpip requests
[LOCAL] : CAP : Remote can do SFTP version 4
[LOCAL] : CAP : Remote x.509v3 uses ASN.1 encoding for DSA signatures
[LOCAL] : CAP : Remote correctly handles zlib@openssh.com
[LOCAL] : SSPI : Requesting full delegation
[LOCAL] : SSPI : [Kerberos] SPN : host@xxx
[LOCAL] : SSPI : Requesting full delegation
[LOCAL] : SSPI : [Kerberos (Group Exchange)] SPN : host@xxx
[LOCAL] : SEND : KEXINIT
[LOCAL] : RECV : Read kexinit
[LOCAL] : Available Remote Kex Methods = gss-gex-sha1-toWM5Slw5Ew8Mqkay+al2g==,gss-group1-sha1-toWM5Slw5Ew8Mqkay+al2g==,gss-group14-sha1-toWM5Slw5Ew8Mqkay+al2g==,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
[LOCAL] : Selected Kex Method = gss-group1-sha1-toWM5Slw5Ew8Mqkay+al2g==
[LOCAL] : Available Remote Host Key Algos = ssh-rsa,ssh-dss,ssh-dss
[LOCAL] : Selected Host Key Algo = ssh-dss
[LOCAL] : Available Remote Send Ciphers = aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
[LOCAL] : Selected Send Cipher = aes256-ctr
[LOCAL] : Available Remote Recv Ciphers = aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
[LOCAL] : Selected Recv Cipher = aes256-ctr
[LOCAL] : Available Remote Send Macs = hmac-md5-etm@openssh.com,hmac-sha1-e...60@openssh.com,hmac-sha1-96,hmac-md5-96
[LOCAL] : Selected Send Mac = hmac-sha2-512
[LOCAL] : Available Remote Recv Macs = hmac-md5-etm@openssh.com,hmac-sha1-e...60@openssh.com,hmac-sha1-96,hmac-md5-96
[LOCAL] : Selected Recv Mac = hmac-sha2-512
[LOCAL] : Available Remote Compressors = none,zlib@openssh.com
[LOCAL] : Selected Compressor = none
[LOCAL] : Available Remote Decompressors = none,zlib@openssh.com
[LOCAL] : Selected Decompressor = none
[LOCAL] : Changing state from STATE_EXPECT_KEX_INIT to STATE_KEY_EXCHANGE
[LOCAL] : SSPI : Requesting full delegation
[LOCAL] : SSPI : SPN : host@xxx
[LOCAL] : SEND : KEXGSS_INIT [2501 bytes]
[LOCAL] : Changing state from STATE_KEY_EXCHANGE to STATE_CLOSED
[LOCAL] : Connected for 0 seconds, 737 bytes sent, 1677 bytes received
[LOCAL] : Stream has closed [CLOSE_TYPE_NONSPECIFIC] : Connection was reset.

Connection was reset.

[bgagnon]

Hello parodymshifter,

When you disabled GSSAPI (auth method), did you also disable the Kerberos key exchange methods (Connection / SSH2 category of Session Options)?

If not, and you do so now, what are the results?

Was Centrify just recently added to the equation?

If you want to disable these auth and key exchange methods in all sessions or the Default Session, see this tip.

[parodymshifter]

Brenda,

Thank you for the suggestion. Disabling both Kerberos options enabled me to login with the interactive password option.

Of course this is not the result I want. I want the GSSAPI option to work like it did before with 7.1.

I also noticed on the servers I'm trying to connect to that I'm getting this error at the time of the failed session starts:

sshd[22818]: 2014-01-14 13:06:15:\tfatal: dh_gen_key: group too small: 1024 (2*need 1024) [preauth]

I don't see this error when I use putty.

[parodymshifter]

I tried disabling GSSAPI and disabling both Kerberos and Kerberos Group Exchange options. That allowed me to use the password option. Then I re-enabled GSSAPI, moved it to the top of the list, and re-enabled Kerberos.

Then I tried connecting again. This time the GSSAPI option worked. So then I turned the Kerberos Group Exchange option back on. Connecting still works.

The one odd thing is that Kerberos and Kerberos Group Exchange were moved automagically from the top to the bottom of the list when I disabled them. When I turned them back on, I left them at the bottom.

I then tried moving them back to the top of the list. That's when the problem re-appears and I get "Session closed." instead of an open session.

I tried moving Kerberos to the 1st position below Diffie-Hellman and then it works again. Why would the order of these options have this kind of effect?

[parodymshifter]

Actually it was Diffie-Hellman-Group that is first in the list with Kerberos and Kerberos (Group Exchange) anywhere after that.

[bgagnon]

Hello parodymshifter,

SecureCRT will try the key exchange methods in order.

I believe what your tests have shown is that Diffie-Hellman-Group succeeds for key exchange, but not Kerberos.

Is Kerberos key exchange needed?

[parodymshifter]

Brenda,

I'm not sure what's required at this point. What I am most concerned about is that without changing any of the settings, when I updated from 7.1 to 7.2 something broke. Either Kerberos was working before with 7.1 and the new release fails to use it properly, or the new software changed the order without my intervention. Either way there is something wrong somewhere.

Why would it work with 7.1 and now be broken with 7.2?

[bgagnon]

Hello parodymshifter,

Unless you can post (or send) trace options output from v7.1, I am not sure that question can be answered.

Look at the available kex methods from the original v7.2 trace options output you posted:

[LOCAL] : Available Remote Kex Methods = gss-gex-sha1-toWM5Slw5Ew8Mqkay+al2g==,gss-group1-sha1-toWM5Slw5Ew8Mqkay+al2g==,gss-group14-sha1-toWM5Slw5Ew8Mqkay+al2g==,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1

Can you confirm the *exact same* key exchange methods were available when you connected with v7.1?

[bgagnon]

Hello parodymshifter,

I have submitted this behavior for investigation by the development team. Should progress be made toward a resolution, or further information be requested, I will post in this thread.

If you prefer direct e-mail notification, contact support@vandyke.com and include "Bug Report - Forum Thread #11353" in the subject line.

[parodymshifter]

On my 7.1 configuration I have
GSSAPI (only) selected for this session with keys in this order:

Kerberos
Kerberos (Group Exchange)
Diffie-hellman-group
diffie-hellman-group14
diffie-hellman

Here's the trace from 7.1

SecureCRT - Version 7.1.0 (x64 build 208)
[LOCAL] : Using protocol SSH2
[LOCAL] : RECV : Remote Identifier = 'SSH-2.0-OpenSSH_6.2'
[LOCAL] : CAP : Remote can re-key
[LOCAL] : CAP : Remote sends language in password change requests
[LOCAL] : CAP : Remote sends algorithm name in PK_OK packets
[LOCAL] : CAP : Remote sends algorithm name in public key packets
[LOCAL] : CAP : Remote sends algorithm name in signatures
[LOCAL] : CAP : Remote sends error text in open failure packets
[LOCAL] : CAP : Remote sends name in service accept packets
[LOCAL] : CAP : Remote includes port number in x11 open packets
[LOCAL] : CAP : Remote uses 160 bit keys for SHA1 MAC
[LOCAL] : CAP : Remote supports new diffie-hellman group exchange messages
[LOCAL] : CAP : Remote correctly handles unknown SFTP extensions
[LOCAL] : CAP : Remote correctly encodes OID for gssapi
[LOCAL] : CAP : Remote correctly uses connected addresses in forwarded-tcpip requests
[LOCAL] : CAP : Remote can do SFTP version 4
[LOCAL] : CAP : Remote x.509v3 uses ASN.1 encoding for DSA signatures
[LOCAL] : CAP : Remote correctly handles zlib@openssh.com
[LOCAL] : SSPI : Requesting full delegation
[LOCAL] : SSPI : [Kerberos] SPN : host@xxx.yyy.com
[LOCAL] : SSPI : Requesting full delegation
[LOCAL] : SSPI : [Kerberos (Group Exchange)] SPN : host@xxx.yyy.com
[LOCAL] : SEND : KEXINIT
[LOCAL] : RECV : Read kexinit
[LOCAL] : Available Remote Kex Methods = gss-gex-sha1-toWM5Slw5Ew8Mqkay+al2g==,gss-group1-sha1-toWM5Slw5Ew8Mqkay+al2g==,gss-group14-sha1-toWM5Slw5Ew8Mqkay+al2g==,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
[LOCAL] : Selected Kex Method = gss-group1-sha1-toWM5Slw5Ew8Mqkay+al2g==
[LOCAL] : Available Remote Host Key Algos = ssh-rsa,ssh-dss,ssh-dss
[LOCAL] : Selected Host Key Algo = ssh-dss
[LOCAL] : Available Remote Send Ciphers = aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
[LOCAL] : Selected Send Cipher = aes256-ctr
[LOCAL] : Available Remote Recv Ciphers = aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
[LOCAL] : Selected Recv Cipher = aes256-ctr
[LOCAL] : Available Remote Send Macs = hmac-md5-etm@openssh.com,hmac-sha1-e...60@openssh.com,hmac-sha1-96,hmac-md5-96
[LOCAL] : Selected Send Mac = hmac-sha1
[LOCAL] : Available Remote Recv Macs = hmac-md5-etm@openssh.com,hmac-sha1-e...60@openssh.com,hmac-sha1-96,hmac-md5-96
[LOCAL] : Selected Recv Mac = hmac-sha1
[LOCAL] : Available Remote Compressors = none,zlib@openssh.com
[LOCAL] : Selected Compressor = none
[LOCAL] : Available Remote Decompressors = none,zlib@openssh.com
[LOCAL] : Selected Decompressor = none
[LOCAL] : Changing state from STATE_EXPECT_KEX_INIT to STATE_KEY_EXCHANGE
[LOCAL] : SSPI : Requesting full delegation
[LOCAL] : SSPI : SPN : host@xxx.yyy.com
[LOCAL] : SEND : KEXGSS_INIT [2353 bytes]
[LOCAL] : RECV : KEXGSS_COMPLETE
[LOCAL] : GSS : The delegation request failed; credentials will not be forwardable.
[LOCAL] : Changing state from STATE_KEY_EXCHANGE to STATE_READY_FOR_NEW_KEYS
[LOCAL] : SEND : NEWKEYS
[LOCAL] : Changing state from STATE_READY_FOR_NEW_KEYS to STATE_EXPECT_NEWKEYS
[LOCAL] : RECV : NEWKEYS
[LOCAL] : Changing state from STATE_EXPECT_NEWKEYS to STATE_CONNECTION
[LOCAL] : SEND: SERVICE_REQUEST[ssh-userauth]
[LOCAL] : RECV: SERVICE_ACCEPT[ssh-userauth] -- OK
[LOCAL] : SENT : USERAUTH_REQUEST [gssapi-keyex]
[LOCAL] : RECV : SSH_MSG_USERAUTH_BANNER
[LOCAL] : RECV : AUTH_SUCCESS
Red Hat Enterprise Linux Server release 6.5 (Santiago)
Kernel 2.6.32-431.1.2.el6.x86_64 on an x86_64 @ Tue Jan 14 2014 18:05:14

[LOCAL] : SEND[0]: SSH_MSG_CHANNEL_OPEN('session')
[LOCAL] : SEND[0]: Pty Request (rows: 45, cols: 175)
[LOCAL] : RECV[0]: pty request succeeded
[LOCAL] : SEND[0]: x11 forwarding request
[LOCAL] : RECV[0]: x11 request failed
[LOCAL] : SEND[0]: agent forwarding request
[LOCAL] : RECV[0]: agent request succeeded
[LOCAL] : SEND[0]: shell request
[LOCAL] : RECV[0]: shell request succeeded

[bgagnon]

Hi parodymshifter,

Thanks for the v7.1 trace options output.

That shows us what we needed to know.

In v7.2, we added SHA-2 MAC support:

Changes in SecureFX 7.2 (Beta 1) -- October 8, 2013
---------------------------------------------------

New features:

/snip

• SSH2: Added support for SHA-2 MAC algorithms.

Always wanting the more secure option, they are the first algorithms tried in SecureCRT.

If you disable/reorder SHA2-512 and SHA2-256 in the Connection / SSH2 / Advanced category of Session Options, MAC section (and re-enable GSSAPI/Kerberos as desired), what are the results?

Can you verify in the server logs that is why the prior configuration in v7.2 failed?

[bgagnon]

Hello parodymshifter,

It has been a few days since my last post and I wanted to touch base to see if you had a chance to try the suggestion:

If you disable/reorder SHA2-512 and SHA2-256 in the Connection / SSH2 / Advanced category of Session Options, MAC section (and re-enable GSSAPI/Kerberos as desired), what are the results?

Also, I wondered if you were able to obtain a server-side log of the SecureCRT v7.1 and v7.2 connection results?

If so, since the log file could contain sensitive data, could you send it to support@vandyke.com (as an attachment to the email) and include "Attn Brenda - Forum Thread #11353" in the subject line?

Or, if you are unable to provide the server logs, were you able to analyze them yourself for the differences between the SecureCRT version 7.1 and version 7.2 connection results?

[sbaker]

I have the same behavior occurring as the person above.
My connection is an ssh to a solaris system and get the 'connection closed' with the error
'dh_gen_key: group too small: 1024 (2*need 1024)'
found in the solaris logs.

I changed the MAC order of the SHA2-512 and SHA2-256,
now SHA2-256 is first, and I no longer receive the error and can login.

Odd.
I have connections to other solaris systems that are using the default order
and they work just fine.
I can't see anything different about this connection.
The solaris systems are supposed to be identically configured.
I check the mac order on the other connection properties hasn't changed.

[jjh]

Hi sbaker.

Are the versions of OpenSSH on each of the Solaris machines
identical?

Thanks

JJH