2 Replies Latest reply on Oct 10, 2016 12:34 AM by r0m

    Broken S2S session (hazelcast cluster)

    r0m

      Hi,

       

      I have just deployed a cluster (hazelcast v2.2.0) of Openfire server (v4.0.3) on my 3 CentOS servers.

      I have also deployed another standalone Openfire server which plays the role of external server which etablished a S2S session with one node of the cluster.

      All work fine except one thing : when I restart service on one node the S2S session seems to be broken with standalone server and client (Spark v2.6.3) which is connected on it can not join again "conference.<domain>" host by cluster.

       

      The only thing that permits to have a steady situation is to disconnect S2S session through web interface on the server which have etablished the session on first.

      For example, if the standalone server is the first which etablished a session with one node of the cluster I must disconnect the S2S from web interface of standalone node on "sessions/server sessions" menu (and vice versa).

       

      Why S2S session can not be etablished again ?

       

      Any help would be appreciated

        • Re: Broken S2S session (hazelcast cluster)
          r0m

          Here I join logs of node and standalone server  (debug mode enabled on all openifre server) which have a broken session after I restarted openfire service on node.

          Client connected on standalone server try to connect to "conference.a.fr" (domain host by cluser).

           

          NODE log :

          2016.10.03 07:22:30 INFO  [Server SR - 20471351]: org.jivesoftware.openfire.net.SocketReadingMode - STARTTLS negotiation (with: org.jivesoftware.openfire.net.SocketConnection@4422ad socket: Socket[addr=/192.168.4.251,port=47297,localport=5269] session: org.jivesoftware.openfire.session.LocalIncomingServerSession@1753ae1 status: 1 address: a.fr/g2ajkz4uw id: g2ajkz4uw) failed.
          javax.net.ssl.SSLHandshakeException: The peer closed the connection while performing a TLS handshake.
                  at org.jivesoftware.openfire.net.TLSStreamHandler.doHandshake(TLSStreamHandler.java:228)
                  at org.jivesoftware.openfire.net.TLSStreamHandler.start(TLSStreamHandler.java:178)
                  at org.jivesoftware.openfire.net.SocketConnection.startTLS(SocketConnection.java:195)
                  at org.jivesoftware.openfire.net.SocketReadingMode.negotiateTLS(SocketReadingMode.java:87)
                  at org.jivesoftware.openfire.net.BlockingReadingMode.readStream(BlockingReadingMode.java:138)
                  at org.jivesoftware.openfire.net.BlockingReadingMode.run(BlockingReadingMode.java:76)
                  at org.jivesoftware.openfire.net.SocketReader.run(SocketReader.java:145)
                  at java.lang.Thread.run(Thread.java:745)
          

           

          STANDALONE server log :

          2016.10.03 07:22:02 DEBUG [NioProcessor-1]: org.apache.mina.filter.ssl.SslFilter - Session Server[9](SSL): Message received : HeapBuffer[pos=0 lim=149 cap=256: 17 03 01 00 90 67 59 C9 5B CC 16 84 84 78 65 1D...]
          2016.10.03 07:22:02 DEBUG [NioProcessor-1]: org.apache.mina.filter.ssl.SslHandler - Session Server[9](SSL) Processing the received message
          2016.10.03 07:22:02 DEBUG [NioProcessor-1]: org.apache.mina.filter.ssl.SslFilter - Session Server[9](SSL): Processing the SSL Data
          2016.10.03 07:22:02 DEBUG [NioProcessor-1]: org.apache.mina.filter.executor.OrderedThreadPoolExecutor - Adding event MESSAGE_RECEIVED to session 9
          Queue : [MESSAGE_RECEIVED, ]
          
          
          2016.10.03 07:22:02 DEBUG [socket_c2s-thread-3]: org.apache.mina.core.filterchain.IoFilterEvent - Firing a MESSAGE_RECEIVED event for session 9
          2016.10.03 07:22:02 DEBUG [socket_c2s-thread-3]: org.apache.mina.filter.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 9
          2016.10.03 07:22:02 DEBUG [socket_c2s-thread-3]: org.apache.mina.filter.ssl.SslFilter - Session Server[9](SSL): Writing Message : WriteRequest: HeapBuffer[pos=0 lim=153 cap=4096: 3C 69 71 20 74 79 70 65 3D 22 72 65 73 75 6C 74...]
          2016.10.03 07:22:02 DEBUG [socket_c2s-thread-3]: org.apache.mina.core.filterchain.IoFilterEvent - Event MESSAGE_RECEIVED has been fired for session 9
          2016.10.03 07:22:02 DEBUG [NioProcessor-1]: org.apache.mina.filter.executor.OrderedThreadPoolExecutor - Adding event MESSAGE_SENT to session 9
          Queue : [MESSAGE_SENT, ]
          
          
          2016.10.03 07:22:02 DEBUG [socket_c2s-thread-2]: org.apache.mina.core.filterchain.IoFilterEvent - Firing a MESSAGE_SENT event for session 9
          2016.10.03 07:22:02 DEBUG [socket_c2s-thread-2]: org.apache.mina.core.filterchain.IoFilterEvent - Event MESSAGE_SENT has been fired for session 9
          2016.10.03 07:22:20 DEBUG [TaskEngine-pool-1621]: org.jivesoftware.util.log.util.CommonsLogFactory - Closing statement a2e6eb (belonging to connection 11) automatically
          2016.10.03 07:22:30 DEBUG [NioProcessor-1]: org.apache.mina.filter.ssl.SslFilter - Session Server[9](SSL): Message received : HeapBuffer[pos=0 lim=149 cap=256: 17 03 01 00 90 78 20 37 91 E7 A5 FE D8 B9 5D C0...]
          2016.10.03 07:22:30 DEBUG [NioProcessor-1]: org.apache.mina.filter.ssl.SslHandler - Session Server[9](SSL) Processing the received message
          2016.10.03 07:22:30 DEBUG [NioProcessor-1]: org.apache.mina.filter.ssl.SslFilter - Session Server[9](SSL): Processing the SSL Data
          2016.10.03 07:22:30 DEBUG [NioProcessor-1]: org.apache.mina.filter.executor.OrderedThreadPoolExecutor - Adding event MESSAGE_RECEIVED to session 9
          Queue : [MESSAGE_RECEIVED, ]
          
          
          2016.10.03 07:22:30 DEBUG [socket_c2s-thread-3]: org.apache.mina.core.filterchain.IoFilterEvent - Firing a MESSAGE_RECEIVED event for session 9
          2016.10.03 07:22:30 DEBUG [socket_c2s-thread-3]: org.apache.mina.filter.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 9
          2016.10.03 07:22:30 DEBUG [socket_c2s-thread-3]: org.apache.mina.core.filterchain.IoFilterEvent - Event MESSAGE_RECEIVED has been fired for session 9
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.session.LocalOutgoingServerSession[Authenticate local domain: 'b.uk' to remote domain: 'conference.a.fr'] - Start domain authentication ...
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.session.LocalOutgoingServerSession[Authenticate local domain: 'b.uk' to remote domain: 'conference.a.fr'] - Searching for pre-existing outgoing sessions to the remote domain (if one exists, it will be re-used) ...
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.session.LocalOutgoingServerSession[Authenticate local domain: 'b.uk' to remote domain: 'conference.a.fr'] - There are no pre-existing outgoing sessions to the remote domain itself. Searching for pre-existing outgoing sessions to super- or subdomains of the remote domain (if one exists, it might be re-usable) ...
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.session.LocalOutgoingServerSession[Authenticate local domain: 'b.uk' to remote domain: 'conference.a.fr'] - There are no pre-existing session to other domains hosted on the remote domain.
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.session.LocalOutgoingServerSession[Authenticate local domain: 'b.uk' to remote domain: 'conference.a.fr'] - Unable to re-use an existing session. Creating a new session ...
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.session.LocalOutgoingServerSession[Create outgoing session for: b.uk to conference.a.fr] - Creating new session...
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.session.LocalOutgoingServerSession[Create outgoing session for: b.uk to conference.a.fr] - Creating plain socket connection to a host that belongs to the remote XMPP domain.
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.net.SocketUtil - Creating a socket connection to XMPP domain 'conference.a.fr' ...
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.net.SocketUtil - Use DNS to resolve remote hosts for the provided XMPP domain 'conference.a.fr' (default port: 5269) ...
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.net.SocketUtil - Found 1 host(s) for XMPP domain 'conference.a.fr'.
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.net.SocketUtil - Trying to create socket connection to XMPP domain 'conference.a.fr' using remote host: chat.a.fr:5269 (blocks up to 120000 ms) ...
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.net.SocketUtil - Successfully created socket connection to XMPP domain 'conference.a.fr' using remote host: chat.a.fr:5269!
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.session.LocalOutgoingServerSession[Create outgoing session for: b.uk to conference.a.fr] - Send the stream header and wait for response...
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.session.LocalOutgoingServerSession[Create outgoing session for: b.uk to conference.a.fr] - Got a response (stream ID: g2ajkz4uw, version: 1.0). Check if the remote server is XMPP 1.0 compliant...
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.session.LocalOutgoingServerSession[Create outgoing session for: b.uk to conference.a.fr] - The remote server is XMPP 1.0 compliant (or at least reports to be).
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.session.LocalOutgoingServerSession[Create outgoing session for: b.uk to conference.a.fr] - Processing stream features of the remote domain...
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.session.LocalOutgoingServerSession[Create outgoing session for: b.uk to conference.a.fr] - Check if both us as well as the remote server have enabled STARTTLS and/or dialback ...
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.session.LocalOutgoingServerSession[Create outgoing session for: b.uk to conference.a.fr] - Both us and the remote server support the STARTTLS feature. Secure and authenticate the connection with TLS & SASL...
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.session.LocalOutgoingServerSession[Secure/Authenticate connection for: b.uk to: conference.a.fr] - Securing and authenticating connection ...
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.session.LocalOutgoingServerSession[Secure/Authenticate connection for: b.uk to: conference.a.fr] - Indicating we want TLS and wait for response.
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.session.LocalOutgoingServerSession[Secure/Authenticate connection for: b.uk to: conference.a.fr] - Received 'proceed' from remote server. Negotiating TLS...
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.keystore.OpenfireX509TrustManager - Constructed trust manager. Number of trusted issuers: 173, accepts self-signed: false, checks validity: true
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.keystore.OpenfireX509TrustManager - Attempting to verify a chain of 1 certificates.
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.keystore.OpenfireX509TrustManager - Validating chain with 1 certificates, using 169 trust anchors.
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.keystore.OpenfireX509TrustManager - ** Chain to be validated:
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.keystore.OpenfireX509TrustManager -    length: 1
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.keystore.OpenfireX509TrustManager -  Certificate[0] (valid from Thu Sep 22 09:27:14 UTC 2016 to Tue Sep 21 09:27:14 UTC 2021):
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.keystore.OpenfireX509TrustManager -    subjectDN: CN=a.fr
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.keystore.OpenfireX509TrustManager -    issuerDN: CN=a.fr
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.session.LocalOutgoingServerSession[Secure/Authenticate connection for: b.uk to: conference.a.fr] - TLS negotiation failed: General SSLEngine problem
          2016.10.03 07:22:30 INFO  [pool-19-thread-2]: org.jivesoftware.openfire.session.LocalOutgoingServerSession[Create outgoing session for: b.uk to conference.a.fr] - STARTTLS negotiation failed. Closing connection (without sending any data such as <failure/> or </stream>).
          javax.net.ssl.SSLHandshakeException: General SSLEngine problem
                  at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1431)
                  at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:535)
                  at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1214)
                  at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1186)
                  at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469)
                  at org.jivesoftware.openfire.net.TLSStreamHandler.doHandshake(TLSStreamHandler.java:293)
                  at org.jivesoftware.openfire.net.TLSStreamHandler.start(TLSStreamHandler.java:178)
                  at org.jivesoftware.openfire.net.SocketConnection.startTLS(SocketConnection.java:195)
                  at org.jivesoftware.openfire.session.LocalOutgoingServerSession.secureAndAuthenticate(LocalOutgoingServerSession.java:407)
                  at org.jivesoftware.openfire.session.LocalOutgoingServerSession.createOutgoingSession(LocalOutgoingServerSession.java:305)
                  at org.jivesoftware.openfire.session.LocalOutgoingServerSession.authenticateDomain(LocalOutgoingServerSession.java:210)
                  at org.jivesoftware.openfire.server.OutgoingSessionPromise$PacketsProcessor.sendPacket(OutgoingSessionPromise.java:267)
                  at org.jivesoftware.openfire.server.OutgoingSessionPromise$PacketsProcessor.run(OutgoingSessionPromise.java:243)
                  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
                  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
                  at java.lang.Thread.run(Thread.java:745)
          Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
                  at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
                  at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1728)
                  at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:304)
                  at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296)
                  at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1509)
                  at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216)
                  at sun.security.ssl.Handshaker.processLoop(Handshaker.java:979)
                  at sun.security.ssl.Handshaker$1.run(Handshaker.java:919)
                  at sun.security.ssl.Handshaker$1.run(Handshaker.java:916)
                  at java.security.AccessController.doPrivileged(Native Method)
                  at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1369)
                  at org.jivesoftware.openfire.net.TLSStreamHandler.doTasks(TLSStreamHandler.java:344)
                  at org.jivesoftware.openfire.net.TLSStreamHandler.doHandshake(TLSStreamHandler.java:254)
                  ... 10 more
          Caused by: java.security.cert.CertificateException: java.security.cert.CertPathBuilderException: Unable to find certificate chain.
                  at org.jivesoftware.openfire.keystore.OpenfireX509TrustManager.checkServerTrusted(OpenfireX509TrustManager.java:109)
                  at sun.security.ssl.AbstractTrustManagerWrapper.checkServerTrusted(SSLContextImpl.java:936)
                  at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1496)
                  ... 18 more
          Caused by: java.security.cert.CertPathBuilderException: Unable to find certificate chain.
                  at org.bouncycastle.jce.provider.PKIXCertPathBuilderSpi.engineBuild(Unknown Source)
                  at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
                  at org.jivesoftware.openfire.keystore.OpenfireX509TrustManager.checkChainTrusted(OpenfireX509TrustManager.java:262)
                  at org.jivesoftware.openfire.keystore.OpenfireX509TrustManager.checkServerTrusted(OpenfireX509TrustManager.java:105)
                  ... 20 more
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.session.LocalOutgoingServerSession[Create outgoing session for: b.uk to conference.a.fr] - Unable to create a new session. Going to try connecting using server dialback as a fallback.
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.server.ServerDialback[Acting as Originating Server: Create Outgoing Session from: b.uk to RS at: conference.a.fr (port: 5269)] - Creating new outgoing session...
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.net.SocketUtil - Creating a socket connection to XMPP domain 'conference.a.fr' ...
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.net.SocketUtil - Use DNS to resolve remote hosts for the provided XMPP domain 'conference.a.fr' (default port: 5269) ...
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.net.SocketUtil - Found 1 host(s) for XMPP domain 'conference.a.fr'.
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.net.SocketUtil - Trying to create socket connection to XMPP domain 'conference.a.fr' using remote host: chat.a.fr:5269 (blocks up to 120000 ms) ...
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.net.SocketUtil - Successfully created socket connection to XMPP domain 'conference.a.fr' using remote host: chat.a.fr:5269!
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.server.ServerDialback[Acting as Originating Server: Create Outgoing Session from: b.uk to RS at: conference.a.fr (port: 5269)] - Send the stream header and wait for response...
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.server.ServerDialback[Acting as Originating Server: Create Outgoing Session from: b.uk to RS at: conference.a.fr (port: 5269)] - Got a response. Check if the remote server supports dialback...
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.server.ServerDialback[Acting as Originating Server: Create Outgoing Session from: b.uk to RS at: conference.a.fr (port: 5269)] - Dialback seems to be supported by the remote server.
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.server.ServerDialback[Acting as Originating Server: Authenticate domain: b.uk with RS: conference.a.fr (id: 2aqbj72aj1)] - Authenticating domain ...
          2016.10.03 07:22:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.server.ServerDialback[Acting as Originating Server: Authenticate domain: b.uk with RS: conference.a.fr (id: 2aqbj72aj1)] - Sending dialback key and wait for the validation response...
          2016.10.03 07:23:04 DEBUG [NioProcessor-1]: org.apache.mina.filter.ssl.SslFilter - Session Server[9](SSL): Message received : HeapBuffer[pos=0 lim=37 cap=256: 17 03 01 00 20 B3 D2 F5 91 9F 30 DF EE CB 33 6D...]
          2016.10.03 07:23:04 DEBUG [NioProcessor-1]: org.apache.mina.filter.ssl.SslHandler - Session Server[9](SSL) Processing the received message
          2016.10.03 07:23:04 DEBUG [NioProcessor-1]: org.apache.mina.filter.ssl.SslFilter - Session Server[9](SSL): Processing the SSL Data
          2016.10.03 07:23:04 DEBUG [NioProcessor-1]: org.apache.mina.filter.executor.OrderedThreadPoolExecutor - Adding event MESSAGE_RECEIVED to session 9
          Queue : [MESSAGE_RECEIVED, ]
          
          
          2016.10.03 07:23:04 DEBUG [socket_c2s-thread-3]: org.apache.mina.core.filterchain.IoFilterEvent - Firing a MESSAGE_RECEIVED event for session 9
          2016.10.03 07:23:04 DEBUG [socket_c2s-thread-3]: org.apache.mina.filter.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 9
          2016.10.03 07:23:04 DEBUG [socket_c2s-thread-3]: org.apache.mina.core.filterchain.IoFilterEvent - Event MESSAGE_RECEIVED has been fired for session 9
          2016.10.03 07:23:34 DEBUG [NioProcessor-1]: org.apache.mina.filter.ssl.SslFilter - Session Server[9](SSL): Message received : HeapBuffer[pos=0 lim=37 cap=256: 17 03 01 00 20 D7 E2 C5 63 E0 8A 4D 4F 59 7E 1A...]
          2016.10.03 07:23:34 DEBUG [NioProcessor-1]: org.apache.mina.filter.ssl.SslHandler - Session Server[9](SSL) Processing the received message
          2016.10.03 07:23:34 DEBUG [NioProcessor-1]: org.apache.mina.filter.ssl.SslFilter - Session Server[9](SSL): Processing the SSL Data
          2016.10.03 07:23:34 DEBUG [NioProcessor-1]: org.apache.mina.filter.executor.OrderedThreadPoolExecutor - Adding event MESSAGE_RECEIVED to session 9
          Queue : [MESSAGE_RECEIVED, ]
          
          
          2016.10.03 07:23:34 DEBUG [socket_c2s-thread-2]: org.apache.mina.core.filterchain.IoFilterEvent - Firing a MESSAGE_RECEIVED event for session 9
          2016.10.03 07:23:34 DEBUG [socket_c2s-thread-2]: org.apache.mina.filter.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 9
          2016.10.03 07:23:34 DEBUG [socket_c2s-thread-2]: org.apache.mina.core.filterchain.IoFilterEvent - Event MESSAGE_RECEIVED has been fired for session 9
          2016.10.03 07:24:04 DEBUG [NioProcessor-1]: org.apache.mina.filter.ssl.SslFilter - Session Server[9](SSL): Message received : HeapBuffer[pos=0 lim=37 cap=128: 17 03 01 00 20 18 55 1C D0 B1 0B 29 AB 87 79 FB...]
          2016.10.03 07:24:04 DEBUG [NioProcessor-1]: org.apache.mina.filter.ssl.SslHandler - Session Server[9](SSL) Processing the received message
          2016.10.03 07:24:04 DEBUG [NioProcessor-1]: org.apache.mina.filter.ssl.SslFilter - Session Server[9](SSL): Processing the SSL Data
          2016.10.03 07:24:04 DEBUG [NioProcessor-1]: org.apache.mina.filter.executor.OrderedThreadPoolExecutor - Adding event MESSAGE_RECEIVED to session 9
          Queue : [MESSAGE_RECEIVED, ]
          
          
          2016.10.03 07:24:04 DEBUG [socket_c2s-thread-3]: org.apache.mina.core.filterchain.IoFilterEvent - Firing a MESSAGE_RECEIVED event for session 9
          2016.10.03 07:24:04 DEBUG [socket_c2s-thread-3]: org.apache.mina.filter.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 9
          2016.10.03 07:24:04 DEBUG [socket_c2s-thread-3]: org.apache.mina.core.filterchain.IoFilterEvent - Event MESSAGE_RECEIVED has been fired for session 9
          2016.10.03 07:24:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.server.ServerDialback[Acting as Originating Server: Authenticate domain: b.uk with RS: conference.a.fr (id: 2aqbj72aj1)] - Failed to authenticate domain: Time out waiting for validation response.
          2016.10.03 07:24:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.server.ServerDialback[Acting as Originating Server: Create Outgoing Session from: b.uk to RS at: conference.a.fr (port: 5269)] - Failed to authenticate the connection with dialback.
          2016.10.03 07:24:30 WARN  [pool-19-thread-2]: org.jivesoftware.openfire.server.ServerDialback[Acting as Originating Server: Create Outgoing Session from: b.uk to RS at: conference.a.fr (port: 5269)] - Unable to create a new outgoing session
          2016.10.03 07:24:30 WARN  [pool-19-thread-2]: org.jivesoftware.openfire.session.LocalOutgoingServerSession[Create outgoing session for: b.uk to conference.a.fr] - Unable to create a new session: Dialback (as a fallback) failed.
          2016.10.03 07:24:30 WARN  [pool-19-thread-2]: org.jivesoftware.openfire.session.LocalOutgoingServerSession[Authenticate local domain: 'b.uk' to remote domain: 'conference.a.fr'] - Unable to authenticate: Fail to create new session.
          2016.10.03 07:24:30 DEBUG [pool-19-thread-2]: org.apache.mina.filter.ssl.SslFilter - Session Server[9](SSL): Writing Message : WriteRequest: HeapBuffer[pos=0 lim=262 cap=4096: 3C 69 71 20 74 79 70 65 3D 22 65 72 72 6F 72 22...]
          2016.10.03 07:24:30 DEBUG [pool-19-thread-2]: org.jivesoftware.openfire.server.OutgoingSessionPromise - OutgoingSessionPromise: Error sending packet to remote server:
          <iq id="4DMc8-27" to="conference.a.fr" type="get" from="user1.uk@b.uk/Spark 2.6.3">
            <query xmlns="http://jabber.org/protocol/disco#info"/>
          </iq>
          java.lang.Exception: Failed to create connection to remote server
                  at org.jivesoftware.openfire.server.OutgoingSessionPromise$PacketsProcessor.sendPacket(OutgoingSessionPromise.java:279)
                  at org.jivesoftware.openfire.server.OutgoingSessionPromise$PacketsProcessor.run(OutgoingSessionPromise.java:243)
                  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
                  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
                  at java.lang.Thread.run(Thread.java:745)
          2016.10.03 07:24:30 DEBUG [NioProcessor-1]: org.apache.mina.filter.executor.OrderedThreadPoolExecutor - Adding event MESSAGE_SENT to session 9
          Queue : [MESSAGE_SENT, ]
          
          
          2016.10.03 07:24:30 DEBUG [socket_c2s-thread-2]: org.apache.mina.core.filterchain.IoFilterEvent - Firing a MESSAGE_SENT event for session 9
          2016.10.03 07:24:30 DEBUG [socket_c2s-thread-2]: org.apache.mina.core.filterchain.IoFilterEvent - Event MESSAGE_SENT has been fired for session 9
          2016.10.03 07:24:34 DEBUG [NioProcessor-1]: org.apache.mina.filter.ssl.SslFilter - Session Server[9](SSL): Message received : HeapBuffer[pos=0 lim=37 cap=128: 17 03 01 00 20 F0 3D 97 89 F3 06 64 93 4A B6 C8...]
          2016.10.03 07:24:34 DEBUG [NioProcessor-1]: org.apache.mina.filter.ssl.SslHandler - Session Server[9](SSL) Processing the received message
          2016.10.03 07:24:34 DEBUG [NioProcessor-1]: org.apache.mina.filter.ssl.SslFilter - Session Server[9](SSL): Processing the SSL Data
          2016.10.03 07:24:34 DEBUG [NioProcessor-1]: org.apache.mina.filter.executor.OrderedThreadPoolExecutor - Adding event MESSAGE_RECEIVED to session 9
          Queue : [MESSAGE_RECEIVED, ]
          
          
          2016.10.03 07:24:34 DEBUG [socket_c2s-thread-3]: org.apache.mina.core.filterchain.IoFilterEvent - Firing a MESSAGE_RECEIVED event for session 9
          2016.10.03 07:24:34 DEBUG [socket_c2s-thread-3]: org.apache.mina.filter.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 9
          2016.10.03 07:24:34 DEBUG [socket_c2s-thread-3]: org.apache.mina.core.filterchain.IoFilterEvent - Event MESSAGE_RECEIVED has been fired for session 9