Check for invalid TLS handshake