CryptoLyzer: A comprehensive cryptographic settings analyzer

CryptoLyzer is a multiprotocol cryptographic settings analyzer with SSL/TLS, SSH, and HTTP header analysis ability. The main purpose of the tool is to tell you what kind of cryptographic-related settings are enabled on a client or server.

If you are not interested in the principles behind the project, but in the practice, feel free to skip the next section and jump to the Practice section.

Principles

Rationale

There are many notable open-source projects (SSLyze, CipherScan, testssl.sh, tls-scan, …) and several SaaS solutions (CryptCheck, CypherCraft, Hardenize, ImmuniWeb, Mozilla Observatory, SSL Labs, …) to do a security setting analysis, especially when we are talking about TLS, which is the most common and popular cryptographic protocol. However, most of these tools heavily depend on one or more versions of one or more cryptographic protocol libraries, like GnuTLS, OpenSSL, or wolfSSL. But why is this such a problem?

The minor problem is the dependency easily stucks them in SSL/TLS/DTLS as other cryptographic protocols (eg: IPSec VPN, Kerberos, OpenVPN, SSH, …) can not be implemented directly by these libraries. Supporting them with the analyzer application needs extra effort. Anyway, most of the analysis of the cryptographic setting does not require any cryptography because before the parties could agree on the cryptographic algorithms they use plain text.

The major problem is the fact that analysis should test special and corner cases of the protocol that are intentionally triggered. It is hard to do that with a cryptographic protocol library, which was designed for production not for penetration testing or settings analysis. During an analysis, connections are tried to be established with hardly supported, experimental, obsoleted, or even deprecated mechanisms or algorithms to identify which ones are supported by the given client or server implementation. These mechanisms and algorithms may or may not be supported by the latest or any version of any cryptographic protocol implementations.

That is why most of the existing tools require special build(s) of the dependent library where all the protocol versions and algorithms of the good old days are reenabled to make the chance to set up these libraries to offer them as clients or as servers. But what if we want to test an algorithm or a mechanism that has never been implemented by the dependent cryptographic library?

It is not just a theory. A special fork of OpenSSL, maintained by Pluralsight author Peter Mosmans, aims to have as many ciphers as possible. This fork is used and recommended by Mozilla Cipherscan, however, it can offer less than two hundred cipher suites, but there are more than three hundred in the different RFCs according to Cipher Suite Info. The majority of them are weak or insecure, which makes it particularly important to be part of the analysis. In addition, it is also true that there are cipher suites that are not on the Cipher Suite Info list, for instance, Russian standard (GOST) cipher suites. These are rarely used cipher suites, but there is an OpenSSL engine that implements them, so they should be checked.

The situation is similar in the case of other parts of the TLS protocol, like extensions. It is unlikely that a cryptographic library will support each extension, but some of them are possibly implemented by some of the libraries. For instance, multiple certificate status request extension and Diffie-Hellman key exchange in TLS 1.3 are not implemented by OpenSSL 1.x but implemented by GnuTLS and WolfSSL. Another example can be the X.509 certificate-based authentication in SSH protocol, which is supported by Tectia SSH and has an open-source implementation but is not supported by the default OpenSSH. Even though they are not so common a comprehensive settings analyzer tool should run checks against them.

Summing up the main reason for establishing a greenfield project instead of contributing to an existing one is the opinion, that an analysis is mostly testing when we trigger special and corner cases, hardly supported, experimental, obsoleted, or even deprecated mechanisms and algorithms, so a comprehensive analyzer should be implemented independently from current cryptographic protocol implementations as much as possible.

Goals

CrypytoLyzer focuses on comprehensiveness, multi-protocol ability, and library independence. To do so cryptographic protocols clients are implemented focusing only on the necessary parts (usually the handshake). This kind of client can check a server against rarely or privately used, deprecated, or completely insecure algorithms and mechanisms.

  • Zero Trust
    • should not require any privilege during the analysis
    • Should not require any credentials (except it is strictly necessary)
    • should not provide any information for 3rd party
  • Completeness
    • should be able to support as many cryptographic protocols (DTLS, IPSec, OpenVPN, TLS, SSL, SSH, …) as possible
    • should be able to handle special/corner cases of the cryptographic protocols
  • Usability
    • should work both in public and private networks
    • should be able to give a human-readable summary (Markdown)
    • should be able to give machine-readable detailed output (JSON)
    • should be customizable to meet special needs (Python library)

The existing solutions are focused on only one cryptographic protocol (TLS or SSH) despite the fact, that all the cryptographic protocols have the same building blocks (authentication, key exchange, symmetric ciphers, integrity), so they can be analyzed in the same (or almost the same) way, only the information which the analysis is based on can be acquired differently.

Features

The goals above are just goals, not the current state of the development, particularly since the actual version number is 0.8.0 at the time of writing. The supported cryptographic protocol families, for now, are SSL/TLS and SSH. You can read the comparative analysis with either open-source or proprietary (SaaS) solutions below. I wanted to stay as objective as an author can with the competitors, but forgive me if I have not managed to.

Transport Layer Security (TLS)

Specialties

Cryptolyzer supports both the early and deprecated Secure Socket Layer (SSL) 2.0 and each Transport Layer Security version up to the TLS 1.3 version including draft versions. Some features although be checked hardly or difficultly using the most popular SSL/TLS implementations (eg: GnuTLS, LibreSSL, OpenSSL, wolfSSL, …) or just not in the scope of them. The checks are the specialties of Cryptolyzer such as supporting each cipher suite discussed on ciphersuite.info (and more) including the Russian governmental standard (GOST) cipher suites, Diffie-Hellman parameter checks, JA3 tag generation, …

  Crypto Lyzer Qualys SSL Labs Immuni Web Hard enize Moz. Obs. testssl .sh SSL-yze  
TLS 1.3 Draft Versions
TLS 1.3 DH Parameters
TLS GOST Ciphers
TLS PQC Curves
TLS DH Param Prime Check
TLS well-known DH Param Recognition
TLS DHE Param Reuse Check
TLS ECDH Param Reuse Check 1.0.x
SSH DH Param Check
SSH host certificate
DNSSEC key Extract and Analysis
DNS TXT Record (DMARC, SPF, MTA-STS, TLSRPT) Parsing
JA3 tag Generation
JA3 tag Decode
HASSH tag Generation
HASSH tag Decode
  94% 19% 0% 19% 0% 0% 25% 6%

Opportunistic TLS

Several application-layer protocols have an extension to upgrade a plain text connection to an encrypted one (TLS) without using a separate port referred to as opportunistic TLS (STARTTLS). This mechanism is supported by the analyzer in the case of most protocols.

  Crypto Lyzer Qualys SSL Labs Immuni Web Hard enize Moz. Obs. testssl .sh SSL-yze
FTP
IMAP
LDAP
LMTP
MySQL
NNTP
OpenVPN
POP3
PostgreSQL
RDP
Sieve
SMTP
XMPP (Jabber)
  100% 0% 8% 15% 0% 85% 61%

X.509 Certificates

There are X.509 related mechanisms in TLS that can have a serious impact on security in particular the revocation checking which is a weak point of the public key infrastructure (PKI). Cryptolyzer performs as well as its competitors in this field.

  Crypto Lyzer Qualys SSL Labs Immuni Web Hard enize Moz. Obs. testssl .sh SSL-yze
OCSP Stapling
OCSP Must Staple
Certificate Transparency
Multiple Trusted Root CA Stores
Revocation Status
  100% 100 80% 100% 40% 80% 80%

Extensions

There are several extensions for the original TLS specifications which have both security and compatibility considerations. A high number of extension types can be analyzed by CryptoLyzer compared to most of the competitors.

TODO

  Crypto Lyzer Qualys SSL Labs Immuni Web Hard enize Moz. Obs. testssl .sh SSL-yze
Fallback SCSV
TLS Clock Skew
Compression
Secure Renegotiation
Insecure Renegotiation 1.0.x
Session Caching
Session Ticketing
Elliptic Curves
EC Point Format
App. Layer Proto. Negotiation
Next Protocol Negotiation
Extended Master Secret
Encrypt-then-mac
GREASE check 1.0.x
Handshake Length Intolerance 1.0.x
  80% 73% 40% 0% 0% 73% 53%

Secure Shell (SSH)

In addition to TLS, there are some other cryptographic protocols, such as SSH. The cryptographic protocols have the same building blocks (authentication, key exchange, symmetric ciphers, integrity), so they can be analyzed in the same manner and it can be done with the same tool using CryptoLyzer without compromise.

  Crypto Lyzer Moz. Obs. Rebex SSH Check SSH Conf. Auditor ssh-audit ssh-scan
Algorithms
Host Keys
Host Certificates
X.509 Certificates
DH Param Check
Banner Analysis
Client Simulation
  86% 71% 14% 29% 29% 43%

HTTP Headers

Security

There are security mechanisms in the application layer too, not only in the transport layer. The HTTP protocol is a good example of that as there are encryption (HSTS), authentication (Expect-CT, Expect-Staple), and content integrity (subresource integrity, content security policy) related headers which are parsed by CryptoLyzer in detail.

  Crypto Lyzer Qualys SSL Labs Immuni Web Hard enize Moz. Obs. testssl .sh SSL-yze
Content Security Policy
CSP Report-To
Expect Certificate Transparency
Expect OCSP Staple
HTTP Strict Transport Security
HTTP Public Key Pinning
Referrer Policy
Subresource Integrity
X Content Type Options
X Frame Options
X XSS Protection
  100% 18% 36% 55% 82% 18% 18%

Generic

Several HTTP headers are not closely related to security. Although, some among them, especially caching headers (Age, Cache-Control, Date, ETag, Expires, Last-Modified), can have significance when the obsolescence of security-related information (eg: CRL) is indicated by caching headers.

  Crypto Lyzer Qualys SSL Labs Immuni Web Hard enize Moz. Obs. testssl .sh SSL-yze
Content-Type
Server
Application banner 1.1.x
Age
Cache-Control
Date
ETag
Expires
Last-Modified
Pragma
Proxy banner 1.1.x
Set-Cookie
  83% 0% 8% 8% 75% 50% 0%

DNS Records

Nowadays information related to security settings is stored in DNS records especially, but not exclusively in the case of the SMTP protocol. These records can be analyzed by CryptoLyzer and the necessary implementations are planned to be done soon.

TODO

  Crypto Lyzer Qualys SSL Labs Immuni Web Hard enize Moz. Obs. testssl .sh SSL-yze
DNS CAA 1.0.x
MTA-STS
DANE 1.0.x
SPF
DMARC
TLS-RPT
DNSKEY
RRSIG
DS
  89% 11% 11% 78% 11% 11% 11%

Generic

Usability is another field where Cryptolyzer should evolve. Comprehensiveness is not enough, weaknesses, and vulnerabilities should be highlighted to make the analysis results more easily understandable.

  Crypto Lyzer Qualys SSL Labs Immuni Web Hard enize Moz. Obs. testssl .sh SSL-yze
Client Simulation
Weakness Check
Vulnerability Check
Highlights in Output
  75% 100% 75% 50% 75% 100% 25%

Other protocols

Some cryptographic protocols use the variant of the TLS protocol. Some protocols wrap the original TLS protocol such as OpenVPN, QUIC, and there is a modified version of TLS, according to the need of UDP, called DTLS. Their support would be implemented relatively easily in CryptoLyzer in the future.

Practice

Installation

CryptoLyzer can be easily installed from The Python Package Index (PyPi)

$ pip install cryptolyzer

or via Docker from Docker Hub Container Image Library (docker hub)

$ docker pull coroner/cryptolyzer

and there are prebuilt packages for DEB (Debian, Ubuntu, …) and RPM (Fedora, RHEL, SUSE) based systems.

Usage

The command-line interface is slightly similar to the OpenSSL command-line tool. On each level of subcommands, comprehensive help can be given using the –help option. The analysis subject can be given multiple times as an argument in URL format. Most of the time scheme and port part of the URL has default values according to the analyzer (eg: tls, ssh).

$ cryptolyze tls all www.example.com

$ cryptolyze tls1_2 ciphers https://www.example.com:443

$ cryptolyze ssh all www.example.com

$ cryptolyze ssh2 ciphers ssh://www.example.com:22

$ cryptolyze dns dnssec example.com

$ cryptolyze dns mail example.com

$ cryptolyze http headers http://www.example.com/path?query#fragment

or can be used via Docker

$ docker run --rm coroner/cryptolyzer tls all www.example.com

$ docker run --rm coroner/cryptolyzer tls1_2 ciphers https://www.example.com:443

$ docker run --rm coroner/cryptolyzer ssh all www.example.com

$ docker run --rm coroner/cryptolyzer ssh2 ciphers ssh://www.example.com:22

$ docker run --rm coroner/cryptolyzer dns dnssec example.com

$ docker run --rm coroner/cryptolyzer dns mail example.com

$ docker run --rm coroner/cryptolyzer http headers http://www.example.com/path?query#fragment

CryptoLyzer: A comprehensive cryptographic settings analyzer is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License

Updated: