The Digital Cat - SSH

Public key cryptography: OpenSSH private keys

2021-06-03T14:00:00+01:00

When you create standard RSA keys with ssh-keygen you end up with a private key in PEM format, and a public key in OpenSSH format. Both have been described in detail in my post Public key cryptography: RSA keys. In 2014, OpenSSH introduced a custom format for private keys that is apparently similar to PEM but is internally completely different. This format is used by default when you create ed25519 keys and it is expected to be the default format for all keys in the future, so it is worth having a look.

While investigating this topic I found a lot of misconceptions and wrong or partially wrong statements on Stack Overflow, so I hope this might be a comprehensive view of what this format is, its relationship with PEM, and the tools that you can use to manipulate it.

I'm not the first programmer to look into this, clearly, and I have to mention two posts that I read before writing this one: OpenSSH ed25519 private key file format written in December 2017 by Peter Lyons and The OpenSSH private key binary format, written in August 2020 by Marin Atanasov Nikolov. I'm sure many others have done this research but these are the resources that I found and I want to say a big thanks to both authors for sharing their findings. I will shamelessly use their results in the following explanation, as I hope others will do with what I'm writing here. Sharing knowledge is one of the best ways to help others.

Please note that all the private keys shown in this post have been trashed after I published it.

Note: as the word "key" can identify several different component of the systems I will describe, I will as much as possible use the words "private key" and "encryption key". The first is the key that we generate to be used in SSH, while the second is a parameter of a (symmetric) encryption algorithm.

KDFs and protection at rest¶

Describing the introduction of the new format, the OpenSSH changelog says

Add a new private key format that uses a bcrypt KDF to better
protect keys at rest. This format is used unconditionally for
Ed25519 keys, but may be requested when generating or saving
existing keys of other types via the -o ssh-keygen(1) option.
We intend to make the new format the default in the near future.
Details of the new format are in the PROTOCOL.key file.

Before we start dissecting the format, then, it is worth briefly discussing what a KDF is, what bcrypt is, and what it means to protect keys at rest.

Key Derivation Functions

Whenever a system is protected by a password you want to store the latter somewhere. This is clearly necessary to check the validity of the passwords that the user inputs and decide if you should grant access, but you shouldn't store the password in clear text, as a breach in the storage might compromise the whole system. The idea behind storing password securely is to run them through a hash function and store the hash: whenever someone inputs a password we can run the hash function again and compare the two hashes. However, we also want to prevent the attacker to be able to reconstruct the password from the hash, so we need a cryptographic hash function, which is a hash function with added requirements to prevent an easy inversion of the process.

The same strategy can be applied when it comes to encryption. An encryption system needs a key (a sequence of bits used to encrypt the message) and we need to derive it from the password given by the user. Encryption keys are required to have a specific length dictated by the encryption algorithm that we use, so hashing looks like a good solution, as all hashes generated by a given algorithm are by definition of the same size. AES, for example, one of the most widespread symmetric block ciphers, uses a key of exactly 128, 192, or 256 bits. Converting the password into a key of predetermined size is called stretching.

Any cryptographic system can be broken using a brute-force attack, as you can always test all possible inputs. In the case of login, we can just input all possible passwords until we get access to the system, while in the case of encryption we can try to decrypt using all possible keys until we obtain a meaningful result. This means that the most important thing we can do to protect such systems is to make brute-force attacks infeasible. This can be done increasing the key size (using more bits) but also using a slow stretching algorithm.

While hash functions created for things like digital signatures should be fast, then, hash functions that we use to obfuscate the password (for storage) or to create the key (for encryption/decryption) have to be very slow. The slowness of the processing can frustrate brute-force attacks and make them less effective is not infeasible. An example: at the current state of technology, you can easily hash 1 trillion passwords a second with a trivial expense, but if each one of those hashes takes 1 second you end up having to wait more than 31,000 years before you test all of them.

The process that converts a password into a key is called Key Derivation Function (KDF) and despite the name it is usually a complex algorithm and not a single mathematical function. PBKDF2 is an important KDF, defined as part of the specification PKCS #5, and it can use any pseudorandom function as part of the key stretching. An important feature of PBKDF2 is that it accepts an iteration count as input, that allows to slow down the process. As we just saw, this is the key to making the algorithm slower in order to adapt to the increasing computing power available to attackers.

bcrypt

The password-hashing function known as bcrypt was created in 1999 and is based on the Blowfish cipher created in 1993. Bcrypt is well know to be an extremely good choice thanks to the simple fact that its slowness can be increased tuning one of the parameters of the algorithm called "cost factor". This represents the number of iterations done in the setup of the underlying cipher, and its logarithmic nature makes easy to adapt the whole process to the increasing computational power available to attackers. This post attempts to estimate the time to hash a password of 15 characters with a cost of 30 (the maximum is actually 31) with a decent 2017 laptop (2.8 GHz Intel Core i7 16 GB RAM). The result turns out to be around 500 days which makes you understand that bcrypt won't die easily. It is important to note here that bcrypt is not a KDF, but a hash function. As such, it might be part of a KDF, but not replace the whole process.

Protection at rest

Protection at rest refers to the scheme that ensures data is secure when it is stored. Practically speaking, when it comes to SSH keys, we refer to the fact that an attacker that can physically access a key, for example stealing a laptop, actually owns an encrypted version of the key, which can't be used without first decrypting it. As the attacker is supposed to ignore the password used to encrypt the key, the only strategy they can use is to brute-force the key, and here is where the concept of protection at rest comes into play. Actually, the other strategy they can employ is to kidnap you and to force you to reveal the password, but this somehow falls outside the sphere of cryptographic security.

PEM format and protection at rest¶

Now that I clarified some terminology, let's have a look at what the standard PEM format does to store encrypted passwords. As I explained in my post Public key cryptography: RSA keys a PEM file contains a text header, a text footer, and some content. The content is always an ASN.1 structure created using DER and encoded using base64.

For encrypted private keys, the ASN.1 structure is created following a standard called PKCS #8. This standard uses an encryption scheme called PBES2 described in the specification PKCS #5, which uses a symmetric cipher and a password, previously converted into an encryption key using the KDF called PBKDF2. I hope at this point some if not all of these names ring a bell.

We can roughly sketch the process with the following steps:

Create the private key using the requested asymmetric algorithm (e.g. RSA or ED25519)
Encrypt the private key following PBES2
- Stretch the password into an encryption key using PBKDF2 with one of the possible hash functions and a random salt value
- Encrypt the private key using the newly created encryption key
Represent the encrypted key and the parameters used for PBKDF2 using ASN.1/DER
Encode the result with base64
Add a header and a footer that specify the nature of the content

Let's create an encrypted key with OpenSSL and analyse it. The command I used is

openssl genpkey -aes-256-cbc -algorithm RSA\
    -pkeyopt rsa_keygen_bits:4096 -pass pass:foobar\
    -out key_rsa_4096_openssl_pw

which creates a 4096 bits RSA key and encrypts it with AES using foobar as password. What I get is a file in the aforementioned PEM format

We can dump the ASN.1 content directly from the PEM format using openssl asn1parse

$ openssl asn1parse -inform pem -in key_rsa_4096_openssl_pw
    0:d=0  hl=4 l=2477 cons: SEQUENCE          
    4:d=1  hl=2 l=  87 cons: SEQUENCE          
    6:d=2  hl=2 l=   9 prim: OBJECT            :PBES2  1
   17:d=2  hl=2 l=  74 cons: SEQUENCE          
   19:d=3  hl=2 l=  41 cons: SEQUENCE          
   21:d=4  hl=2 l=   9 prim: OBJECT            :PBKDF2  2
   32:d=4  hl=2 l=  28 cons: SEQUENCE          
   34:d=5  hl=2 l=   8 prim: OCTET STRING      [HEX DUMP]:5BE04AE9442D08F0  4
   44:d=5  hl=2 l=   2 prim: INTEGER           :0800  5
   48:d=5  hl=2 l=  12 cons: SEQUENCE          
   50:d=6  hl=2 l=   8 prim: OBJECT            :hmacWithSHA256  6
   60:d=6  hl=2 l=   0 prim: NULL              
   62:d=3  hl=2 l=  29 cons: SEQUENCE          
   64:d=4  hl=2 l=   9 prim: OBJECT            :aes-256-cbc  3
   75:d=4  hl=2 l=  16 prim: OCTET STRING      [HEX DUMP]:88BD4E050F7D6691847BEAE813121BB0
   93:d=1  hl=4 l=2384 prim: OCTET STRING      [HEX DUMP]:93C719E39B382D[...]

Please note that I truncated the final OCTET STRING that contains the encrypted key as it is pretty long.

You can clearly see that this key is encrypted using PBES2 1 and PBKDF2 2. The algorithm used to encrypt the key is aes-256-cbc 3, as I asked. Specifically, this is AES with a key of 256 bits in CBC mode).

According to the PKCS #5 specification, the PBES2 block contains

PBES2-params ::= SEQUENCE {
       keyDerivationFunc AlgorithmIdentifier {{PBES2-KDFs}},
       encryptionScheme AlgorithmIdentifier {{PBES2-Encs}} }

and indeed we have PBKDF2 1 for keyDerivationFunc, and aes-256-cbc 3 for encryptionScheme. The sequence PBKDF2 is specified in the same document as

PBKDF2-params ::= SEQUENCE {
       salt CHOICE {
           specified OCTET STRING,
           otherSource AlgorithmIdentifier {{PBKDF2-SaltSources}}
       },
       iterationCount INTEGER (1..MAX),
       keyLength INTEGER (1..MAX) OPTIONAL,
       prf AlgorithmIdentifier {{PBKDF2-PRFs}} DEFAULT
       algid-hmacWithSHA1 }

As you can see in the ASN.1 dump the salt is 5BE04AE9442D08F0 4, the iteration count is 2048 (0x800) 5, and the hash function (prf, pseudorandom function) is hmacWithSHA256 6 without any additional parameters. The value 2048 for the iterations is a default value in OpenSSL (see the definition of PKCS5_DEFAULT_ITER).

OpenSSH's private key format¶

As we saw at the beginning of the post, the OpenSSH team came up with a custom format to store the private keys, so now that we are familiar with the nomenclature and with the way PEM stores encrypted keys, lets see what this new format can do.

The best starting point for our investigation is the tool ssh-keygen which we can use to create private keys. The source can be found in the OpenSSH repository in the file ssh-keygen.c. This file uses two different functions, sshkey_private_to_blob2 (source code) for the new format and sshkey_private_to_blob_pem_pkcs8 (source code) for keys in PKCS #8 format. The former calls bcrypt_pbkdf which comes from OpenBSD (source code).

This function contains a modified implementation of PBKDF2 that uses bcrypt as the core hash function. The comment that you can find at the top of the file bcrypt_pbkdf.c says

/*
 * pkcs #5 pbkdf2 implementation using the "bcrypt" hash
 *
 * The bcrypt hash function is derived from the bcrypt password hashing
 * function with the following modifications:
 * 1. The input password and salt are preprocessed with SHA512.
 * 2. The output length is expanded to 256 bits.
 * 3. Subsequently the magic string to be encrypted is lengthened and modified
 *    to "OxychromaticBlowfishSwatDynamite"
 * 4. The hash function is defined to perform 64 rounds of initial state
 *    expansion. (More rounds are performed by iterating the hash.)
 *
 * Note that this implementation pulls the SHA512 operations into the caller
 * as a performance optimization.
 *
 * One modification from official pbkdf2. Instead of outputting key material
 * linearly, we mix it. pbkdf2 has a known weakness where if one uses it to
 * generate (e.g.) 512 bits of key material for use as two 256 bit keys, an
 * attacker can merely run once through the outer loop, but the user
 * always runs it twice. Shuffling output bytes requires computing the
 * entirety of the key material to assemble any subkey. This is something a
 * wise caller could do; we just do it for you.
 */

As you can see, this is intended to be a pkcs #5 pbkdf2 implementation that uses bcrypt as its underlying hash function. It also mentions some modifications, and it's worth noting that when you modify a standard you are not following the standard any more. I won't run through all the details of the implementation, though, as it's beyond the scope of the post.

So, the OpenSSH private key format ultimately contains a private key encrypted with a non-standard version of PBKDF2 that uses bcrypt as its core hash function. The structure that contains the key is not ASN.1, even though it's base64 encoded and wrapped between header and footer that are similar to the PEM ones. A description of the structure can be found in https://github.com/openssh/openssh-portable/blob/2dc328023f60212cd29504fc05d849133ae47355/PROTOCOL.key.

Cost factor and rounds

PBKDF2 uses the concept of rounds to make the key stretching slower. This is the number of times the hash function is called internally (using as salt the output of the previous iteration), so in PBKDF2 the number of rounds or iterations is directly proportional to the slowness of the stretching operation.

Bcrypt implements a similar mechanism with its cost factor. The cost factor in the standard bcrypt implementation is defined as the binary logarithm of the number of iterations of a specific part of the process (the repeated expansion of the password and the salt). Using the binary logarithm means that a cost factor of 4 (the minimum) corresponds to 16 iterations, while 31 (the maximum) corresponds to 2,147,483,648 (more than 2 billion) iterations.

In the OpenSSH/OpenBSD implementation things are a bit different.

OpenBSD's version of bcrypt runs with a fixed cost of 6, that creates 64 iterations of the key expansion (source code), but being an implementation of PBKDF2 it can still be hardened increasing the number of rounds (source code). Those rounds correspond to the value given to the parameter -a of the ssh-keygen command line.

How many rounds?

When it comes to KDFs, the advice is always to run as much iterations as possible while keeping the specific application usable, so you need to tune your SSH keys testing different values in your system. To give you some rough estimations, Wikipedia mentions that for PBKDF2 the number of iterations used by Apple and Lastpass is between 2k and 100k. It is worth reiterating though that you shouldn't aim to use other people's figures, in this case. Instead, run tests of your software and hardware.

On my laptop, an i7-8565U with 32GiB of RAM running Kubuntu 20.04 I get the following results, which are pretty linear:

ssh-keygen -a 100 -t ed25519	0.667s
ssh-keygen -a 500 -t ed25519	3.148s
ssh-keygen -a 1000 -t ed25519	6.331s
ssh-keygen -a 5000 -t ed25519	31.624s

A sensible value for me might be between 100 and 500, then, so that I don't have to wait too long every time I push and pull my branches from GitHub.

Can we convert private OpenSSH keys into PEM?¶

As OpenSSL doesn't understand the OpenSSH private keys format, a common question among programmers and devops is if it is possible to convert it into a PEM format. As you might have guessed reading the previous sections, the answer is no. The PEM format for private keys uses PKCS#5, so it supports only the standard implementation of PBKDF2.

It's interesting to note that the OpenSSL team also specifically decided not to support this new format as it is not standard (see https://github.com/openssl/openssl/issues/5323).

A poorly documented format¶

PEM, PKCS #8, ASN.1, and all other formats that we use every day, included the OpenSSH public key format, are well documented and standardised in RFCs or similar documents. The OpenSSH private key format is documented in a tiny file that you can find in the source code, but doesn't offer more than a quick overview. To have a good understanding of what is going on I had to read the source code, not only of OpenSSH, but also of OpenBSD.

I think poor documentation like this might be acceptable in personal projects or in new tools, but SSH is used by the whole world, and when the team decides to come up with a completely new format for one of its most important elements I would expect them to detail every single bit of it, or at least try to be more open about the reasons and the implementation. I also personally believe that standards can't but benefit intercommunication between systems and, in cryptography, improve security, since they are reviewed and discussed by a wider audience.

The claim is that the new SSH private key format offers a better protection of keys at rest. I'd be very interested to see a cryptanalysis made by some expert (which I'm not). Cryptography is a tricky field, and often things that are apparently smart end up being tragically wrong.

Resources¶

OpenSSL documentation: asn1parse, genpkey
The Base64 encoding
The Abstract Syntax Notation One ASN.1 interface description language
RFC 4251 - The Secure Shell (SSH) Protocol Architecture
RFC 4253 - The Secure Shell (SSH) Transport Layer Protocol
RFC 4716 - The Secure Shell (SSH) Public Key File Format
RFC 5208 - PKCS #5: Password-Based Cryptography Specification Version 2.0
RFC 5208 - Public-Key Cryptography Standards (PKCS) #8: Private-Key Information Syntax Specification Version 1.2
RFC 5958 - Asymmetric Key Packages
RFC 7468 - Textual Encodings of PKIX, PKCS, and CMS Structures

Feedback¶

Feel free to reach me on Twitter if you have questions. The GitHub issues page is the best place to submit corrections.

Public key cryptography: SSL certificates

2020-11-04T23:00:00+01:00

In the context of public key cryptography, certificates are a way to prove the identity of the owner of a public key.

While public key cryptography allows us to communicate securely through an insecure network, it leaves the problem of identity untouched. Once we established an encrypted communication we can be sure that the data we send and receive cannot be read or tampered with by third parties. But how can we be sure that the entity on the other side of the communication channel, with which we initiated the communication, is what it claims to be?

In other words, the messages cannot be read or modified by malicious third-parties, but what if we established communication with a malicious actor in the first place? Such a situation can arise during a man-in-the-middle attack, where the low-level network communication is hijacked by a malicious actor who pretends to be the desired recipient of the communication.

In the context of the Internet, and in particular of the World Wide Web, the main concern is that the server that provides services we log into (think of every service that has your personal or financial data like you bank, Google, Facebook, Netflix, etc.) is run by the company that we trust and not by an attacker who wants to steal our data.

In this post I will try to clarify the main components of the certificates system and to explain the meaning of the major acronyms and names that you might hear when you deal with this part of web development.

Clarification: SSL vs TLS¶

In the world of web development and infrastructure management, we normally speak of SSL protocol and of SSL certificates, but it has to be noted that SSL (Secure Sockets Layer) is the name of a deprecated protocol. The current implementation of the protocol used to secure web applications is TLS (Transport Layer Security).

The story of SSL and TLS is rich of events and spans 25 years since its inception by Taher Elgamal at Netscape. In short, SSL had 3 major versions (the first of which was never publicly used), and was replaced by TLS in 1999. TLS itself has gone through 3 revisions at the time of writing, TLS 1.3 being the latest version available.

The TLS/SSL nomenclature is one of many sources of confusion in the complicated world of security and applied cryptography. In this article I will use only the acronym TLS, but I went for SSL in the title because I wanted the subject matter to be recognisable also by developers that are not much into security and cryptography.

X.509 certificates¶

While the problem of the identity in an insecure network can be solved in several ways, the solution embraced to secure the World Wide Web is based on a standard called X.509. When we mention TLS certificates, we usually mean X.509 certificates used in a TLS connection, such as that created by HTTPS.

X.509 is the ITU-T standard used to represent certificates, and has been chosen to be the standard used in the TLS protocol. The standard doesn't only define the binary structure of the certificate itself, but it also defines procedures to revoke the certificates, and establishes a hierarchical system of certification known as certificate path, or certificate chain.

The structure of an X.509 certificate is expressed using ASN.1, a notation used natively by the PEM format (discussed here). You can read the full specification in RFC 2459, in particular Section 4 "Certificate and Certificate Extensions Profile". I will refer to this later when I will have a look at a real certificate.

Before I discuss how certificates solve the problem of identity (or ownership of a public key), let's clarify the relationship between them and HTTPS.

HTTPS stands for HTTP Secure, and the core of the protocol consists of running HTTP over TLS. When we access a web site with HTTPS the browser first establishes a TLS connection with the server and then communicates with it using pure HTTP. This means that the whole HTTP protocol is encrypted, as the secure channel is established outside it, and also means that, aside from the different URI scheme https:// instead of http://, there are no differences between the two protocols.

Certificates come into play when the browser establishes the TLS connection, which is why you need to set-up HTTPS as part of your infrastructure and not in your web application. By the time the HTTP requests reach your application they are already decrypted and accessible in plain text, as the HTTP protocol mandates. We usually say that we "terminate TLS" when a component of our infrastructure manages certificates and decrypts HTTPS into HTTP.

How do certificates work?¶

The X.509 standard establishes entities called Certificate Authorities (CAs), and creates a hierarchy of trust called chain between them. The idea is that there is a set of entities that are trusted worldwide by operating systems, browsers, and other network-related software, and that these entities can trust other entities, thus creating a trust network.

While the market of Certificate Authorities is dominated by three major commercial players (see the usage statistics) there are approximately 100 organisations operating worldwide, among which some non-profit ones. Not all of these are trusted by all operating systems or browsers, though.

The set of CAs trusted by an organisation is called root program. The Mozilla community runs a program that is independent from the hardware/software platform, aptly called Mozilla's CA Certificate Program and uses data contained in the Common CA Database (CCADB). Private companies such as Microsoft, Apple, and Oracle run their own root programs and software running on the respective platforms (Windows, macOS/iOS, Java) can decide to trust the CAs provided by those programs.

In the open-source world, the Mozilla root program is by far the most influential and important source of information, being used by other software packages and Linux distributions.

It is possible to create certificates that are not signed by any CA, and these are called self-signed certificates. Such certificates can be used with any software that relies on certificates, but it requires such a software to disable certificate checking with the Certificate Authorities. Self-signed certificates are obviously useful for testing purposes, but there are scenarios in which it might be desirable not to rely on the CAs and establish a private network of trust.

Example: CA root certificate¶

The certificates for root CAs that are part of the Mozilla root program can be retrieved from the Common CA Database web page, or can be seen in the Firefox source code directly. On a running Firefox browser you can open the Privacy & Security menu and click on "View Certificates" at the bottom of the page. The CAs are listed under the tab "Authorities".

The interesting thing you can do here is to export a CA certificate. If you do it Firefox will save it in a file with extension .crt, that contains data in PEM format. I exported the certificate for Amazon Root CA 1 and I ended up with the file AmazonRootCA1.crt. If, instead of exporting, you view the certificate, you will end up in a page that allows you to download the certificate and the chain, both in PEM format, in files with the extension .pem. As you see, you are not the only one who is confused.

I described the PEM format in a post on RSA keys so I won't repeat here the whole discussion about it. The RFC 7468 ("Textual Encodings of PKIX, PKCS, and CMS Structures") describes certificates in section 5. Section 4 mentions the module id-pkix1-e for Certificate, CertificateList, and SubjectPublicKeyInfo RFC 5280 ("Internet X.509 Public Key Infrastructure Certificate and Certificate Revocation List (CRL) Profile").

The identifier id-pkix1-e is part of a registry of objects to be used in ASN.1 data created in the framework of the Public-Key Infrastructure using X.509 (PKIX) Working Group, that defined the infrastructure around the X.509 certificates system. Basically it's a standard way to identify binary objects and their structure. You can see a full list of all the objects in RFC 7299 ("Object Identifier Registry for the PKIX Working Group"). Not a very exciting one to read, if you ask me.

I can dump the content of the Amazon Root CA 1 certificate with OpenSSL

$ openssl asn1parse -inform pem -in amazon-root-ca-1.pem
    0:d=0  hl=4 l= 833 cons: SEQUENCE
    4:d=1  hl=4 l= 553 cons: SEQUENCE
    8:d=2  hl=2 l=   3 cons: cont [ 0 ]
   10:d=3  hl=2 l=   1 prim: INTEGER           :02
   13:d=2  hl=2 l=  19 prim: INTEGER           :066C9FCF99BF8C0A39E2F0788A43E696365BCA
   34:d=2  hl=2 l=  13 cons: SEQUENCE
   36:d=3  hl=2 l=   9 prim: OBJECT            :sha256WithRSAEncryption
   47:d=3  hl=2 l=   0 prim: NULL
   49:d=2  hl=2 l=  57 cons: SEQUENCE
   51:d=3  hl=2 l=  11 cons: SET
   53:d=4  hl=2 l=   9 cons: SEQUENCE
   55:d=5  hl=2 l=   3 prim: OBJECT            :countryName
   60:d=5  hl=2 l=   2 prim: PRINTABLESTRING   :US
   64:d=3  hl=2 l=  15 cons: SET
   66:d=4  hl=2 l=  13 cons: SEQUENCE
   68:d=5  hl=2 l=   3 prim: OBJECT            :organizationName
   73:d=5  hl=2 l=   6 prim: PRINTABLESTRING   :Amazon
   81:d=3  hl=2 l=  25 cons: SET
   83:d=4  hl=2 l=  23 cons: SEQUENCE
   85:d=5  hl=2 l=   3 prim: OBJECT            :commonName
   90:d=5  hl=2 l=  16 prim: PRINTABLESTRING   :Amazon Root CA 1
  108:d=2  hl=2 l=  30 cons: SEQUENCE
  110:d=3  hl=2 l=  13 prim: UTCTIME           :150526000000Z
  125:d=3  hl=2 l=  13 prim: UTCTIME           :380117000000Z
  140:d=2  hl=2 l=  57 cons: SEQUENCE
  142:d=3  hl=2 l=  11 cons: SET
  144:d=4  hl=2 l=   9 cons: SEQUENCE
  146:d=5  hl=2 l=   3 prim: OBJECT            :countryName
  151:d=5  hl=2 l=   2 prim: PRINTABLESTRING   :US
  155:d=3  hl=2 l=  15 cons: SET
  157:d=4  hl=2 l=  13 cons: SEQUENCE
  159:d=5  hl=2 l=   3 prim: OBJECT            :organizationName
  164:d=5  hl=2 l=   6 prim: PRINTABLESTRING   :Amazon
  172:d=3  hl=2 l=  25 cons: SET
  174:d=4  hl=2 l=  23 cons: SEQUENCE
  176:d=5  hl=2 l=   3 prim: OBJECT            :commonName
  181:d=5  hl=2 l=  16 prim: PRINTABLESTRING   :Amazon Root CA 1
  199:d=2  hl=4 l= 290 cons: SEQUENCE
  203:d=3  hl=2 l=  13 cons: SEQUENCE
  205:d=4  hl=2 l=   9 prim: OBJECT            :rsaEncryption
  216:d=4  hl=2 l=   0 prim: NULL
  218:d=3  hl=4 l= 271 prim: BIT STRING
  493:d=2  hl=2 l=  66 cons: cont [ 3 ]
  495:d=3  hl=2 l=  64 cons: SEQUENCE
  497:d=4  hl=2 l=  15 cons: SEQUENCE
  499:d=5  hl=2 l=   3 prim: OBJECT            :X509v3 Basic Constraints
  504:d=5  hl=2 l=   1 prim: BOOLEAN           :255
  507:d=5  hl=2 l=   5 prim: OCTET STRING      [HEX DUMP]:30030101FF
  514:d=4  hl=2 l=  14 cons: SEQUENCE
  516:d=5  hl=2 l=   3 prim: OBJECT            :X509v3 Key Usage
  521:d=5  hl=2 l=   1 prim: BOOLEAN           :255
  524:d=5  hl=2 l=   4 prim: OCTET STRING      [HEX DUMP]:03020186
  530:d=4  hl=2 l=  29 cons: SEQUENCE
  532:d=5  hl=2 l=   3 prim: OBJECT            :X509v3 Subject Key Identifier
  537:d=5  hl=2 l=  22 prim: OCTET STRING      [HEX DUMP]:04148418CC8534ECBC0C94942E08599CC7B2104E0A08
  561:d=1  hl=2 l=  13 cons: SEQUENCE
  563:d=2  hl=2 l=   9 prim: OBJECT            :sha256WithRSAEncryption
  574:d=2  hl=2 l=   0 prim: NULL
  576:d=1  hl=4 l= 257 prim: BIT STRING

Let's read part of it using the aforementioned section 4 of RFC 5280.

The signed certificate is a sequence of three main components

   Certificate  ::=  SEQUENCE  {
        tbsCertificate       TBSCertificate,
        signatureAlgorithm   AlgorithmIdentifier,
        signatureValue       BIT STRING  }

and the TBSCertificate structure represents the unsigned certificate (TBS = To Be Signed)

TBSCertificate  ::=  SEQUENCE  {
        version         [0]  EXPLICIT Version DEFAULT v1,
        serialNumber         CertificateSerialNumber,
        signature            AlgorithmIdentifier,
        issuer               Name,
        validity             Validity,
        subject              Name,
        subjectPublicKeyInfo SubjectPublicKeyInfo,
        issuerUniqueID  [1]  IMPLICIT UniqueIdentifier OPTIONAL,
                             -- If present, version MUST be v2 or v3
        subjectUniqueID [2]  IMPLICIT UniqueIdentifier OPTIONAL,
                             -- If present, version MUST be v2 or v3
        extensions      [3]  EXPLICIT Extensions OPTIONAL
                             -- If present, version MUST be v3
        }

Comparing this with the output of OpenSSL we can find fields such as version

   10:d=3  hl=2 l=   1 prim: INTEGER           :02

which according to the documentation is 3 (binary 02). Many values are of type PRINTABLESTRING, so they are readable already in the ASN.1 dump.

The validity of the certificate is

  110:d=3  hl=2 l=  13 prim: UTCTIME           :150526000000Z
  125:d=3  hl=2 l=  13 prim: UTCTIME           :380117000000Z

and following section 4.1.2.5.1 of the RFC we find out that the certificate is valid between 26 May 2015 and 17 Jan 2038. You can easily read these values in the certificate page in the browser without getting an headache trying to decode ASN.1.

The CA signed the certificate using a certain algorithm. The algorithm identifier is repeated twice, first in the structure Certificate (signatureAlgorithm AlgorithmIdentifier) and then in the structure TBSCertificate (signature AlgorithmIdentifier). The two fields must have the same value.

   34:d=2  hl=2 l=  13 cons: SEQUENCE
   36:d=3  hl=2 l=   9 prim: OBJECT            :sha256WithRSAEncryption
   47:d=3  hl=2 l=   0 prim: NULL

[...]

  561:d=1  hl=2 l=  13 cons: SEQUENCE
  563:d=2  hl=2 l=   9 prim: OBJECT            :sha256WithRSAEncryption
  574:d=2  hl=2 l=   0 prim: NULL

For this certificate, the algorithm used by Amazon is sha256WithRSAEncryption. This label is described in RFC 4055 ("Additional Algorithms and Identifiers for RSA Cryptography for use in the Internet X.509 Public Key Infrastructure Certificate and Certificate Revocation List (CRL) Profile") as "PKCS #1 version 1.5 signature algorithm with SHA-256". The specific algorithm can be found in RFC 2313 ("PKCS #1: RSA Encryption Version 1.5"). As the name of the algorithm suggests, the certificate is first digested with SHA-256 and then encrypted using RSA and the private key of the signer.

Speaking of keys, the public key the CA used for the certificate can be found in the field subjectpublickeyinfo, which is again made of a field type AlgorithmIdentifier and a bit string with the value of the key. In this case the fields are

  205:d=4  hl=2 l=   9 prim: OBJECT            :rsaEncryption
  216:d=4  hl=2 l=   0 prim: NULL
  218:d=3  hl=4 l= 271 prim: BIT STRING

The algorithm rsaEncryption is described in RFC 3279 ("Algorithms and Identifiers for the Internet X.509 Public Key Infrastructure Certificate and Certificate Revocation List (CRL) Profile"), section 2.3.1 as

      RSAPublicKey ::= SEQUENCE {
         modulus            INTEGER,    -- n
         publicExponent     INTEGER  }  -- e

(sic) or in RFC 8017 ("PKCS #1: RSA Cryptography Specifications Version 2.2")

RSAPublicKey ::= SEQUENCE {
    modulus           INTEGER,  -- n
    publicExponent    INTEGER   -- e
}

We can then use the option -strparse of the module asn1parse to find the actual values

$ openssl asn1parse -inform pem -in amazon-root-ca-1.pem -strparse 218
    0:d=0  hl=4 l= 266 cons: SEQUENCE
    4:d=1  hl=4 l= 257 prim: INTEGER           :B2788071CA78D5E371AF478050747D6ED8D78876F4
9968F7582160F97484012FAC022D86D3A0437A4EB2A4D036BA01BE8DDB48C80717364CF4EE8823C73EEB37F5B5
19F84968B0DED7B976381D619EA4FE8236A5E54A56E445E1F9FDB416FA74DA9C9B35392FFAB02050066C7AD080
B2A6F9AFEC47198F503807DCA2873958F8BAD5A9F948673096EE94785E6F89A351C0308666A14566BA54EBA3C3
91F948DCFFD1E8302D7D2D747035D78824F79EC4596EBB738717F2324628B843FAB71DAACAB4F29F240E2D4BF7
715C5E69FFEA9502CB388AAE50386FDBFB2D621BC5C71E54E177E067C80F9C8723D63F40207F2080C4804C3E3B
24268E04AE6C9AC8AA0D
  265:d=1  hl=2 l=   3 prim: INTEGER           :010001

As we already saw for RSA keys), OpenSSL has a specific module for important structures, and the X.509 certificates are definitely worth a module aptly called x509. using that we can easily decode any certificate

$ openssl x509 -inform pem -in amazon-root-ca-1.pem -noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            06:6c:9f:cf:99:bf:8c:0a:39:e2:f0:78:8a:43:e6:96:36:5b:ca
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = US, O = Amazon, CN = Amazon Root CA 1
        Validity
            Not Before: May 26 00:00:00 2015 GMT
            Not After : Jan 17 00:00:00 2038 GMT
        Subject: C = US, O = Amazon, CN = Amazon Root CA 1
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus:
                    00:b2:78:80:71:ca:78:d5:e3:71:af:47:80:50:74:
                    7d:6e:d8:d7:88:76:f4:99:68:f7:58:21:60:f9:74:
                    84:01:2f:ac:02:2d:86:d3:a0:43:7a:4e:b2:a4:d0:
                    36:ba:01:be:8d:db:48:c8:07:17:36:4c:f4:ee:88:
                    23:c7:3e:eb:37:f5:b5:19:f8:49:68:b0:de:d7:b9:
                    76:38:1d:61:9e:a4:fe:82:36:a5:e5:4a:56:e4:45:
                    e1:f9:fd:b4:16:fa:74:da:9c:9b:35:39:2f:fa:b0:
                    20:50:06:6c:7a:d0:80:b2:a6:f9:af:ec:47:19:8f:
                    50:38:07:dc:a2:87:39:58:f8:ba:d5:a9:f9:48:67:
                    30:96:ee:94:78:5e:6f:89:a3:51:c0:30:86:66:a1:
                    45:66:ba:54:eb:a3:c3:91:f9:48:dc:ff:d1:e8:30:
                    2d:7d:2d:74:70:35:d7:88:24:f7:9e:c4:59:6e:bb:
                    73:87:17:f2:32:46:28:b8:43:fa:b7:1d:aa:ca:b4:
                    f2:9f:24:0e:2d:4b:f7:71:5c:5e:69:ff:ea:95:02:
                    cb:38:8a:ae:50:38:6f:db:fb:2d:62:1b:c5:c7:1e:
                    54:e1:77:e0:67:c8:0f:9c:87:23:d6:3f:40:20:7f:
                    20:80:c4:80:4c:3e:3b:24:26:8e:04:ae:6c:9a:c8:
                    aa:0d
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Basic Constraints: critical
                CA:TRUE
            X509v3 Key Usage: critical
                Digital Signature, Certificate Sign, CRL Sign
            X509v3 Subject Key Identifier: 
                84:18:CC:85:34:EC:BC:0C:94:94:2E:08:59:9C:C7:B2:10:4E:0A:08
    Signature Algorithm: sha256WithRSAEncryption
         98:f2:37:5a:41:90:a1:1a:c5:76:51:28:20:36:23:0e:ae:e6:
         28:bb:aa:f8:94:ae:48:a4:30:7f:1b:fc:24:8d:4b:b4:c8:a1:
         97:f6:b6:f1:7a:70:c8:53:93:cc:08:28:e3:98:25:cf:23:a4:
         f9:de:21:d3:7c:85:09:ad:4e:9a:75:3a:c2:0b:6a:89:78:76:
         44:47:18:65:6c:8d:41:8e:3b:7f:9a:cb:f4:b5:a7:50:d7:05:
         2c:37:e8:03:4b:ad:e9:61:a0:02:6e:f5:f2:f0:c5:b2:ed:5b:
         b7:dc:fa:94:5c:77:9e:13:a5:7f:52:ad:95:f2:f8:93:3b:de:
         8b:5c:5b:ca:5a:52:5b:60:af:14:f7:4b:ef:a3:fb:9f:40:95:
         6d:31:54:fc:42:d3:c7:46:1f:23:ad:d9:0f:48:70:9a:d9:75:
         78:71:d1:72:43:34:75:6e:57:59:c2:02:5c:26:60:29:cf:23:
         19:16:8e:88:43:a5:d4:e4:cb:08:fb:23:11:43:e8:43:29:72:
         62:a1:a9:5d:5e:08:d4:90:ae:b8:d8:ce:14:c2:d0:55:f2:86:
         f6:c4:93:43:77:66:61:c0:b9:e8:41:d7:97:78:60:03:6e:4a:
         72:ae:a5:d1:7d:ba:10:9e:86:6c:1b:8a:b9:59:33:f8:eb:c4:
         90:be:f1:b9

Now I'm pretty sure you want to kill me because I could have shown you this from the start. But I like to understand things, and the easy path doesn't always make everything clear. At any rate, here you have a way to read an X.509 certificate in PEM format.

Please note that in this certificate the Issuer and the Subject are the same entity, as this is a root certificate, which is signed by the same entity that creates it.

        Issuer: C = US, O = Amazon, CN = Amazon Root CA 1
[...]
        Subject: C = US, O = Amazon, CN = Amazon Root CA 1

Moreover, one of the version 3 extensions of the self-signed certificate is a basic constraint with the boolean CA set to true. It also has the extension Key Usage set to Digital Signature, Certificate Sign, CRL Sign, which means that the certificate can be used to sign other certificates.

Example: self-signed certificate¶

You can use OpenSSL to create a self-signed certificate using the module req that you would normally use to create certificate requests. As a self-signed certificate doesn't need approval, the module can directly output the certificate.

$ openssl req -x509 -newkey rsa:2048 -keyout self-signed-key.pem -out self-signed.pem -days 365 -nodes -subj '/CN=localhost'
Generating a RSA private key
....+++++
................+++++
writing new private key to 'self-signed-key.pem'
-----

(note that for simplicity's sake I specified the option -nodes that prevents the key to be protected with a password, but this is a bad practice). This command creates the two files I mentioned, self-signed-key.pem (the private key) and self-signed.pem.

We can read the certificate using the module x509

$ openssl x509 -inform pem -in self-signed.pem  -noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            46:e5:2f:8e:42:82:43:b8:ac:88:cb:6d:0c:2f:71:28:a9:fe:00:ec
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN = localhost
        Validity
            Not Before: Nov  3 00:23:34 2020 GMT
            Not After : Nov  3 00:23:34 2021 GMT
        Subject: CN = localhost
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus:
                    00:b7:14:ef:3b:eb:8b:a9:40:18:c5:d2:eb:1d:4f:
                    5d:e4:a3:17:f3:df:ce:b7:d3:3f:52:58:eb:61:02:
                    a2:68:0a:cd:0f:97:ae:e0:a5:ac:a7:88:cf:a1:15:
                    0a:97:ca:e7:03:8a:a5:c0:66:38:ef:bb:59:4d:48:
                    17:db:a7:bd:fa:4b:50:2a:be:e9:5b:bb:59:65:71:
                    dc:99:73:9c:bc:4d:3b:42:97:91:e9:3b:1a:8a:9d:
                    cc:41:38:ba:8b:8f:df:65:ff:5b:1f:ef:8a:b7:c5:
                    93:07:ce:15:4c:13:72:78:59:64:9a:5b:95:20:b6:
                    b3:8e:aa:c3:29:c3:7f:28:39:43:81:59:e4:0f:26:
                    7c:3f:49:d2:06:05:d9:54:ab:09:65:96:01:cc:c2:
                    72:be:85:1f:40:ea:94:35:04:09:9d:87:eb:a1:90:
                    36:ce:d2:55:f9:ee:08:db:52:78:e8:70:d0:25:89:
                    13:8e:0f:9d:98:98:d1:4d:67:06:8f:8a:61:9e:3a:
                    73:89:aa:0a:0a:1b:05:a7:52:32:ef:1b:78:5a:5f:
                    4b:b6:c9:a7:4e:15:10:04:50:99:00:09:2f:60:8e:
                    aa:20:af:6b:ee:f5:60:0b:29:da:38:1c:b2:73:14:
                    99:a4:ee:5e:89:e6:77:0b:ba:cf:d3:5d:d7:a3:ea:
                    c4:bf
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Subject Key Identifier: 
                64:7B:C1:FC:99:74:56:B7:82:D1:4F:E7:2D:94:77:1A:09:52:26:5C
            X509v3 Authority Key Identifier: 
                keyid:64:7B:C1:FC:99:74:56:B7:82:D1:4F:E7:2D:94:77:1A:09:52:26:5C

            X509v3 Basic Constraints: critical
                CA:TRUE
    Signature Algorithm: sha256WithRSAEncryption
         43:7b:0b:c8:98:b8:6f:72:af:39:4a:d9:76:ce:e3:9d:3a:c7:
         9f:14:b0:4f:20:0a:45:b3:b4:8c:e5:37:4c:bf:15:ad:8e:5c:
         45:4f:3e:b7:ef:8d:60:57:bb:6f:d9:5e:6a:d3:04:05:4a:ff:
         f2:66:b1:76:66:59:7e:24:89:0a:50:28:c9:d5:f5:7a:00:07:
         8a:79:9c:6e:53:43:66:e5:9a:10:d8:f8:e1:f2:c1:f1:17:d0:
         d2:9e:50:80:fe:2a:ca:08:b6:98:e9:b5:a4:82:23:31:45:35:
         33:da:2c:e3:fe:54:f2:bd:f2:61:91:f4:32:e3:7d:4c:3a:e5:
         3a:0f:cd:36:b0:8b:af:9f:8e:3d:0e:0b:a5:df:4a:3a:91:83:
         b3:b2:5f:3c:47:81:73:4f:a2:c1:49:06:75:17:25:fa:5a:8d:
         30:e5:55:7f:9c:3e:15:a8:b5:ab:f7:45:38:e3:76:8e:d4:0d:
         60:fc:42:17:3d:85:72:41:1d:53:9d:58:b0:e9:29:0c:e4:6b:
         14:c2:22:c4:d5:7b:de:36:da:df:d8:a0:4f:a4:0a:f2:3e:ca:
         7e:66:a6:10:38:97:24:73:5b:db:eb:0b:6c:a8:f8:37:15:2c:
         0e:b1:82:44:cc:fe:85:b0:cb:6c:26:4b:4a:70:33:dc:7e:f5:
         84:ba:07:db

As you can see this certificate has the same value in Issuer and Subject, as happened before for the Amazon Root one. It also has the flag CA set to true but it doesn't have the extension Key Usage meaning that this certificate can't be used to sign other certificates.

Example: this site's certificate¶

You can see TLS certificates and the chain of trust in action in this very website. Following the documentation of you browser (instructions for Firefox are here), you can see the certificate used by The Digital Cat. At the time of writing the blog is hosted on GitHub Pages, even tough I'm using a custom domain, and GitHub partnered with Let's Encrypt to provide certificates for such a configuration (details here).

Indeed, the certificate for thedigitalcatonline.com is provided by "Let's Encrypt Authority X3", which in turn is trusted by Digital Signature Trust Co. with its root CA "DST Root CA X3".

Let's have a look at the three certificates. The one for The Digital Cat is

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            03:93:02:bb:9a:c9:ed:a5:c3:d1:16:00:8b:15:76:af:e5:d9
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
        Validity
            Not Before: Oct 22 04:53:28 2020 GMT
            Not After : Jan 20 04:53:28 2021 GMT
        Subject: CN = www.thedigitalcatonline.com
[...]
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage: 
                TLS Web Server Authentication, TLS Web Client Authentication
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Subject Key Identifier: 
                63:4E:15:85:56:5A:A4:94:02:C2:16:42:A4:A5:97:9A:38:02:57:97
            X509v3 Authority Key Identifier: 
                keyid:A8:4A:6A:63:04:7D:DD:BA:E6:D1:39:B7:A6:45:65:EF:F3:A8:EC:A1

            Authority Information Access: 
                OCSP - URI:http://ocsp.int-x3.letsencrypt.org
                CA Issuers - URI:http://cert.int-x3.letsencrypt.org/

            X509v3 Subject Alternative Name: 
                DNS:www.thedigitalcatonline.com
[...]

And you can see that this time the Subject is www.thedigitalcatonline.com, but the Issuer is Let's Encrypt Authority X3. The certificate provided by the organisation Let's Encrypt is

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            0a:01:41:42:00:00:01:53:85:73:6a:0b:85:ec:a7:08
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: O = Digital Signature Trust Co., CN = DST Root CA X3
        Validity
            Not Before: Mar 17 16:40:46 2016 GMT
            Not After : Mar 17 16:40:46 2021 GMT
        Subject: C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
[...]
        X509v3 extensions:
            X509v3 Basic Constraints: critical
                CA:TRUE, pathlen:0
            X509v3 Key Usage: critical
                Digital Signature, Certificate Sign, CRL Sign
            Authority Information Access: 
                OCSP - URI:http://isrg.trustid.ocsp.identrust.com
                CA Issuers - URI:http://apps.identrust.com/roots/dstrootcax3.p7c

            X509v3 Authority Key Identifier: 
                keyid:C4:A7:B1:A4:7B:2C:71:FA:DB:E1:4B:90:75:FF:C4:15:60:85:89:10

            X509v3 Certificate Policies: 
                Policy: 2.23.140.1.2.1
                Policy: 1.3.6.1.4.1.44947.1.1.1
                  CPS: http://cps.root-x1.letsencrypt.org

            X509v3 CRL Distribution Points: 

                Full Name:
                  URI:http://crl.identrust.com/DSTROOTCAX3CRL.crl

            X509v3 Subject Key Identifier: 
                A8:4A:6A:63:04:7D:DD:BA:E6:D1:39:B7:A6:45:65:EF:F3:A8:EC:A1
[...]

Here, the Subject is Let's Encrypt Authority X3 (the Issuer of the previous certificate), and the Issuer is DST Root CA X3. Last, the certificate provided by the organisation Digital Signature Trust Co. is

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            44:af:b0:80:d6:a3:27:ba:89:30:39:86:2e:f8:40:6b
        Signature Algorithm: sha1WithRSAEncryption
        Issuer: O = Digital Signature Trust Co., CN = DST Root CA X3
        Validity
            Not Before: Sep 30 21:12:19 2000 GMT
            Not After : Sep 30 14:01:15 2021 GMT
        Subject: O = Digital Signature Trust Co., CN = DST Root CA X3
[...]
        X509v3 extensions:
            X509v3 Basic Constraints: critical
                CA:TRUE
            X509v3 Key Usage: critical
                Certificate Sign, CRL Sign
            X509v3 Subject Key Identifier: 
                C4:A7:B1:A4:7B:2C:71:FA:DB:E1:4B:90:75:FF:C4:15:60:85:89:10
[...]

As happened for the certificate Amazon Root CA 1 that we discussed before, this one is self-signed, having the same value for Subject and Issuer.

How to verify certificates with OpenSSL¶

To verify if a certificate is valid we can use the module verify of OpenSSL. By default, OpenSSL doesn't trust anything, and verify relies on a default path in the system to find root certificates. You can see the path running

$ openssl version -d
OPENSSLDIR: "/usr/lib/ssl"

On Ubuntu 20.04, the directory /usr/lib/ssl/certs is a symbolic link to /etc/ssl/certs that is installed by the package ca-certificates which is linked to the Mozilla's CA Certificate Program (details on that package can be found in the source code).

So, if a root certificate is included in the Mozilla program, it is trusted by OpenSSL

$ openssl verify amazon-root-ca-1.pem 
amazon-root-ca-1.pem: OK

while a self-signed certificate is not

$ openssl verify self-signed.pem 
CN = localhost
error 18 at 0 depth lookup: self signed certificate
error self-signed.pem: verification failed

A non-root certificate can be verified specifying which root certificate signed it. So, the certificate for this website is not trusted automatically

$ openssl verify www-thedigitalcatonline-com.pem 
CN = www.thedigitalcatonline.com
error 20 at 0 depth lookup: unable to get local issuer certificate
error www-thedigitalcatonline-com.pem: verification failed

But it is verified specifying the certificate for Let's Encrypt that signed it

$ openssl verify -CAfile lets-encrypt-x3.pem www-thedigitalcatonline-com.pem 
www-thedigitalcatonline-com.pem: OK

because the certificate lets-encrypt-x3.pem is signed by DST_Root_CA_X3.pem which is included in the Mozilla program, and thus included in my Linux distribution.

If I remove the default certificates path OpenSSL doesn't accept the certificate for Let's Encrypt any more

$ openssl verify -no-CApath -CAfile lets-encrypt-x3.pem www-thedigitalcatonline-com.pem
C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
error 2 at 1 depth lookup: unable to get issuer certificate
error www-thedigitalcatonline-com.pem: verification failed

Low-level certificate validation process¶

Let's have a look at the signature process for x.509 certificates. The process depends on the specific algorithm used to sign the certificate, so I will use the certificate Amazon Root CA 1 as an example, leaving to the reader the investigation about other algorithms.

A signed certificate is made of two parts, the certificate itself and the signature. The signature contains an encrypted hash of the certificate, so the verification is done in three steps:

Decrypt the encrypted hash using the public key
Compute the hash of the certificate using the same algorithm
Compare the hashes

For the Amazon root certificate, we know the signature algorithm and value from the output of openssl x509

$ openssl x509 -inform pem -in amazon-root-ca-1.pem  -noout -text
[...]
    Signature Algorithm: sha256WithRSAEncryption
         98:f2:37:5a:41:90:a1:1a:c5:76:51:28:20:36:23:0e:ae:e6:
         28:bb:aa:f8:94:ae:48:a4:30:7f:1b:fc:24:8d:4b:b4:c8:a1:
         97:f6:b6:f1:7a:70:c8:53:93:cc:08:28:e3:98:25:cf:23:a4:
         f9:de:21:d3:7c:85:09:ad:4e:9a:75:3a:c2:0b:6a:89:78:76:
         44:47:18:65:6c:8d:41:8e:3b:7f:9a:cb:f4:b5:a7:50:d7:05:
         2c:37:e8:03:4b:ad:e9:61:a0:02:6e:f5:f2:f0:c5:b2:ed:5b:
         b7:dc:fa:94:5c:77:9e:13:a5:7f:52:ad:95:f2:f8:93:3b:de:
         8b:5c:5b:ca:5a:52:5b:60:af:14:f7:4b:ef:a3:fb:9f:40:95:
         6d:31:54:fc:42:d3:c7:46:1f:23:ad:d9:0f:48:70:9a:d9:75:
         78:71:d1:72:43:34:75:6e:57:59:c2:02:5c:26:60:29:cf:23:
         19:16:8e:88:43:a5:d4:e4:cb:08:fb:23:11:43:e8:43:29:72:
         62:a1:a9:5d:5e:08:d4:90:ae:b8:d8:ce:14:c2:d0:55:f2:86:
         f6:c4:93:43:77:66:61:c0:b9:e8:41:d7:97:78:60:03:6e:4a:
         72:ae:a5:d1:7d:ba:10:9e:86:6c:1b:8a:b9:59:33:f8:eb:c4:
         90:be:f1:b9

You can see the signed certificate binary values with cat amazon-root-ca-1.pem | tail -n+2 | head -n-1 | base64 -di | hexdump -ve '/1 "%02x "' -e '2/8 "\n"'. While we can recognise the signature in the last 256 bytes we can't easily separate the bytes with the signature algorithm. If we open the signed certificate with an ASN.1 parser, instead, we can easily find the binary value of the certificate part

30 82 03 41 30 82 02 29 a0 03 02 01 02 02 13 06
6c 9f cf 99 bf 8c 0a 39 e2 f0 78 8a 43 e6 96 36
5b ca 30 0d 06 09 2a 86 48 86 f7 0d 01 01 0b 05
00 30 39 31 0b 30 09 06 03 55 04 06 13 02 55 53
31 0f 30 0d 06 03 55 04 0a 13 06 41 6d 61 7a 6f
6e 31 19 30 17 06 03 55 04 03 13 10 41 6d 61 7a
6f 6e 20 52 6f 6f 74 20 43 41 20 31 30 1e 17 0d
31 35 30 35 32 36 30 30 30 30 30 30 5a 17 0d 33
38 30 31 31 37 30 30 30 30 30 30 5a 30 39 31 0b
30 09 06 03 55 04 06 13 02 55 53 31 0f 30 0d 06
03 55 04 0a 13 06 41 6d 61 7a 6f 6e 31 19 30 17
06 03 55 04 03 13 10 41 6d 61 7a 6f 6e 20 52 6f
6f 74 20 43 41 20 31 30 82 01 22 30 0d 06 09 2a
86 48 86 f7 0d 01 01 01 05 00 03 82 01 0f 00 30
82 01 0a 02 82 01 01 00 b2 78 80 71 ca 78 d5 e3
71 af 47 80 50 74 7d 6e d8 d7 88 76 f4 99 68 f7
58 21 60 f9 74 84 01 2f ac 02 2d 86 d3 a0 43 7a
4e b2 a4 d0 36 ba 01 be 8d db 48 c8 07 17 36 4c
f4 ee 88 23 c7 3e eb 37 f5 b5 19 f8 49 68 b0 de
d7 b9 76 38 1d 61 9e a4 fe 82 36 a5 e5 4a 56 e4
45 e1 f9 fd b4 16 fa 74 da 9c 9b 35 39 2f fa b0
20 50 06 6c 7a d0 80 b2 a6 f9 af ec 47 19 8f 50
38 07 dc a2 87 39 58 f8 ba d5 a9 f9 48 67 30 96
ee 94 78 5e 6f 89 a3 51 c0 30 86 66 a1 45 66 ba
54 eb a3 c3 91 f9 48 dc ff d1 e8 30 2d 7d 2d 74
70 35 d7 88 24 f7 9e c4 59 6e bb 73 87 17 f2 32
46 28 b8 43 fa b7 1d aa ca b4 f2 9f 24 0e 2d 4b
f7 71 5c 5e 69 ff ea 95 02 cb 38 8a ae 50 38 6f
db fb 2d 62 1b c5 c7 1e 54 e1 77 e0 67 c8 0f 9c
87 23 d6 3f 40 20 7f 20 80 c4 80 4c 3e 3b 24 26
8e 04 ae 6c 9a c8 aa 0d 02 03 01 00 01 a3 42 30
40 30 0f 06 03 55 1d 13 01 01 ff 04 05 30 03 01
01 ff 30 0e 06 03 55 1d 0f 01 01 ff 04 04 03 02
01 86 30 1d 06 03 55 1d 0e 04 16 04 14 84 18 cc
85 34 ec bc 0c 94 94 2e 08 59 9c c7 b2 10 4e 0a
08

The signature algorithm part is

30 0d 06 09 2a 86 48 86 f7 0d 01 01 0b 05 00
03 82 01 01 00

and the ASN.1 parser tells us that those bytes represent an OBJECT IDENTIFIER which value is 2.16.840.1.101.3.4.2.1. Now, object identifiers are not complicated per se, they are just a way to identify algorithms and other well known components in ASN.1 structures. The description of the field signatureAlgorithm of an x.509 certificate mentions three other RFCs that contains descriptions of the available algorithms. In particular, RFC 4055 contains the description of PKCS #1 one-way hash functions, one of which is

id-sha256  OBJECT IDENTIFIER  ::=  { joint-iso-itu-t(2)
                     country(16) us(840) organization(1) gov(101)
                     csor(3) nistalgorithm(4) hashalgs(2) 1 }

You can see the values in the object identifier between parentheses. Since these are PKCS #1 (a.k.a. RSA) has functions, OpenSSL identifies it as sha256WithRSAEncryption (see again RFC 4055).

RSA encryption is described in RFC 2313 ("PKCS #1: RSA Encryption Version 1.5") and the signature algorithm based on RSA is described there in section 10. In particular, section 10.2 details the verification process, which is the one we are interested in. The steps are

Bit-string-to-octet-string conversion of the signature
RSA decryption
Digest decoding (ASN.1)
Message digesting and comparison

As for the signature conversion, the sentence

Specifically, assuming that the length in bits of the
signature S is a multiple of eight, the first bit of the signature
shall become the most significant bit of the first octet of the
encrypted data, and so on through the last bit of the signature,
which shall become the least significant bit of the last octet of the
encrypted data.

is a very verbose way to say that the signature is big-endian.

So, the hexadecimal value of the signature is

98f2375a4190a11ac57651282036230eaee628bbaaf894ae48a4307f1bfc248d
4bb4c8a197f6b6f17a70c85393cc0828e39825cf23a4f9de21d37c8509ad4e9a
753ac20b6a897876444718656c8d418e3b7f9acbf4b5a750d7052c37e8034bad
e961a0026ef5f2f0c5b2ed5bb7dcfa945c779e13a57f52ad95f2f8933bde8b5c
5bca5a525b60af14f74befa3fb9f40956d3154fc42d3c7461f23add90f48709a
d9757871d1724334756e5759c2025c266029cf2319168e8843a5d4e4cb08fb23
1143e843297262a1a95d5e08d490aeb8d8ce14c2d055f286f6c49343776661c0
b9e841d7977860036e4a72aea5d17dba109e866c1b8ab95933f8ebc490bef1b9

And reading the field Subject Public Key Info of the certificate we find the public key. Remember that this is a root certificate, so it is signed using the same key that it contains, which is not true in general.

The public key's modulus is

b2788071ca78d5e371af478050747d6ed8d78876f49968f7582160f97484012f
ac022d86d3a0437a4eb2a4d036ba01be8ddb48c80717364cf4ee8823c73eeb37
f5b519f84968b0ded7b976381d619ea4fe8236a5e54a56e445e1f9fdb416fa74
da9c9b35392ffab02050066c7ad080b2a6f9afec47198f503807dca2873958f8
bad5a9f948673096ee94785e6f89a351c0308666a14566ba54eba3c391f948dc
ffd1e8302d7d2d747035d78824f79ec4596ebb738717f2324628b843fab71daa
cab4f29f240e2d4bf7715c5e69ffea9502cb388aae50386fdbfb2d621bc5c71e
54e177e067c80f9c8723d63f40207f2080c4804c3e3b24268e04ae6c9ac8aa0d

and the exponent is 0x10001 (default choice).

RSA public-key signature decryption is performed with signature ^ exponent mod modulus, and this operation returns

1fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
fffffffffffffffffffff003031300d0609608648016503040201050004206fc
4b8ac3d2b52c08baf56255e43d22c762962e4facab01ace16d48ec008be0a

Once the padding is removed, we are left with an ASN.1 binary structure that represents the digest

DigestInfo ::= SEQUENCE {
  digestAlgorithm DigestAlgorithmIdentifier,
  digest Digest }

(see RFC 2313 - Section 10.1.2)

The value of Digest can be extracted with an ASN.1 parser or by taking the last 256 bits and is 6fc4b8ac3d2b52c08baf56255e43d22c762962e4facab01ace16d48ec008be0a.

At this point we need to process the certificate bytes (without signature) with the SHA-256 hash function and we will find a matching value of 6fc4b8ac3d2b52c08baf56255e43d22c762962e4facab01ace16d48ec008be0a.

This process (for the specific case of this certificate) can be easily done in Python

from cryptography import x509
from hashlib import sha256

certificate_pem_file = "amazon-root-ca-1.pem"

with open(certificate_pem_file, "rb") as f:
    certificate_pem = f.read()

certificate = x509.load_pem_x509_certificate(certificate_pem)

modulus = certificate.public_key().public_numbers().n
exponent = certificate.public_key().public_numbers().e

signature = int.from_bytes(certificate.signature, "big")

verification = pow(signature, exponent, modulus)

digest = bytes().fromhex(str(hex(verification))[-64:])

calculated_digest = sha256(certificate.tbs_certificate_bytes)

print(digest.hex() == calculated_digest.hexdigest())

This is arguably not the best Python code ever, but it's a simple way to demonstrate the process. As I said, this is far from being general, as it assumes the signature is sha256WithRSAEncryption, which might not be the case.

What I showed you here is what happens when we validate a root certificate. When we validate a non-root certificate the process is exactly the same (taking into account that the algorithms involved might be different), only the public key used to sign the certificate doesn't come from the certificate itself, but from the signer one. So, in the case of this blog, the certificate for www.thedigitalcat.com has a signature encrypted with the public key of Let's Encrypt. And the certificate for Let's Encrypt will be signed using the public key of Digital Signature Trust Co. This is what creates the chain of trust.

Algorithms used by root certificates¶

A quick scan of the certificates that are part of the Mozilla program reveals that the vast majority of them is using RSA to self-sign them

$ for i in /etc/ssl/certs/*.pem; do openssl x509 -inform pem -in ${i} -noout -text | grep -E "Public Key Algorithm"; done | sort | uniq -c
     25             Public Key Algorithm: id-ecPublicKey
    114             Public Key Algorithm: rsaEncryption

while part of them are using id-ecPublicKey which is the identifier of elliptic curves algorithms.

When it comes to signature algorithms, instead, there is more variety

$ for i in /etc/ssl/certs/*.pem; do openssl x509 -inform pem -in ${i} -noout -text | grep -E "^    Signature Algorithm"; done | sort | uniq -c
      7     Signature Algorithm: ecdsa-with-SHA256
     18     Signature Algorithm: ecdsa-with-SHA384
     47     Signature Algorithm: sha1WithRSAEncryption
     57     Signature Algorithm: sha256WithRSAEncryption
      9     Signature Algorithm: sha384WithRSAEncryption
      1     Signature Algorithm: sha512WithRSAEncryption

Even here, elliptic curves are slowly being adopted.

If you are using AWS, you can create certificates with ACM, the AWS Certificate Manager. Such certificates cannot be downloaded, they can only be attached to other AWS components. For this reason, the generation process requires you to create any request, as you might have to do with other authorities. Certificates created in the ACM are free.

Certificates created in the ACM can be attached to several AWS components, most notably Load Balancers, CloudFront, and API Gateway.

Traditionally, load balancers are the place where TLS is terminated for HTTPS, requiring a connection to port 443. While Application Load Balancers can do that, in 2019 AWS announced support for certificates in Network Load Balancers as well.

Let's encrypt¶

In an effort to push for HTTP encryption of any public server, the Internet Security Research Group founded in 2016 a non-profit CA named Let's Encrypt, which provides at no charge TLS certificates valid for 90 days. Such certificates can be renewed automatically as part of the setup (certbot) and represent a viable alternative to certificates issued by other CA, in particular for open source projects. This blog uses a certificate issued by Let's Encrypt (provided by GitHub Pages) and will thus expire in less than 3 months (but also automatically renewed).

Final words¶

I hope this post helped to clarify some of the most obscure points of certificates, that definitely bugged be when I first approached them. As always when standards are involved, the risk is to get lost in the myriad of documents where information is scattered, and not to realise that some (if not many) parts of the systems we run every day have a long history and thus a big burden of legacy code or nomenclature.

Resources¶

The Wikipedia article on TLS
The Wikipedia article on Certificate authority
The Wikipedia article on X.509
The Wikipedia article on Let's Encrypt
OpenSSL documentation: asn1parse, x509, verify
The Abstract Syntax Notation One ASN.1 interface description language
RFC 2313 - "PKCS #1: RSA Encryption Version 1.5"
RFC 2459 - "Internet X.509 Public Key Infrastructure Certificate and CRL Profile"
RFC 3279 - "Algorithms and Identifiers for the Internet X.509 Public Key Infrastructure Certificate and Certificate Revocation List (CRL) Profile"
RFC 4055 - "Additional Algorithms and Identifiers for RSA Cryptography for use in the Internet X.509 Public Key Infrastructure Certificate and Certificate Revocation List (CRL) Profile"
RFC 5280 - "Internet X.509 Public Key Infrastructure Certificate and Certificate Revocation List (CRL) Profile"
RFC 7299 - "Object Identifier Registry for the PKIX Working Group"
RFC 7468 - "Textual Encodings of PKIX, PKCS, and CMS Structures"
RFC 8017 - "PKCS #1: RSA Cryptography Specifications Version 2.2"
RFC 8446 - "The Transport Layer Security (TLS) Protocol Version 1.3"
pyca/cryptography - The Python pyca/cryptography package

Feedback¶

Feel free to reach me on Twitter if you have questions. The GitHub issues page is the best place to submit corrections.

Public key cryptography: RSA keys

2018-04-25T13:00:00+01:00

I bet you created at least once an RSA key pair, usually because you needed to connect to GitHub and you wanted to avoid typing your password every time. You diligently followed the documentation on how to create SSH keys and after a couple of minutes your setup was complete.

But do you know what you actually did?

Do you know what the file ~/.ssh/id_rsa really contains? Why did ssh create two files with such a different format? Did you notice that one file begins with ssh-rsa, while the other begins with -----BEGIN RSA PRIVATE KEY-----? Have you noticed that sometimes the header of the second file misses the RSA part and just says BEGIN PRIVATE KEY?

I believe that a minimum level of knowledge regarding the various formats of RSA keys is mandatory for every developer nowadays, not to mention the importance of understanding them deeply if you want to pursue a career in the infrastructure management world.

RSA algorithm and key pairs¶

Since the invention of public-key cryptography, various systems have been devised to create the key pair. One of the first ones is RSA, the creation of three brilliant cryptographers, that dates back to 1977. The story of RSA is quite interesting, as it was first invented by an English mathematician, Clifford Cocks, who was however forced to keep it secret by the British intelligence office he was working for.

Keeping in mind that RSA is not a synonym for public-key cryptography but only one of the possible implementations, I wanted to write a post on it because it is still, more than 40 years after its publication, one of the most widespread algorithms. In particular it is the standard algorithm used to generate SSH key pairs, and since nowadays every developer has their public key on GitHub, BitBucket, or similar systems, we may arguably say that RSA is pretty ubiquitous.

I will not cover the internals of the RSA algorithm in this article, however. If you are interested in the gory details of the mathematical framework you may find plenty of resources both on Internet and in the textbooks. The theory behind it is not trivial, but it is definitely worth the time if you want to be serious about the mathematical part of cryptography.

In this article I will instead explore two ways to create RSA key pairs and the formats used to store them. Applied cryptography is, like many other topics in computer science, a moving target, and the tools change often. Sometimes it is pretty easy to find out how to do something (StackOverflow helps), but less easy to get a clear picture of what is going on.

All the examples shown in this post use a 2048-bits RSA key created for this purpose, so all the numbers you see come from a real example. The key has been obviously trashed after I wrote the article.

The PEM format¶

Let's start the discussion about key pairs with the format used to store them. Nowadays the most widely accepted storage format is called PEM (Privacy-enhanced Electronic Mail). As the name suggests, this format was initially created for e-mail encryption but later became a general format to store cryptographic data like keys and certificates. It is described in RFC 7468 ("Textual Encodings of PKIX, PKCS, and CMS Structures").

An example private key in PEM format is the following

Basically, you can tell you are dealing with a PEM format from the typical header and footer that identify the content. While the hyphens and the two words BEGIN and END are always present, the PRIVATE KEY part describes the content and can change if the PEM file contains something different from a key, for example an X.509 certificate for SSL.

The PEM format specifies that the the body of the content (the part between the header and the footer) is encoded using Base64.

If the private key has been encrypted with a password the header and the footer are different

When the PEM format is used to store cryptographic keys the body of the content is in a format called PKCS #8. Initially a standard created by a private company (RSA Laboratories), it became a de facto standard so has been described in various RFCs, most notably RFC 5208 ("Public-Key Cryptography Standards (PKCS) #8: Private-Key Information Syntax Specification Version 1.2").

The PKCS #8 format describes the content using a description language called ASN.1 (Abstract Syntax Notation One) and the relative binary encoding DER (Distinguished Encoding Rules) to serialise the resulting structure. This means that Base64-decoding the content will return some binary content that can be processed only by an ASN.1 parser.

Let me visually recap the structure

-----BEGIN label-----
+--------------------------- Base64 ---------------------------+
|                                                              |
| PKCS #8 content:                                             |
| ASN.1 language serialised with DER                           |
|                                                              |
+--------------------------------------------------------------+
-----END label-----

Please note that, due to the structure of the underlying ASN.1 structure, RSA PEM bodies start always with the same characters: MIG for 1024 bit keys, MII for 2048 and 4096 bit ones.

OpenSSL and ASN.1

OpenSSL can directly decode a key in PEM format and show the underlying ASN.1 structure with the module asn1parse

$ openssl asn1parse -inform pem -in private.pem
    0:d=0  hl=4 l=1214 cons: SEQUENCE          
    4:d=1  hl=2 l=   1 prim: INTEGER           :00
    7:d=1  hl=2 l=  13 cons: SEQUENCE          
    9:d=2  hl=2 l=   9 prim: OBJECT            :rsaEncryption
   20:d=2  hl=2 l=   0 prim: NULL              
   22:d=1  hl=4 l=1192 prim: OCTET STRING      [HEX DUMP]:308204A40201000282010100B2F5FD3F9F0917112
   CE42F8BF87ED676E15258BE443F36DEAFB0B69BDE2496B495EAAD1B01CAD84271B014E96F79386C636D348516DA74A68
   A8C70FBA882870C47B4218D8F49186DDF72727B9D80C21911C3E337C6E407FFB47C2F2767B0D164D8A1E9AF95F6481BF
   8D9EDFB2E3904B2529268C460256FAFD0A677D29898F10B1D15128A695839FC08EDD584E8335615B1D1D7277BE65C532
   DCA92DDC7050374868B117EA9154914EF9292B8443F13696E4FAD50DED6BD90E5A6F7ED33BE2ECE31C6DD7A4253EE6CD
   C56787DDD1D5CD776614022DB87D03BB22F23285B5A3167AF8DACABBEA40004471337D3781E8C5CCA0EA5E27799B510E
   4EF938C61CAA60D02030100010282010100B24255000A6A03901827333539511E4F4C21BA43CBB72BF0A51060D4E1719
   0AC50A871C57503986696D7CDFCB80D0726EFE2D76DBA55DFDC0425E064CC753810035C6A0F97AA37AB39E7C6215BC1E
   595131D0C3782E5A11213B59F42A1067F8CF43C538992D6BEFD1DE3F6293CE18ECC1173C4E7D6DD7362AD7323E7A218B
   5FFB0F245EB796327CC87493EDD134234ED5F3B14A4C4D92374597F64A6D3CB2C10F0CD2D57E99F58C8D28F2049D1433
   CC4BD677017AD1BDD1C83CFB8FB7E8C8FDCF0B4FB77DE7B8285749CEDFBFD6878F7F7930073F0F42ADDCBA8385D7ED05
   CDFCAA2A2BA757601723A96201FECCC2E65C65E14F65F1D34D6ECDFE3F85401800102818100E1D16389BF6EFF7AE44F6
   57106ED81C81A48B5FB356F83DD4A229E8654BDC036716BBD9D46DFD1498132545054958ACA5CFDA709D97CC8C6A9E92
   03D05F7B9D45E685A19A5F58267FCB17FCF502B32CFEDB94CAEA58EE5F63EBA5F33D09946C8652132344410D3D658748
   BCAE256F24896C2A9AD9340D3C8392652DA8ED7346D02818100CAE155C9B3A4546B5FC3CF4CC80D539D531C406BAC5ED
   82818E977B496F9F614CEFB1179E3BFBFAB22BCA7F88EBB8C9B1327AE70113242DFF0866370B6C76782DBD50DBE1FEE9
   B3316B9AAC7BABB7CFA0A9EF26C3C976CF62DA8F41EFE065458DC7C1CBCA78FB1CB4FF7AA50D116CE1640956A4E89EAD
   F5293FAA13A2349F42102818100BC3B93324E6D92EE7883AA366624F28ABF461ED3B0BE2CF7F805158939F815D20C075
   83E52C6DCA8DDD5FB2C1EE5AC9474A1476CD16ACFDDB1E24EEA2F204939BA1C58068B2D342FC4169D484D36451BC7B82
   F306176D53FC71809A5A25B320277320DAC3D949D504DD9907164EC3EF7BD1BB4DEA82160A7C4E3AA2ADEE88A9D02818
   02915E921A7D7A7A0F70BD8775C2C16BACD91F319DB1679FFE4CBA30A5768D784EF45B90C4E2B0ECDC18323211B06B03
   AD76E39CD482E3D8CCC50EAE270A1813CE6F80688723F07FF18A3110AD1AE16692CAD73BAA7AAA2CE5800D72F4F92489
   296542C1DA87159382B41A4A42933CD18848BBDB39A0A8E9F5288770E27075B010281803AB4E3B841AB234515BF0A8D2
   E40FB6E95389702D834474E9AD849124DC6C1D342738D4E7510265DF6B744EBAA4A88A7995346BEEF047DB024CE8B2A4
   E3923B0566389948AB0BBB031879770DA14F4418AEB75AE98349122A2D9535117B05BEF938A1211A3BE6E882957BC2A5
   F1DE5CA50C26F42EE0A383A2A2B6340D52E1A36

This that you see in the code snippet is then the private key in ASN.1 format. Remember that DER is only used to go from the text representation of ASN.1 to binary data, so we don't see it unless we decode the Base64 content into a file and open it with a binary editor.

Note that the ASN.1 structure contains the type of the object (rsaEncryption, in this case). You can further decode the OCTET STRING field, which is the actual key, specifying the offset

$ openssl asn1parse -inform pem -in private.pem -strparse 22
    0:d=0  hl=4 l=1188 cons: SEQUENCE          
    4:d=1  hl=2 l=   1 prim: INTEGER           :00
    7:d=1  hl=4 l= 257 prim: INTEGER           :B2F5FD3F9F0917112CE42F8BF87ED676E15258BE443F36DEAFB
    0B69BDE2496B495EAAD1B01CAD84271B014E96F79386C636D348516DA74A68A8C70FBA882870C47B4218D8F49186DDF
    72727B9D80C21911C3E337C6E407FFB47C2F2767B0D164D8A1E9AF95F6481BF8D9EDFB2E3904B2529268C460256FAFD
    0A677D29898F10B1D15128A695839FC08EDD584E8335615B1D1D7277BE65C532DCA92DDC7050374868B117EA9154914
    EF9292B8443F13696E4FAD50DED6BD90E5A6F7ED33BE2ECE31C6DD7A4253EE6CDC56787DDD1D5CD776614022DB87D03
    BB22F23285B5A3167AF8DACABBEA40004471337D3781E8C5CCA0EA5E27799B510E4EF938C61CAA60D
  268:d=1  hl=2 l=   3 prim: INTEGER           :010001
  273:d=1  hl=4 l= 257 prim: INTEGER           :B24255000A6A03901827333539511E4F4C21BA43CBB72BF0A51
    060D4E17190AC50A871C57503986696D7CDFCB80D0726EFE2D76DBA55DFDC0425E064CC753810035C6A0F97AA37AB39
    E7C6215BC1E595131D0C3782E5A11213B59F42A1067F8CF43C538992D6BEFD1DE3F6293CE18ECC1173C4E7D6DD7362A
    D7323E7A218B5FFB0F245EB796327CC87493EDD134234ED5F3B14A4C4D92374597F64A6D3CB2C10F0CD2D57E99F58C8
    D28F2049D1433CC4BD677017AD1BDD1C83CFB8FB7E8C8FDCF0B4FB77DE7B8285749CEDFBFD6878F7F7930073F0F42AD
    DCBA8385D7ED05CDFCAA2A2BA757601723A96201FECCC2E65C65E14F65F1D34D6ECDFE3F854018001
  534:d=1  hl=3 l= 129 prim: INTEGER           :E1D16389BF6EFF7AE44F657106ED81C81A48B5FB356F83DD4A2
    29E8654BDC036716BBD9D46DFD1498132545054958ACA5CFDA709D97CC8C6A9E9203D05F7B9D45E685A19A5F58267FC
    B17FCF502B32CFEDB94CAEA58EE5F63EBA5F33D09946C8652132344410D3D658748BCAE256F24896C2A9AD9340D3C83
    92652DA8ED7346D
  666:d=1  hl=3 l= 129 prim: INTEGER           :CAE155C9B3A4546B5FC3CF4CC80D539D531C406BAC5ED82818E
    977B496F9F614CEFB1179E3BFBFAB22BCA7F88EBB8C9B1327AE70113242DFF0866370B6C76782DBD50DBE1FEE9B3316
    B9AAC7BABB7CFA0A9EF26C3C976CF62DA8F41EFE065458DC7C1CBCA78FB1CB4FF7AA50D116CE1640956A4E89EADF529
    3FAA13A2349F421
  798:d=1  hl=3 l= 129 prim: INTEGER           :BC3B93324E6D92EE7883AA366624F28ABF461ED3B0BE2CF7F80
    5158939F815D20C07583E52C6DCA8DDD5FB2C1EE5AC9474A1476CD16ACFDDB1E24EEA2F204939BA1C58068B2D342FC4
    169D484D36451BC7B82F306176D53FC71809A5A25B320277320DAC3D949D504DD9907164EC3EF7BD1BB4DEA82160A7C
    4E3AA2ADEE88A9D
  930:d=1  hl=3 l= 128 prim: INTEGER           :2915E921A7D7A7A0F70BD8775C2C16BACD91F319DB1679FFE4C
    BA30A5768D784EF45B90C4E2B0ECDC18323211B06B03AD76E39CD482E3D8CCC50EAE270A1813CE6F80688723F07FF18
    A3110AD1AE16692CAD73BAA7AAA2CE5800D72F4F92489296542C1DA87159382B41A4A42933CD18848BBDB39A0A8E9F5
    288770E27075B01
1061:d=1  hl=3 l= 128 prim: INTEGER           :3AB4E3B841AB234515BF0A8D2E40FB6E95389702D834474E9AD8
    49124DC6C1D342738D4E7510265DF6B744EBAA4A88A7995346BEEF047DB024CE8B2A4E3923B0566389948AB0BBB0318
    79770DA14F4418AEB75AE98349122A2D9535117B05BEF938A1211A3BE6E882957BC2A5F1DE5CA50C26F42EE0A383A2A
    2B6340D52E1A36

Being this an RSA key the fields represent specific components of the algorithm. We find in order: the modulus n = pq, the public exponent e, the private exponent d, the two prime numbers p and q, and the values d_p, d_q, and q_inv (for the Chinese remainder theorem speed-up).

If the key has been encrypted there are fields with information about the cipher, and the OCTET STRING fields cannot be further parsed because of the encryption.

$ openssl asn1parse -inform pem -in private-enc.pem
    0:d=0  hl=4 l=1311 cons: SEQUENCE          
    4:d=1  hl=2 l=  73 cons: SEQUENCE          
    6:d=2  hl=2 l=   9 prim: OBJECT            :PBES2
   17:d=2  hl=2 l=  60 cons: SEQUENCE          
   19:d=3  hl=2 l=  27 cons: SEQUENCE          
   21:d=4  hl=2 l=   9 prim: OBJECT            :PBKDF2
   32:d=4  hl=2 l=  14 cons: SEQUENCE          
   34:d=5  hl=2 l=   8 prim: OCTET STRING      [HEX DUMP]:7FBE6B5C86A4B922
   44:d=5  hl=2 l=   2 prim: INTEGER           :0800
   48:d=3  hl=2 l=  29 cons: SEQUENCE          
   50:d=4  hl=2 l=   9 prim: OBJECT            :aes-256-cbc
   61:d=4  hl=2 l=  16 prim: OCTET STRING      [HEX DUMP]:7FC1CC749F456498F01E43108E4340DE
   79:d=1  hl=4 l=1232 prim: OCTET STRING      [HEX DUMP]:A5581EDC2797FC4E1AD0B66A00B765900AF1164D8
   F67458C1A4E72F54A65F2B8C0C5AD7E42584B95161FD98FBECA07D8E1049687C365ED157C45F1B57B175D2EF778A1FE7
   D12E50C0DF4248F0E1469DA40F9948581F16546F9582D9DCA83AC07C9466A6E3E6CE98CC241C44DAB32F5891B96DE302
   4B6E6A0F4980C6286D6EB8AA1680AD132810EEFB127DE42968142F4F9A4A2CE55A560C054C54DFFBB720A81F3F50A2A6
   3D748CE06309F55340BD4C74980C48F4C9D41650568A62BBE8E0337653BD4A2F7D47C3A24514B5D3100ED40C164831C6
   5A96DC90AD20F4AEF02E00203B0F0B2D550987AEE8F4C7E0E7C0CFF426B465D3CF568D02EE86AF043345954B0AAA649F
   A9F80E026E2A189EC60772A058615DCFEC9EC4D2D12CDEB7844EAA00202E435A0B9B0A28AC4F2DA213214F773A2319D5
   5A560D5C99246F9895F5EF04D97FF1CE26EFC2FF82249F6E94253CB92EE0A74AE3942285C2DFFC77883709E7FF2569FD
   9C8F58C112CD4A125E40E7BC8599242D71DE7D48416B6A36FBE0B90BA9A05AFB982CAE9AD337C2318582AA328ABC341F
   BB1C036DE334DE327DEC97BA757CBBAED26F25DD74BD8BE9215B479CD49D8357AFA5289A0265ADE025F9FC0CDB1CDBF0
   4C812F20B7CEB58BF12C1FD1756AABD7F557B87E1D245E8062D1DF4078D77AD98BFDF0C0F3A06A7FA11BFAE0EBF8F3EE
   1F8AB0D6D7C905D4D238E2738613EA753E044589CEBDF3714CACEC298653FA45AF5977BDCF23B5DD60B479C7958B8AC1
   8CAA4AA4A79C283805246675BBB8D2D0E5B714320E7E6FE8B2EF73DB9839095229B9653726AB9689B19AB47113F70204
   83B2D1A82FE2EB9ABAB429DDF5ACDEBCAB62BABD48D2DBA1D398B03F9919F1DAC8CDA19D39BBAF2B5FE96C43E78F565C
   465019DF88E71BCE35C6F7F8BE87EB384FA1193345E47CA9382BCEFFC2E6B37681E8D95EB48BC7044F7DCA743217D4C0
   81200502E98EC2CFAA9D17277D5385E65CC8104DA999E31532A8B9B3B4D3E219613AE09BC9F10553CC4E5F135ACD3FB4
   A3BBAB21839CEFBBC0D4BB16AE4FBD7407E6E3709B059BD86AFFE032805CE5FB0B8005009B5964B79E478DA7FE88C20D
   D2FEDA10A0EB3433ADC90AF5DD8772B840A5CD7C5E32D96153E41F12BA501EF1F48C4E20CB0120CFBB6F546C2B6E22E0
   834CB9DFBFA4834FEB4B7374788F781A1634ABF9D1FD014E6DB3749E6A086155521ADB9F271D6BF6F60455903B1D913D
   A639EE9F5CA5135FD2A1873FF35EAB8C151C5B90826E4303233D4BB053EBD929107874CDCCADFFF492A7CB595EADF03E
   4C0FE15326752898F1B9AA3EAC9907D9F276E6AB37AFA34FF8F3DBAB7B009754CF1A13029CD6857686105830F0CF6E99
   476CB07ECAAEA8B5CCC2720479423F8504E783D6712E424C636DAB41203D9EC76F47C4B56F453C42E5626048C24CC585
   F0710514EEF6D4C9644E0721CEAE9F885FBD672742A555095A895C7F0D4E814BEF4D223B13285E95BEDF7357D3545784
   32C1EBB63A6EF1D83E21A08DADA073BF9419C7A3185BB492A13569F262683E7CD86EC66CF671C919789038598EFEC22B
   C8EA1E265A4E0864F9E7253BE32457AC1B186722F3D0FF4AD450D04BA97D5B7DC1AA617DBD25EE8EC912072ABCBF5394
   D08AA276732666D4C349196940BFE869DA909EC03A8E25B23339EE50453CB5F81400B1380CA46AF0FC012CA55F322C1C
   5806E5D76D4CD8308B8FDFE

OpenSSL and RSA keys

Another way to look into a private key with OpenSSL is to use the module rsa. While the module asn1parse is a generic ASN.1 parser, the module rsa knows the structure of an RSA key and can properly output the field names

$ openssl rsa -in private.pem -noout -text
Private-Key: (2048 bit)
modulus:
    00:b2:f5:fd:3f:9f:09:17:11:2c:e4:2f:8b:f8:7e:
    d6:76:e1:52:58:be:44:3f:36:de:af:b0:b6:9b:de:
    24:96:b4:95:ea:ad:1b:01:ca:d8:42:71:b0:14:e9:
    6f:79:38:6c:63:6d:34:85:16:da:74:a6:8a:8c:70:
    fb:a8:82:87:0c:47:b4:21:8d:8f:49:18:6d:df:72:
    72:7b:9d:80:c2:19:11:c3:e3:37:c6:e4:07:ff:b4:
    7c:2f:27:67:b0:d1:64:d8:a1:e9:af:95:f6:48:1b:
    f8:d9:ed:fb:2e:39:04:b2:52:92:68:c4:60:25:6f:
    af:d0:a6:77:d2:98:98:f1:0b:1d:15:12:8a:69:58:
    39:fc:08:ed:d5:84:e8:33:56:15:b1:d1:d7:27:7b:
    e6:5c:53:2d:ca:92:dd:c7:05:03:74:86:8b:11:7e:
    a9:15:49:14:ef:92:92:b8:44:3f:13:69:6e:4f:ad:
    50:de:d6:bd:90:e5:a6:f7:ed:33:be:2e:ce:31:c6:
    dd:7a:42:53:ee:6c:dc:56:78:7d:dd:1d:5c:d7:76:
    61:40:22:db:87:d0:3b:b2:2f:23:28:5b:5a:31:67:
    af:8d:ac:ab:be:a4:00:04:47:13:37:d3:78:1e:8c:
    5c:ca:0e:a5:e2:77:99:b5:10:e4:ef:93:8c:61:ca:
    a6:0d
publicExponent: 65537 (0x10001)
privateExponent:
    00:b2:42:55:00:0a:6a:03:90:18:27:33:35:39:51:
    1e:4f:4c:21:ba:43:cb:b7:2b:f0:a5:10:60:d4:e1:
    71:90:ac:50:a8:71:c5:75:03:98:66:96:d7:cd:fc:
    b8:0d:07:26:ef:e2:d7:6d:ba:55:df:dc:04:25:e0:
    64:cc:75:38:10:03:5c:6a:0f:97:aa:37:ab:39:e7:
    c6:21:5b:c1:e5:95:13:1d:0c:37:82:e5:a1:12:13:
    b5:9f:42:a1:06:7f:8c:f4:3c:53:89:92:d6:be:fd:
    1d:e3:f6:29:3c:e1:8e:cc:11:73:c4:e7:d6:dd:73:
    62:ad:73:23:e7:a2:18:b5:ff:b0:f2:45:eb:79:63:
    27:cc:87:49:3e:dd:13:42:34:ed:5f:3b:14:a4:c4:
    d9:23:74:59:7f:64:a6:d3:cb:2c:10:f0:cd:2d:57:
    e9:9f:58:c8:d2:8f:20:49:d1:43:3c:c4:bd:67:70:
    17:ad:1b:dd:1c:83:cf:b8:fb:7e:8c:8f:dc:f0:b4:
    fb:77:de:7b:82:85:74:9c:ed:fb:fd:68:78:f7:f7:
    93:00:73:f0:f4:2a:dd:cb:a8:38:5d:7e:d0:5c:df:
    ca:a2:a2:ba:75:76:01:72:3a:96:20:1f:ec:cc:2e:
    65:c6:5e:14:f6:5f:1d:34:d6:ec:df:e3:f8:54:01:
    80:01
prime1:
    00:e1:d1:63:89:bf:6e:ff:7a:e4:4f:65:71:06:ed:
    81:c8:1a:48:b5:fb:35:6f:83:dd:4a:22:9e:86:54:
    bd:c0:36:71:6b:bd:9d:46:df:d1:49:81:32:54:50:
    54:95:8a:ca:5c:fd:a7:09:d9:7c:c8:c6:a9:e9:20:
    3d:05:f7:b9:d4:5e:68:5a:19:a5:f5:82:67:fc:b1:
    7f:cf:50:2b:32:cf:ed:b9:4c:ae:a5:8e:e5:f6:3e:
    ba:5f:33:d0:99:46:c8:65:21:32:34:44:10:d3:d6:
    58:74:8b:ca:e2:56:f2:48:96:c2:a9:ad:93:40:d3:
    c8:39:26:52:da:8e:d7:34:6d
prime2:
    00:ca:e1:55:c9:b3:a4:54:6b:5f:c3:cf:4c:c8:0d:
    53:9d:53:1c:40:6b:ac:5e:d8:28:18:e9:77:b4:96:
    f9:f6:14:ce:fb:11:79:e3:bf:bf:ab:22:bc:a7:f8:
    8e:bb:8c:9b:13:27:ae:70:11:32:42:df:f0:86:63:
    70:b6:c7:67:82:db:d5:0d:be:1f:ee:9b:33:16:b9:
    aa:c7:ba:bb:7c:fa:0a:9e:f2:6c:3c:97:6c:f6:2d:
    a8:f4:1e:fe:06:54:58:dc:7c:1c:bc:a7:8f:b1:cb:
    4f:f7:aa:50:d1:16:ce:16:40:95:6a:4e:89:ea:df:
    52:93:fa:a1:3a:23:49:f4:21
exponent1:
    00:bc:3b:93:32:4e:6d:92:ee:78:83:aa:36:66:24:
    f2:8a:bf:46:1e:d3:b0:be:2c:f7:f8:05:15:89:39:
    f8:15:d2:0c:07:58:3e:52:c6:dc:a8:dd:d5:fb:2c:
    1e:e5:ac:94:74:a1:47:6c:d1:6a:cf:dd:b1:e2:4e:
    ea:2f:20:49:39:ba:1c:58:06:8b:2d:34:2f:c4:16:
    9d:48:4d:36:45:1b:c7:b8:2f:30:61:76:d5:3f:c7:
    18:09:a5:a2:5b:32:02:77:32:0d:ac:3d:94:9d:50:
    4d:d9:90:71:64:ec:3e:f7:bd:1b:b4:de:a8:21:60:
    a7:c4:e3:aa:2a:de:e8:8a:9d
exponent2:
    29:15:e9:21:a7:d7:a7:a0:f7:0b:d8:77:5c:2c:16:
    ba:cd:91:f3:19:db:16:79:ff:e4:cb:a3:0a:57:68:
    d7:84:ef:45:b9:0c:4e:2b:0e:cd:c1:83:23:21:1b:
    06:b0:3a:d7:6e:39:cd:48:2e:3d:8c:cc:50:ea:e2:
    70:a1:81:3c:e6:f8:06:88:72:3f:07:ff:18:a3:11:
    0a:d1:ae:16:69:2c:ad:73:ba:a7:aa:a2:ce:58:00:
    d7:2f:4f:92:48:92:96:54:2c:1d:a8:71:59:38:2b:
    41:a4:a4:29:33:cd:18:84:8b:bd:b3:9a:0a:8e:9f:
    52:88:77:0e:27:07:5b:01
coefficient:
    3a:b4:e3:b8:41:ab:23:45:15:bf:0a:8d:2e:40:fb:
    6e:95:38:97:02:d8:34:47:4e:9a:d8:49:12:4d:c6:
    c1:d3:42:73:8d:4e:75:10:26:5d:f6:b7:44:eb:aa:
    4a:88:a7:99:53:46:be:ef:04:7d:b0:24:ce:8b:2a:
    4e:39:23:b0:56:63:89:94:8a:b0:bb:b0:31:87:97:
    70:da:14:f4:41:8a:eb:75:ae:98:34:91:22:a2:d9:
    53:51:17:b0:5b:ef:93:8a:12:11:a3:be:6e:88:29:
    57:bc:2a:5f:1d:e5:ca:50:c2:6f:42:ee:0a:38:3a:
    2a:2b:63:40:d5:2e:1a:36

The fields are the same we found in the ASN.1 structure, but in this representation we have a better view of the specific values of the RSA key. You can compare the two and see that the value of the fields are the same.

If you want to learn something about RSA, try to investigate the historical reasons behind the choice of 65537 as a common public exponent (as you can see here in the section publicExponent).

PKCS #8 vs PKCS #1

The first version of the PKCS standard (PKCS #1) was specifically tailored to contain an RSA key. Its ASN.1 definition can be found in RFC 8017 ("PKCS #1: RSA Cryptography Specifications Version 2.2")

RSAPublicKey ::= SEQUENCE {
    modulus           INTEGER,  -- n
    publicExponent    INTEGER   -- e
}

RSAPrivateKey ::= SEQUENCE {
    version           Version,
    modulus           INTEGER,  -- n
    publicExponent    INTEGER,  -- e
    privateExponent   INTEGER,  -- d
    prime1            INTEGER,  -- p
    prime2            INTEGER,  -- q
    exponent1         INTEGER,  -- d mod (p-1)
    exponent2         INTEGER,  -- d mod (q-1)
    coefficient       INTEGER,  -- (inverse of q) mod p
    otherPrimeInfos   OtherPrimeInfos OPTIONAL
}

Subsequently, as the need to describe new types of algorithms increased, the PKCS #8 standard was developed. This can contain different types of keys, and defines a specific field for the algorithm identifier. Its ASN.1 definition can be found in RFC 5958 ("Asymmetric Key Packages")

OneAsymmetricKey ::= SEQUENCE {
     version                   Version,
     privateKeyAlgorithm       PrivateKeyAlgorithmIdentifier,
     privateKey                PrivateKey,
     attributes            [0] Attributes OPTIONAL,
     ...,
     [[2: publicKey        [1] PublicKey OPTIONAL ]],
     ...
   }

PrivateKey ::= OCTET STRING
                     -- Content varies based on type of key. The
                     -- algorithm identifier dictates the format of
                     -- the key.

The definition of the field PrivateKey for the RSA algorithm is the same used in PKCS #1.

If the PEM format uses PKCS #8 its header and footer are

-----BEGIN PRIVATE KEY-----
[...]
-----END PRIVATE KEY-----

If it uses PKCS #1, however, there has to be an external identification of the algorithm, so the header and footer are

-----BEGIN RSA PRIVATE KEY-----
[...]
-----END RSA PRIVATE KEY-----

The structure of PKCS #8 is the reason why we had to parse the field at offset 22 to access the RSA parameters when using the module asn1parse of OpenSSL. If you are parsing a PKCS #1 key in PEM format you don't need this second step.

Private and public key¶

In the RSA algorithm the public key is built using the modulus and the public exponent, which means that we can always derive the public key from the private key. OpenSSL can easily do this with the module rsa, producing the public key in PEM format

$ openssl rsa -in private.pem -pubout
writing RSA key
-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAsvX9P58JFxEs5C+L+H7W
duFSWL5EPzber7C2m94klrSV6q0bAcrYQnGwFOlveThsY200hRbadKaKjHD7qIKH
DEe0IY2PSRht33Jye52AwhkRw+M3xuQH/7R8LydnsNFk2KHpr5X2SBv42e37LjkE
slKSaMRgJW+v0KZ30piY8QsdFRKKaVg5/Ajt1YToM1YVsdHXJ3vmXFMtypLdxwUD
dIaLEX6pFUkU75KSuEQ/E2luT61Q3ta9kOWm9+0zvi7OMcbdekJT7mzcVnh93R1c
13ZhQCLbh9A7si8jKFtaMWevjayrvqQABEcTN9N4Hoxcyg6l4neZtRDk75OMYcqm
DQIDAQAB
-----END PUBLIC KEY-----

You can dump the information in the public key specifying the flag -pubin

$ openssl rsa -in public.pem -noout -text -pubin
Public-Key: (2048 bit)
Modulus:
    00:b2:f5:fd:3f:9f:09:17:11:2c:e4:2f:8b:f8:7e:
    d6:76:e1:52:58:be:44:3f:36:de:af:b0:b6:9b:de:
    24:96:b4:95:ea:ad:1b:01:ca:d8:42:71:b0:14:e9:
    6f:79:38:6c:63:6d:34:85:16:da:74:a6:8a:8c:70:
    fb:a8:82:87:0c:47:b4:21:8d:8f:49:18:6d:df:72:
    72:7b:9d:80:c2:19:11:c3:e3:37:c6:e4:07:ff:b4:
    7c:2f:27:67:b0:d1:64:d8:a1:e9:af:95:f6:48:1b:
    f8:d9:ed:fb:2e:39:04:b2:52:92:68:c4:60:25:6f:
    af:d0:a6:77:d2:98:98:f1:0b:1d:15:12:8a:69:58:
    39:fc:08:ed:d5:84:e8:33:56:15:b1:d1:d7:27:7b:
    e6:5c:53:2d:ca:92:dd:c7:05:03:74:86:8b:11:7e:
    a9:15:49:14:ef:92:92:b8:44:3f:13:69:6e:4f:ad:
    50:de:d6:bd:90:e5:a6:f7:ed:33:be:2e:ce:31:c6:
    dd:7a:42:53:ee:6c:dc:56:78:7d:dd:1d:5c:d7:76:
    61:40:22:db:87:d0:3b:b2:2f:23:28:5b:5a:31:67:
    af:8d:ac:ab:be:a4:00:04:47:13:37:d3:78:1e:8c:
    5c:ca:0e:a5:e2:77:99:b5:10:e4:ef:93:8c:61:ca:
    a6:0d
Exponent: 65537 (0x10001)

Generating key pairs with OpenSSL¶

If you want to generate an RSA private key you can do it with OpenSSL

$ openssl genpkey -algorithm RSA -out private.pem \
  -pkeyopt rsa_keygen_bits:2048
......................................................................+++
..........+++

Since OpenSSL is a collection of modules, we specify genpkey to generate a private key. The option -algorithm specifies which algorithm we want to use to generate the key (RSA in this case), -out specifies the name of the output file, and -pkeyopt allows us to set the value for specific key options. In this case the length of the RSA key in bits.

If you want an encrypted key you can generate one specifying the cipher (for example -aes-256-cbc)

$ openssl genpkey -algorithm RSA -out private-enc.pem \
  -aes-256-cbc -pkeyopt rsa_keygen_bits:2048
...........................+++
..........+++
Enter PEM pass phrase:
Verifying - Enter PEM pass phrase:

You can see the list of supported ciphers with openssl list-cipher-algorithms. In both cases you can then extract the public key with the method shown previously. OpenSSL private keys are created using PKCS #8, so unencrypted keys will be in the form

-----BEGIN PRIVATE KEY-----
[...]
-----END PRIVATE KEY-----

and encrypted ones in the form

-----BEGIN ENCRYPTED PRIVATE KEY-----
[...]
-----END ENCRYPTED PRIVATE KEY-----

Generating key pairs with OpenSSH¶

Another tool that you can use to generate key pairs is ssh-keygen, which is a tool included in the SSH suite that is specifically used to create and manage SSH keys. As SSH keys are standard asymmetrical keys we can use the tool to create keys for other purposes.

To create a key pair just run

ssh-keygen -m PEM -t rsa -b 2048 -f key

The option -m specifies the key format. By default OpenSSH uses its own format specified in RFC 4716 ("The Secure Shell (SSH) Public Key File Format").

The option -t specifies the key generation algorithm (RSA in this case), while the option -b specifies the length of the key in bits.

The option -f sets the name of the output file. If not present, ssh-keygen will ask the name of the file, offering to save it to the default file ~/.ssh/id_rsa. The tool always asks for a password to encrypt the key, but you are allowed to enter an empty one to skip the encryption.

This tool creates two files. One is the private key file, named as requested, and the second is the public key file, named like the private key one but with the extension .pub.

The value PEM specified for the option -m writes the private key using the PKCS #1 format, so the key will be in the form

-----BEGIN RSA PRIVATE KEY-----
[...]
-----END RSA PRIVATE KEY-----

Using -m PKCS8 instead uses PKCS #8 and the key will be in the form

-----BEGIN PRIVATE KEY-----
[...]
-----END PRIVATE KEY-----

The OpenSSH public key format

The public key saved by ssh-keygen is written in the so-called SSH-format, which is not a standard in the cryptography world. It's structure is ALGORITHM KEY COMMENT, where the KEY part of the format is encoded with Base64.

For example

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCy9f0/nwkXESzkL4v4ftZ24VJYvkQ/Nt6vsLab3iSWtJXqrRsBythCcbAU6W9
5OGxjbTSFFtp0poqMcPuogocMR7QhjY9JGG3fcnJ7nYDCGRHD4zfG5Af/tHwvJ2ew0WTYoemvlfZIG/jZ7fsuOQSyUpJoxGAlb6
/QpnfSmJjxCx0VEoppWDn8CO3VhOgzVhWx0dcne+ZcUy3Kkt3HBQN0hosRfqkVSRTvkpK4RD8TaW5PrVDe1r2Q5ab37TO+Ls4xx
t16QlPubNxWeH3dHVzXdmFAItuH0DuyLyMoW1oxZ6+NrKu+pAAERxM303gejFzKDqXid5m1EOTvk4xhyqYN user@host

To manually decode the central part of the key you can use base64 and hexdump

$ cat key.pub | cut -d " " -f2 | \
  base64 -d | hexdump -ve '/1 "%02x "' -e '2/8 "\n"'
00 00 00 07 73 73 68 2d 72 73 61 00 00 00 03 01
00 01 00 00 01 01 00 b2 f5 fd 3f 9f 09 17 11 2c
e4 2f 8b f8 7e d6 76 e1 52 58 be 44 3f 36 de af
b0 b6 9b de 24 96 b4 95 ea ad 1b 01 ca d8 42 71
b0 14 e9 6f 79 38 6c 63 6d 34 85 16 da 74 a6 8a
8c 70 fb a8 82 87 0c 47 b4 21 8d 8f 49 18 6d df
72 72 7b 9d 80 c2 19 11 c3 e3 37 c6 e4 07 ff b4
7c 2f 27 67 b0 d1 64 d8 a1 e9 af 95 f6 48 1b f8
d9 ed fb 2e 39 04 b2 52 92 68 c4 60 25 6f af d0
a6 77 d2 98 98 f1 0b 1d 15 12 8a 69 58 39 fc 08
ed d5 84 e8 33 56 15 b1 d1 d7 27 7b e6 5c 53 2d
ca 92 dd c7 05 03 74 86 8b 11 7e a9 15 49 14 ef
92 92 b8 44 3f 13 69 6e 4f ad 50 de d6 bd 90 e5
a6 f7 ed 33 be 2e ce 31 c6 dd 7a 42 53 ee 6c dc
56 78 7d dd 1d 5c d7 76 61 40 22 db 87 d0 3b b2
2f 23 28 5b 5a 31 67 af 8d ac ab be a4 00 04 47
13 37 d3 78 1e 8c 5c ca 0e a5 e2 77 99 b5 10 e4
ef 93 8c 61 ca a6 0d

The structure of this binary file is pretty simple, and is described in two different RFCs. RFC 4253 ("SSH Transport Layer Protocol") states in section 6.6 that

The "ssh-rsa" key format has the following specific encoding:

      string    "ssh-rsa"
      mpint     e
      mpint     n

while the definition of the types string and mpint can be found in RFC 4251 ("SSH Protocol Architecture"), section 5

string

    [...] They are stored as a uint32 containing its length
    (number of bytes that follow) and zero (= empty string) or more
    bytes that are the value of the string.  Terminating null
    characters are not used. [...]

mpint

    Represents multiple precision integers in two's complement format,
    stored as a string, 8 bits per byte, MSB first. [...]

This means that the above sequence of bytes is interpreted as 4 bytes of length (32 bits of the type uint32) followed by that number of bytes of content.

(4 bytes)   00 00 00 07          = 7
(7 bytes)   73 73 68 2d 72 73 61 = "ssh-rsa" (US-ASCII)
(4 bytes)   00 00 00 03          = 3
(3 bytes)   01 00 01             = 65537 (a common value for the RSA exponent)
(4 bytes)   00 00 01 01          = 257
(257 bytes) 00 b2 .. ca a6 0d    = The key modulus

Please note that since we created a key of 2048 bits we should have a modulus of 256 bytes. Instead this key uses 257 bytes prefixing the number with a byte 00 to avoid it being interpreted as negative (two's complement format).

The structure shown above is the reason why all the RSA public SSH keys start with the same 12 characters AAAAB3NzaC1y. This string, converted in Base64 gives the initial 9 bytes 00 00 00 07 73 73 68 2d 72 (Base64 characters are not a one-to-one mapping of the source bytes). If the exponent is the standard 65537 the key starts with AAAAB3NzaC1yc2EAAAADAQAB, which encoded gives the fist 18 bytes 00 00 00 07 73 73 68 2d 72 73 61 00 00 00 03 01 00 01.

Converting between PEM and OpenSSH format¶

We often need to convert files created with one tool to a different format, so this is a list of the most common conversions you might need. I prefer to consider the key format instead of the source tool, but I give a short description of the reason why you should want to perform the conversion.

PEM/PKCS#1 to PEM/PKCS#8

This is useful to convert OpenSSH private keys to a newer format.

openssl pkcs8 -topk8 -inform PEM -outform PEM -in pkcs1.pem -out pkcs8.pem

OpenSSH public to PEM/PKCS#8

To convert public OpenSSH keys in a PEM format using PKCS #8 (prints to stdout)

ssh-keygen -e -f public.pub -m PKCS8

This is easy to remember because -e stands for export. Note that you can also use -m PEM to convert the key into a PEM format that uses PKCS #1.

PEM/PKCS#8 to OpenSSH public

If you need to use in SSH a key pair created with another system

ssh-keygen -i -f public.pem -m PKCS8

This is easy to remember because -i stands for import. As happened when exporting the key, you can import a PEM/PKCS #1 key using -m PEM.

Reading RSA keys in Python¶

In Python you can use the package pycrypto to access a PEM file containing an RSA key with the function RSA.importKey. Now you can hopefully understand the documentation that says

externKey (string) - The RSA key to import, encoded as a string.

An RSA public key can be in any of the following formats:
    * X.509 subjectPublicKeyInfo DER SEQUENCE (binary or PEM encoding)
    * PKCS#1 RSAPublicKey DER SEQUENCE (binary or PEM encoding)
    * OpenSSH (textual public key only)

An RSA private key can be in any of the following formats:
    * PKCS#1 RSAPrivateKey DER SEQUENCE (binary or PEM encoding)
    * PKCS#8 PrivateKeyInfo DER SEQUENCE (binary or PEM encoding)
    * OpenSSH (textual public key only)

For details about the PEM encoding, see RFC1421/RFC1423.

In case of PEM encoding, the private key can be encrypted with DES or 3TDES
according to a certain pass phrase. Only OpenSSL-compatible pass phrases are
supported.

In practice what you can do with a file private.pem is

from Crypto.PublicKey import RSA

f = open('private.pem', 'r')
key = RSA.importKey(f.read())

and the variable key will contain an instance of _RSAobj (not a very pythonic name, to be honest). This instance contains the RSA parameters as attributes as stated in the documentation

modulus = key.n
public_exponent = key.e
private_exponent = key.d
first_prime_number = key.p
second_prime_number = key.q
q_inv_crt = key.u

Final words¶

I keep finding on StackOverflow (and on other boards) messages of users that are confused by RSA keys, the output of the various tools, and by the subtle but important differences between the formats, so I hope this post helped you to get a better understanding of the matter.

Resources¶

The Wikipedia article on RSA
OpenSSL documentation: asn1parse, rsa, genpkey
The Base64 encoding
The Abstract Syntax Notation One ASN.1 interface description language
RFC 4251 - The Secure Shell (SSH) Protocol Architecture
RFC 4253 - The Secure Shell (SSH) Transport Layer Protocol
RFC 4716 - The Secure Shell (SSH) Public Key File Format
RFC 5208 - Public-Key Cryptography Standards (PKCS) #8: Private-Key Information Syntax Specification Version 1.2
RFC 5958 - Asymmetric Key Packages
RFC 7468 - Textual Encodings of PKIX, PKCS, and CMS Structures
RFC 8017 - PKCS #1: RSA Cryptography Specifications Version 2.2
PyCrypto - The Python Cryptography Toolkit

Feedback¶

Feel free to reach me on Twitter if you have questions. The GitHub issues page is the best place to submit corrections.

The Digital Cat - SSH

Public key cryptography: OpenSSH private keys

KDFs and protection at rest¶

Key Derivation Functions

bcrypt

Protection at rest

PEM format and protection at rest¶

OpenSSH's private key format¶

Cost factor and rounds

How many rounds?

Can we convert private OpenSSH keys into PEM?¶

A poorly documented format¶

Resources¶

Feedback¶

Public key cryptography: SSL certificates

Clarification: SSL vs TLS¶

X.509 certificates¶

How are certificates related to HTTPS?¶

How do certificates work?¶

Example: CA root certificate¶

Example: self-signed certificate¶

Example: this site's certificate¶

How to verify certificates with OpenSSL¶

Low-level certificate validation process¶

Algorithms used by root certificates¶

AWS components related to certificates¶

Let's encrypt¶

Final words¶

Resources¶

Feedback¶

Public key cryptography: RSA keys

RSA algorithm and key pairs¶

The PEM format¶

OpenSSL and ASN.1

OpenSSL and RSA keys

PKCS #8 vs PKCS #1

Private and public key¶

Generating key pairs with OpenSSL¶

Generating key pairs with OpenSSH¶

The OpenSSH public key format

Converting between PEM and OpenSSH format¶

PEM/PKCS#1 to PEM/PKCS#8

OpenSSH public to PEM/PKCS#8

PEM/PKCS#8 to OpenSSH public

Reading RSA keys in Python¶

Final words¶

Resources¶

Feedback¶