Virtual Private Networks

This article is Part 3 in a series of three (3) articles about how Virtual Private Networks (VPNs) work. While I get into some technical details, the intent is to scratch the surface just enough to convey the most important security and privacy concepts about each subject.

Part 1 of this series, How VPNs Work outlines VPN functionality at a high level. Part 2 (Protocols) in this series examines VPN protocols and VPN server compatibility. And this article (Part 3) explores security and encryption risks in VPNs, such as encryption ciphers and Forward Secrecy.

Topics

• Terminology (security related)
• Encapsulated Tunnel Formation
• Key Negotiation
• Authentication
• Data Encryption
• Obfuscation
• Perfect Forward Secrecy (PFS)
• Privacy and Confidentiality Through Cryptography
• Key Agreement and Authentication Protocols
• Payload Protection
• Common Encryption Ciphers
• Poor Habits of VPN Service Providers

Terminology

When researching VPN features, you will find terms such as authentication, confidentiality, privacy, anonymity, and integrity. What do these words mean in the context of VPN protection? And btw, what is a hash?

Anonymity: Refers to concealing the IP header, disguising the true source identity of the original packet or payload. This can be achieved through encryption, authentication, both, or neither. It depends on the security protocol methods.

Authentication: Key negotiation and validating a packet's sender via previously exchanged security keys.

Authenticity: See definition for integrity.

Confidentiality: Data encryption. Whether or not an IP packet or payload's contents (data) can be read in transit. In other words, are the communication's contents encrypted?

Integrity: Validation the IP packet and/or payload were not tampered with in transit (sometimes referred to as tunnel integrity)

Hash: A hash is a checksum value. It is derived from a one-way calculation (called hashing). A checksum value is calculated by applying an algorithm to all the bits in a string of characters.

Privacy: Usually refers to confidentiality. Occasionally used to reference anonymity.

Encapsulated Tunnel Formation

A VPN connection between two (2) network peers is a secure, bi-directional communications channel. The first step in establishing a VPN is both network peers must determine a compatible protocol by which they will establish a VPN framework and secure connection. If the parties cannot establish a mutually agreeable VPN protocol, then obviously there won't be a VPN. This can be a very simple process - such as if one (1) peer only supports a certain protocol - or it can be very flexible, such as when the VPN server is running Streisand or SoftEther, both of which support a wide variety of VPN protocols. Acceptable VPN protocols are determined by the server.

This initial dialogue between network peers normally often takes place over an insecure connection. Sometimes it will occur over a secure connection, but this requires the peers to have previously agreed upon a pre-shared key or the VPN server uses PKI (Public Key Infrastructure) with a private key, and the connecting peer has ability to acquire the public key. A VPN server might also use X.509 certificates, a form of PKI where an independent Certificate Authority acts as an intermediary to validate the authentication keys. If any of these conditions exist then the VPN negotiation dialogue will begin over a secure connection.

Depending on the system architecture, the peer-to-peer dialogue discussing the VPN protocol that will be used could still happen under an insecure channel. The exact details and steps of agreeing on a VPN protocol are completely dependent on the VPN server's implementation.

GRE: When is a VPN not a VPN?

I'll briefly mention a non-VPN protocol that can be mistaken for a VPN. It's called Generic Routing Encapsulation (GRE). Defined by RFC 2784 and RFC 2890, GRE is a common tunneling protocol and an alternative to IPsec tunnels.

Why would anyone want to create a GRE tunnel? Two (2) main reasons: Faster data throughput compared with IPsec; and GRE is a Layer 2 encapsulation protocol, meaning any traffic conforming to the Ethernet standard is acceptable. The latter is not true with IPsec, which is a Layer 3 protocol that can only ferret IP packets. Thus, GRE has a broader range of usefulness compared with many other secure tunneling protocols. However, this comes at a cost: GRE performs authentication only on the encapsulated transmission. That means no privacy protection and no data protection. And GRE provides authentication only through a checksum header that is part of the GRE protocol. It's not a very sophisticated protocol, and may be easily manipulated via Man-in-the-Middle (MitM) attack vectors.

Key Negotiation

Once a VPN protocol is agreed upon, a security policy must be determined between the peers. The first step is key negotiation.

Key negotiation solidifies the encryption standard that will be used by both network peers to encrypt and decrypt the IP packets send back and forth to one another. The implementation of the agreed encryption key along with an agreed encryption algorithm will together create the secure tunnel that forms the outer layer of the VPN.

Most types of key negotiation are single phase or dual phase.

Single-Phase: A single key is negotiated and implemented. Both peers (client and server) negotiate a framework and set of ciphers they will use to implement a secure communication between them, called a security policy.

Dual-Phase: In the dual-phase model the first phase is the same as the single-phase model. The 2nd phase uses a different key negotiation process. The advantage of the two-phase approach is it is much more difficult for a malicious actor to break the encryption. The 2nd phase is processed immediately after the first, making it nearly impossible for a third party to have any chance of decrypting the VPN's encapsulation layer.¹

• Phase 1: Same as the single-phase model. Both peers (client and server) negotiate a set of ciphers they will use to continue a conversation about security.
• Phase 2: A secure connection is created between the peers, and another negotiation occurs where the peers agree on the actual security methods that will be implemented to establish a secure communication between them. This is the security policy between the peers.

Why Single-Phase Key Negotiation is a Bad Idea

Single phase key negotiation protocols should be avoided. They are inherently less secure for several reasons:

1. The key encryption method is negotiated by the peers on insecure channels.
2. The same encryption key is maintained until the connection is closed.
3. In some cases, the same encryption key is re-used for future VPN connections.

If you are interested in implementing Perfect Forward Secrecy, the client and server must both be configured to use the 2-phase key negotiation model. Unfortunately, the single-phase method is a common strategy with VPN service providers. This issue and some of the ways it increases the risk to customers are discussed below in the section, Poor Habits of VPN Service Providers.

Authentication

Authentication provides integrity. The VPN peers may or may not have agreed to utilize an authentication security protocol. If they have, during packet exchanges over the secure communication link, a hash is calculated from a portion of the packet and inserted into the data stream. The hash value is used to validate the authenticity of transmitted data on the receiving end. When the IP packet arrives at the receiving peer, the receiver runs the an authentication check on the packet using whichever cipher algorithm was agreed to by the peers during their security protocol negotiation (before the data communication channel was established). The hash value is deciphered from the received packet. The receiver then performs its own calculation on the received packet, subject to the agreed auth cipher, which produces a corresponding hash. The transmitted (received) hash is compared to the calculated hash value. If they match, the packet/payload is valid. If not, the data is presumed to have been tampered with or damaged in transit and is rejected. The exact formula varies depending on the hash algorithm, but will consist of part or all of a pre-defined portion of the IP packet. This could be the whole packet or part of the packet only, such as the payload. More sophisticated hash algorithms incorporate the hash itself into the equation, thereby protecting not only the data, but also the hash.

Data encryption

If the agreed security policy includes a requirement for confidentiality, to attain this goal the data must be encrypted. Ciphers and algorithms used to encrypt data are different than those used for authentication.

Obfuscation

Obfuscation is the act of hiding or masking something. Associated with confidentiality, anonymity, or privacy. Obfuscation is the process of anonymizing the address history of the data transmission such that if the IP packet is intercepted by a third-party in transit, the true source address of the underlying data packet cannot be ascertained. It's important to understand that by virtue of how network traffic functions, any IP packet must have a clearly specified source and destination address. Obviously, without a clear destination address, the packet won't arrive at its intended destination. Furthermore, due to how information traverses networks and the Internet, they must also have a valid source IP address. However, what obfuscation does in the case of a VPN is to hide the source and/or destination address of the payload.

Since a VPN encapsulates the true data traffic payload inside a secure channel, by definition shouldn't there always be obfuscation of the data including its source and destination? That seems logical, but no. The encapsulated packet has an IP header and other information just like a normal packet (i.e. one not encapsulated by a VPN). If only the data portion of the payload is encrypted, then the source and destination of the data payload are not concealed. Even if they are authenticated, the source/destination can still be read by any device facilitating the data transfer from peer to peer.

Whether or not obfuscation is present depends on the security policy of the VPN, the VPN protocol, and whether the VPN operates on OSI Layer 2 or Layer 3. For example, some VPNs transport data with authentication and encrypt the data payload, but do not encrypt the header of the payload. An example of this is a Layer 3 IPsec secured channel in Transport mode with AH (Authentication Header) and without ESP (Encapsulating Security Payload). Such traffic is not encrypted and acts like a Layer 2 tunnel. The IP packet is delivered as-is with the exception of the AH appended to the end of the original packet. Thus, the source/destination IP addresses are easily read in transit.

Perfect Forward Secrecy (PFS)

Perfect Forward Secrecy (PFS) deserves its own section. PFS is a process of establishing a secure communication key while two (2) network peers are already in the midst of an existing secure communication. An encryption system has the property of Forward Secrecy if a plain-text (decrypted) inspection of the data exchange occurring during key agreement phase of session initiation does not reveal the key was used to encrypt the remainder of the session.

An encryption system has the property of forward secrecy when:

1. The data exchange between two (2) network peers during key agreement negotiation is encrypted
2. If the data exchange during the key agreement phase of session initiation were to be decrypted by a third-party, the separate key used to encrypt the remainder of the session would not be revealed

In other words, if the encryption key used during the key negotiation phase is exposed, the independent key used for the data exchange - which was negotiated during the key agreement phrase - could not be ascertained by a hacker. This is because the key used during the data exchange portion of the connection is an independent key, AND because the key negotiation in the prior phase (that established the encryption key to be used during the data exchange) is not exposed by the decryption of the 1st phase because there isn't sufficient information available from simply decrypting that communication for a third-party to determine what the negotiated data exchange key is.

Data exchange keys should be symmetric, not public/private key pairs.

PFS Process Flow

Perfect Forward Secrecy (PFS) (also referred to as simply Forward Secrecy) is a multi-step secure key exchange process. PFS only protects encryption keys, not the ciphers used to encrypt and decrypt underlying messages (data).

If an existing secure channel between the two (2) network peers does not exist:

1. A bi-directional insecure (non-encrypted) tunnel is created between the peers for negotiation and management of keying material.
2. Keying material authentication occurs between the peers, setting up the basic keying material for step 3.
3. A bi-directional secure tunnel is created. The peers use the keys from step 1 only to authenticate one another. This is sometimes referred to as the management channel.
4. The peers use a key exchange algorithm such as Diffie–Hellman, to securely agree on an ephemeral (temporary) symmetric session key.
5. A new bi-directional secure tunnel is created using the ephemeral symmetric keys. This is sometimes referred to as the data channel.
6. When the current session terminates, the keys are discarded.

If at the beginning, a secure channel already exists between the two (2) network peers then the process becomes:

1. The peers use a key exchange algorithm such as Diffie–Hellman over the existing secure channel to agree on an ephemeral (temporary) symmetric session key for step 2.
2. A new bi-directional secure tunnel is created using the ephemeral symmetric keys.
3. When the current session terminates, the keys are discarded.

Even More Security with Timers

Forward Secrecy can be taken a step further for an even greater level of security by setting a timer when a new PFS symmetric key is generated. When the timer expires, a new symmetric key is negotiated between the peers. The secure channel is terminated and a new secure bi-directional channel is opened using the newly generated key. A timer is set again, and the process repeats.

This process is utilized by the more robust VPN services - such as OpenVPN and WireGuard - when PFS is activated. The purpose is to make it extraordinarily difficult to eavesdrop on an existing VPN connection in realtime. The downside to this approach is the more frequently the session key is replaced, the more often there will be a slight disruption in performance while a new key is negotiated, though with modern hardware users are unlikely to notice under most circumstances.

The idea behind PFS is a malicious actor will not have enough time to crack the encryption of the existing key and decrypt the contents of the communication before the key is changed between the peers. However, it does not prevent the possibility of an eavesdropping third-party from cracking the communication if they were able to record the transmission. Given enough time and resources, eventually any transmission can be decrypted. The question is one of time, resources, and whether or not the information gleaned will still be relevant at the time it is decrypted.

Even so, it is a daunting task. As long as session keys are replaced periodically and neither peer's key is exposed during data exchanges, a malicious actor would have to decode an encryption key from every new session in order to rebuild the entire original conversation between the network peers. This is because as soon as a new key is used, the process of brute-force decryption would start all over again for the next session. Since the key negotiation process does not reveal the new key of either peer, the decryption attack process must be repeated for the new session. If they key replacement interval is sufficiently short, this makes the process of reconstructing the entire communication exceptionally difficult. In fact, it is impossible using current technologies, if the peers have handled their data exchange sessions correctly.

That is now, but what about the future? For instance, the U.S.A.'s National Security Agency (NSA) is widely believed to record and store certain digital communications indefinitely for analysis. No one outside that agency knows their criteria for which electronic conversations are recorded and under what circumstances they are retained. One may logically presume there must be some set of criteria as it's just not realistic - even for a large and secretive government organization with vast resources - to store everything from everyone forever. There's no point. Most of the information retained would be useless. However, if you are a monitoring target for whatever reason, it is reasonable to presume your data could be recorded and stored indefinitely until such time as it was reasonable to expect that the Agency could decrypt and analyze it.

This brings me to the subject of Quantum Computing; a real-life game changer that at some point will turn the tables on encryption by dramatically reducing the timeline on brute-force decryption from thousands of years to minutes or seconds.

Quantum Computing: A Real Threat to PFS

In a recent blog post (Google Just Blew a Giant Hole in Cryptography. Your Privacy is Next.), I explain why quantum computing is a real threat to Forward Secrecy. A recent experiment by Google proved quantum computers 1) can be created; and 2) do indeed live up to their hype. While quantum computing is in its infancy, it presents a very real threat to modern encryption techniques such as PFS.

Normally, a brute force decryption attempt targetted at modern ciphers is completely unrealistic. It would take even a super computer over 10,000 years to process every possible permutation of brute-forcing a strong encryption key. However, that is theoretically not the case with a quantum computer, because it is capable of the simultaneous calculation of multiple outcomes from every equation. Imagine if a bit in a computer were simultaneously 0, 1, and both. That is how a quantum computer behaves. It essentially calculates the outcome of all three (3) states at the same time. Furthermore, it introduces a new, third state (both on and off at the same time) of existence into the digital world.

Privacy and Confidentiality Through Cryptography

When it comes to VPNs, layers of cryptographic protection are implemented at various stages of the VPN process. As described above, the beginning of a VPN connection involves a crytographic key exchange. The network peers reach an agreement on how they will communicate securely with one another, using a script dicated by the VPN protocol. However, in reality it is the VPN protocol agreement that is the first step. The client and server must find common ground in terms of a communication framework before their discussion can move forward. VPN protocols utilize a variety of cryptographic protection techniques, and they don't all use the same methods.

It's not just about encrypting data. Various techniques are deployed to safeguard communications, such as the selective use of security frameworks and protocols comprised of different cryptographic algorithms. This section explores these methodologies in more detail.

Key Agreement and Authentication Protocols

The most common key agreement and authentication protocol is IKE.

IKE

Internet Key Exchange (IKE) is a dual-phase key authentication framework and network protocol. IKE is used by several VPN programs during key negotiation to facilitate the establishment of secure connections. IKEv2 is the current version of IKE, preceded by IKEv1.

IKEv2 and NAT-T: IKEv2 is neutral toward NAT (Network Address Translation). It does not block NAT Traversal (NAT-T), nor does it support it explicitly. The IKEv2 integrity checksum does not protect addresses in the IP header.

You can read more about IKEv2 in the discussion of IPsec.

MOBIKE

MOBIKE (IKEv2 Mobility and Multi-homing Protocol) is an extension to IKEv2, governed by RFC 4555. It is especially common on mobile VPN platforms).

Payload Protection

The encryption keys used in the key agreement phase are related strictly toward creating the encapsulation tunnel itself; the outer layer of the VPN connection. While that may sufficient for some needs, any knowledgeable and security conscious engineer will inform you the best cyber defenses are multi-layered. No matter how improbable, there are ways of snooping keys generated in the encapsulation tunnel formed in the prior steps. If one end of the connection is compromised, the underlying transmission is revealed.

What if the data payload inside the tunnel is encrypted to begin with, such as transmitting HTTPS traffic over a VPN? That solves one problem. The data stream is likely protected from prying eyes (though whether or not that is true in reality depends on many other factors such as the "S" method used in HTTPS), but the metadata associated with the payload is not. This is why other protective methods exist to safeguard the data payload and the encapsulated IP packet as a whole.

This is where anonymity, confidentiality, and integrity come into play. At this stage, we have authenticated the participants of the communication through key exchanges. Now, it is wise to take additional steps to protect the IP packet and the data payload itself. These are two different subjects. Let's examine them at a high level:

• Anonymity: Techniques may be utilized to obfuscate the true source and destination IP addresses of the data payload. To do that, one must obscure the IP packet, and in particular the IP header for the real data payload. A good example of this is Encapsulating Security Payload (ESP), though this is only applicable when ESP is used in Tunnel mode.
• Integrity: Authenticating the IP packet and/or payload ensures the integrity of the data and potentially some metadata are retained through the transmission. Integrity (authentication) is used to validate the information passed from one peer to another was not altered in transit. Authentication Header (AH) of IPsec is a good example of providing integrity to a VPN connection.
• Confidentiality: Sometimes referred to as Privacy, confidentiality and/or privacy refer to anonymity. They all mean the same thing: obfuscation of the identity if the source IP address (and in some cases, also the destination IP address).

IPsec Encapsulating Security Payload (ESP)

I'll use IPsec's ESP as an example of a security protocol designed to provide anonymity and confidentiality through encryption. ESP = Encapsulating Security Payload. ESP behaves exactly as it sounds. It is an IPsec protocol that encrypts and authenticates. When used in Tunnel mode, the original IP header, transport layer headers, and data payload are encrypted.

When used in Transport mode, it only encrypts the transport layer headers (OSI Layer 4), data payload, and the ESP tail (used in the ESP authentication scheme).

You can learn more about ESP in What Is IPsec?. ESP is governed by RFC 4303.

IPsec Authentication Header (AH)

Another feature of the IPsec security framework and protocol, the Authentication Header (AH) is a protocol that provides authentication only (widely referred to indirectly as integrity as it maintains the integrity of a connection). As its name implies, AH is a header field in the encapsulated transmission. And just like ESP, its scope of protection differs slightly depending on whether it is applied during Transport or Tunnel mode. Here are illustrations showing an embedded TCP/IP packet as the data payload.

What is interesting about AH is its behavior is - unlike ESP - it's identical regardless of whether the VPN mode is tunnel or transport.

AH is governed by RFC 4302.

Common Encryption Ciphers

Like the old expression, "Once in a blue moon," new cryptographic algorithms in IT security are a rarity. Many ciphers used today are over 20 years old (e.g. HMAC, circa 1998).² Even some rising stars are old, such as algorithms (the concept dates back to 2002).³ Even ChaCha20 - which has only come into favor very recently - dates back to 2008 as part of the Salsa20 group of ciphers.⁴ In fact, both ChaCha20 and Poly1305 only gained industry-wide recognition in 2016 as an option for TLS⁵ and more recently appear to be up-and-coming standards for a broader set of use-cases across the IT industry.⁶

As far as VPN protocols are concerned, there are just a few prevalent ciphers. The selection available to VPNs is also often shaped by third-party encryption libraries such as OpenSSL that the VPN protocols rely upon. Let's explore some of the most common ciphers used to secure communications.

Authentication Only

Ciphers that provide authentication only. This may be the data payload, the entire IP packet, or Ethernet frame. The cipher doesn't care. Its data protection is a function of the cipher's application, not the cipher's capability unto itself.

MAC, HMAC, and HMAC-SHA

MAC is an acronym for Message Authentication Code. HMAC is an acronym for Hashed Message Authentication Code.

A MAC is a short piece of information used to authenticate a message (IP packet, data payload, or Ethernet frame, depending on protocol type). A HMAC is a specific type of MAC involving a cryptographic hash function and a secret cryptographic key. MACs authenticate a message's data, protecting its integrity (also known as authenticity). HMAC is commonly referenced as HMAC-SHA. It is widely used in SSH, SSL, TLS, and VPN protocols.

SHA stands for Secure Hash Algorithm. The SHA portion refers to SHA1, SHA2, or SHA3. The number suffix indicates the generation of SHA. The SHA algorithms consist of pre-defined hash functions and hash values (called digests). HMAC with SHA is described in a way that indicates the number of bits used to calculate the hash value, which ranges from 224-512 for SHA2 and up to 1024-bits for SHA3. The higher the number of hash bits, ostensibly the more difficult it would be for a malicious actor to decrypt and/or manipulate the hash successfully.

Adoption of SHA3 - released in 2015 - has proven to be controversial from a performance viewpoint and due to adoption constraints.⁷

The SHA algorithms are derived from MD5, with improvements to the original MD5 construct such as greater complexity and hash size (bits).⁸

Poly1305

Poly1305 is an authentication cipher. It is a MAC (Message Authentication Code). It is used to validate the authenticity of a message (whether or not it has been altered in transit from source to destination).

Encryption Only

These ciphers provide confidentiality (encryption) only. They do not provide integrity (authentication) protection.

AES

AES - which means Advanced Encryption Standard - is a family of block ciphers with different key and block sizes. A block cipher is a deterministic algorithm; meaning given a particular input, the result after the algorithm is run will always be the same. AES ciphers use symmetric keys as the block in their algorithm. Thus, in order to decrypt a message that was encryped with a particular AES method, not only do you need to know the message was encrypted with AES, but you also need to know which algorithm was used and what the cipher block was (which in this case is a symmetric key determined prior to the AES calculations).

AES encryption is widely used to encrypt TLS web traffic and VPN data.

ChaCha20

ChaCha20 is a 20-computational round streaming cipher (a type of symmetric key cipher) based on a pseudorandom cipher digit stream called a keystream. ChaCha20 is a modification of the Salsa20 algorithm published in 2008.⁹ ChaCha20 was incorporated into the IKE and IPsec standards in 2015.¹⁰

Key Agreement Negotiation: Diffie-Hellman (DH)

Diffie-Hellman (DH) is an anonymous key agreement algorithm used to create encryption keys between two communicating parties. It is a staple of numerous cryptographic protocols, including IKE. DH allows the generation of symmetric keys and is designed for short-term shared key creation. Mathematically, DH allows both parties to derive a symmetrical secret key without explicitly sharing it. Instead, characteristics that describe the key are shared, and through a series of complicated calculations, both parties arrive at the same result independently. The resulting symmetric key is sometimes referred to as a "shared secret."

Diffie-Hellman ciphers are divided into groups, described in several RFCs (RFC 3526, 5114, 5903, and 7296). Each DH group contains one or more algorithms. As of this writing, there are 24 DH groups. The distinguishing characteristics between DH groups are:

1. Type of algorithm (e.g. modulus MODP or Elliptic Curve)
2. Number of bits used to calculate the key

Diffie-Hellman key exchanges provide encryption only and provide no authentication. This makes them vulnerable to sophisticated Man-in-the-Middle (MitM) attacks.¹¹ Prior to 2015, conventional thinking was if the DH key were robust enough (1024-bits or greater), it was impossible for anyone to crack the key before a later stage of security key generation would make knowledge of the original key obsolete. Unless an attacker could reasonably expect to decipher the first DH key in a key exchange agreement process in real-time, DH was sufficient and basically unbreakable with modern technology and into the foreseeable future.

Curve25519

Curve25519 is an encryption algorithm. It is a variation on Diffie-Hellman. It implements DH with an elliptic curve group called X25519 to create what is called an Elliptic Curve Diffie-Hellman (ECDH) model. Like DH, ECDH algorithms are used in key agreement negotiation protocols.

AEAD Ciphers: Encryption + Authentication

The new kid on the block in the IT security neighborhood is the AEAD cipher. AEAD stands for Authenticated Encryption With Associated Data and is a relatively new class of ciphers that began rolling out around 2008.¹² AEAD ciphers perform both encryption and authentication in one round, usually consisting of a block cipher and single key. An AEAD means you don't need a separate authentication process, such as Hash MAC (HMAC-SHA), making AEAD ciphers simpler to implement. The 2-stage AEAD process is somewhat controversial due to the intermingling of the functions (encryption and authentication) by the same process and in the same step.

ChaCha20 + Poly1305 Combo

A slightly different model that still qualifies as an AEAD cipher is the combination of ChaCha20 + Poly1305. The former is a streaming cipher (a type of symmetric key cipher) based on a pseudorandom cipher digit stream called a keystream. The latter provides authentication after the encryption process has completed. This model simply combines the two (2) methods into a single process.

Algorithm Modes of Operation

You may see an algorithm (especially encryption algorithms) written as ALG_NAME-XXX, where "XXX" is usually GCM, CTR, or CBC. What do those letters mean after the AES? The letters after the dash (-) designate the mode of operation for a symmetric-key cryptographic block cipher. I'm going to skip the details on how these operating modes differentiate from one another. The most important thing to know is they tweak the process by which the algorithm is applied.

• CBC: Cipher Block Chaining
• CTR: Counter Mode
• GCM: Galois/Counter Mode

Each combination of cipher and operating mode is guided by an RFC document. For instance, AES-GCM implemented with TLS (Transport Layer Security) is governed by RFC 5288 [2008], while AES-CTR is governed by RFC 3686 [2004].

Security Protocol Must's, May's, Should's, and Don'ts

When it comes to encryption algorithms, security protocols have lists of ciphers they must support, may support, should support, and cannot support. This is one of the most important parts of a security protocol's operational framework and it has a major impact on which cryptographic algorithms may be utilized by both peers to establish a VPN. During the negotiation of security agreements for example, when a VPN is established, the starting point is dictated by what either peer is capable of supporting in terms of encryption algorithms.

This is just scratching the surface on how ciphers are agreed upon during a VPN security protocol exchange, but you can think of this phase as agreeing on the 'big buckets' in terms of how encrypting the conversation between peers will be managed. As an example, let's take a look at an IPsec connection's requirements, which are laid out in RFC 8221 but summarized here:

• Encryption must be authenticated. There are 3 methods you may use. Choose One:
  1. ESP + AEAD cipher
  2. ESP + non-AEAD cipher + authentication algorithm
  3. ESP + non-AEAD cipher + AH

Acceptable IPsec Cipher Suites

So, the starting point of security protocol negotiations of IPsec VPN connections is described above, and as you can see each peer needs to abide by these rules. Now, how would you prioritize these? If a peer can support all three (3) methods, according RFC 8221, they should be chosen in the descending order shown above. Next up is the heart of the matter, which is the question of which cipher should be chosen. Obviously, in an ideal world, both VPN peers would always choose the most robust cipher. However, for various reasons that is not always possible. So, rather than automatically reject the connection, most VPN peers will attempt to negotiate a cipher suite that is amenable to both peers. This is where the list of ciphers each peer is capable of using and is allowed to use become important.

So, there we have a table of acceptable ciphers for use with IPsec. Any not listed in that table are automatically not acceptable and IPsec must reject them. Notice that 3DES is still acceptable, even though it is widely regarded as insecure. This leaves the door open to a potential downgrade attack by an attacker (i.e. the attacker attempts to force the IPsec channel to use 3DES encryption, which the hacker knows they can break).

Pretty clear cut, eh? Unfortunately, no. It isn't. For example, AES-CTR is not listed in the table above. It cannot be used. Are there IPsec servers in the wild using it? Yes. How is that possible?
At one time AES-CTR was acceptable for use in IPsec (see RFC 3686) [2004]. That RFC was never obsoleted explicitly, yet by definition in RFC 8221's language it became obsolete in 2017. The IETF (Internet Engineering Task Force) aware of the fact IPsec has so many related RFC documents it's become quite confusing. In July 2018, the IETF published RFC 8423, which seeks to clear up some of that confusion. Unfortunately, it doesn't mention RFC 3686 at all (which approved the use of AES-CTR in IPsec back in 2004). Numerous other historic IPsec RFC's aren't mentioned either, but I find the ommission of RFC 3686 to be a particularly good example because AES-CTR is an AEAD cipher. This leaves the door open for old implementations of IPsec to exist where their owners may believe (incorrectly) their IPsec implementation is in compliance with current standards, when it isn't. We'll get into this issue further below, in Poor Habits of VPN Service Providers.

IKEv2 Acceptable Cipher Suites

What about the IKE key exchange process? Since IPsec outsources the key agreement protocols to IKE, we need to know what IKE is capable of and allowed to do as well. This information is found in RFC 6071: IP Security (IPsec) and Internet Key Exchange (IKE) Document Roadmap, which basically says, "Do what the security framework does." RFC 6071 effectively mirrors the requirements of other RFC documents relative to various security frameworks (e.g. IPsec), such that IKE falls in-line with whatever the higher framework allows (when in Rome...).

Interestingly, RFC 6071 mentions Camellia and SEED; lesser known, but sometimes valid encryption algorithms. Both are explicitly allowed with IKE, though their implementation depends upon the overarching security framework (whether IPsec or something else).

Poor Habits of VPN Service Providers

Not all VPN service providers are created equally. Here are some examples of business practices by VPN service providers you should watch out for. If there is evidence a provider is using any of these poor methods, avoid them.

Authentication Without Encryption

Beware outer encapsulation layer protocols that provide no confidentiality (encryption)!

You should never encounter this with a VPN, but sadly it can happen. Have you read the section above on GRE? Make sure your service provider is not using GRE or IPsec over GRE to establish the encapsulation tunnel, as it provides authentication only!

When using authentication, employing encryption in some fashion is a wise move as it will drastically reduce the likelihood of the information being altered while in transit. In order for a malicious actor to modify the authentication scheme without detection, they would have to decrypt the packet, then modify it and its hash value (authentication), and re-encrypt the data before sending it on its way. Using modern cryptographic algorithms, this is an impossible feat if the encryption key is unknown to the bad actor and sufficient security practices are employed.

But what if there is no data encryption? Then the possibility of an authentication hash being tampered with in transit becomes a legitimate concern. Conceptually, it is a relatively easy problem to solve. An attacker would have to recalculate the authentication hash and save a new hash value before passing the packet along toward its destination. While this is a difficult feat if the data is encrypted, it's a rather simple task if the data is transmitted in plain text. Under that circumstance, the attacker only needs to deduce the authentication algorithm used in order to correctly reform the new hash value. This is a much easier task than attempting to do so in addition to cracking encryption at the same time. If the data payload is transmitted in plain text (not encrypted) and the authentication method can be deduced quickly (e.g. is the whole packet encrypted or just part of it?), the process of identifying the authentication algorithm utilized by the target is not particularly difficult. There are only a small number of possible algorithms. Even if the network protocol used to transmit the data is unknown, again there is a limit to how many possible variations exist.

The bottom line is utilizing authentication without encryption sets the protection bar rather low. A skilled hacker will be able to overcome such a defense relatively easily.

Key Recycling (Encapsulation Tunnel PKI Keys)

Does your VPN service provider practice PKI key recycling? Watch out for this terrible concept, which demonstrates poor judgment on the part of a VPN service provider's personnel. Some VPN providers re-use the same encapsulation tunnel key for every re-connection; others are even worse and provide the same public key for every customer and never change the key!

Why would anyone do this? And how is this possible? One word: Convenience. This business strategy involves using a single public key when establishing the encapsulation layer of the VPN connection. The real key agreement negotiation is carried out afterwards (or should be).

Why is this approach a problem?

This poor practice effectively exposes the "front door" of the VPN service to the whole world. A clever malicious actor may use this information to crack the associated private key used by the server. The hacker gains the ability to decrypt ALL incoming connections to the server. At that point they are also in a position to potentially - given enough time - unlock any client-side encapsulation layer transmission as well.

This problem manifests itself with the public key being shared in one of these ways:

1. Prominently disclosed on a public-facing website
2. Emailed to the user during new account creation process
3. Provided in prepared configuration file(s)

The point here is the same public key is shared with EVERY user who signs up for the service, and in the worst cases the public key is shared with the whole world whether you have an account or not.

This practice means if a hacker successfully implements a Man-in-the-Middle (MitM) attack, they will have the capability to vacuum up all the traffic sent to and from the VPN server. Even if the hacker has not yet cracked the server private key, knowing the same public key is used for ALL inbound connections means collecting all those transmissions will provide them with more data and allow them to more rapidly crack the private key. Cracking the return direction (client) private key for any given connection is more challenging, but if a similar security practice is followed where the same client-side private/public key combination is recycled with every connection to the VPN, the decryption task for the hacker becomes substantially easier. Even if that is not the case, the fact the hacker can decrypt 50% of the transactions will help them crack the other key more quickly. Overall, this places a substantial risk of compromise for any and ALL users of the VPN service!

Footnotes