IKEv1 vs IKEv2 · netskillup

Two phases, in both versions

AH and ESP both need shared keys to work with, and IKE is the negotiation that produces them, in two phases:

Phase 1: establish a protected channel between the two peers, called the IKE SA (Security Association), that the rest of the negotiation can safely happen over.
Phase 2: use that protected channel to negotiate the actual keys AH/ESP will use to protect real traffic.

How IKE actually delivers confidentiality, integrity, and authentication

Every message below is doing one of three jobs, the same three that show up across this entire security module. IKE's whole Phase 1 negotiation exists to set these three properties up before anything sensitive gets sent.

Property	What it guarantees	How IKE provides it
Confidentiality	Nobody watching the wire can read the content	An encryption algorithm negotiated in the first message pair, applied to every message from that point on
Integrity	Nobody altered the message in transit without it being noticed	An HMAC (Hashed Message Authentication Code) algorithm, negotiated alongside the encryption algorithm
Authentication	Each peer actually proves it is who it claims to be, not just that it can do math	A HASH payload (or a digital signature, with certificates) computed from the negotiated keys and each peer's identity, exchanged and checked at the end of Phase 1

That last row is worth sitting with. The Diffie-Hellman exchange earlier in Phase 1 gives both sides a shared secret, but math alone doesn't prove who's on the other end, an attacker could run the same math. Authentication is the separate, explicit step that turns "we agree on a number" into "I know it's actually you."

Phase 1: Main mode's six messages

Main mode is the default, careful version of Phase 1: six messages, three round trips, and it never exposes either peer's identity before encryption has already started.

Encryption only starts after message 4. Messages 1 through 4 are visible to anyone watching the link; only 5 and 6, carrying identity and the authentication proof, are ever encrypted.

Messages 1 and 2: the SA proposal plaintext

SPIA Security Parameter Index the initiator generates to label this negotiation; the responder's own SPI starts at zero and gets filled in once it replies

ProposalEncryption algorithm, HMAC algorithm, Diffie-Hellman group, and authentication method (pre-shared key or certificate), offered by the initiator and matched or rejected by the responder

ResultBoth peers now agree on every algorithm they'll use for the rest of Phase 1, but nothing is encrypted or authenticated yet

Messages 3 and 4: the key exchange still plaintext

KE payloadEach side's Diffie-Hellman public value, in the group agreed on in messages 1 and 2

NonceA fresh random value from each side, mixed into the key derivation so the resulting keys are unique to this exchange, not just this DH group

ResultBoth sides can now independently compute the same shared secret and derive the encryption and HMAC keys from it, but these two messages themselves went out unencrypted and unauthenticated, exactly the raw material an eavesdropper sees

Messages 5 and 6: identity and authentication encrypted

ID payloadEach peer's identity, an IP address, hostname, or distinguished name, now encrypted with the keys derived from messages 3 and 4

HASH payloadA value computed from the negotiated keys, both nonces, and each peer's own identity, proving the sender actually derived the correct key material and really is who the ID payload claims

ResultOnce the responder's HASH in message 6 checks out, both peers are mutually authenticated and the IKE SA (Phase 1's protected channel) is complete

Phase 1: Aggressive mode's three messages

Aggressive mode collapses the same negotiation into three messages by combining what main mode spread across six, trading away main mode's one real protection: it sends identity and authentication data before encryption has started.

Half the round trips of main mode, at the cost of message 2 going out completely in the clear, identity, key material, and the responder's authentication proof all included.

Why message 2 is the actual weak point security note

Main mode never sends identity or authentication data until messages 5 and 6, both already encrypted. Aggressive mode's message 2 bundles the responder's identity and its HASH payload into the very first reply, sent before any encryption key has been established at all. Anyone capturing that traffic gets the responder's identity for free, and pre-shared-key deployments using aggressive mode have historically been targeted with offline attacks against that exposed hash. This is exactly why main mode is the safer default whenever both peers can use it.

Phase 2: Quick mode

Once Phase 1 finishes, either mode, both peers have one protected channel (the IKE SA) they can now use as many times as needed. Quick mode is what actually runs inside that channel to negotiate the real IPsec SAs that AH or ESP will use.

Quick mode's three messages all encrypted, inside the Phase 1 SA

QM1 (initiator → responder)Proposes the IPsec SA: which protocol (AH or ESP), which transform, and a fresh nonce

QM2 (responder → initiator)Accepts the proposal and returns its own fresh nonce

QM3 (initiator → responder)A final confirmation, proving the initiator is still live and really did receive QM2

The two nonces get mixed into the key derivation so the IPsec session keys are fresh, distinct from the Phase 1 keys, and not simply reused traffic-protection keys borrowed from the negotiation channel itself.

IKEv2: One fixed exchange

IKEv2 replaced the whole main/aggressive split with a single design: four messages, two named exchanges, no mode to accidentally pick wrong.

Everything main mode needed six messages for, encryption starting only after the DH exchange, identity and authentication protected, happens here in four, with a CHILD_SA (the equivalent of a Quick Mode negotiation) often piggybacked onto IKE_AUTH itself.

IKE_SA_INIT: messages 1 and 2 plaintext

SA proposalEncryption, integrity, and Diffie-Hellman group choices, negotiated in a single round trip instead of main mode's two separate messages

KE and nonceBoth sides' Diffie-Hellman public values and nonces, bundled into the same exchange that negotiated the algorithms

ResultBoth peers can now derive shared keys, exactly what took messages 1 through 4 in main mode, done here in one round trip

IKE_AUTH: messages 3 and 4 encrypted

ID payloadEach peer's identity, encrypted, never exposed before encryption exists (unlike aggressive mode)

AUTH payloadIKEv2's equivalent of the HASH payload: proof each side derived the right keys and really controls the claimed identity

SA, TS payloadsOften included right here too: the actual CHILD_SA (the traffic-protecting IPsec SA) and traffic selectors, so a fresh IKEv2 session can be fully ready after just these four messages, no separate Quick Mode round trip required

	IKEv1	IKEv2
Initial exchange	6 messages (main) or 3 (aggressive)	4 messages, always
Identity exposed before encryption	No (main mode) or yes (aggressive mode)	Never
NAT traversal	Added later, inconsistently	Built in
Reliability (retransmission)	Not built in	Built in
Mobility (MOBIKE)	No	Yes
Standard variations	Many optional, vendor-specific extensions	Simpler, more consistent across vendors

WHY MOST NEW DEPLOYMENTS DEFAULT TO IKEV2

Fewer messages, no risky aggressive-mode shortcut, built-in NAT traversal, and mobility support add up to a simpler, more reliable protocol with fewer ways to misconfigure it. IKEv1 still shows up in older or legacy equipment, but IKEv2 is the default choice for anything new.