Two protocols, two different jobs
Think of a shipping container. A tamper-evident seal on the outside proves nobody opened it in transit and confirms who sent it, but anyone can still see straight through the container's mesh sides at what's inside if they look. A fully sealed, opaque, locked container does both: nobody can see the contents without the key, and it can carry that same tamper-proof seal too. IPsec's two protocols map directly onto that difference.
- AH (Authentication Header): authenticates the packet and checks its integrity, including parts of the IP header itself, but doesn't encrypt anything. The contents stay fully visible to anyone watching.
- ESP (Encapsulating Security Payload): encrypts the payload, and typically authenticates it too. Contents are hidden, not just verified.
| AH | ESP | |
|---|---|---|
| Confidentiality (encryption) | No | Yes |
| Integrity / authentication | Yes | Yes, typically |
| Protects the outer IP header | Yes, its immutable fields | No |
| Works through NAT | No, without extra help | Yes, with NAT-T (NAT Traversal) |
| IP protocol number | 51 | 50 |
Transport mode vs tunnel mode
Transport mode protects just the original payload, leaving the original IP header alone; it's a host-to-host arrangement. Tunnel mode wraps the entire original packet, header and all, inside a brand new outer IP header, which is what makes site-to-site VPNs possible: the two gateways' IP addresses are all that's visible from the outside, and every internal host behind either gateway is completely unaware the tunnel even exists.
AH authenticates fields in the IP header that NAT (Network Address Translation) rewrites in transit, which breaks AH's integrity check the moment a NAT device touches the packet. Combined with offering no confidentiality at all, that's why almost every real-world IPsec deployment today uses ESP, either alone or with its own authentication enabled, and AH is mostly a historical footnote.