Symmetric · 04

Key Management Basics

The cipher is the easy part. Everything around the cipher is where systems actually fail. This page covers generating, deriving, storing, rotating, and destroying keys, plus the KEK and DEK hierarchy that real systems use to scale all of it.

01

Why Key Management Is The Hard Part

AES has not been broken in 25 years. The same cannot be said for the systems that hold AES keys. The vast majority of real-world cryptographic failures happen not in the algorithm but in the surrounding plumbing.

Every symmetric cipher assumes the key is secret, random, and known only to the parties who need it. Every one of those three properties is something an engineer has to actively enforce. A key written to a config file, checked into Git, mailed in plaintext, or generated from the system clock fails the assumption before AES even runs.

Key management is the discipline of answering five questions for every key in a system:

  1. How was it generated, and is the source of randomness trustworthy?
  2. How is it distributed to the parties that need it?
  3. Where is it stored while in use, and who can read it?
  4. When and how is it rotated or revoked?
  5. How is it destroyed when it is no longer needed?

Every modern compliance framework (FIPS 140-3, PCI DSS, HIPAA, SOC 2, NIST SP 800-57) is, at its core, a checklist for these five questions.

02

Generating Keys

A symmetric key is just a string of random bits. AES-128 needs 128 of them, AES-256 needs 256. The hard part is the word random.

Keys must come from a cryptographically secure pseudo-random number generator (CSPRNG). The operating system provides one. On Linux it is /dev/urandom or the getrandom() syscall. On Windows it is BCryptGenRandom. Most language standard libraries wrap these: Python secrets.token_bytes(32), Go crypto/rand, Node crypto.randomBytes(32).

What never works:

Failure mode

In 2008, Debian shipped an OpenSSL with a patched random number generator that produced only 32,767 distinct values. Every SSH key, TLS certificate, and DNSSEC key generated on a Debian system for almost two years was one of about 32,000 possible keys. They were instantly enumerable. The cipher was fine. The randomness was not.

03

Key Derivation Functions

Users do not pick 256-bit random keys. They pick passwords. The job of a key derivation function (KDF) is to convert a low-entropy human password into a high-entropy, fixed-length symmetric key.

A KDF takes three inputs: the password, a salt (random bytes stored alongside the output), and a cost parameter that controls how expensive the derivation is. The expense is the point. If deriving a key takes 100 milliseconds for a legitimate user, it also takes 100 milliseconds for an attacker testing each password from a dictionary.

KDFYearCost knobStatus
PBKDF22000Iteration countAcceptable, but GPU-friendly. Use only with 600,000+ iterations.
bcrypt1999Work factor (log2)Still solid for passwords. Limited to 72-byte input.
scrypt2009N, r, p (CPU and memory)Memory-hard. Resists custom hardware better than PBKDF2.
Argon2id2015Time, memory, parallelismThe current recommendation. Winner of the Password Hashing Competition.

Argon2id is the default to reach for in any new system. PBKDF2 still appears in protocols (WPA2, TLS 1.2, many disk encryption tools) because it predates the others and is built into countless standards.

The salt

The salt is not secret. It exists to make two users with the same password produce two different stored values. Without a unique salt per user, an attacker can compute a single dictionary against all users at once. The salt should be at least 16 random bytes per password and stored alongside the derived key.

04

Storing Keys

Once a key exists, the question becomes where to put it. The wrong answers are obvious in retrospect and embarrassingly common in practice.

Never store keys in

Source code. Configuration files committed to version control. Environment variables logged by error trackers. Database tables alongside the data they protect. Shared network drives. Slack messages. Email attachments. Screenshots in tickets.

The right answers all share a property: the key is held in a place that exposes only the operations you can perform with it, never the bytes themselves.

MechanismWhat it isWhen to use it
HSMHardware Security Module. A tamper-resistant box that generates and uses keys but refuses to export them.Root keys, certificate authority signing keys, anything regulated (PCI, FIPS).
Cloud KMSAWS KMS, Azure Key Vault, Google Cloud KMS. Managed HSM-backed services with an API.Application-level encryption keys, envelope encryption schemes.
TPMTrusted Platform Module. A small chip on the motherboard that holds keys tied to the machine.Disk encryption keys (BitLocker, LUKS with TPM), platform attestation.
Secure enclaveApple Secure Enclave, Android StrongBox, Intel SGX. A separate processor with its own memory.Mobile device keys, biometric template protection, Touch ID and Face ID.
Secrets managerHashiCorp Vault, AWS Secrets Manager, 1Password Service Accounts.Application secrets, database credentials, API tokens. Not a replacement for a true HSM.

For a typical application: cloud KMS holds the root key, the secrets manager holds the day-to-day credentials, and individual disk volumes use TPM-sealed keys. No human reads any of those bytes directly.

05

The Key Lifecycle

Every key goes through the same six phases from creation to destruction. NIST SP 800-57 formalizes them; the diagram below shows the practical version.

Figure 4.1: The key lifecycle A circular diagram showing six lifecycle phases connected in order: Generate, Distribute, Use, Rotate, Revoke, Destroy. Arrows flow clockwise around a central label. Key Lifecycle NIST SP 800-57 01 Generate 02 Distribute 03 Use 04 Rotate 05 Revoke 06 Destroy dashed: replacement key begins a new cycle
Fig 4.1 · The six phases of a symmetric key's life
PhaseWhat happens
GenerateA CSPRNG produces the key. It is registered in a key store and tagged with an ID, an algorithm, and a creation timestamp.
DistributeThe key, or a wrapped copy of it, is delivered to the systems that need it. Usually over an authenticated channel like TLS, or via a KMS that exposes the operation rather than the bytes.
UseThe key encrypts and decrypts data. Usage is logged. The key is held only in protected memory.
RotateA new key is generated and takes over for new operations. The old key stays available to decrypt anything it previously encrypted.
RevokeThe old key is marked as no longer trusted. It cannot be used for new operations. Existing data may still need to be decrypted and re-encrypted under the new key.
DestroyThe key bytes are securely erased from every store. Auditable. Irreversible. After this point, anything still encrypted under the key is unrecoverable.
06

Key Hierarchies: KEK and DEK

A system encrypting a million files does not use one key for all of them, and it does not put a million keys in the HSM. It uses a hierarchy.

The pattern is called envelope encryption. One master key, the Key Encryption Key (KEK), lives in the HSM or KMS and never leaves. It is used only to encrypt other keys. The keys it encrypts are called Data Encryption Keys (DEKs), and the DEKs are the ones that actually encrypt user data.

Figure 4.2: KEK and DEK hierarchy A diagram showing a single Key Encryption Key inside an HSM at the top, fanning out to three Data Encryption Keys, each of which encrypts a separate data object. HSM / CLOUD KMS KEK never leaves the boundary wraps DEK 1 stored encrypted DEK 2 stored encrypted DEK 3 stored encrypted encrypts User database tenant A File storage tenant B Backups tenant C one KEK protects many DEKs; one DEK protects one data scope
Fig 4.2 · Envelope encryption with a KEK protecting many DEKs

The payoff is enormous:

AWS S3 server-side encryption, Google Drive, Apple FileVault, and almost every modern database encryption feature use this pattern. When AWS says "encryption is on by default," what they mean is that a KEK in KMS is wrapping per-object DEKs.

07

Rotation In Practice

Rotating a key means generating a new one and shifting future operations to it. The old key sticks around long enough to decrypt anything it previously encrypted, then is destroyed.

Why rotate at all?

Modern KMS systems automate this. AWS KMS supports automatic annual rotation of customer master keys. The application code does not need to know the rotation happened; the KMS keeps both versions and uses the appropriate one based on the encryption context tag baked into the ciphertext.

08

Where Keys Actually Live

The abstract lifecycle becomes concrete in a few well-known places. Each of these is worth being able to picture when you read the term.

SystemWhere the key livesHow it gets there
TLS 1.3 sessionEphemeral. In RAM only, for the duration of the connection.Derived from an ECDHE handshake; thrown away when the connection closes (forward secrecy).
BitLocker / LUKSSealed to the TPM on the local machine.Released to the OS only when the boot measurements match the expected values.
iMessage / SignalPer-device keys in the Secure Enclave; per-message keys derived via the Double Ratchet.Generated on device enrollment. The server never sees them.
AWS S3 SSE-KMSPer-object DEK encrypted under a KEK in AWS KMS.Generated at upload time; the wrapped DEK is stored next to the object.
SSH server host keyOn disk in /etc/ssh/, owned by root, mode 0600.Generated once at install time. Rotated only on compromise or migration.
Application secretsHashiCorp Vault or AWS Secrets Manager.Fetched at startup over an authenticated channel; kept in memory only.
The pattern

Notice what is missing from every row of that table: source code, environment files, and Git history. If you can name the file path on a developer's laptop where a production key lives, the key is in the wrong place.