Unicode Normalisation: NFC, NFD, NFKC, and NFKD Explained

Unicode allows the same visual character to be represented by multiple distinct sequences of code points. The letter é can be stored as a single precomposed character (U+00E9 LATIN SMALL LETTER E WITH ACUTE) or as two separate code points—a plain e (U+0065) followed by a combining acute accent (U+0301). Both render identically in most fonts, but they're different byte sequences. Unicode normalisation eliminates this ambiguity by converting text to a canonical form before comparison or storage.

The Four Normal Forms

• NFD (Canonical Decomposition): Decomposes precomposed characters into their base character plus combining marks. é → e + ´
• NFC (Canonical Composition): First applies NFD, then recomposes where possible. e + ´ → é. NFC is the preferred form for storage and exchange—it's compact and canonical.
• NFKD (Compatibility Decomposition): Like NFD but also decomposes compatibility variants. The full-width letter Ａ (U+FF21) and the mathematical bold A 𝐀 (U+1D400) both decompose to plain A (U+0041). Useful for search indexing.
• NFKC (Compatibility Composition): NFKD followed by canonical composition. The most aggressive normalisation—it strips stylistic distinctions. Ｇｏｏｇｌｅ normalises to Google. Used for identifier comparison and security checks.

When to Normalise

Always normalise before comparing strings or storing to a database. NFC is the right choice for storage and display. NFKC is the right choice for usernames, search queries, and anywhere you want to treat stylistically different but semantically equivalent forms as the same. Failing to normalise is one reason why "find/replace" operations can miss matches and why password hashes for visually identical passwords don't match.

Language Support

Most modern languages and frameworks have normalisation built in:

• Python: unicodedata.normalize('NFC', s)
• JavaScript: str.normalize('NFC') (ES6+)
• Java: java.text.Normalizer.normalize(s, Form.NFC)
• Ruby: str.unicode_normalize(:nfc)

Use the character comparison tool to compare a precomposed character with its decomposed equivalent and confirm they map to the same glyph.