When it comes to Unicode implementations, there’s a rich set of test
cases to perform. Realizing it is the start. Automating it is the next
step.
At a high-level Unicode-related security bugs can be categorized into the following root-causes:
Canonicalization
- Interpreting non-shortest form (e.g .UTF-8 encoding trickery)
- Other decoding issues
Absorption (over-consumption)
- Over-consuming invalid byte sequences or correcting rather than failing
- When <41 C2 C3 B1 42> becomes <41 42>
Character deletion and swallowing
- “deletion of noncharacters” (UTR-36)
- <scr[U+FEFF]ipt> becomes <script>
- Use replacement characters instead!
Interpreting Syntax replacements
- white space and line feeds
- E.g. when U+180E acts like U+0020
Best-fit mappings
- When σ becomes s
- When ′ becomes ‘
Buffer overruns
- Incorrect assumptions about string sizes (chars vs. bytes)
- Improper width calculations
Timing issues
- handling Unicode after security gates
- Sometimes handling Unicode before a gate can be a problem too! E.g. BOM handling
Tags: test cases
