Investigate Unicode and bidirectional source rules

Archived MVP specification. For ASPL 0.5.0, see the current reference and native HTTP guide.

Type: research Status: resolved Parent: Find the ASPL MVP specification

Question

Which Unicode normalization, identifier, bidirectional-text, and confusable-character rules should an Arabic-first source language consider, according to Unicode standards and relevant language specifications, before ASPL chooses its MVP lexer and diagnostic policy?

Answer

Pin Unicode 17.0 and decode .aspl files as strict UTF-8. Use a documented Arabic-or-Latin profile of XID_Start and XID_Continue, add _, compare case-sensitive identifiers after NFC normalization, retain combining marks, and reject mixed-script names, non-ASCII digits, default-ignorable characters, and characters outside the UTS #39 General Security Profile. Use direction-aware UTS #39 confusable checks only for warnings, never for name equality. Parse source in logical order; admit LRM and RLM only as semantically invisible token-boundary controls, render all bidi controls visibly in diagnostics, and require the website editor to follow UTS #55 token-atom ordering for RTL and LTR display.

See Unicode and bidirectional source rules for the ASPL MVP for the cited rationale, diagnostic contract, and conformance cases.