phonemes

14 files changed, 472 insertions, 72 deletions

diff --git a/docs/index.html b/docs/index.html
index d6a0817..7017da5 100644
--- a/docs/index.html
+++ b/docs/index.html
@@ -60,7 +60,7 @@ entirely from the samples.</p>
     font renderer, and the every-other-frame cursor.</span></li>
 
   <li><a href="phonology.html">4. The phonology engine</a>
-    <span><code>Na_VoiceSe</code> in full: the 55-entry digraph table, short/long
+    <span><code>Na_VoiceSe</code> in full: the 53-entry digraph table, short/long
     vowel tables, silent&nbsp;e, magic&nbsp;e, and punctuation flush.</span></li>
 
   <li><a href="playback.html">5. Playback</a>
diff --git a/docs/phonology.html b/docs/phonology.html
index ebbc872..075f3a2 100644
--- a/docs/phonology.html
+++ b/docs/phonology.html
@@ -47,8 +47,13 @@ rewritten.</p>
 
 <h3>0. Lone mute</h3>
 <pre>if (a == 0x85 &amp;&amp; b == 0xFF &amp;&amp; c == 0xFF) return;</pre>
-<p>Returns before touching any state, so a lone punctuation mark does not disturb the
-previous-character history.</p>
+<p><code>sou_num_org</code>, <code>sou_num2_org</code> and <code>sou_num3_org</code> are assigned
+<em>before</em> this test, so they <strong>are</strong> overwritten; what survives is
+<code>sou_last_num</code> / <code>sou_last_num2</code>, which are written further down. A lone
+punctuation mark therefore leaves the digraph history intact, but still clobbers the
+context bytes a pending <a href="#mode2">mode&nbsp;2</a> call will read —
+<code>sou_num3_org</code> for its short-vowel lookup, <code>sou_num_org</code> for its dedup
+comparison.</p>
 
 <h3>1. Silent e (<code>:2612</code>)</h3>
 <pre>if (a == 'e' &amp;&amp; b &gt; 0x77 &amp;&amp; class(last_num) == vowel &amp;&amp; class(last_num2) == consonant) {
@@ -71,6 +76,17 @@ are consumed by one sample.</p>
 <h4>The digraph table — <code>Sou_ConnectCheck</code> (<code>:2316</code>)</h4>
 <p>53 entries. All are two characters except <code>you</code>, the only rule that reads <code>c</code>; one pairs a letter with a space rather than another letter (<code>i␣</code>).</p>
 
+<div class="note">
+<strong><code>i␣</code> is very nearly unreachable.</strong> Its <code>b</code> is <code>0x81</code>,
+which is also a punctuation-flush trigger — so rule&nbsp;3 below sets <code>r28 = 3</code> and
+unconditionally reassigns <code>a</code>, discarding the <code>0x05</code>. It survives only when
+the previous call's <code>b</code> was <em>also</em> punctuation, which forces <code>r28 = 1</code>
+instead; <code>0x05</code> then falls through <code>Sou_TanboinHenkan</code>'s
+<code>default: return a</code> and reaches playback unchanged. Every other digraph has a letter
+in <code>b</code> and so never meets rule&nbsp;3 — for those, "<code>r28 = 2</code> falls straight
+through to playback" holds as written.
+</div>
+
 <div class="grid2">
 <code>ai</code>→0x0F<br><code>al</code>→0x02<br><code>an</code>→0x39<br><code>ar</code>→0x16<br><code>as</code>→0x3A<br>
 <code>be</code>→0x3B<br><code>by</code>→0x3C<br>
@@ -110,9 +126,23 @@ each one a recorded syllable.</p>
 <pre>a = Sou_ConnectCheck(sou_last_num2, sou_num_org, sou_num2_org);
 if (a &lt; 0x53 || a &gt; 0x76) break;      // it merged, or wasn't a letter — leave it
 a = Sou_TanboinHenkan(a, b);</pre>
-<p>This gives the final letter of a word a chance to pair with the letter before it
-instead of being dropped against the following space. Note the guard also drops the
-whole branch if <code>a &gt; 0x77</code> before it starts.</p>
+<p>The <em>intent</em> is to give the final letter of a word a chance to pair with the letter
+before it instead of being dropped against the following space. What it does is narrower.
+<code>a</code> is reloaded from <code>sou_last_num2</code> — the <em>previous</em> call's
+<code>b</code> — and if <code>Sou_ConnectCheck</code> finds no merge it returns that byte
+unchanged, so <code>Sou_TanboinHenkan</code> speaks the previous letter a second time and the
+word-final letter is never voiced at all.</p>
+
+<p>Worked example, <code>cat␣</code>. Window <code>(c, a, t)</code> doesn't merge, so
+<code>r28 = 1</code> plays hard <code>c</code> and queues <code>a</code>; the deferred call plays
+<code>a</code>. Window <code>(t, ␣, …)</code> then hits rule&nbsp;3, reloads
+<code>a = sou_last_num2 = 'a'</code>, fails to merge <code>at</code>, and plays <code>a</code>
+again. The <code>t</code> makes no sound. The branch only pays off when the reloaded pair
+<em>does</em> merge — then the merged code plays and the <code>0x53</code>–<code>0x76</code>
+guard breaks out before <code>Sou_TanboinHenkan</code> can touch it.</p>
+
+<p>Note the guard also drops the whole branch if <code>a &gt; 0x77</code> before it starts, and
+that nothing is queued on this path.</p>
 
 <h3>4. Vowel length (<code>:2658</code>, for <code>r28 == 1</code> or <code>4</code>)</h3>
 <pre>if (class(a) == vowel &amp;&amp; class(b) == consonant &amp;&amp; c == 'e')
@@ -168,7 +198,7 @@ with the first syllable of the spoken number: <code>0</code>=<code>ze</code>(ro)
 get codes nothing else reaches. The digraph names here are just labels for shared
 samples — the recording itself presumably says the number.</p>
 
-<h2>The deferred second syllable — mode 2</h2>
+<h2 id="mode2">The deferred second syllable — mode 2</h2>
 
 <p><code>Sou_SpecialRoutine10</code> (<code>:1708</code>) runs once per game frame inside
 <code>Na_GameFrame</code>:</p>
diff --git a/docs/playback.html b/docs/playback.html
index 1cf9464..73be9a2 100644
--- a/docs/playback.html
+++ b/docs/playback.html
@@ -33,7 +33,7 @@ into the phoneme path — the pitch and volume bumps below never apply to the pl
 <tr><th>Code</th><th>Char</th><th>Effect</th></tr>
 <tr><td><code>0x86</code></td><td>click</td><td><code>NA_SE_BEBE</code>, then mute</td></tr>
 <tr><td><code>0x81</code> <code>0x84</code> <code>0x85</code></td><td>space, newline, mute</td><td>mute</td></tr>
-<tr><td><code>0x82</code></td><td><code>?</code></td><td>mute, and pitch × 1.24 / 1.30 / 1.13 for sequence 243 / 244 / 245</td></tr>
+<tr><td><code>0x82</code></td><td><code>?</code></td><td>mute, and pitch × 1.24 / 1.30 / 1.13 for <code>sou_now_voice_seq</code> 1 / 2 / 3 — the small index <code>Na_SpecChange</code> sets alongside sequence 243 / 244 / 245, not the sequence number itself</td></tr>
 <tr><td><code>0x83</code></td><td><code>!</code></td><td>mute, and volume × 1.15</td></tr>
 <tr><td>anything else</td><td></td><td>plays; sets <code>sou_now_boin</code> (dead, see <a href="quirks.html">quirks</a>)</td></tr>
 </table>
diff --git a/docs/quirks.html b/docs/quirks.html
index bb860c9..c46fdb2 100644
--- a/docs/quirks.html
+++ b/docs/quirks.html
@@ -145,7 +145,7 @@ the config. See <a href="gating.html">gating</a>.</td></tr>
 </table>
 
 <p>Everything else in <code>extraction.md</code> §1 was re-verified against the current tree
-and holds: the voice table, the 55 digraph entries, both vowel tables, the per-spec
+and holds: the voice table, the 53 digraph entries, both vowel tables, the per-spec
 timbre values, and the spec→sequence→bank mapping.</p>
 
 <div class="pager">
diff --git a/docs/scheduling.html b/docs/scheduling.html
index 581dff6..07f1e52 100644
--- a/docs/scheduling.html
+++ b/docs/scheduling.html
@@ -111,9 +111,12 @@ volume adjustment is skipped entirely. (What happens for other values is
 <a href="quirks.html">not what it looks like</a>.)</p>
 
 <p>It also contains one very specific special case: if the character is ♪ at exactly
-normal scale in colour <code>rgba(50, 60, 50, 255)</code>, the NPC id is forced to 0,
-suppressing the melody. This is presumably a specific piece of dialogue where a
-literal music note glyph should stay silent.</p>
+normal scale in colour <code>rgba(50, 60, 50, 255)</code> and the NPC id is non-zero, that id
+is forced to 0. This does <strong>not</strong> silence it — <code>Na_VoiceSe</code>'s ♪ escape
+only bails on <code>voice == 0xFA</code>, and <code>melody_seq_offset[0]</code> /
+<code>melody_seq_size[0]</code> are a real <code>0x120</code>-byte entry. The effect is to play
+the figure on instrument&nbsp;0 rather than the speaker's own, presumably so one particular
+line reads as a generic note instead of that character's timbre.</p>
 
 <h2>End of message</h2>
 
diff --git a/src/main.rs b/src/main.rs
index d35753c..44d0887 100644
--- a/src/main.rs
+++ b/src/main.rs
@@ -10,8 +10,8 @@ struct Args {
     timbre: f64,
 }
 
+mod phonemes;
 mod tokenizer;
-use tokenizer::VoiceCode;
 
 fn main() {
     let _args = Args::parse();
diff --git a/src/phonemes/consonant.rs b/src/phonemes/consonant.rs
new file mode 100644
index 0000000..bdf07ee
--- /dev/null
+++ b/src/phonemes/consonant.rs
@@ -0,0 +1,53 @@
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub enum Consonant {
+    B,
+    CHard,
+    CSoft,
+    D,
+    F,
+    G,
+    H,
+    J,
+    K,
+    L,
+    M,
+    P,
+    Q,
+    R,
+    S,
+    T,
+    V,
+    W,
+    X,
+    Y,
+    Z,
+}
+
+impl Consonant {
+    pub const fn code(self) -> u8 {
+        use Consonant::*;
+        match self {
+            B => 0x07,
+            CHard => 0x18,
+            CSoft => 0x23,
+            D => 0x09,
+            F => 0x11,
+            G => 0x12,
+            H => 0x00,
+            J => 0x17,
+            K => 0x18,
+            L => 0x19,
+            M => 0x1A,
+            P => 0x21,
+            Q => 0x18,
+            R => 0x22,
+            S => 0x23,
+            T => 0x26,
+            V => 0x2D,
+            W => 0x2E,
+            X => 0x31,
+            Y => 0x15,
+            Z => 0x30,
+        }
+    }
+}
diff --git a/src/phonemes/digit.rs b/src/phonemes/digit.rs
new file mode 100644
index 0000000..a694016
--- /dev/null
+++ b/src/phonemes/digit.rs
@@ -0,0 +1,31 @@
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub enum Digit {
+    D0,
+    D1,
+    D2,
+    D3,
+    D4,
+    D5,
+    D6,
+    D7,
+    D8,
+    D9,
+}
+
+impl Digit {
+    pub const fn code(self) -> u8 {
+        use Digit::*;
+        match self {
+            D0 => 0x32,
+            D1 => 0x33,
+            D2 => 0x26,
+            D3 => 0x27,
+            D4 => 0x34,
+            D5 => 0x35,
+            D6 => 0x36,
+            D7 => 0x37,
+            D8 => 0x10,
+            D9 => 0x38,
+        }
+    }
+}
diff --git a/src/phonemes/digraph.rs b/src/phonemes/digraph.rs
new file mode 100644
index 0000000..c332520
--- /dev/null
+++ b/src/phonemes/digraph.rs
@@ -0,0 +1,118 @@
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+#[allow(non_camel_case_types)]
+pub enum Digraph {
+    AI,
+    AL,
+    AN,
+    AR,
+    AS,
+    BE,
+    BY,
+    CH,
+    DO,
+    EA,
+    ER,
+    FA,
+    FO,
+    GE,
+    GO,
+    HA,
+    HE,
+    HI,
+    HO,
+    I_,
+    IF,
+    IN,
+    IR,
+    IS,
+    IT,
+    KS,
+    ME,
+    MY,
+    NA,
+    NG,
+    NO,
+    OF,
+    OH,
+    OK,
+    ON,
+    OO,
+    OR,
+    OU,
+    OW,
+    OY,
+    RE,
+    SE,
+    SH,
+    SI,
+    SO,
+    SU,
+    TH,
+    TO,
+    UP,
+    WE,
+    WO,
+    YOU,
+    ZE,
+}
+
+impl Digraph {
+    pub const fn code(self) -> u8 {
+        use Digraph::*;
+        match self {
+            AI => 0x0F,
+            AL => 0x02,
+            AN => 0x39,
+            AR => 0x16,
+            AS => 0x3A,
+            BE => 0x3B,
+            BY => 0x3C,
+            CH => 0x08,
+            DO => 0x3D,
+            EA => 0x15,
+            ER => 0x0D,
+            FA => 0x35,
+            FO => 0x34,
+            GE => 0x17,
+            GO => 0x3E,
+            HA => 0x3F,
+            HE => 0x40,
+            HI => 0x41,
+            HO => 0x42,
+            I_ => 0x05,
+            IF => 0x43,
+            IN => 0x44,
+            IR => 0x0C,
+            IS => 0x45,
+            IT => 0x46,
+            KS => 0x31,
+            ME => 0x47,
+            MY => 0x48,
+            NA => 0x38,
+            NG => 0x1C,
+            NO => 0x49,
+            OF => 0x4A,
+            OH => 0x4B,
+            OK => 0x4C,
+            ON => 0x4D,
+            OO => 0x2A,
+            OR => 0x2C,
+            OU => 0x20,
+            OW => 0x06,
+            OY => 0x1F,
+            RE => 0x0E,
+            SE => 0x37,
+            SH => 0x24,
+            SI => 0x36,
+            SO => 0x4F,
+            SU => 0x25,
+            TH => 0x28,
+            TO => 0x50,
+            UP => 0x51,
+            WE => 0x52,
+            WO => 0x33,
+            YOU => 0x77,
+            ZE => 0x32,
+        }
+    }
+}
diff --git a/src/phonemes/mod.rs b/src/phonemes/mod.rs
index e69de29..3f0276e 100644
--- a/src/phonemes/mod.rs
+++ b/src/phonemes/mod.rs
@@ -0,0 +1,11 @@
+mod consonant;
+mod digit;
+mod digraph;
+mod phonemes;
+mod vowel;
+
+pub use consonant::Consonant;
+pub use digit::Digit;
+pub use digraph::Digraph;
+pub use phonemes::Phoneme;
+pub use vowel::{Vowel, VowelLen, VowelPhoneme};
diff --git a/src/phonemes/phonemes.rs b/src/phonemes/phonemes.rs
index 9313c21..3ec5e56 100644
--- a/src/phonemes/phonemes.rs
+++ b/src/phonemes/phonemes.rs
@@ -1,6 +1,32 @@
-#[derive(Debug, PartialEq, Eq)]
+use super::consonant::Consonant;
+use super::digit::Digit;
+use super::digraph::Digraph;
+use super::vowel::VowelPhoneme;
+
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
 pub enum Phoneme {
-    
+    Vowel(VowelPhoneme),
+    Consonant(Consonant),
+    Digit(Digit),
+    Digraph(Digraph),
+    Silent,
+    Raw(u8),
 }
 
+impl Phoneme {
+    pub const fn code(self) -> u8 {
+        match self {
+            Phoneme::Vowel(v) => v.code(),
+            Phoneme::Consonant(c) => c.code(),
+            Phoneme::Digit(d) => d.code(),
+            Phoneme::Digraph(d) => d.code(),
+            Phoneme::Silent => 0x00,
+            Phoneme::Raw(n) => n,
+        }
+    }
 
+    pub const fn plays(self) -> bool {
+        let n = self.code();
+        n != 0 && n <= 0x77
+    }
+}
diff --git a/src/phonemes/vowel.rs b/src/phonemes/vowel.rs
new file mode 100644
index 0000000..622185d
--- /dev/null
+++ b/src/phonemes/vowel.rs
@@ -0,0 +1,38 @@
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub enum Vowel {
+    A,
+    I,
+    U,
+    E,
+    O,
+    N,
+}
+
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub enum VowelLen {
+    Short,
+    Long,
+}
+
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub struct VowelPhoneme(pub Vowel, pub VowelLen);
+
+impl VowelPhoneme {
+    pub const fn code(self) -> u8 {
+        use Vowel::*;
+        use VowelLen::*;
+        match (self.0, self.1) {
+            (A, Short) => 0x01,
+            (A, Long) => 0x02,
+            (I, Short) => 0x14,
+            (I, Long) => 0x15,
+            (U, Short) => 0x29,
+            (U, Long) => 0x2A,
+            (E, Short) => 0x0A,
+            (E, Long) => 0x0E,
+            (O, Short) => 0x1D,
+            (O, Long) => 0x1E,
+            (N, _) => 0x1B,
+        }
+    }
+}
diff --git a/src/tokenizer/mod.rs b/src/tokenizer/mod.rs
index 54ea9a3..1651f76 100644
--- a/src/tokenizer/mod.rs
+++ b/src/tokenizer/mod.rs
@@ -1,2 +1,2 @@
 mod tokens;
-pub use tokens::VoiceCode;
+pub use tokens::{ControlCode, Digit, Letter, VoiceCode};
diff --git a/src/tokenizer/tokens.rs b/src/tokenizer/tokens.rs
index fa1d7e7..d013497 100644
--- a/src/tokenizer/tokens.rs
+++ b/src/tokenizer/tokens.rs
@@ -1,74 +1,164 @@
-#[derive(Debug, PartialEq, Eq)]
+//! The voice-code alphabet.
+//!
+//! `mMsg_sound_voice_get` folds the game's 256-entry character set down to these 34
+//! values. Everything after this point — word boundaries, capitalisation, sentence
+//! structure, and any letter outside `a-z0-9` — is gone.
+
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
 pub enum ControlCode {
-    Melody,     // 0x80
-    Space,      // 0x81
-    Question,   // 0x82
-    Bang,       // 0x83
-    Newline,    // 0x84
-    Punct,      // 0x85
-    Bebe,       // 0x86,
-    Control,    // -1
+    /// ♪ — plays the speaker's melody and bypasses the whole engine.
+    Melody, // 0x80
+    Space,    // 0x81
+    Question, // 0x82
+    Bang,     // 0x83
+    /// Injected by `mMsg_sound_voice_get2`; never present in the raw table.
+    Newline, // 0x84
+    /// The catch-all for punctuation and symbols.
+    Punct, // 0x85
+    /// The typing click. Only the interpunct (char 133) maps here.
+    Bebe, // 0x86
+    /// An empty `b` / `c` slot in a half-built window.
+    Empty, // 0xFF
+    /// `mMsg_sound_voice_get` returning `-1` for `CHAR_CONTROL_CODE` /
+    /// `CHAR_MESSAGE_TAG`. Never reaches the audio engine: `mMsg_sound_CodeVoice`
+    /// drops it, and the `c` lookahead folds it to [`ControlCode::Empty`].
+    Control, // -1
+}
+
+impl ControlCode {
+    /// [`ControlCode::Control`] has no code of its own; it reports `0xFF` because that is
+    /// what the lookahead substitutes, and it can never reach playback by any other path.
+    pub const fn code(self) -> u8 {
+        match self {
+            ControlCode::Melody => 0x80,
+            ControlCode::Space => 0x81,
+            ControlCode::Question => 0x82,
+            ControlCode::Bang => 0x83,
+            ControlCode::Newline => 0x84,
+            ControlCode::Punct => 0x85,
+            ControlCode::Bebe => 0x86,
+            ControlCode::Empty | ControlCode::Control => 0xFF,
+        }
+    }
 }
 
-#[derive(Debug, PartialEq, Eq)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
 pub enum Letter {
-    A,          // 0x5D
-    B,          // 0x5E
-    C,          // 0x5F
-    D,          // 0x60
-    E,          // 0x61
-    F,          // 0x62
-    G,          // 0x63
-    H,          // 0x64
-    I,          // 0x65
-    J,          // 0x66
-    K,          // 0x67
-    L,          // 0x68
-    M,          // 0x69
-    N,          // 0x6A
-    O,          // 0x6B
-    P,          // 0x6C
-    Q,          // 0x6D
-    R,          // 0x6E
-    S,          // 0x6F
-    T,          // 0x70
-    U,          // 0x71
-    V,          // 0x72
-    W,          // 0x73
-    X,          // 0x74
-    Y,          // 0x75
-    Z,          // 0x76
+    A,
+    B,
+    C,
+    D,
+    E,
+    F,
+    G,
+    H,
+    I,
+    J,
+    K,
+    L,
+    M,
+    N,
+    O,
+    P,
+    Q,
+    R,
+    S,
+    T,
+    U,
+    V,
+    W,
+    X,
+    Y,
+    Z,
 }
 
-#[derive(Debug, PartialEq, Eq)]
+impl Letter {
+    /// `0x5D + (letter - 'a')`.
+    pub const fn code(self) -> u8 {
+        use Letter::*;
+        match self {
+            A => 0x5D,
+            B => 0x5E,
+            C => 0x5F,
+            D => 0x60,
+            E => 0x61,
+            F => 0x62,
+            G => 0x63,
+            H => 0x64,
+            I => 0x65,
+            J => 0x66,
+            K => 0x67,
+            L => 0x68,
+            M => 0x69,
+            N => 0x6A,
+            O => 0x6B,
+            P => 0x6C,
+            Q => 0x6D,
+            R => 0x6E,
+            S => 0x6F,
+            T => 0x70,
+            U => 0x71,
+            V => 0x72,
+            W => 0x73,
+            X => 0x74,
+            Y => 0x75,
+            Z => 0x76,
+        }
+    }
+}
+
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
 pub enum Digit {
-    D0,         // 0x53
-    D1,         // 0x54
-    D2,         // 0x55
-    D3,         // 0x56
-    D4,         // 0x57
-    D5,         // 0x58
-    D6,         // 0x59
-    D7,         // 0x5A
-    D8,         // 0x5B
-    D9,         // 0x5C
+    D0,
+    D1,
+    D2,
+    D3,
+    D4,
+    D5,
+    D6,
+    D7,
+    D8,
+    D9,
+}
+
+impl Digit {
+    /// `0x53 + digit`. This is the *voice* code, not the phoneme code — see
+    /// `phonemes::digit_code` for what the conversion tables turn it into.
+    pub const fn code(self) -> u8 {
+        use Digit::*;
+        match self {
+            D0 => 0x53,
+            D1 => 0x54,
+            D2 => 0x55,
+            D3 => 0x56,
+            D4 => 0x57,
+            D5 => 0x58,
+            D6 => 0x59,
+            D7 => 0x5A,
+            D8 => 0x5B,
+            D9 => 0x5C,
+        }
+    }
 }
 
-#[derive(Debug)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
 pub enum VoiceCode {
     Letter(Letter),
     ControlCode(ControlCode),
-    Digit(Digit)
+    Digit(Digit),
 }
 
-
-impl Letter {
-    fn is_shiin(&self) -> bool {
-        !self.is_boin()
+impl VoiceCode {
+    pub const fn code(self) -> u8 {
+        match self {
+            VoiceCode::Letter(l) => l.code(),
+            VoiceCode::ControlCode(c) => c.code(),
+            VoiceCode::Digit(d) => d.code(),
+        }
     }
 
-    fn is_boin(&self) -> bool {
-        use Letter::*;
-        [A, E, I, O, U, N].contains(&self)
+    /// `> 0x77` — the test several rules use to mean "punctuation, not a letter".
+    pub const fn is_punctuation(self) -> bool {
+        self.code() > 0x77
     }
 }