5. Playback

A phoneme code becomes an instrument number, written into a port on a looping JAudio2 sequence. Everything expressive happens through two float multipliers.

Punctuation, applied at play time

By the time the rule engine finishes, punctuation codes are still in a. The sou_voice_mode switch (game64.c_inc:2810) resolves them, and splits on whether this is the player's voice:

Specs 5 and 6 — the player

switch (a) {
    case 0x81: case 0x84: case 0x85: case 0x86:
        Na_SysTrgStart(NA_SE_BEBE);  a = 0;  break;
}

Every punctuation code clicks. ? and ! fall through unmodified into the phoneme path — the pitch and volume bumps below never apply to the player.

All other specs

Both multipliers are applied to the spec's base value, which was just reset at the top of the call, so they last exactly one utterance. The pitch factor differing per sequence is a per-bank tuning correction, not an expressive choice.

The emotion LFO

Six emotion states are set by text control codes and stored in sou_message_status:

Each drives a short random-period modulation of _18 (pitch multiplier) and _0C (volume multiplier), recomputed per utterance (game64.c_inc:2880). It is skipped for punctuation (r29 > 0x77) and for specs 5 and 6, in which case the previous values persist.

Angry (1)

if (counter == 0) counter = (rand() >> 6) + 10;   // 10–13
else counter--;
switch (counter) {
    case 0: case 1: case 2:  pitch = 1.25, vol = 1.25;  break;
    case 3: case 4:          pitch = 1.20, vol = 1.20;  break;
    case 9: case 10:         pitch = 1.25, vol = 1.25;  break;
    default:                 pitch = 0.90, vol = 1.05;  break;
}

Mostly flat and slightly quiet, with two bursts near the top and bottom of each 10–13 syllable cycle — a jagged, shouty contour.

Sad (2), Sleepy (4), Gloomy (5)

if (counter == 0) { effect = (rand() >> 6) + 14; counter = 1; }   // 14–17
else if (counter == effect) { counter = 0; sad_toguru ^= 1; }
else counter++;

pitch = (sad_toguru ? 1.1 : 1.0) - counter * 0.022;
vol   = (sad_toguru ? 1.1 : 1.0) - counter * 0.025;

A sawtooth: pitch and volume slide down over 14–17 syllables, then jump back up, alternating between two baselines. The three states share the code exactly — they differ only in what the writers chose to tag.

Fun (3)

if (counter == 0) counter = (rand() >> 6) + 10;
else counter--;
switch (counter) {
    case 0: case 1:  pitch = 1.05, vol = 1.05;  break;
    case 2:          pitch = 0.90, vol = 1.15;  break;
    default:         pitch = 1.15, vol = 1.05;  break;
}

Consistently bright with a dip at the end of each cycle.

Normal (0)

pitch = 1.0;  vol = 1.0;

Reaching the DSP

Two subtracks in NA_GROUP4 alternate, toggled by sou_voice_se_toguru on every call, so consecutive syllables can overlap rather than cutting each other off:

void Sou_VoiceStart(u8 subtrack, u8 b) {
    NA_COMMAND_AUDIO_SUBTRACK_SET_PORT(VOICE_GROUP, subtrack, 0, b);
}

The guard before it is if (a != 0 && a <= 0x77) — code 0 never plays, which is why h is silent. The call is also skipped entirely when the raw code was space, newline or mute (r29 != 0x85 && r29 != 0x81 && r29 != 0x84).

The sequence side

Each voice sequence's subtrack sits in a spin loop:

L_0019: delay 1 frame
        macro_read_port0
        branch_rel_not_eq_zero L_0019   ; wait until Sou_VoiceStart writes the port
        set_dyntbl D_0026
        dyntbl_call                     ; index the table with the phoneme code
        branch L_0019

The dynamic table has 128 entries pointing at 10-byte routines C1 <instrument> / 88 <note> / FF. Entry v selects instrument v − 1. Entry 0 is an immediate FF — nothing.

Per-frame parameter push

Sou_VoiceMake (game64.c_inc:1174) runs for both subtracks once per game frame, inside Na_GameFrame:

volumeScale    = _08 * _0C;      // spec base × emotion volume
frequencyScale = _14 * _18;      // spec base × emotion pitch
NA_COMMAND_AUDIO_SUBTRACK_SET_VOL_SCALE (VOICE_GROUP, index, volumeScale);
NA_COMMAND_AUDIO_SUBTRACK_SET_FREQ_SCALE(VOICE_GROUP, index, frequencyScale);
NA_COMMAND_AUDIO_SUBTRACK_SET_PAN       (VOICE_GROUP, index, pan);
NA_COMMAND_AUDIO_SUBTRACK_REVERB_VOLUME (VOICE_GROUP, index, reverbVolume);

pan is initialised to 0x3F (centre) and reverbVolume to 0, and neither is ever written for voice subtracks. Animalese is always centred, dry, and unaffected by the speaker's distance from the camera — unlike every other positional SE in the game.

Because both subtracks share sou_voice_se[0] and [1] which are always written together, an in-flight syllable's volume and pitch are retroactively changed by the next syllable's settings.

Frame ordering

graph.c runs game_main (including the draw pass that calls Na_VoiceSe) and then sAdo_GameFrame. So within one frame:

game_main   → mFontSentence_animal_voice_se → Na_VoiceSe  (sets params, writes port)
Na_GameFrame → Sou_VoiceMake       (pushes params to the DSP)
             → Sou_SpecialRoutine10 (fires any queued second syllable)

Sou_VoiceMake runs before Sou_SpecialRoutine10, so a deferred syllable's parameters land one frame late.

Banks

From src/static/jaudio_NES/game/audioheaders.c:

param1 >> 8 is the instrument count, param0 >> 8 the sample bank id (5 for all three), and all have zero drums and zero SFX. 119 instruments = phoneme codes 0x01–0x77 exactly.

Bank 157 stopping at 83 is not an omission: instruments 0–82 cover codes 0x01–0x53, and the phonology engine never produces a code above 0x52 apart from 0x77. The missing range 0x53–0x76 is the editor's letter-name block, which spec 3 can never reach. The only thing spec-3 voices genuinely lack is you.

Sample details are in ../extraction.md: N64 VADPCM, tuning 0.6891 (22050 Hz), order 2, two predictors, no loop, ~190 ms each.