The sounds of language

A continuous signal, heard as discrete sounds.

Say “ten” five times and record it. The spectrograms will all differ: one is longer, one higher in pitch, one was said with a smile. A child, a woman and a man differ even more, because their vocal tracts have different sizes. In a sentence there are not even gaps between the words. Physically, speech is a continuous, never-repeating stream. Yet every English listener hears “ten” every time, as three sounds in a row, and hears at once that “den” is a different word. To do that, the listener throws away almost all of the variation (speaker, speed, pitch, mood) and keeps a short string of discrete sounds. An actual speech sound, as one speaker produced it at one moment, is called a phone, and it is written in square brackets: [t], [ɛ], [n].

A real sentence, sound by sound

Below is a sentence read by ksp, an Indian-English speaker in the CMU ARCTIC corpus (Kominek & Black 2004): “Tom Spink has a harpoon.” (utterance a0480, 2.1 s). The corpus comes with phone labels made by an automatic aligner, which places a dictionary pronunciation on the audio. Three views of the same signal: the raw signal (waveform and spectrogram), the string of 18 phones, and the phones grouped into 5 words.

Two lessons hide in this demo. First, the boundaries are fuzzy: play a single phone and it often sounds clipped or carries a trace of its neighbours, because the sounds overlap. Second, the labels were produced with a General American dictionary, so they are that dictionary's (the /ɑ/ of Tom, the /ɹ/ of harpoon), not necessarily what ksp says. The aligner can only use symbols its dictionary has: it has no symbol for a retroflex [ʈ], however the speaker pronounces /t/.

The smallest sound that can change a word.

The listener does not keep every phone apart, only the differences that can turn one word into another. Those categories are the phonemes of a language. Two kinds of unit need two notations. A phone is an actual sound, written in square brackets: [pʰ] is a p with a puff of air. A phoneme is a category of a particular language, written between slashes: English /p/ covers both [pʰ] and [p]. (Spellings go in angle brackets: ⟨ph⟩.) The test that decides whether two sounds are different phonemes is the minimal pair: two words that differ in exactly one sound and mean different things. pin and bin make /p/ and /b/ two English phonemes; pin, pen and pan do the same for /ɪ/, /ɛ/ and /æ/.

Minimal pairs from one speaker

ksp reads 1132 sentences containing 2747 different words. Comparing their dictionary pronunciations finds 2492 pairs of words that differ in exactly one phoneme. Here are eight, cut from the recordings (each word with 40 ms of context, faded in and out). The small print under each word is a measurement on that very token, as a check that the audio really carries the contrast the labels claim.

The checks show the contrasts are real in his speech, sometimes with a different recipe from American English. His /b/ and /ɡ/ are voiced right through the closure (pre-voiced, as voiced stops are in Hindi), while his /p/ and /k/ start voicing soon after the release. His /v/ in very has almost no hiss: it is closer to the approximant [ʋ] of Hindi व than to an English fricative, and still clearly different from the voiceless /f/ of ferry. His steel/still and men/man vowels differ more in length than in formants (his /æ/ is not lower than his /ɛ/ in these tokens), and /s/ versus /ʃ/ differ in noise centroid by only a few hundred hertz: listen for the difference in the hiss. Contrasts survive; the cues that carry them vary from speaker to speaker.

Phonemes and allophones: one category, several pronunciations

A phoneme is not one sound. It is a category in the listener's mind, and in real speech it comes out in several slightly different ways depending on its neighbours. Each of those ways is an allophone of the phoneme. The surprising part is that native speakers use them all the time without noticing.

Take English /t/. Say these words slowly, with a hand in front of your mouth:

• top: a clear puff of air after the t, an aspirated [tʰ];
• stop: no puff after the s, a plain [t];
• butter in American English: a quick tap of the tongue, almost a d, a flap [ɾ];
• button in many British accents: no tongue contact at all, a catch in the throat, a glottal stop [ʔ].

Four physically different sounds, and an English listener hears “t” every time. Swap them around (say stop with a puff of air) and the word sounds odd or foreign, but it is still stop. That is the mark of allophones: they are variants of one phoneme, and which one you get is predictable from the context. An allophone rule states the pattern, for example “/t/ is aspirated at the start of a stressed syllable, plain after /s/” (Hayes 2009).

Now compare a language that draws the line in a different place. Hindi keeps the plain and the aspirated stop apart: ताल /t̪aːl/ ‘rhythm’ and थाल /t̪ʰaːl/ ‘plate’ are different words. The same puff of air that English ignores is, in Hindi, the whole difference between two phonemes:

Three quick tests tell the two situations apart:

Where, how, and with or without voice.

A language has a few dozen phonemes, and they are not an unordered list. Every speech sound is made by pushing air from the lungs through the vocal tract and shaping it on the way, so sounds can be sorted by how that shaping is done (Ladefoged & Johnson 2015). The first split is between consonants, made by narrowing or closing the tract somewhere, and vowels, made with the tract open and only the shape of the tongue and lips changing. (How the vocal folds and the tract turn air into sound is the subject of A buzz and a tube; you can move a model tongue and lips yourself in Vocal tract, live.)

Consonants: place, manner, voicing

Three questions describe almost any consonant. Where is the tract narrowed (the place of articulation)? How is it narrowed (the manner)? And do the vocal folds vibrate (voicing)? The answers are a consonant's name: /b/ is a voiced bilabial stop, /s/ a voiceless alveolar fricative, Hindi /ɳ/ (ण) a voiced retroflex nasal.

Vowels: height, backness, rounding

Vowels have no closure to locate, so they are described by the position of the tongue and the shape of the lips. Height: is the tongue high in the mouth (close vowels, /i/ of see, /u/ of food) or low with the jaw open (open vowels, /aː/ of Hindi आम)? Backness: is the highest point of the tongue at the front (/i e/), in the centre (/ə/, the first vowel of about, the inherent vowel of Hindi क), or at the back (/u o/)? Rounding: are the lips rounded (/u o/) or spread (/i a/)? So /i/ is a close front unrounded vowel and /u/ a close back rounded one. Two further properties matter in many languages: length (Tamil /a/ vs /aː/) and nasalisation (Hindi माँ /mãː/ ‘mother’). Acoustically, height and backness show up as the first two formants: close vowels have a low \(F_1\), front vowels a high \(F_2\), which is why the minimal-pair cards of section 2 measure \(F_1\) and \(F_2\).

The IPA chart: the classification as a map

To compare sounds across languages, linguists need a script whose symbols mean the same thing everywhere. That is the International Phonetic Alphabet (IPA), published by the International Phonetic Association since 1888 (IPA 1999) on one principle: one symbol for each distinctive sound. Its consonant chart is exactly the classification above laid out as a grid: place in the columns, from the lips on the left to the glottis on the right; manner in the rows; and where a cell holds two symbols, the voiceless one comes first and the voiced one second. Hatched cells are articulations judged impossible or unused. Vowels go in a quadrilateral, a stylised side view of the mouth: height vertically, backness horizontally. Below is a compact version, coloured by which symbols English and Hindi use (one PHOIBLE inventory each, named under the chart; the same two inventories are used in section 4).

Each cell pools the variants a language builds on one base symbol: Hindi's /p/ cell stands for p pʰ and its /b/ cell for b bʱ, so the four-way Hindi stop system fits into two cells. The rest is done by diacritics, small marks that refine a symbol. The ones you will meet in this module:

Distinctive features: the finer classification

Place, manner and voicing already show that phonemes are not unrelated symbols: /p/ and /b/ are made almost the same way, /p/ and /a/ very differently. Phonologists push the idea further by describing each phoneme as a bundle of distinctive features (Jakobson, Fant & Halle 1952; Chomsky & Halle 1968): binary properties such as voice (do the vocal folds vibrate?), labial (are the lips involved?), continuant (does air keep flowing?), nasal or spread glottis (aspiration). In machine-learning terms, a phoneme symbol is a one-hot code, while its feature vector is a hand-made embedding in which similar sounds sit close together. PHOIBLE (Moran & McCloy 2019), the database of phoneme inventories that page 2 explores, stores such a vector, with values +, − or 0 (not applicable), for every phoneme of every inventory. A natural distance is the Hamming distance, the number of features in which two phonemes differ:

Minimal pairs usually differ in one or two features: /p/–/b/ in 1 (voice), /p/–/pʰ/ in 1 (spread glottis), Hindi /t̪/–/ʈ/ in 2 (anterior, distributed), /i/–/ɪ/ in 1 (tense). A consonant and a vowel such as /p/ and /a/ differ in 15. Features also explain why sounds pattern together: English aspirates the [−voice, −continuant] consonants, /p t k/ and /tʃ/ (chin), with one rule over a class rather than four separate rules.

Speech technology uses the same idea. Classical recognisers model each phone in the context of its neighbours and share data between contexts by asking feature questions (“is the left neighbour a nasal?”); multilingual recognisers map a universal phone set onto each language's inventory (page 2, section 8), and some predict articulatory features directly, which lets them handle a sound they have never seen as a new combination of familiar features.

Hindi splits what English merges, and the other way round.

Section 3 gave the dimensions along which two sounds can differ. Whether a given difference separates words is a fact about a language, and that is why a contrast obvious to one listener can be nearly inaudible to another: English speakers learning Hindi struggle to hear ताल versus टाल (dental vs retroflex, a difference of place), and many Hindi speakers learning English merge vine and wine. Test yourself: for each pair of sounds below (they differ in aspiration, breathy voice, place or manner), guess whether English and Hindi treat them as one phoneme or two. The answers are computed from PHOIBLE: a pair counts as “two phonemes” if the language's inventory lists both sounds. For English we use General American, inventory 2175 (source ph: Western and Mid-Western US; Southern California), one of 9; all nine give the same answers here. For Hindi we use its median inventory (the rule explained on page 2, section 1), inventory 354 (source upsid, no dialect given), one of 5. These are the two inventories coloured in the IPA chart of section 3. The answers also say how many of the language's other inventories agree.

Mind the notation of the sources. PHOIBLE's English inventories write the voiceless stops as aspirated /pʰ tʰ kʰ/ (the phone you hear word-initially) and have no plain /p/, so “p vs pʰ” comes out as one phoneme. English /t/ is alveolar (tongue on the gum ridge), Hindi /t̪/ dental (tongue on the teeth) next to a retroflex /ʈ/: three different places, three different columns of the IPA chart, and English has neither of the two Hindi sounds as such. UPSID (Maddieson 1984) marks Hindi /ʃ/ and /z/ as marginal (mainly in loanwords), and writes some coronals as s̪|s, “dental or alveolar, undecided”.

Spelling is not sound.

The IPA of section 3 was designed so that one symbol stands for one sound, in every language. Ordinary spelling is a separate system, and its relation to the phonemes can be loose. The unit of writing is the grapheme: one letter (⟨b⟩), a group of letters acting together (⟨sh⟩, ⟨ough⟩), or in Indian scripts a consonant letter, a vowel sign or a mark that removes a vowel. English spelling kept many old spellings after the pronunciation changed: ⟨ough⟩ is said differently in though, through, tough, cough and bough; /iː/ is spelt six ways in see, sea, me, key, field, receive; the ⟨k⟩ of knight has been silent for centuries.

Writing systems differ in what their graphemes stand for. Alphabets (Latin, Greek, Cyrillic) write consonants and vowels as separate letters. Abugidas, which include nearly all scripts of India, write each consonant with a built-in vowel: क is ka, a vowel sign (mātrā) changes it (कि ki), and the virāma removes it (क् k). Abjads (Arabic, and the Perso-Arabic script of Urdu) mostly leave vowels unwritten, and logographic scripts (Chinese) write units of meaning. The Brahmi-derived scripts were organised by ancient Indian grammarians who had analysed how sounds are made, so their consonant letters already form the place-and-manner grid of section 3 (Hindi values):

Indian scripts are therefore much closer to sound than English spelling, but still not a transcription. In Hindi the inherent vowel is often silent (schwa deletion): कमल is written ka-ma-la but said /kəməl/. Assamese writes ⟨স⟩ (an s letter) for the sound /x/, so অসম ‘Assam’ is said /ɔxɔm/. Loan sounds get a dot (nuqtā): ज़ /z/, फ़ /f/. Converting spelling into phonemes is a task of its own, grapheme-to-phoneme conversion (G2P), and every text-to-speech system needs it.

Six questions.