How pitch shifting and time stretching work

Pitch and time in plain English

Play a record at the wrong speed and two things change together. At 45 rpm, a 33⅓ record runs 1.35 times too fast, so the song finishes sooner and every note comes out 5.2 semitones higher. Slow a tape machine down and the reverse happens: the music drags and the voices drop. On any machine that plays a recording by moving it past a head or a needle, speed and pitch are one control.

The link comes from how those machines read sound. A note's pitch is its frequency, the number of times its waveform repeats each second. Play the recording 10% faster and every repeat arrives 10% sooner, so every frequency rises by 10% and the whole thing ends sooner too.

Digital audio lets you pull the two apart. Time stretching changes how long a sound lasts while each note keeps its pitch, so a 120 BPM drum loop can play at 100 BPM. Pitch shifting moves the notes and leaves the timing exactly where it was, so a vocal recorded in A minor can sit in a song in C minor. Both usually start the same way. The sound is cut into short overlapping slices, called grains, and those grains are spaced out or packed together.

Reading the lab

The three cards along the top are three ways of changing a recording. Varispeed is the tape machine, with speed and pitch locked together. Plain grains and Matched grains both let you move speed and pitch on their own, and they differ in one detail you can hear, covered further down.

The top screen draws the original loop and the processed loop on the same time scale. Slow the loop down and the coloured lane gets longer than the pale one, and the dashed line at the end of each lane marks where it loops round. Under it, the spectrum shows how much of each loop sits at each frequency, with the original in pale behind. Change the pitch and the coloured shape slides sideways; change only the speed and it stays where it is.

The close-up takes the 250 ms around the loudest moment of the original and finds the same moment of music in the processed loop, then draws both across the full width so they line up. The small ticks are where each grain starts. Next to it, the Right now panel describes your settings using numbers measured from the processed audio: its length, its tempo and key if the loop has one, and on the sine and the drums, how cleanly the grains are joining.

A/B switches to the original at its own speed and pitch, matched in loudness. Most of the loops come from two Sounds Like You packs and the vocal is a public domain recording from Freesound. You can also upload your own sound or record eight seconds with your microphone. Nothing leaves your device.

Varispeed: pitch and speed together

Varispeed is the oldest way to change pitch and it's still the cleanest. Nothing is cut up: the recording is simply read faster or slower, so the result sounds exactly like the original played on a faster or slower machine. Ableton Live calls this warp mode Re-Pitch, Logic Pro's Flex Time calls it Speed (FX), and Pro Tools has an Elastic Audio mode called Varispeed.

Doubling the speed doubles every frequency, which is one octave up, and halving it drops an octave. In between, each semitone is a speed change of 5.95%, because twelve equal steps have to multiply up to 2. That's why the ±8% pitch fader on a Technics SL-1200 turntable only covers about 1.3 semitones up and 1.4 down: plenty for matching the tempo of two records, nowhere near enough to change a song's key by much.

Plenty of records use varispeed on purpose. The Chipmunks records of the late 1950s were made by recording voices with the tape running at half speed and playing them back at normal speed, which put the voices up an octave. Nightcore edits do the same to whole songs, and slowed and reverb edits go the other way, dropping the tempo and the key together.

How much speed is a semitone?

Musical steps are ratios. Going up one semitone multiplies the frequency by the twelfth root of 2, which is 1.0595, so in varispeed it means playing 5.95% faster whatever the note. A cent is a hundredth of a semitone, and it's the unit tuners and pitch plug-ins use for fine adjustments.

The table works out a few common moves for a loop recorded at 120 BPM.

Work it the other way and you can see the problem with varispeed in a real project. Say you want a 120 BPM sample in a 128 BPM track. That's 106.7% speed, which in varispeed is 1.12 semitones up, so the sample ends up 12 cents sharp of B♭ minor and out of tune with everything else. Time stretching solves exactly this.

What is time stretching?

Time stretching changes the length of a sound and leaves its pitch alone. The simplest version works on the waveform directly. It cuts the sound into grains a few tens of milliseconds long, gives each one a fade in and a fade out, and lays them down again overlapping each other by half. To make the sound last longer, the grains are laid down further apart than they were taken; to make it shorter, closer together.

Every grain still holds a short piece of the original wave at its original speed, so the pitch inside each one stays the same and only the spacing between them changes. Stretch a loop to 79.4% speed and each 60 ms grain is taken from 23.8 ms further along the original but placed 30 ms after the last one, which adds up to a loop 26% longer at the same pitch.

This is a close relative of granular synthesis, which uses the same grains for sound design, scattering and layering them from any point in a sample. A time stretcher uses them to sound as close to the original as it can.

How does pitch shifting work?

Many pitch shifters are a time stretcher followed by varispeed. To go up 5 semitones, the shifter first stretches the sound to 1.335 times its length at the same pitch, then plays that result 1.335 times faster. The varispeed step raises the pitch by 5 semitones and brings the length back to where it started, so the timing lines up with the original exactly. The lab does it in that order.

Because the last step is a change of speed, every frequency is multiplied by the same ratio. The fundamental of a note and all its harmonics move together and stay in proportion, which is why the result still sounds like the same instrument playing in a new key. A frequency shifter adds the same number of hertz to everything instead, and that turns notes metallic.

On the spectrum screen, a pitch shift moves the whole shape sideways by the same distance everywhere, because the screen's frequency scale is logarithmic like your hearing. Going up an octave moves 100 Hz to 200 Hz and 5 kHz to 10 kHz, and both moves are the same width on the screen.

Speeding up has one technical catch. When a sound is played faster, anything near the top of the frequency range gets pushed above what the sample rate can hold, and it folds back down as aliasing. A good pitch shifter filters the top end off before it speeds the sound up, and this lab does too. The aliasing lesson shows what happens when that filter is missing.

Why plain grains sound rough

Laying grains down at new positions has a cost. Where two grains overlap, both are playing at once, and unless their waves happen to be in step, they partly cancel. On a held note that cancellation comes round once per grain, so the level flutters at the grain rate and the note gets a rough, warbling edge.

A sine wave makes this easy to hear. At half speed with 60 ms plain grains, each grain is placed 30 ms after the last but taken from only 15 ms further along the original. The 15 ms difference is 1.65 cycles of a 110 Hz sine, so every overlap meets the next grain 0.65 of a cycle out of phase. The lab measures the level of each cycle of the processed sine, and it swings by about 5 dB. The spectrum shows the same thing a different way: one line at 110 Hz has become a row of lines 33 Hz apart, the grain rate, with the loudest at 122 Hz.

The fix is to let each grain move a little. Before placing a grain, the stretcher looks at the end of the grain before it and slides the new one a few milliseconds earlier or later, to wherever its wave matches best. The grains then meet in step and add up cleanly. This method is called WSOLA, short for waveform similarity overlap-add, and the Matched grains card in the lab is a straightforward version of it. On the same half-speed sine, the level swing drops to almost nothing and the spectrum goes back to a single line at 110 Hz.

Matching works well on one note at a time, like a sine, a bass line or a single voice, because there's one wave to line up. Chords and full mixes have several notes with different cycle lengths, and no single slide lines all of them up at once. That's why DAWs have separate modes for monophonic and polyphonic material.

Grain size: flams against buzz

The length of the grains is the biggest trade-off in time stretching, and Ableton Live puts it on a Grain Size control in its Tones and Texture modes.

Long grains suit held notes. A 100 ms grain holds eleven cycles of a 110 Hz note, which gives the matching plenty of wave to compare, and the joins come round less often. The cost is on transients. At half speed, every moment of the original lands in several overlapping grains, and with long grains those copies are far enough apart to hear separately. A snare hit comes out twice, a fraction of a second apart, which drummers call a flam. In the lab, half-speed drums in 150 ms grains have about 100 attacks against 40 in the original.

Short grains fix the drums and hurt the bass. At 20 ms the copies of each hit sit so close together that they blur into one, and the same half-speed drums come out with close to the original 40 attacks. But a 20 ms grain holds barely two cycles of a 110 Hz note, and the grains now repeat 100 times a second, fast enough for the joins themselves to become a buzz.

Formants and the chipmunk sound

Pitch a voice up 7 semitones and the singer seems to shrink into a much smaller person, or a cartoon. Formants are the reason. They're the resonances of the throat, mouth and nose, which boost certain frequency ranges whatever note is being sung, and they're a big part of what makes an "ee" sound different from an "oo" and one person's voice recognisable.

A plain pitch shift multiplies every frequency by the same ratio, formants included. Up 7 semitones, every resonance sits 1.5 times higher than before, as if the singer's whole vocal tract had shrunk by a third. Down 7 semitones and the resonances drop by the same ratio, which gives the slow, giant voice. The lab doesn't correct formants, so the vocal step shows exactly that.

Formant-preserving pitch shifters separate the two. They measure the overall shape of the spectrum, move the notes, then put the shape back where it was. Celemony Melodyne and Soundtoys Little AlterBoy can also move the formants on their own, which changes the size of a voice without changing the note. The Formants control in Ableton Live's Complex Pro mode only does the preserving, keeping the resonances in place when you transpose.

Slicing, phase vocoders and other methods

Grain-based stretching is one family. DAWs use several others, often more than one inside a single mode.

Slicing is the drum specialist. It finds each transient, cuts the audio there, and moves each slice to its new place in time without stretching it at all. Slow a loop down and gaps open after each hit, which the stretcher fills by fading the slice out or looping its tail; speed it up and each slice is cut short where the next one begins. Propellerhead's ReCycle, which made REX files, worked this way, as does Logic's Slicing mode, and Ableton's Beats mode is also built around the transients it finds.

A phase vocoder works in the frequency domain. It breaks the sound into hundreds or thousands of narrow frequency bands, tracks how each band's phase moves, and rebuilds the sound at a new length while keeping each band's frequency. It copes better than grains with chords and full mixes, but it smears transients and can sound "phasey", a slightly hollow, swirling tone. Paulstretch, a free program by Paul Nasca, pushes the frequency-domain approach to the extreme and is made for stretching sounds to many times their length as an effect.

For a single voice there's PSOLA, short for pitch-synchronous overlap-add. It finds every cycle of the note and cuts one grain around each, then moves the grains closer together to raise the pitch or further apart to lower it. Each grain keeps its own shape, so the formants stay where they were, but it needs one clear pitch at a time to work.

Choosing a warp mode in your DAW

Every DAW names its modes differently, but they line up by the kind of sound they're built for. Pick the mode for the material first, then adjust its settings by ear.

Logic's Rhythmic mode is for parts like strummed guitar and keyboard comping, which have a beat and notes at once. Logic also has Tempophone (FX), an effect mode with an obviously grainy, artificial sound, and Pro Tools has X-Form, a slower, higher-quality polyphonic mode that renders the result instead of playing it back live.

How to fit a sample to your tempo and key

This order works in any DAW and keeps the artefacts as small as they can be.

1. Find the sample's own tempo and keySample packs usually put both in the file name. If not, your DAW's tempo detection is a starting point, but check it, because detectors often report half or double the real tempo.
2. Decide whether the pitch is allowed to moveA drum loop with no clear key that only needs a few percent of speed change sounds cleanest in varispeed. Anything with notes in it needs time stretching so it stays in key.
3. Pick the mode for the materialUse the table above. A drum loop in a polyphonic mode or a vocal in a beats mode is the most common reason a stretch sounds bad.
4. Line up the first downbeatPut the start of the sample, or its first warp marker, exactly on the first beat, so the stretcher counts bars from the right place.
5. Transpose in whole semitonesTo move a sample from A minor to C minor, transpose it +3, or −9 if you'd prefer it lower. Smaller moves leave fewer artefacts, so take the shorter route when the part allows it.
6. Listen to the hits and the held notesSolo the sample and listen for doubled attacks, warble on long notes and a hollow tone. Try the mode's grain or transient settings before you give up on it.

Common pitch shifting and time stretching mistakes

Speeding a sample up and leaving it out of tune

Re-Pitch, Speed and Varispeed modes all move the key along with the tempo. A sample pulled from 120 to 128 BPM that way ends up 1.12 semitones higher, between two keys. If it has notes in it, use a stretching mode instead.

Stretching an already stretched file

Every pass adds its own artefacts, and they build up. Keep the original file and stretch from it each time you change your mind about the tempo.

Using the wrong mode for the material

Grain-based modes double drum hits when you slow them down, and slicing modes chop held notes into stutters. If a stretch sounds wrong, the mode is the first thing to change.

Pitching vocals a long way without formant correction

A few semitones on a backing vocal often passes. Push a lead vocal much further and the singer stops sounding like themselves, so turn on formant preservation or move the formants back by hand.

Trusting the detected tempo

A loop detected at 60 BPM is often a 120 BPM loop, and one detected at 170 might be 85. Stretch it with the wrong original tempo and it plays at twice or half the length it should, with all the artefacts of a huge stretch.

Training your ears to hear pitch and speed

On a busy mix it's easy to mistake a change of tempo for a change of key. The quiz under the lab plays a loop with its pitch moved, its speed moved, or both together the way varispeed does, with the screens covered and the loudness matched. Listen to the groove for the speed and to the lowest note for the pitch, and use A/B to check against the original. Eight rounds take a few minutes, and your best score is saved in your browser.

Questions people ask

Keep going