1 /* -*- c-basic-offset: 4 indent-tabs-mode: nil -*- vi:set ts=8 sts=4 sw=4: */
5 An audio time-stretching and pitch-shifting library.
6 Copyright 2007-2008 Chris Cannam.
8 This program is free software; you can redistribute it and/or
9 modify it under the terms of the GNU General Public License as
10 published by the Free Software Foundation; either version 2 of the
11 License, or (at your option) any later version. See the file
12 COPYING included with this distribution for more information.
15 #ifndef _RUBBERBANDSTRETCHER_H_
16 #define _RUBBERBANDSTRETCHER_H_
18 #define RUBBERBAND_VERSION "1.3.0-gpl"
19 #define RUBBERBAND_API_MAJOR_VERSION 2
20 #define RUBBERBAND_API_MINOR_VERSION 0
26 * @mainpage RubberBand
28 * The Rubber Band API is contained in the single class
29 * RubberBand::RubberBandStretcher.
31 * Threading notes for real-time applications:
33 * Multiple instances of RubberBandStretcher may be created and used
34 * in separate threads concurrently. However, for any single instance
35 * of RubberBandStretcher, you may not call process() more than once
36 * concurrently, and you may not change the time or pitch ratio while
37 * a process() call is being executed (if the stretcher was created in
38 * "real-time mode"; in "offline mode" you can't change the ratios
41 * So you can run process() in its own thread if you like, but if you
42 * want to change ratios dynamically from a different thread, you will
43 * need some form of mutex in your code. Changing the time or pitch
44 * ratio is real-time safe except in extreme circumstances, so for
45 * most applications that may change these dynamically it probably
46 * makes most sense to do so from the same thread as calls process(),
47 * even if that is a real-time thread.
53 class RubberBandStretcher
57 * Processing options for the timestretcher. The preferred
58 * options should normally be set in the constructor, as a bitwise
59 * OR of the option flags. The default value (DefaultOptions) is
60 * intended to give good results in most situations.
62 * 1. Flags prefixed \c OptionProcess determine how the timestretcher
63 * will be invoked. These options may not be changed after
66 * \li \c OptionProcessOffline - Run the stretcher in offline
67 * mode. In this mode the input data needs to be provided
68 * twice, once to study(), which calculates a stretch profile
69 * for the audio, and once to process(), which stretches it.
71 * \li \c OptionProcessRealTime - Run the stretcher in real-time
72 * mode. In this mode only process() should be called, and the
73 * stretcher adjusts dynamically in response to the input audio.
75 * The Process setting is likely to depend on your architecture:
76 * non-real-time operation on seekable files: Offline; real-time
77 * or streaming operation: RealTime.
79 * 2. Flags prefixed \c OptionStretch control the profile used for
80 * variable timestretching. Rubber Band always adjusts the
81 * stretch profile to minimise stretching of busy broadband
82 * transient sounds, but the degree to which it does so is
83 * adjustable. These options may not be changed after
86 * \li \c OptionStretchElastic - Only meaningful in offline
87 * mode, and the default in that mode. The audio will be
88 * stretched at a variable rate, aimed at preserving the quality
89 * of transient sounds as much as possible. The timings of low
90 * activity regions between transients may be less exact than
91 * when the precise flag is set.
93 * \li \c OptionStretchPrecise - Although still using a variable
94 * stretch rate, the audio will be stretched so as to maintain
95 * as close as possible to a linear stretch ratio throughout.
96 * Timing may be better than when using \c OptionStretchElastic, at
97 * slight cost to the sound quality of transients. This setting
98 * is always used when running in real-time mode.
100 * 3. Flags prefixed \c OptionTransients control the component
101 * frequency phase-reset mechanism that may be used at transient
102 * points to provide clarity and realism to percussion and other
103 * significant transient sounds. These options may be changed
104 * after construction when running in real-time mode, but not when
105 * running in offline mode.
107 * \li \c OptionTransientsCrisp - Reset component phases at the
108 * peak of each transient (the start of a significant note or
109 * percussive event). This, the default setting, usually
110 * results in a clear-sounding output; but it is not always
111 * consistent, and may cause interruptions in stable sounds
112 * present at the same time as transient events.
114 * \li \c OptionTransientsMixed - Reset component phases at the
115 * peak of each transient, outside a frequency range typical of
116 * musical fundamental frequencies. The results may be more
117 * regular for mixed stable and percussive notes than
118 * \c OptionTransientsCrisp, but with a "phasier" sound. The
119 * balance may sound very good for certain types of music and
120 * fairly bad for others.
122 * \li \c OptionTransientsSmooth - Do not reset component phases
123 * at any point. The results will be smoother and more regular
124 * but may be less clear than with either of the other
127 * 4. Flags prefixed \c OptionPhase control the adjustment of
128 * component frequency phases from one analysis window to the next
129 * during non-transient segments. These options may be changed at
132 * \li \c OptionPhaseLaminar - Adjust phases when stretching in
133 * such a way as to try to retain the continuity of phase
134 * relationships between adjacent frequency bins whose phases
135 * are behaving in similar ways. This, the default setting,
136 * should give good results in most situations.
138 * \li \c OptionPhaseIndependent - Adjust the phase in each
139 * frequency bin independently from its neighbours. This
140 * usually results in a slightly softer, phasier sound.
142 * 5. Flags prefixed \c OptionThreading control the threading
143 * model of the stretcher. These options may not be changed after
146 * \li \c OptionThreadingAuto - Permit the stretcher to
147 * determine its own threading model. Usually this means using
148 * one processing thread per audio channel in offline mode if
149 * the stretcher is able to determine that more than one CPU is
150 * available, and one thread only in realtime mode.
152 * \li \c OptionThreadingNever - Never use more than one thread.
154 * \li \c OptionThreadingAlways - Use multiple threads in any
155 * situation where \c OptionThreadingAuto would do so, except omit
156 * the check for multiple CPUs and instead assume it to be true.
158 * 6. Flags prefixed \c OptionWindow control the window size for
159 * FFT processing. The window size actually used will depend on
160 * many factors, but it can be influenced. These options may not
161 * be changed after construction.
163 * \li \c OptionWindowStandard - Use the default window size.
164 * The actual size will vary depending on other parameters.
165 * This option is expected to produce better results than the
166 * other window options in most situations.
168 * \li \c OptionWindowShort - Use a shorter window. This may
169 * result in crisper sound for audio that depends strongly on
170 * its timing qualities.
172 * \li \c OptionWindowLong - Use a longer window. This is
173 * likely to result in a smoother sound at the expense of
174 * clarity and timing.
176 * 7. Flags prefixed \c OptionFormant control the handling of
177 * formant shape (spectral envelope) when pitch-shifting. These
178 * options may be changed at any time.
180 * \li \c OptionFormantShifted - Apply no special formant
181 * processing. The spectral envelope will be pitch shifted as
184 * \li \c OptionFormantPreserved - Preserve the spectral
185 * envelope of the unshifted signal. This permits shifting the
186 * note frequency without so substantially affecting the
187 * perceived pitch profile of the voice or instrument.
189 * 8. Flags prefixed \c OptionPitch control the method used for
190 * pitch shifting. These options may be changed at any time.
191 * They are only effective in realtime mode; in offline mode, the
192 * pitch-shift method is fixed.
194 * \li \c OptionPitchHighSpeed - Use a method with a CPU cost
195 * that is relatively moderate and predictable. This may
196 * sound less clear than OptionPitchHighQuality, especially
197 * for large pitch shifts.
199 * \li \c OptionPitchHighQuality - Use the highest quality
200 * method for pitch shifting. This method has a CPU cost
201 * approximately proportional to the required frequency shift.
203 * \li \c OptionPitchHighConsistency - Use the method that gives
204 * greatest consistency when used to create small variations in
205 * pitch around the 1.0-ratio level. Unlike the previous two
206 * options, this avoids discontinuities when moving across the
207 * 1.0 pitch scale in real-time; it also consumes more CPU than
208 * the others in the case where the pitch scale is exactly 1.0.
213 OptionProcessOffline = 0x00000000,
214 OptionProcessRealTime = 0x00000001,
216 OptionStretchElastic = 0x00000000,
217 OptionStretchPrecise = 0x00000010,
219 OptionTransientsCrisp = 0x00000000,
220 OptionTransientsMixed = 0x00000100,
221 OptionTransientsSmooth = 0x00000200,
223 OptionPhaseLaminar = 0x00000000,
224 OptionPhaseIndependent = 0x00002000,
226 OptionThreadingAuto = 0x00000000,
227 OptionThreadingNever = 0x00010000,
228 OptionThreadingAlways = 0x00020000,
230 OptionWindowStandard = 0x00000000,
231 OptionWindowShort = 0x00100000,
232 OptionWindowLong = 0x00200000,
234 OptionFormantShifted = 0x00000000,
235 OptionFormantPreserved = 0x01000000,
237 OptionPitchHighSpeed = 0x00000000,
238 OptionPitchHighQuality = 0x02000000,
239 OptionPitchHighConsistency = 0x04000000
245 DefaultOptions = 0x00000000,
246 PercussiveOptions = 0x00102000
250 * Construct a time and pitch stretcher object to run at the given
251 * sample rate, with the given number of channels. Processing
252 * options and the time and pitch scaling ratios may be provided.
253 * The time and pitch ratios may be changed after construction,
254 * but most of the options may not. See the option documentation
255 * above for more details.
257 RubberBandStretcher(size_t sampleRate,
259 Options options = DefaultOptions,
260 double initialTimeRatio = 1.0,
261 double initialPitchScale = 1.0);
262 ~RubberBandStretcher();
265 * Reset the stretcher's internal buffers. The stretcher should
266 * subsequently behave as if it had just been constructed
267 * (although retaining the current time and pitch ratio).
272 * Set the time ratio for the stretcher. This is the ratio of
273 * stretched to unstretched duration -- not tempo. For example, a
274 * ratio of 2.0 would make the audio twice as long (i.e. halve the
275 * tempo); 0.5 would make it half as long (i.e. double the tempo);
276 * 1.0 would leave the duration unaffected.
278 * If the stretcher was constructed in Offline mode, the time
279 * ratio is fixed throughout operation; this function may be
280 * called any number of times between construction (or a call to
281 * reset()) and the first call to study() or process(), but may
282 * not be called after study() or process() has been called.
284 * If the stretcher was constructed in RealTime mode, the time
285 * ratio may be varied during operation; this function may be
286 * called at any time, so long as it is not called concurrently
287 * with process(). You should either call this function from the
288 * same thread as process(), or provide your own mutex or similar
289 * mechanism to ensure that setTimeRatio and process() cannot be
290 * run at once (there is no internal mutex for this purpose).
292 void setTimeRatio(double ratio);
295 * Set the pitch scaling ratio for the stretcher. This is the
296 * ratio of target frequency to source frequency. For example, a
297 * ratio of 2.0 would shift up by one octave; 0.5 down by one
298 * octave; or 1.0 leave the pitch unaffected.
300 * To put this in musical terms, a pitch scaling ratio
301 * corresponding to a shift of S equal-tempered semitones (where S
302 * is positive for an upwards shift and negative for downwards) is
303 * pow(2.0, S / 12.0).
305 * If the stretcher was constructed in Offline mode, the pitch
306 * scaling ratio is fixed throughout operation; this function may
307 * be called any number of times between construction (or a call
308 * to reset()) and the first call to study() or process(), but may
309 * not be called after study() or process() has been called.
311 * If the stretcher was constructed in RealTime mode, the pitch
312 * scaling ratio may be varied during operation; this function may
313 * be called at any time, so long as it is not called concurrently
314 * with process(). You should either call this function from the
315 * same thread as process(), or provide your own mutex or similar
316 * mechanism to ensure that setPitchScale and process() cannot be
317 * run at once (there is no internal mutex for this purpose).
319 void setPitchScale(double scale);
322 * Return the last time ratio value that was set (either on
323 * construction or with setTimeRatio()).
325 double getTimeRatio() const;
328 * Return the last pitch scaling ratio value that was set (either
329 * on construction or with setPitchScale()).
331 double getPitchScale() const;
334 * Return the processing latency of the stretcher. This is the
335 * number of audio samples that one would have to discard at the
336 * start of the output in order to ensure that the resulting audio
337 * aligned with the input audio at the start. In Offline mode,
338 * latency is automatically adjusted for and the result is zero.
339 * In RealTime mode, the latency may depend on the time and pitch
340 * ratio and other options.
342 size_t getLatency() const;
345 * Change an OptionTransients configuration setting. This may be
346 * called at any time in RealTime mode. It may not be called in
347 * Offline mode (for which the transients option is fixed on
350 void setTransientsOption(Options options);
353 * Change an OptionPhase configuration setting. This may be
354 * called at any time in any mode.
356 * Note that if running multi-threaded in Offline mode, the change
357 * may not take effect immediately if processing is already under
358 * way when this function is called.
360 void setPhaseOption(Options options);
363 * Change an OptionFormant configuration setting. This may be
364 * called at any time in any mode.
366 * Note that if running multi-threaded in Offline mode, the change
367 * may not take effect immediately if processing is already under
368 * way when this function is called.
370 void setFormantOption(Options options);
373 * Change an OptionPitch configuration setting. This may be
374 * called at any time in RealTime mode. It may not be called in
375 * Offline mode (for which the transients option is fixed on
378 void setPitchOption(Options options);
381 * Tell the stretcher exactly how many input samples it will
382 * receive. This is only useful in Offline mode, when it allows
383 * the stretcher to ensure that the number of output samples is
384 * exactly correct. In RealTime mode no such guarantee is
385 * possible and this value is ignored.
387 void setExpectedInputDuration(size_t samples);
390 * Ask the stretcher how many audio sample frames should be
391 * provided as input in order to ensure that some more output
392 * becomes available. Normal usage consists of querying this
393 * function, providing that number of samples to process(),
394 * reading the output using available() and retrieve(), and then
397 * Note that this value is only relevant to process(), not to
398 * study() (to which you may pass any number of samples at a time,
399 * and from which there is no output).
401 size_t getSamplesRequired() const;
404 * Tell the stretcher the maximum number of sample frames that you
405 * will ever be passing in to a single process() call. If you
406 * don't call this function, the stretcher will assume that you
407 * never pass in more samples than getSamplesRequired() suggested
408 * you should. You should not pass in more samples than that
409 * unless you have called setMaxProcessSize first.
411 * This function may not be called after the first call to study()
414 * Note that this value is only relevant to process(), not to
415 * study() (to which you may pass any number of samples at a time,
416 * and from which there is no output).
418 void setMaxProcessSize(size_t samples);
421 * Provide a block of "samples" sample frames for the stretcher to
422 * study and calculate a stretch profile from.
424 * This is only meaningful in Offline mode, and is required if
425 * running in that mode. You should pass the entire input through
426 * study() before any process() calls are made, as a sequence of
427 * blocks in individual study() calls, or as a single large block.
429 * "input" should point to de-interleaved audio data with one
430 * float array per channel. "samples" supplies the number of
431 * audio sample frames available in "input". If "samples" is
432 * zero, "input" may be NULL.
434 * Set "final" to true if this is the last block of data that will
435 * be provided to study() before the first process() call.
437 void study(const float *const *input, size_t samples, bool final);
440 * Provide a block of "samples" sample frames for processing.
441 * See also getSamplesRequired() and setMaxProcessSize().
443 * Set "final" to true if this is the last block of input data.
445 void process(const float *const *input, size_t samples, bool final);
448 * Ask the stretcher how many audio sample frames of output data
449 * are available for reading (via retrieve()).
451 * This function returns 0 if no frames are available: this
452 * usually means more input data needs to be provided, but if the
453 * stretcher is running in threaded mode it may just mean that not
454 * enough data has yet been processed. Call getSamplesRequired()
455 * to discover whether more input is needed.
457 * This function returns -1 if all data has been fully processed
458 * and all output read, and the stretch process is now finished.
460 int available() const;
463 * Obtain some processed output data from the stretcher. Up to
464 * "samples" samples will be stored in the output arrays (one per
465 * channel for de-interleaved audio data) pointed to by "output".
466 * The return value is the actual number of sample frames
469 size_t retrieve(float *const *output, size_t samples) const;
472 * Return the value of internal frequency cutoff value n.
474 * This function is not for general use.
476 float getFrequencyCutoff(int n) const;
479 * Set the value of internal frequency cutoff n to f Hz.
481 * This function is not for general use.
483 void setFrequencyCutoff(int n, float f);
486 * Retrieve the value of the internal input block increment value.
488 * This function is provided for diagnostic purposes only.
490 size_t getInputIncrement() const;
493 * In offline mode, retrieve the sequence of internal block
494 * increments for output, for the entire audio data, provided the
495 * stretch profile has been calculated. In realtime mode,
496 * retrieve any output increments that have accumulated since the
497 * last call to getOutputIncrements, to a limit of 16.
499 * This function is provided for diagnostic purposes only.
501 std::vector<int> getOutputIncrements() const;
504 * In offline mode, retrieve the sequence of internal phase reset
505 * detection function values, for the entire audio data, provided
506 * the stretch profile has been calculated. In realtime mode,
507 * retrieve any phase reset points that have accumulated since the
508 * last call to getPhaseResetCurve, to a limit of 16.
510 * This function is provided for diagnostic purposes only.
512 std::vector<float> getPhaseResetCurve() const;
515 * In offline mode, retrieve the sequence of internal frames for
516 * which exact timing has been sought, for the entire audio data,
517 * provided the stretch profile has been calculated. In realtime
518 * mode, return an empty sequence.
520 * This function is provided for diagnostic purposes only.
522 std::vector<int> getExactTimePoints() const;
525 * Return the number of channels this stretcher was constructed
528 size_t getChannelCount() const;
531 * Force the stretcher to calculate a stretch profile. Normally
532 * this happens automatically for the first process() call in
535 * This function is provided for diagnostic purposes only.
537 void calculateStretch();
540 * Set the level of debug output. The value may be from 0 (errors
541 * only) to 3 (very verbose, with audible ticks in the output at
542 * phase reset points). The default is whatever has been set
543 * using setDefaultDebugLevel, or 0 if that function has not been
546 void setDebugLevel(int level);
549 * Set the default level of debug output for subsequently
550 * constructed stretchers.
554 static void setDefaultDebugLevel(int level);