Commuted Piano Synthesis

While the piano-hammer is a nonlinear element, it is nevertheless possible to synthesize a piano using commuted synthesis with high fidelity and low computational cost [471,524]. The key observation is to note that the interaction between the hammer and string is essentially discrete at only one or a few time instants per hammer strike. That is, the hammer-string interaction can be modeled as one or a few discrete impulses which are filtered in a velocity-dependent way.

When the hammer first strikes the string at rest, it ``sees'' the wave impedance of an infinitely long string. As a result, the collision pulse can be well modeled as a filtered impulse.^6.2 This disturbance must propagate to the agraffe and back before any further interaction can take place. Since what comes back is a slightly filtered version of the original interaction pulse, it is also representable as a filtered impulse (just filtered a little more). This secondary pulse passes over the hammer. If the hammer has fallen away from contact with the string by then, there is no further interaction, and the original force pulse is the complete excitation. In cases in which the hammer is still in contact with the string, a new interaction pulse is issued which travels outward from the striking point in a manner similar to the original force pulse. In real pianos, there are typically only a few interaction pulses before the hammer falls away, except at very high pitches.

The nature of the hammer-string interaction is predictable to first order. To a large extent, for example, the number of interaction impulses is determined by which string is being struck. Thus, given key number and hammer velocity, together with current string velocity, one can predict rather well the amplitude and timing of all interaction impulses. There is a slight amount of unpredictability which we neglect having to do with the fact that when the hammer strikes an already vibrating string, the entire history of string vibration influences the exact details of the hammer-string interaction; however, this is a second-order effect which may not even be desirable in the simulation. A pianist has observed that sometimes a note is unexpectedly bright, corresponding perhaps to striking the string while it is rapidly approaching the hammer, and as a result, he would have to play a little brighter thereafter to keep the dynamic level smooth. Perhaps we can actually improve the piano by removing unpredictability in the hammer-string interaction. If not, perhaps unpredictability can be restored to an adequate degree by introducing random perturbations of the interaction impulse levels as a function of amplitude of vibration prior to the hammer strike. More accurately, the unforced string velocity $v_s$ at the time of each interaction can be used to obtain the correctly modified interaction impulse amplitude. A rigorous model can compute the hammer-string interaction force from a real-time model of hammer-string contact [527].

The height of each interaction impulse determines the setting of a filter which converts the interaction impulse into the appropriate force pulse which is physically accurate for the given conditions. For the initial strike, the force pulse may be computed using a model of the felt-covered piano hammer [78] striking a string with collision velocity $v_c = v_h-v_s$, where $v_h$ denotes hammer velocity and $v_s$ denotes the string velocity. Since we will neglect prior string vibrations in this treatment, we have $v_s=0$ and $v_c=v_h$. Each velocity differential gives rise to a force pulse over time which fully characterizes the hammer strike. If the hammer were a point mass and the string were the ideal lossless string, the force pulse would have a decaying exponential shape, and its amplitude would be a linear function of collision velocity. The nonlinear spring (felt) surrounding the hammer mass gives rise to a force pulse which rises and decays, looking more like the convolution of two exponentials. The harder the strike, i.e., the larger the collision velocity, the more narrow and more wideband is the force pulse. This is the main effect of the felt nonlinearity, and it cannot be neglected in medium to high quality piano synthesis.

The second interaction pulse, if any, is a little more complicated. In this case, the hammer is assumed to still be in contact with the string when the first echo of the original hammer strike returns, and this force pulse passes over the hammer. Thus, we have a ``collision'' between a stationary hammer and a traveling force pulse on the string. If the felt spring were linear, we could simply compute the reflected and transmitted force pulses from the impedance of the hammer (which would be simply a linear mass-spring system) using the loaded waveguide junction formulas [437]. However, since the spring is nonlinear, impedance descriptions do not apply, and we must numerically compute the reflected force pulse using a nonlinear model.

The assumption of a zero initial string velocity gives us the important result that all interaction impulses are predetermined by the initial collision velocity, even when they overlap. (Again, random perturbations can be used to simulate interaction with prior string vibrations.) Examples of the appearance of the synthesized overlapping force pulses are shown in Fig. 5.1. Such waveforms can be well approximated by an impulse located at the peak of each force pulse convolved with a lowpass filter which provides the appropriate smoothing (which decreases with increasing dynamic level).

Figure 5.1: Example of overlapping hammer-string interaction force pulses. The vertical lines indicate the locations and amplitudes of three single-sample impulses which passed through one to three digital filters to produce the overlapping pulses shown.

The creation of a force pulse from a single impulse for a specific dynamic level is shown in Fig. 5.2. Without loss of generality, we consider force units; dividing by the string wave impedance gives the corresponding velocity injection for the string loop. Other physical variables may be chosen, as discussed in [442].

Figure 5.2: Illustration of the creation of a single hammer-string interaction force pulse as the impulse response of a digital filter. The input to the filter is a single nonzero sample (impulse), and the output is the desired hammer-string force pulse. When the amplitude of the input impulse increases, the output pulse increases in amplitude and decreases in width, which means the filter is nonlinear. However, on each specific impulse, the filter operates as a normal linear, time-invariant filter. In this way, the entire piano is ``linearized'' with respect to a given hammer velocity.

One method of creating multiple force pulses from multiple impulses at a specific dynamic level is shown in Fig. 5.3. The multiple interaction impulses become multiple overlapping impulse responses which feed the summer, and this goes on to the string.

Figure 5.3: Illustration of the creation of a multiple hammer-string interaction force pulse as the superposition of impulse-responses of a bank of digital filters. The input to each filter is a single impulse, and the sum of their outputs gives the desired superposition of hammer-string force pulses. When the input impulses increase in amplitude, the output pulses become taller and thinner, showing less overlap.

At a specific dynamic level, we have obtained the critical feature that the model is linear and time invariant. That means we may now commute the soundboard/enclosure filter with not only the string, but with the hammer lowpass filter as well. These operations are depicted in Fig. 5.4 which shows the natural schematic diagram of the complete piano synthesis system, and Fig. 5.5 which shows the results of commuting the hammer-string assembly with the soundboard and enclosure.

Figure 5.4: Piano synthesis as described using natural ordering of all elements.

Figure 5.5: Piano synthesis using commuted ordering. The soundboard and piano enclosure are commuted such that we only need a stored recording of the impulse response of their series combination. The large digital filter required to implement the soundboard and the piano enclosure is thus removed.

While the commuted result is valid only for a fixed hammer-string collision velocity $v_c$, that is all we need. For different collision velocities, we simply alter the filters applied to the soundboard/enclosure impulse response. This works because the nonlinearity is confined to the hammer-string collision, and these are discrete, non-overlapping events which can be modeled individually using linear, time-invariant elements, indexed by collision velocity. Note, however, that if a virtual piano key is restruck before the excitation table has finished playing out, that playback must either be prematurely terminated (the low-cost solution), or multiple, overlapping playbacks must be supported, as in commuted bowed-string synthesis [444]. The main approximation in the commuted technique is that detailed reflections generated on the string during hammer-string contact are neglected; that is, the hammer force pulse depends only on the hammer-string velocity at the time of their initial contact, and the string velocity is treated as remaining constant throughout the contact duration.

Figure 5.6: Commuted piano synthesis in the simple case of a single hammer-string interaction pulse. The excitation table contains essentially the impulse response of the soundboard and piano enclosure, perhaps filtered in a way that alleviates requirements on all later filters.

In the simplest case of a single interaction pulse, the final synthesis structure appears as shown in Fig. 5.6. This is essentially the same as that in the body-plucked string of §4.15 [444] with a new nonlinear filter between the excitation table and string. The filter changes with excitation amplitude which makes it nonlinear. Since the excitation table playback must be triggered somehow, we will use the convention that when $v_c$ changes value, the table playback is triggered. The numerical value of $v_c$ is used to set the lowpass filter coefficients, providing both an amplitude-dependent gain and an amplitude-dependent impulse-response width. It is good also (in fixed-point arithmetic) to choose a final output scaling based on $v_c$ so that the string is always fully excited.^6.3

A further simplified, single-filter structure supporting three hammer-string interaction pulses (during a single hammer strike) is illustrated in Fig. 5.7.

Figure 5.7: Commuted piano synthesis supporting three hammer-string pulses using a shared filter. The Excitation Table playback is triggered when $v_c$ changes, possibly preempting an existing playback.

In this case, we require three separate read-pointers, traversing the excitation table, which implement the equivalent of the three-tap delay line indicated in Fig. 5.4. (Alternatively, a tapped delay line can of course be used.) The shared lowpass filter is set according to the amplitude, $v_c$, of the initial collision velocity. Sharing the lowpass filter in this way corresponds to saying that the interaction impulses are all about the same height. While this is not true in the physical situation, it is an approximation which simplifies the implementation by eliminating all hammer filters but one. As in the previous case, triggering of the first read-pointer occurs when $v_c$ changes, the second read-pointer starts at a fixed delay after the first, (corresponding to the time it takes a wave to travel from the hammer to the agraffe and back), and the third read-pointer starts at the same delay relative to the second. (Thus, the impulses are uniformly spaced in time.) The time delay needed between read-pointer launches can be implemented, for example, using a simple counter-to-zero in software or hardware.

Figure 5.8: Commuted piano synthesis supporting three hammer-string interaction pulses with separate force-pulse filters for each pulse.

A separate-filter version for three pulses is shown in Fig. 5.8. In this case, we require three separate read-pointers reading the excitation table, (or the excitation can be fed through a tapped delay line with three taps), and three separate filters which are set according to the amplitude of the initial collision velocity. The outputs of these filters are summed and fed into the string. Use of this structure allows each interaction impulse to be shaped by its own filter, giving an arbitrarily high degree of accuracy in approximating the overlapping force pulses.

To save computation in the filtering, we can make use of the observation that, under the assumption of a string initially at rest, each interaction pulse is smoother than the one before it. That suggests sharing each filter application with all later force pulse computations. This arrangement is shown in Fig. 5.9. Thus, each filter need only supply the mild smoothing (and perhaps dispersion) associated with traveling from the hammer to the agraffe and back, plus the mild attenuation associated with reflection off of the felt-covered hammer (a nonlinear mass-spring system). The first filter LPF1 can be used to shape a velocity-independent excitation signal to obtain the appropriate ``shock spectrum'' for that hammer velocity. Alternatively, the excitation table itself can be varied with velocity to produce the needed signal. In this case, filter LPF1 can be eliminated entirely by applying it in advance to the excitation signal. It is possible to interpolate between tables for two different striking velocities; in this case, the tables should be pre-processed to eliminate phase cancellations during cross-fade.

Figure 5.9: Commuted piano synthesis supporting three hammer-string interaction pulses using separate filters for each pulse and implementing the filters successively. Each new delay is equal to the travel from the hammer, to the agraffe, and back to the hammer.

Assuming the first filter in Fig. 5.9 is ``weakest'' at the highest hammer velocity (MIDI velocity $127$), that filtering can be applied to the excitation table in advance, and the first filter then becomes no computation for MIDI velocity $127$, and as velocity is lowered, the filter only needs to make up the difference between what was done in advance to the table and what is desired at that velocity.

Since, for most keys, only a few interaction impulses are observed, per hammer strike, in real pianos, this computational model of the piano achieves a high degree of realism for a price comparable to the cost of the strings only. The soundboard and enclosure filtering have been eliminated and replaced by a look-up table using a few read-pointers per note, and the hammer costs only one or a few low-order filters which in principle convert the interaction impulse into an accurate force pulse.