Linear, time-invariant systems
 

• Let us consider a dynamical system with input  and output :




• Such a system is said to be a linear, time-invariant system if it obeys the laws of superposition and scaling over time. That is, if you observe an output signal  in response to an input signal , and you later observe an output  in response to input , then the response to the combination  is just .

 
• One way to characterize a linear, time-invariant system is by measuring its impulse response function. This is the response you would obtain to a small pulse of unit amplitude, , where




Let's call the output signal we measure in response to such an input .
 

• Once we have measured the impulse response function, , we can compute the output  in response to any complex input . The first step is to consider the signal  to be composed of a superposition of unit impulses of different amplitudes:




Now since the system is also time-invariant (i.e., it does not change its behavior over time), then we know that the response to a shifted impulse  is just . And so the response to a weighted sum of such shifted impulses is just a weighted sum of the resulting shifted impulse response functions. Or in the language of mathematics:


 

• Thus, the output  in response to the input signal  may be written as


(1)

And in the limit that the spacing between time samples becomes infinitesimally small, this relation becomes exact and the sum turns into an integral:

(2)
 
 

• The expression above is known as the convolution sum (1) or convolution integral (2). It tells us how to predict the output of a linear, time-invariant system in response to any arbitrary input signal.

 
• The mathematical shorthand notation for the convolution operation is to use the  symbol as follows:



 

• One way of interpreting the convolution sum is just as we developed it above - i.e., it is simply a linear superposition of impulse response functions  each of which is multiplied by .

 
• The other (more common way) of interpreting the convolution sum is that it tells us that the output is computed by taking a weighted sum of the present and past input values. We can see this by writing out the sum in (1) above:




where we have assumed here for now that the times  are spaced by one unit of time. Note also that we do not sum over values of  for which . The reason is that for any physical system,  is defined only for . This indeed makes sense, because otherwise we would need to know future values of the input in order to compute the present output.
 

• Note that computing the convolution sum may be computationally intensive if the impulse response function extends over a long period of time. Typically, though, the impulse response function decays away with time, and there is a point at which we can consider it to be essentially zero and so we can truncate the expansion above at a certain number of "taps" (discrete samples).

 
• The convolution operation may also be thought of as a filtering operation on the signal , where the impulse response function  is acting as the filter. The shape of  determines which properties of the original signal  are "filtered out." The design of filters is usually best thought of in the frequency domain, which we turn to next....