Olshausen BA, Field DJ (1997). Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Research, 37: 3311-3325.

Previously, we have shown that the receptive field properties of simple-cells in area V1 may be accounted for in terms of a strategy for producing a sparse distribution of output activity in response to natural images (Nature, 381:607-609). Here, in addition to describing this work in a more expansive fashion, we examine the neurobiological implications of sparse coding. Of particular interest is the case when the code is overcomplete---i.e., when the number of code elements is greater than the effective dimensionality of the input space. Because the basis functions overlap (i.e., are non-orthogonal and not linearly independent of each other) sparsifying the code will recruit only those basis functions necessary for representing a given input, and so the input-output function will deviate from being purely linear. Interestingly, these deviations from linearity provide a potential explanation for the weak forms of non-linearity observed in the response properties of cortical simple cells. Further predictions of the model and proposed experimental tests are discussed.