This paper deals with the following question: Suppose that there exist an integer or a non-negative integer solution \(\varvec{x}\) to a system \(\varvec{A}\varvec{x}= \varvec{b}\) , where the number of non-zero components of \(\varvec{x}\) is n. The target is, for a given natural number \(k < n\) , to approximate \(\varvec{b}\) with \(\varvec{A}\varvec{y}\) where \(\varvec{y}\) is an integer or non-negative integer solution with at most k non-zero components. We establish upper bounds for this question in general. In specific cases, these bounds are tight. If we view the approximation quality as a function of the parameter k, then the paper explains why the quality of the approximation increases exponentially as k goes to n. This paper is a complete version of an extended abstract that appeared at the 26th International Conference on Integer Programming and Combinatorial Optimization (IPCO) [28].