We propose a novel method for image gradient upsampling that leverages adaptive convolution with learnable kernels to add weighted permutations of the input image pixels back to the image in a constrained manner in order to enhance the image gradient resolution. The proposed approach dynamically adjusts pixel values based on local patterns of pixels in the input image that define the gradients in the image. Traditional upsampling techniques, such as unpooling and nearest-neighbor interpolation rely on predefined rules and often result in blurred or jagged edges due to their inability to adapt to image content. Moreover, these and other image upsampling techniques like those based on deep learning, focus on enhancing the spatial information in the input with the goal of enhancing the human visual perceptual experience. However, the goal of our approach is to enhance the gradient information in the image in order to help gradient based learners learn better latent space features for inference tasks. Our approach uses trainable convolutional kernels that learn to extract and amplify intricate gradient based features during training by subdividing the input images into overlapping patches and applying element-wise transformations. We show the efficacy of the proposed method by enhancing the gradient information in images of the Fashion MNIST dataset.