Few-Shot Class-Incremental Learning (FSCIL) suffers two main problems, namely 1. the model optimization difficulty on few-shot samples of new classes, and 2. the catastrophic forgetting of formerly learned old classes caused by a limitation to reuse the samples of former training. To solve these problems in image classification task, we proposed improving the input feature of the classification layer by integrating a visual-semantic network for projecting an image feature onto a sentence-embeddings feature. The network applied a pre-trained language model and image-text descriptions to distill the multi-modal prior knowledge. The projected feature is called distilled-Word-Embeddings (dWE). We conducted experiments on a benchmark open dataset CUB-200-2011 to compare the effects of the three features: 1) image feature as a baseline, 2) sentence-embeddings feature taken from image descriptions, and 3) our proposed feature dWE. We found that utilizing multiple features outperformed a single feature of the same type. Compared with the baseline, using a combination of an image feature and dWE improved the average accuracy of all sessions from 48.73% to 49.94%. An average rate-of-change (ROC) of the classification accuracy per session was employed to evaluate the catastrophic forgetting. The ROC improved from baseline −6.15% to −3.16% with dWE and to −2.99% with the combination of an image feature and dWE. These can be considered as more than 40% of improvement from the baseline. Moreover, using the combination of an image feature and dWE gave higher ROC than most of the previous FSCIL techniques.