Continuous blood pressure estimation is a critical issue in the medical field, and photoplethysmography (PPG) signals can be utilized to achieve this goal. However, most existing studies do not adequately address the issue of imbalanced label distribution and heavily rely on large amounts of real-world data. Therefore, this paper proposes a Dual-Loss-GCN (Dual Loss-Graph Convolutional Neural Network) framework. The GCNs layer enables the extraction of features from converted graph-structured PPG signals. The loss function combines with a conventional error term weighted by kernel density estimation, which addresses the imbalanced distribution problem, and a physics-informed error term to mitigate dependency on large datasets. Experiments are conducted on the publicly available MIMIC-III dataset. The Mean Absolute Errors for Systolic Blood Pressure and Diastolic Blood Pressure are 1.99 ± 2.23 mmHg and 2.66 ± 2.52 mmHg, respectively, which outperformed other comparison methods, showcasing its potential for reliable blood pressure monitoring.