Homomorphic encryption (HE) is a powerful cryptographic tool that allows direct computation on encrypted data. A fundamental operation in the HE domain is secure matrix multiplication, which serves as a building block for numerous computation tasks such as machine learning. Current studies utilize HE schemes including BGV/BFV and CKKS to perform this operation. However, BGV/BFV is only applicable to integer homomorphic evaluation, and CKKS computes approximated fixed-point numbers. The support for secure outsourced floating point number (FPN) matrix multiplication remains lacking. FPNs are essential in cloud computing for accurately supporting scientific computation and data analysis, where high precision is critical. In this paper, we propose a secure outsourced FPN matrix multiplication scheme. We adopt secure FPN protocols to ensure computation over precise FPN. In addition, we extend the duplicate method to compute matrix multiplication in the encrypted domain, supporting matrices with arbitrary size. The experimental results demonstrate that our proposed scheme can effectively address the secure outsourced matrix multiplication task of FPNs, meanwhile achieving high accuracy with a reduction of units in last place (ULP) error by 3 to 4 orders of magnitude compared with prior work.