Implementing secure two-party computation (2PC) protocols with floating-point numbers, balancing accuracy and efficiency, has been challenging. Despite advancements with SecFloat (in IEEE S&P’22) and Beacon (in USENIX Security’23), there is room for improvement in precision, efficiency and functionality. This paper introduces SecFloatPlus, which complements SecFloat with a new spline generation approach for implementing efficient and accurate 2PC-enabled mathematical functions. It significantly enhances accuracy, achieving up to \(1000 \times \) better performance for the hyperbolic tangent function (i.e., tanh) and \(6 \times \) better performance for the Gaussian Error Linear Unit (GELU) function, compared to Beacon. It also introduces precise inverse trigonometric functions (i.e., arcsin, arccos and arctan) which are never implemented by previous works. Besides, when integrated into secure transformer inference, SecFloatPlus outperforms SecFloat in runtime efficiency and communication cost for the SoftMax activation function, while maintaining precision.