<p>Optimal error estimates on an improved weak Galerkin spectral element method for a model problem of second order elliptic are established. Approximation spaces of weak gradients on physical elements are established from high-order orthogonal polynomials defined on reference elements by the Piola transform, and the Galerkin spectral element approximation space consisting of weak functions for the unknowns is defined through one-to-one mappings. A modified penalty term is then supplemented to the weak Galerkin approximation scheme to guarantee its wellposedness and to achieve the optimal <i>hp</i> error estimates with respect to the mesh size as well as to the polynomial degree. Additionally, a close relationship between spectral gradients and classic gradients are further investigated with the aid of orthogonal projection operators. In the sequel, the optimal <i>hp</i> error estimates performed on triangular as well as parallelogram meshes are obtained, which is optimal both in the mesh size and in the polynomial degree. Numerical experiments are performed to show the effectiveness of the modified weak Galerkin spectral element method.</p>