Acoustic scene classification (ASC) under low-complexity constraints poses significant challenges for accurate and efficient audio representation. Existing lightweight models often employ unified convolutional operations across the time-frequency dimension, ignoring their distinct semantic roles. To address this limitation, we propose Time-Frequency Attention Net (TF-AttNet), which explicitly decouples temporal and frequency modeling, enabling temporal and frequency dimension feature extraction from audio spectrograms. TF-AttNet consists of two specialized modules. The FreqConvBlock (FCB) uses learnable frequency positional encoding and frequency attention. This helps the model focus on semantically important frequency bands and enhances spectral discrimination. The TimeConvBlock (TCB) employs depthwise convolutions and temporal attention. It captures both short-term variations and long-range dependencies in the time dimension. This enables richer temporal representations while keeping the computational cost low. Extensive experiments on the TAU Urban ASC 2022 Mobile dataset show that TF-AttNet outperforms existing lightweight models in classification accuracy, while significantly reducing parameter count and computational cost. These results demonstrate the effectiveness and practicality of TF-AttNet for deployment in real-world resource-constrained scenarios.