Transfer learning from Mandarin-pretrained large language models (LLMs) to Cantonese tasks such as sarcasm detection remains challenging due to lexical mismatches and token-level representation gaps across dialects. Existing pretrained LLMs, mostly trained on Mandarin, often fail to properly encode dialect-specific expressions, causing unreliable meanings in subtle and highly context-dependent tasks like sarcasm detection. To address this, we propose PGATA (Phonology-and-Glyph-Aware Token Alignment for Transfer Learning), a new token alignment framework that explicitly bridges dialect gaps by combining pretrained semantic embeddings with phonological Jyutping and glyph-based radical embeddings. These complementary features are fused using a multilayer perceptron with residual connections, further enhancing dialect-specific semantic signals. The aligned representations are then passed into a frozen LLM backbone equipped with LoRA adapters for efficient and effective fine-tuning under limited supervision. Extensive experiments on Cantonese sarcasm detection show that PGATA significantly outperforms all baselines, confirming its strength across diverse real-world Cantonese scenarios.