In recent research, point-based architectures demonstrate competitive performance in 3D understanding. However, the unstructured nature of point clouds limits the application of effective operators such as convolutions in feature extraction. Although many works have attempted to address the issues of unstructured data and introduce convolutions or transformers, the complex spatial mappings of point clouds and cumbersome convolution implementations limit real-time performance. In addition, voxel-based transformers and group-based convolutions are unable to extract finer-grained features. To address these problems, we propose Dynamic Sequence Convolution (DSConv), a fine-grained convolution based on local serialization. We propose Axis-Group serialization to structure point clouds and preserve direction information without encoding, improving processing efficiency while retaining richer local geometric details. Moreover, we introduce Adaptive Convolution (AdaConv) to achieve translation invariance in sequence convolution, which avoids the complex mapping in point cloud convolution and enables local feature extraction at a finer granularity (each convolution depends on only a few points rather than a group of points). Furthermore, we propose a dynamic position refinement method based on parameter decoupling of AdaConv, continuously enriching the feature representation of point cloud sequences. By combining DSConv with new architectural designs, we outperform the current state-of-the-art methods on ScanObjectNN and S3DIS datasets, while maintaining high scalability and achieving efficient inference speed. The source code will be released.