Graphics Processing Units (GPUs) are now widely used in computing systems, not only for graphics processing but also for general-purpose computing. Programming GPUs is challenging, as it is primarily done using low-level languages such as CUDA and OpenCL. Many approaches to simplifying GPU programming rely on algorithmic skeletons, i.e., higher-order functions that encapsulate common patterns of parallel computing. In these frameworks, programmers are provided with a set of skeletons, which must be combined to solve problems using the GPU. However, new skeletons can only be implemented if they can be expressed as a combination of the available skeletons. Otherwise, extending skeleton libraries may require good knowledge of the underlying compiler/runtime system that supports them. This paper presents PolyHok, a low-level imperative domain-specific language (DSL) for GPU computing embedded in the Elixir functional language. PolyHok enables the implementation of polymorphic higher-order GPU kernels, i.e., GPU kernels that can accept device functions, including anonymous functions, as arguments and that are dynamically typed, based on the arguments they receive, and JIT compiled at runtime. With such kernels, programmers can implement high-level abstractions typically associated with higher-order functions, such as algorithmic skeletons and array comprehensions. This paper details the design and current implementation of PolyHok and compares its performance with pure CUDA through experiments with six benchmarks.