As important as being able to know the words in a text, being able to identify melodic patterns in musical scores is of utmost importance in many applications, such as indexing for information search and retrieval. We explore here methods to set up appropriate musical units that extend beyond individual notes into higher-level units of communication (i.e., melodic patterns, dubbed “musical words”). This prospect is studied following two main ideas for musical encoding, understood as the process of transforming musical content into musical text based on a musical word vocabulary: 1) based on the association of notes of musical scores and corresponding lyrics text (available in some datasets annotated with such explicit alignments); 2) based on the unsupervised grouping of individual notes into note sequences using a technique known as Byte Pair Encoding (BPE). To assess the appropriateness of these musical encodings, we resort to Zipf’s curves of the encoded musical texts and measure how close such curves are to a Zipfian law. Results suggest that the musical words derived from the alignment with lyrics words produce the most natural melodic patterns and that better results are achieved if the encoding is based on note pitch intervals rather than notes encoded as absolute pitch symbols. Additionally, BPE appears to be a promising unsupervised way to encode musical content into sequences of fully automatically discovered musical words.