Creating a musical video that contains a sequence of several clips can be a hard task. Selecting, ordering, and simultaneously considering the tension, feeling, or emotions happening in a musical piece can be difficult. This work relates the tonal tension of a musical piece with the visual and color characteristics of a set of clips. Moreover, the motion of components, the saturation level, and the colorfulness are considered in each clip. The idea is to tackle this problem as a combinatorial optimization problem, where the distance between tonal tension and visual features can be optimized considering different criteria. Here, a local search method is proposed and evaluated to optimize visual features considering real clips recorded with a cellphone and well-known musical pieces. Also, the K-predominant color distance between neighbor clips is minimized to avoid abrupt changes in the selected clip sequence. Results show that the proposal can successfully select and order a subset of clips relating the tonal tension with the mentioned features. Also, scenarios with different importance of the visual clip features are presented and analyzed to evaluate the flexibility of the proposed approach.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Automated Selection and Ordering of Clip Sequences for Music Videos Based on Tonal Tension and Visual Features

  • Nicolás Rojas-Morales

摘要

Creating a musical video that contains a sequence of several clips can be a hard task. Selecting, ordering, and simultaneously considering the tension, feeling, or emotions happening in a musical piece can be difficult. This work relates the tonal tension of a musical piece with the visual and color characteristics of a set of clips. Moreover, the motion of components, the saturation level, and the colorfulness are considered in each clip. The idea is to tackle this problem as a combinatorial optimization problem, where the distance between tonal tension and visual features can be optimized considering different criteria. Here, a local search method is proposed and evaluated to optimize visual features considering real clips recorded with a cellphone and well-known musical pieces. Also, the K-predominant color distance between neighbor clips is minimized to avoid abrupt changes in the selected clip sequence. Results show that the proposal can successfully select and order a subset of clips relating the tonal tension with the mentioned features. Also, scenarios with different importance of the visual clip features are presented and analyzed to evaluate the flexibility of the proposed approach.