When designing novel technology, many of the important design decisions tend to be taken early in the process, which is also when it is usually easier to modify the design. Thus, it is important to learn as much as possible, as early as possible in any design process. This work is carried out within the EU project ABILITY, which aims at developing novel refreshable tactile displays. Previously, we conducted a literature overview and interviews which showed that it is possible to digitally display visual graphics non-visually in a way that can be at least as good as existing tactile materials. The interviews also showed that it was difficult for users to envision interacting with such novel system. Thus, we worked with different types of prototype materials to create a deeper foundation for our future designs with our target users. To achieve this, we adopted the well-established philosophy of lo-fi prototyping, while focusing on functionality and concepts, not the detailed finished designs. In the presented studies, we investigated the AI system envisioned to support the device as well as the ABILITY displays, focusing on the novel pin display, with the aim of finding out what is important and how the interaction should work.