CUI Creative Collaborator

Overview

With the rise of artificially intelligent conversational user interfaces (CUI's) in our homes and pockets come new opportunities for human/computer interaction. Our relationship with these virtual assistants, the Siri's and Alexa's of the world, are essentially one-sided however; we give instructions or ask questions and expect answers that satisfy us. What would happen if Alexa talked back? For this project, I was interested in exploring the potential of a virtual assistant that exhibited signs of its own agency. In which contexts would a CUI with a sense of agency be useful and not just frustrating? Creative collaboration with a CUI, something like writing a song together, would prove challenging for the human user but implies a new set of creative possibilities.

Exploration

In my first attempt, I created a system that would ostensibly deny access to functions of the technology if it was dissatisfied with the commands and/or attitude of the human user (fig. 1). The feedback I received from the first round of critique was that the CUI behaved more akin to an assistant and did not effectively demonstrate a sense of its own agency. Questions that arose from the discussion included:

• How does the AI/CUI’s knowledge or ability (contextual affordances) affect the experience of interacting with it?
• What are the different ways an embodied personality (the appearance of AI agency) of the CUI can impact the experience?
• How much control does the AI/CUI have over a given task or interaction?

Development (Round 1)

From here, I decided to pursue the idea of a creative human/computer collaboration. Human to human collaboration implies that each collaborator exhibit some degree of agency for the relationship to be mutual. Similarly, a human/computer collaboration would would need to demonstrate the computer’s agency. In order to demonstrate this relationship I decided on songwriting/composing as a creative task to be completed through the human/machine collaboration. This brought up a number of other questions:

• What’s the AI’s agenda?
• What does a computer value, compared to a human?
• Do those goals need to be the same?

Conceptual questions aside, it was necessary to map out the CUI's interaction states. Following a schematic created for Google's Assistant AI/CUI, I diagrammed my own CUI's different interaction modes (fig. 2). Here, I envisioned the CUI moving seamlessly from conversation to music playing. Being an avid user of the Voice Memos app on the iPhone, a technology I regularly use to record and develop song sketches for my own music, I considered the ways a CUI might adapt to that environment (fig. 3).

Development (Round 2)

The critique from the second iteration was that the CUI still felt too much like an assistant, particularly in its language patterns, congeniality and accommodation of the user. There was still work to be done in order to promote the idea that the CUI was able to carry out its own agenda. Another concept that came up during this time was the idea of authorship or ownership of the collaborative work. How is the work distributed? Who is credited? The overall feedback was to explore the constraints and roles within the human/computer relationship. What would it be like for the human to take the backseat during the interaction? In order to clarify the themes to address in the interface, I went back to the drawing board and created a concept map of creative collaboration (fig. 4), highlighting those characteristics that were important to address in the design.

Refinement

It was evident from the previous feedback that there needed to be a way to differentiate between a CUI with assistant-like behavior versus one with agency. I outlined two different interaciton flows (fig. 5), one that would function more like tradition human/computer interaction where the CUI acts as an assistant and the other where the computer took on a more active role in the relationship.

This led me to prototype the two collaboration paths. The final prototypes include a dashboard where the user can look at their quantified collaboration status with the computer. Ostensibly, if the user wanted to properly pair with the computer in the creation of a work, they would need to satisfy a defined level of collaborative balance between contributors. The program would block publication of the work if the proper collaboration level was not met.