Koala

A Conversational Bus Service Assistant

project info

team

collaborating partner

15 weeks

Service design course

Umeå Institute of Design

Shigeo Katsura-Gordon

Christoph Zobl

Eduardo Pereira

Microsoft

 

design question

How can we design a Conversational User Interface that achieves human-computer symbiosis through interpretation of language and intent?

PROPOSAL

Koala &

The 6 Principles of

Conversational User Interfaces.

The 6 Principles of Conversational User Interfaces establish a base for a CUI to be considered a legitimate actor in a conversation with a human, it must show competence and abide to social norms and conventions.

 

Koala is the embodiment of these principles, in the context of public transportation.

1. Presence

Create a sense of "personal" space for the CUI and acknowledge human physical presence as a conversation initiator.

2. Not a human

Users should be aware they are not speaking to a human because it will affect their expectations and how they treat the system.

3. Turn Taking

Define the speaking order between the human and the conversational user interface.

4. System Limits

Use language to guide the flow of the conversation and limit it towards the system's reach of action.

5. Unknown Problems

Inquiries outside of the system's scope will arise, the CUI should be equipped to deal with contextually relevant problems and establish when something is out of it's reach. Additionally, giving an alternative solution is ideal but saying "no" is acceptable.

6. Beneficial Use

As soon as users' requests increase in complexity a CUI becomes more convenient.

 

Consider buying a single bus ticket, it might result easier in a GUI. Now think of buying 5, and of different types (adult, student)... that would just be too many clicks and/or cycles through the interface, as opposed to just saying "I'd like 2 adult tickets and 3 youth" and getting the system to do the processing for you.

 

We made our first interview at Umeå Airport's coffee shop. We discovered an interesting relationship between the kiosk's employees and the bus drivers; not only do they talk to the point where they know each other by name but they sometimes even exchange free coffee for a free bus ride or viceversa. 
Very early on our exploration we started finding signs of a reciprocal symbiosis in human relationships.

Samuel can start a conversation with anyone and make them feel welcome in the city but he has very clear the limits he must have with his customers. We learned from him that there are different "spaces" that must be respected or opened depending on the (usually unspoken) terms of a relationship.

Inger is another bus driver in Umeå and Bengt works at the workshop where they give maintenance to the buses. They had known each other for a long time and were an example of a friendship in which they knew they would help each other if necessary.

We met many other people, found out many important things that define how humans interpret each other and how they build relationships; how their expectations change and how this redefines their behaviours and interpretations.

We mapped out connections and found patterns which we used for a role play exercise.

The situation we had seen at the Airport coffee kiosk in which coffee was exchanged for a free bus ride was played out, we asked the audience to interpret why this had happened.

We were explored how the behaviours that communicate acknowledgment of another person's presence are so crucial for the flow of a conversation. Our research had shown how paying attention is a key component of respect and how this is a base of any relationship.

We played a version where the bus driver went the extra mile in being helpful with the customer versus a bus driver that didn't really go further than his basic obligations and discussed how this makes a newcomer feel.

We started by exploring our environment through ethnography, looking to observe and understand different types of relationships and what role conversations play in them.

We chose the public transportation system because of how many people, transactions and machines live in it.

Here we realize the usual, single person request for a simple ticket is generally too simple to need to be tackled with a CUI so, Koala, our last iteration, responds to situations in which there are more variables to handle; that a person in a window could easily do, but a GUI in a ticket machine would require too many steps for.

We created a workshop in which we tried to elicit discussion and conversation through role playing. We also used this opportunity to present our research to our collaborators.

Then we went out and implemented 3 different cultural probes, with the objective of being disruptive to prove right or wrong our previous paradigms.

We learned about social norms, the different forms personal space can take and how sometimes body language is opposite to what a person is saying.

Once we had done enough research, it was time to do some more; but this time, with a prototype.

Iteration# 1 was designed as a first approach, to explore how it felt for people to make a transaction through a non-human object.

This prototype had a human voice behind it, the possibility to contact a human and a more open script.

We found with it that people treat the machine differently once they know there is another person involved.

Therefore, Iteration#2 got a "robotic" voice with a very strict script, no screen or visible connection to a human.

It also got "talking" lights and "listening lights", to define speaking order.

We didn't count with the possibility that the script would be "unfollowed" by the user with an unexpected inquiry.

For our Wizard of Oz prototype we wrote the CUI's script on Notepad and used the Text to Speech function to get the "robotic" voice.

We manually chose what the CUI would say, but only from the options that were already written as part of the Conversation Map.

SIDENOTE

So, Iteration#3 got a modification in the conversation map in which this was taken into account, in order to have an answer in case the inquiry had nothing to do with the contextual capabilities of the machine.

 Also, upgraded light feedback to define the turn taking in the conversation.

The process

The journey of designing Koala.

Ethnographic Research

Role Play Workshop

Prototyping and Testing

Cultural Probes

final thoughts

Reflexions and My Role

What's next? We talked about multimodality of the interface, Koala could be more efficient if it became a hybrid of a Graphical UI and a Conversational UI. Of course, for the purposes of this project we focused on exploring conversation both in content and form but we also concluded that integrating other ways to display information would be ideal in specific cases, like giving directions, for example.

 

Another step forward would be to explore interruptions in turn taking. More often than not, our human conversations are very quick and dynamic, we interrupt each other and constantly modify who's turn it is to speak, we detect changes in the course of a conversation and adapt immediately because we can listen and process information at the same time as we speak, therefore, it would be natural to be able to do the same with a CUI.

 

Conversational conventions, social norms and paradigms and relationships and all human affairs are topics that could be discussed for weeks if not months, they are also very interesting to observe and investigate out in the field. For sure, as a talkative person I do enjoy these philosophical journeys, but during this project I focused on taking the theory and transform it into something concrete. So I took initiative with early but simple prototyping that would send us in a direction where we could explore by interacting with other people and from there it was very natural to follow with iterations. I also had a part in integrating the results from the testing sessions to the next iterations and the final concept.

 

We shouldn’t try to replicate a human or a human conversation, what we should strive for is fitting within the constraints humans consider acceptable within one.

 

During this project we gave special importance to research, prototyping and testing. Our final outcome is not a flawless conversation map or a functional language processing machine, but a series of design principles and learnings that we concluded would be key when implementing a Conversational User Interface (CUI) in order for it to be useful but not overwhelming and smart but not omnipotent

© All rights reserved. Melissa Hellmund.

2018.

Contact:

1. Presence 2. Not a human