Making Sense of Robots in Public Spaces:
A Study of Trash Barrel Robots

Fanjun Bu [email protected] 0000-0002-9953-7347 Cornell University, Cornell Tech2 W Loop RdNew YorkNew YorkUSA10044 , Kerstin Fischer 0000-0003-1987-5344 Southern Denmark UniversityDenmark [email protected] and Wendy Ju [email protected] 0000-0002-3119-611X Jacobs Technion-Cornell Institute, Cornell TechNew YorkUSA

Abstract.

In this work, we analyze video data and interviews from a public deployment of two trash barrel robots in a large public space to better understand the sensemaking activities people perform when they encounter robots in public spaces. Based on an analysis of 274 human-robot interactions and interviews with N=65 individuals or groups, we discovered that people were responding not only to the robots or their behavior, but also to the general idea of deploying robots as trashcans, and the larger social implications of that idea. They wanted to understand details about the deployment because having that knowledge would change how they interact with the robot. Based on our data and analysis, we have provided implications for design that may be topics for future human-robot design researchers who are exploring robots for public space deployment. Furthermore, our work offers a practical example of analyzing field data to make sense of robots in public spaces.

Sensemaking, public interaction, field experiment, urban spaces, behavioral elicitation, robots in public space, robots in the wild

Refer to caption — Figure 1. People in public space trying to make sense of the trash barrel robots.

1. Introduction

How should we design robots to be deployed in public spaces? From more than two decades of research in human-robot interaction, we have a good idea of how people make sense of robots in one-on-one lab encounters. For instance, we know that people tend to interact with robots as if they were social agents (e.g. (Clark and Fischer, 2023), among many others), and much research in robot (behavior) design addresses how such interactions can be facilitated.

Much less is known about robots in the wild (Jung and Hinds, 2018); people engage in the sensemaking process when novel technologies are introduced to their spaces (Forlizzi, 2007), but so far it is not clear how this sensemaking differs from lab contexts and what design implications emerge from that. Consequently, in this paper, we ask how we can design robots for public spaces to address the sensemaking processes that occur in the field, whereby sensemaking we mean observable traces of how people make sense of the robots and what they are doing in the public space, what actions people carry out to decide how to interact with a robot – if at all – as well as the cognitive and cultural resources that they evoke in understanding what the robots are up to.

Specifically, we investigate the deployment of two trash barrel robots in Astor Place, Manhattan, New York City. We focus on how the general public reacted to the introduction of service robots in public spaces, what resources they draw on, and how they make sense of the robots’ presence. The analysis of both post-interaction interviews and video footage reveals that in a public space like Astor Place, people draw on a very broad range of resources, making use of cultural knowledge, knowledge of human nature, communal common ground, and personal experience from interactions with the robots to make sense of the robot deployment, taking into account not only the robots themselves but also their operators, possible deployers, the city, vulnerable co-citizens and society at large when interpreting the novel situation, to mention just a few resources that contribute to the sensemaking process. These results have consequences for how robots should be designed for deployment in public spaces. Our analysis also contributes to the methodology of sensemaking analysis through in-field deployments, serving as an example for future studies and practices in understanding human-robot interactions in real-world contexts.

2. Related Work

When people run into robots, they need to turn the encountered circumstances into situations where they can act upon, a process called sensemaking (Weick et al., 2005). The concepts involved in sensemaking research are informed by research from a broad range of disciplines, such as HCI, systems engineering, knowledge management, organizational communication, and user studies (Turner et al., 2023). Here we discuss sensemaking as a possible method for providing designerly ways (Cross, 2006; Lupetti et al., 2021) to address the emergent design interactions people will have with robots in their public spaces.

2.1. Public deployments of robots

While many researchers in HRI have focused on conducting robot studies ”in the wild,” their arguments for these often focus on the public environments as being the ”ultimate test” for robots—to understand how they will be used (Sabanovic et al., 2006), what people consider to be normal or breaches of expectation (Weiss et al., 2008), or what features will lead people to engage with robots more (Moshkina et al., 2014), outside of the confines of the lab.

Beyond the ’ultimate test’-view, roboticist Pericle Salvini offers an ”urban robotics perspective,” which suggests that the focus of such studies should be on the ”emerging properties” that result from the in-situ interactions (Salvini, 2018). Instead of testing hypotheses, profiling user population behaviors, or identifying qualities of robots fit or unfit for the urban environment, as has been the case in HRI research to date, the urban robotics perspective centers on the discovery of unintended forms of interactions that robots might have with bystanders or passersby. In this view, the naive and untrained responses of people incidentally interacting with the technology are the most important to understand (Dix, 2002).

For example, Babel et al. (2022) investigated the use of cleaning robots and their interactions with passersby in German train stations, inspiring designs for predictive autonomous actions in public settings. Lee et al. deploy BubbleBot in public settings to interact with pedestrians, providing guidelines on the robot’s motion design in serendipitous interactions (Lee et al., 2019; Lee and Jung, 2020). Yang et al. (2015) deployed a robotic trash barrel at a campus cafe, exploring socially acceptable robotic behaviors and investigating how individuals interpret the robot’s movements. By studying autonomous delivery robots on the streets, Pelikan et al. (2024) showed how the robots become a part of everyday street life both physically and socially.

In While et al. (2021)’s comparison of robots deployed in the urban environments of San Francisco, Tokyo, and Dubai, they found strong differences in the rationales and responses to and for robots. The diversity in the rationales for robotic application reflects differences in the economic, social, and political contexts in each of these urban centers, which, fascinatingly, manifest themselves in the attitudes and responses that citizens have to the robots themselves. The 2021 failure of the Sidewalk Labs Smart City ”experiment” in Toronto, which famously featured waste-disposal robots running around, highlights the importance of understanding these factors even in the planning of urban technology deployments (Bozikovic, 2022; Filion et al., 2023). The ways that urban robots are taken up—or not—will be related to larger factors that undergird the reasons and context for urban robot deployment, and the way those reasons are communicated.

2.2. Sensemaking in Human-Robot Interaction

What, then, should be the lens that HRI research should take on public robot deployments? This work proposes sensemaking as a lens to apply to these contexts. Sensemaking has been studied in various contexts in the human-robot interaction literature, from robotic coworkers in factories to on-road delivery robots, to investigate how people would adapt to the introduction of robots in different environments (Sauppé and Mutlu, 2015; Weinberg et al., 2023). People have investigated how sensemaking plays an important role in the understanding of social robots by non-expert users, and how this process can improve trust towards robots (Papagni and Koeszegi, 2020). Hoggenmueller et al. (2020) unveiled the factors that influence people’s understanding of robots’ emotional expressions in urban spaces. Furthermore, Lyons et al. studied how sensemaking could help restore trust when robots performed unexpected actions (Lyons et al., 2023). Sensemaking has also been exploited to study expectations over robots, and how it ultimately leads to decision-making (Szafir and Szafir, 2021).

In the book Sensemaking, Madsbjerg (2019) argues that when making sense of a technology, people reason through ’thick’ data, relying on context and culture; for instance, when robots are introduced in public spaces, the acceptance and perception of robots may vary not only due to robots’ appearances and capabilities but also based on users’ gender, social status, or cultural background (Siino and Hinds, 2005; Lee and Sabanović, 2014). Thus, sensemaking is truly situated and embodied and can be the first step for HRI designers to build interaction patterns and blueprints (Lee and Forlizzi, 2009; Kahn et al., 2008). Sensemaking is hence important for HRI researchers to design understandable and intuitive robots. We feel this is a strong argument for focusing on sensemaking in studying public deployments of robots in urban spaces.

2.3. Resources of Sensemaking

In interactions between humans, people have been found to draw on a broad range of knowledge resources, which they use to interpret their interaction partners’ utterances, to design their own actions, and to make sense of the joint activity (Clark, 1996). Specifically, people make use of what is and has been shared with the interaction partner in prior interactions, including the current perceptually available surroundings, but also on what can be considered to be shared communal common ground, like cultural facts, human nature, and ’ineffable background’, i.e. factual knowledge that necessarily comes with being in a certain place, for instance. Fischer (Fischer, 2016) shows that people, when talking to robots, make use of similar categories as those suggested by Clark (Clark, 1996), attending, for instance to a common language, a common vocabulary, a common perspective, common perception and to a similar cultural background (cf. also (Fischer et al., 2021)). Thus, participants attend to certain aspects of the shared situation, even when interacting with robots, and update their common ground with every turn. Recently, Fantasia et al. (Fantasia et al., 2022) argue that the ”situated and dynamic coordination and negotiation of meanings, intentions expectations and interpretations that human beings experience on a daily basis in their social contexts” is part of the meaningful engagements between humans and robots, yet is not appropriately accounted for in HRI research.

Similarly, when making sense of new technology products like the Roomba robot, people take more into account than the technology itself – Forlizzi (Forlizzi, 2007) argues that social dynamics, economics, and environmental issues can play important roles in informing people’s engagement with the new technology. From a reversed perspective, Gretzel & Murphy (Gretzel and Murphy, 2019) argue that the anthropomorphism in the individual interactions with a robot takes sensemaking beyond the realm of technology-related ideologies. On the other hand, robots also invite sensemaking that rests on the complexities of discourses around social justice and equity.

Moreover, in a recent paper on trust in robots, Cameron et al. (Cameron et al., 2023) find people to include the ’deployer’, i.e. the individual or organization deploying the robot in a given context, in their considerations of the trustworthiness of robots and argue that trust towards robots is happening ”within a human-human interaction social context.” They find a substantial role of the ’deployer’ in shaping people’s attitudes towards, and trust in, the technology.

To sum up, while previous work on sensemaking in HRI has focused on the iterative, dynamic, and constructive nature of sensemaking in interactions between humans and robots (Rudaz et al., 2023), and while some resources were identified that people draw on when engaging in a sensemaking process, it still remains unclear to date what kinds of resources people who encounter robots in public spaces evoke, how they perceive such robots and how they make sense of their behavior, appearance, and presence.

2.4. Previous Work on Trash Barrel Robots

The deployment of everyday robotic objects in public spaces has been a core theme of our research; multiple research papers have been published on this topic. The first trash barrel robot was deployed about ten years before the deployment discussed in this paper, where a trash barrel was mounted on a Roomba Create base at an on-campus cafe (Yang et al., 2015). Through the original deployment, Fischer et al. looked at the initiation and negotiation between the robots and users, highlighting that the ostensive lack of social signals itself is also crucial, a sign of unwillingness to interact (Fischer et al., 2015). Ten years later, we brought the project back with upgraded hardware and deployed the robots in open plazas in New York City (Bu et al., 2023). We published the deployment process, including the ethical approval procedure, and opened a call for similar deployment protocols to promote in-the-wild deployments (Bu et al., 2024). The data collected through the latest deployments are made available upon request (Bu and Ju, 2024). With this dataset, Brown et al. (2024) investigated the emergent interactions between the plaza users and the robots that naturally come about, where they applied ethnomethodological and conversation analysis to scrutinize the turn-taking nature of the interactions without delving deep into the users’ mental models. Different from previous publications on trash barrel robots, this paper focuses on the sensemaking process that the plaza users went through when they came across the robots in everyday settings.

3. Method

In alignment with our previous Trash Barrel robot study, we aim to elicit behaviors people have in response to service robots in a public space, without the intervention of signage or design features to explain the robot or how to interact (Yang et al., 2015; Fischer et al., 2015). Based on the prior study, we had some ideas about how people might respond, but since this current study took place in a more public setting and many years after the original study, we did not attempt to make changes in the overall methodological approach. Wizards were instructed to operate the robots around the public space and interact with people passing by and sitting at cafe tables on-site, based on the video from the other study¹¹1Available online at https://vimeo.com/114106601. (Details about the instructions to the wizard can be found in Appendix A.1). We found that the wizards had to improvise what to do in various scenarios that emerged.

We would like to note that the choice of robots taking the form of trash barrels is a mutual agreement between the researchers and plaza owners through a series of discussions. In fact, the original proposal features chairbots, a fleet of robotic chairs that invite people to sit. However, the moving chairs may be a safety hazard for the general public. On the other hand, the plaza owner points out that a robotic trash barrel would be ideal. People at the plaza usually leave their trash on the table even if there’s a trashcan close by (robotic trash barrels have the potential to provide a service); there would be no direct physical contact involved in interacting with trash barrel robots (interactions are safe); and the robots will not replace the janitors onsite, but rather facilitating their work (enabling rather than replacing).

3.1. Data Collection

In our Wizard-of-Oz study, two trash barrel robots, one for standard landfill and one for recycling, were remotely controlled by two on-site research assistants (Zamfirescu-Pereira et al., 2021). (This differs from the Fischer et al. study, where only one trash barrel robot was deployed (Fischer et al., 2015).) The robots were instrumented with sensors to record the surroundings and interactions with participants.

The robots were deployed in late September and early October 2022 over two weeks, as weather permitted. The robots were deployed for one and a half to two hours, starting around 2 pm each day, during the work week. To collect natural human behavioral data, we did not put up any flyers to prime people. Instead, we administered the consent process after the users interacted with the robots.

3.2. Site

We picked the southern portion of Astor Place, a popular plaza in New York City, as our study site. The trapezium-shaped area hosts a coffee shop at one corner and has staggered tables and chairs for public seating. The plaza was surrounded by numerous shops, restaurants, and offices, which resulted in heavy foot traffic during the day. With occasional art installations and interactive sculptures, Astor Place was a welcoming place for people of all ages to work, eat, relax, and socialize, which made Astor Place a suitable site to study challenges to autonomous everyday objects in public spaces due to the high variety of use, audience, and physical obstacles.

3.3. System

The trash barrel robots are visually similar to the robot deployed by Yang et al. (2015), where 32-gallon BRUTE trash barrels are mounted on a mobile chassis. The main difference is that our mobile chassis is powered by recycled hoverboards, which provide more power and higher speed than the iRobot Create-based platform used in Yang’s study. This enables the robot to navigate on various terrains in urban cities.

We delineate robots’ roles using the standard municipal color scheme: a blue barrel for recycling and a gray barrel for landfill. We affix vinyl recycling decals to the front and back of the blue barrel to emphasize its role. The mobile platform is propelled by a re-engineered hoverboard attached to its center. The original hoverboard PCB is switched out for an ODrive v3.6 motor driver controlled by a Raspberry Pi 4 (RPi4) computer.

3.4. Data

A 360° camera was mounted in front of each barrel above the edge. People are aware of these cameras at first glance. The camera records both 360° video and audio of the activities all around the robot. Two standard GoPro cameras were mounted high up on the wall of the coffee shop at the plaza’s southeast corner, each covering half of the plaza, mostly unnoticed by the participants.

3.5. Wizard

For each deployment, two members of the lab controlled the robots as the hidden wizards on-site. Each wizard controlled one robot through a remote joystick for non-verbal behavioral processing. The wizards were trained in the lab before deployment to familiarize themselves with the joystick controller and practice how to navigate the robots in crowded spaces. During the study, wizards sat near the edge of the public sitting area and were encouraged to communicate with each other to keep them aware of the robots’ surroundings. Wizards were instructed to keep their robots socially appropriate. For example, the robots should not run into tables or passersby, should not accelerate towards people at short distances, and should respond to people’s requests accordingly. The robots could also ”offer” their service if the wizards thought it appropriate (e.g., a person is taking the last bite of their burger). In addition, the wizards were instructed to keep the two robots close to each other during the deployment.

3.6. Interviews

Besides the two hidden wizards, another on-site researcher was in charge of interviewing people after they interacted with the robots. The researcher would approach people and invite them to participate in a very short semi-structured interview regarding their opinions towards the robots. The participation is completely voluntary. The on-site researcher tried their best to interview as many users as possible, but it was unrealistic to interview everyone, given that interactions were happening frequently. The interview questions were designed to be maximally open to allow participants to provide their most honest feelings towards the robots. Thus, we asked them what words came to their minds when they thought of the robots, what things they noticed that they approved or disapproved of, how they would rate the robots, and whether there was anything else they wanted to say – see Appendix A.2 for the interview script, and Appendix A.3 for details on the relevant permissions and consent procedure.

4. Analysis

To understand people’s sensemaking activities when encountering the two trash barrel robots, we pursued two paths to data analysis. First, we performed a thematic analysis on the transcripts from interviews of people who had interacted with the trash barrel robots, to see what topics people raised when we asked them about the robot. See Appendix A.2 for the interview script. Second, we performed a video analysis to find moments of observable sensemaking occurring between members of the public and the robot. Methodologically, we chose to separately analyze the interview transcripts and interaction footage because people’s answers from interviews may not always strongly correlate with their behavior. In addition, due to the nature of the data collection process, we have more video footage than interview data. However, while we begin with analyzing the interview and video footage independently, we try to link data sources at a per-interaction level, if possible, to better ground our findings. This mixed-method approach helps us to identify both overt sensemaking behaviors occurring in the public setting and also to discern less obvious sensemaking that people engage in as they watch and reflect on the robots in their space.

4.1. Interview Analysis

4.1.1. Protocol

To analyze the interview data, we use a qualitative thematic analysis approach called ’affinity diagramming’ (Lucero, 2015). In this bottom-up approach, first, all interviews are transcribed and printed, the interview responses are identified and excerpted, and then the responses are organized in several rounds of coding to develop themes. Specifically, all responses were cut apart so that each response to an interview question was taken as at least one data point; if participants mentioned different topics, they were treated separately. Next, all the segmented interview responses were assembled based on the similarity of topics in a bottom-up iterative process. After clear categories had emerged, all data were coded by two different researchers using these categories. That is, we subsequently used the themes to code each of the responses, such that some responses could be included under multiple themes. Two researchers separately coded the transcripts based on the developed themes. Since the themes are not mutually exclusive, we compute Cohen’s Kappa coefficient for each theme individually and average the coefficient across themes. The resulting multi-label kappa coefficient is 0.62, which shows substantial similarity (Hallgren, 2012).

4.1.2. Themes

Here we describe each theme, ordered by frequency, and illustrate exemplar quotes in each. Some responses spanned multiple themes. For example, the following quote touches on five themes: the robots, the concept, people, trash, the city, and participant sentiment: ”Well, it was startling [sentiment] to see nobody with them. And then I thought, what a good idea. [concept] So people can be putting their trash in because New Yorkers are [the city], sometimes they leave trash at the table. [trash] So I thought too, ’Yeah, it’s really cute.’ [robot]”

Robot Design

In this theme are interview statements that focused on the robot’s design (”It just looks like a regular trash can with the camera on it.”, or adjectives to describe it (”wonderful,” ”creepy,” ”cute”, ”dystopian,” ”awkward,” ”adorable”). Some of the comments were somewhat anthropomorphic, using terms that are usually applied to humans, like friendly, nice and beautiful. Multiple people suggested that we accentuate the anthropomorphism: ”I feel like you should add googly eyes.” Other comments were more in line with the way we talk about objects, focusing on affordances or the experiences they evoke, such as fun or amusement, cool, fantastic, or awesome.

Role/Idea

The role theme pertained to the function, job, or purpose the robot was fulfilling: ”I thought it was going to pick up trash, but then it didn’t. It was just walking around.” Sometimes people evoked other robots–Wall-E, R2D2, Roomba–in these statements. There was some variance in how people interpreted the robot: ”Obviously, it’s a trashcan, so put the trash in.” ”It’s like I don’t think of them as trash cans even.”

Interestingly, many interviewees gave feedback not on the robot itself, but on the idea of the robot: ”I think it’s, um, I don’t think it’s necessary.” Examples are ”The concept is good, very good in fact: It’s modern, it’s ecological and everything.”; ”I like the idea,”; ”It’s very nice and I think it’s recommendable.” and ”It’s a reminder. It’s a reminder that we’re all working together to keep the place clean.”

Interaction with the Robots

The interaction theme focused on the robots’ behavior and on the exchanges the robot had with participants. We included both descriptions of actual interactions, and speculations on interactions offered by the participants: ”You signal it, right? It comes to you. If you’re lazy, you’ll still be able to throw out the trash.” ”It was active. It wasn’t just standing, still moving around. It was perky.” ”Um, um, you should make it say swear words sometimes.” ”It’s a little weird, so I wouldn’t be surprised if somebody just felt like kicking it.”

Early on, people seemed not to know if the interaction was intentional or accidental: ”…I saw it in the corner of my eye. I thought maybe I was tired, and then I was seeing things.”;

Another person commented, ”At first you see it moving, because I thought it was just the breeze blowing, and I’m like, wait, no, it’s moving. At first, you’re like, okay, it’s moving, it’s a robot.”

Once people confirmed that the trash barrels’ movements were intentional, it seemed they either assumed the robots acted for themselves (pragmatic) or demanded people’s responses (imperative). People who assumed the robots were pragmatic stated, ”I like that they were kind of making their way around and seeing who had what and stopping by tables.”; ”Well, I mean if they were running around all the time, it would be a little disconcerting” ( $P_{prag}$ ). On the other hand, people who thought the robots were imperative commented, ”and it moved on… I knew exactly what I was supposed to do if I [had] trash to put it in.”; ”They drive to people and ask for trash”; ”I didn’t know if they [could see] me or acknowledged me or if I should just go and do something else” ( $P_{impe}$ ). These assumptions can be reflected in the ways people interact with the robots. When $P_{prag}$ interacted with the robot, he sat in his chair looking down at his phone. He did not show any sign of interaction until the robot drove past him, quickly tossed trash into the robot, and resumed his normal activity right after (see Fig. 2(a)). He assumed the robot had its own plans and, hence, did not need to actively engage with the robot. On the other hand, $P_{impe}$ actively waved trash at the robot, trying to get its attention (see Fig. 2(b)).

People/Society

The interviewees often discussed ”people,” ”somebody,” ”you,” or ”nobody” in the abstract when they discussed both the interactions, but also the need for trash pick-up, or changes in society: ”I think the trash can is a good idea because it makes people think.” ”It’s brilliant, but as we say, you know, nobody’s going to have jobs.” ”I mean, how much lazier can you get?” ”They can spend money more for people than the trash”. There were also concerns about homeless people, people who have autism, and people in wheelchairs – both ways, such that the robots could assist them but also hinder them.

Purpose of the Deployment

The deployment theme focused on what the robots were doing in the public space. Since the participants did not know why there was a robot in their environment, many of the statements focused on speculation: ”I don’t know why it’s like that. Before you came, we were asking, is it a joke?” ”Is that an idea that they have going forward to maybe have remote-controlled garbage cans? Or just an idea to see how people feel?” ”My first thought was, oh, it’s some kind of art project.” ”I guess I’m wondering, does the data get, like, the footage get saved anywhere? Is it like rogue going around?”

There were also many questions directed at the interviewer: ”Is it just for school? Awesome.”; ”That was you who created that?”; ”Yeah, what’s the organization?”

Sentiment

Sentiment included statements about how participants said they felt about the robot, its interactions or the deployment, what they said they were thinking, or what they speculated other people felt: ”If it was like smooth rolling the entire time, I would be very creeped out.” ”I never wanted anything more than a trash can.” ”Do people get scared? Do they get up and move?” Sometimes the sentiment was implied: ”F*cking killer. Thank you.”

At the same time, seven people described their experience as ”interesting,” and five people indicated some kind of shock, e.g., ”speechless” or ”just shocked.” Furthermore, nine people used terms like ”kind of freaky,” ”a little weird,” ”strange,” or even ”scary.”

Trash

People talked about the trash that the trash barrel collected, but as a general problem rather than in specificity: ”It’s pushing people more to recycle or throw trash out. Especially in New York. It’s a big city.” ”I thought it was going to pick up trash, but then it didn’t. It was just walking around.” ”I like this idea too because it prevents littering.” ”I like that there were two of them. One was for trash and one was for recycling.”

Cameras and surveillance

Fourteen people raised privacy concerns given the cameras mounted on the robots. For instance:”I’m wondering, does the data get, like the footage, get saved anywhere?”; ”It’s kind of intrusive and perhaps non-consensual with like the camera aspect of it,”; ”It’s a little surveillance-y.”

4.1.3. Results

While the absolute counts in each category have little importance, the relative weight of each theme shows what aspects of the robot encounter are striking to them. They notice and comment on the robot itself, but also, to a large degree, communicate about the robots such that they are taken to stand for the underlying idea or concept that the robots embody. They focus on the mechanics of the interaction and on how to beckon or shoo away the robot, but they are also curious about the deployment and why the robot is in their space.

Many people (N=34) report on processes of sensemaking directly. For instance, they report on initial confusion, for example when they first see a trash can move, and how they overcome this initial confusion, for instance:”I didn’t know if it was just an experiment or if I’m allowed to put something on there,” or ”…and then you go like, What? and you want to use it,” or ”Once you kind of get used to it, it’s fun just waving it over.” or ”We didn’t know whether to speak to it or wave it over or what.”

Several people state that they were uncertain about how to interact with the trashcans, e.g.: ”There’s a little more information on how I’m supposed to interact,” ”If there was more of an indication as to what the purpose is,” or ”We don’t know how to do that. We don’t know how to coax a garbage can to me.” Many other people report on their hypotheses about how the robots work, for instance, or ”It was funny that it followed me like a dog,” or ”It responds really well to my hand signals,” or ”It was coming when people beckoned it and were like, hey, I have trash,” or ”I like how it kind of felt like it was interacting with us,” or ”It seemed like the can cared about me doing what I wanted to.”

Finally, people make many suggestions for improvement of the robot design and behavior, asking to add dialog capabilities or anthropomorphic features (e.g. ”maybe with eyes”), and they start brainstorming about (other) applications of the robots, like helping the homeless, blind, or autistic people. Furthermore, there are comments that concern the commercialization of the robots, like ”You could sell it to the city” or ”make a lot of money?” or ”it could be positive for someone like you.”

To sum up, there are comments that concern some of the sensemaking processes that have been heavily discussed in previous HRI work, where people draw on animate beings to make sense of robots’ behavior; these are comments that evaluate the robots using adjectives or attributions from the human domain, like cute, adorable or friendly, or when talking about feeding the robot etc. However, the majority of the interview responses do not concern one-on-one interactions with the robots, even though there are many comments that explicitly refer to active sensemaking activities. In many cases, people consider the robots an idea or concept, and they spend much effort on understanding how they are operated, who the people behind the deployment of the robots are, who is going to see the video footage and what role the city may play in this. In addition, they make use of cultural common ground when making sense of the robots, like when referring to other robots or to cultural knowledge concerning the recycling symbols. Thus, people see far beyond the one-on-one human-robot interaction and attempt to make sense of the robot being deployed in a certain place by certain people for certain social purposes.

4.2. Video Analysis

4.2.1. Protocol

We analyzed video footage captured from both on-robot and in-situ cameras for instances of sense-making activity. Two researchers, one was the interviewer and the other a wizard during the deployment, went through the videos and annotated every interaction, both explicit and implicit, between the robots and people at the plaza. For each interaction, the researchers logged the time, which robots were involved, the group size, and a brief description. Two other researchers, who were not involved during the robot deployment, went through the annotations to refine the timing and description further. This is to make sure the annotations are clear to the general public.

The robots were out in the field for 10 hours, producing a total of 40 hours of footage from all cameras. We focused on the 20 hours of footage captured by the onboard cameras, where interactions were dense and captured from a first-person view. In total, we identified 274 interactions, from which we made 166 sub-clips for detailed analysis.

To focus on the sensemaking activities, the research team scrutinized the videos for instances of observable sensemaking processes. Subsequently, the team re-viewed the clips to develop distinctions between different types of sensemaking activities.

4.2.2. Vignettes

We picked out three video clips to showcase how sensemaking was performed during the deployment, i.e., moments where people have extended engagement with the robot and with other bystanders and observably engage in sensemaking of the robots’ behavior and function. We narrate each of the encounters below, and the original video footage is included in the supplementary material.

This qualitative selection of sensemaking behaviors is not an exhaustive list but is intended to provide fine-grained illustrations of the ways that sensemaking activity unfolds.

Clip 1: Experimentation

Two girls are sitting at a table facing each other.
[Girl A sits on the outside, and Girl B sits across from her.]
Girl A starts to film the robot, while Girl B starts waving
to call the robot over.
As the robot approaches them, Girl B reaches for her
backpack, looking for things to throw away.
Girl B passes Girl A some trash and says, "Throw this
sh*t out."
Girl A ignores her and says "Hello" to the robot
while waving at the robot.
Girl B repeats herself, but Girl A keeps waving.
Girl B says, "It’s not going to wave back to you, it’s not."
In the meantime, Girl A finally takes the trash
from Girl B.
Girl A turns to the robot and asks, "Can I throw this in?"
while raising her hand to show the trash.
Girl B says, "Yeah."
Girl A shows the trash to the robot.
Girl A peeks into the trash barrel and raises her hand to
show the trash to the robot again. Then she retracts
her arm.
Girl B says, "Are you scared of a moving trashcan?"
Girl A chuckles and says, "No."
Girl A tosses the trash into the barrel.
The robot prepares to leave and shakes a little bit.
Girl A looks at the robot and asks, "No?"
As the robot backs up and drives away, both girls say,
"Thank you."

In Clip 1, we can observe sensemaking happen during the interaction between Girl A and the robot. The requesting action performed by Girl B is treated as natural by all parties at play. While the robot approached the table, Girl A was still waving at the robot and expecting more feedback until Girl B interrupted her, saying the robot was not going to wave back. Girl A kept trying to interact with the robot by asking a question about whether she was allowed to throw her trash into the robot. Her subsequent behaviors—peeking, raising her trash, and then retracting—showed that she was waiting for confirmation from the robot, displaying uncertainty about the functionality of the robot. It was not until her friend made fun of her that she finally threw out the trash. The peeking action showed that she was looking for reassurance from other trash in the barrel that the robot was usable. But once she threw out her trash, she misunderstood the delay in the robot’s response as a sign of trouble. The girls ended their part of the interaction by saying ”thank you” only after the robot made a clear leaving gesture. Throughout the interaction, Girl A monitored the robot’s subtle motions, visibly trying to make sense of the meaning behind them.

In the subsequent interviews, both girls verbally described their hesitation, confirming what we observed from the video vignettes. ”I didn’t think it would, like, recognize it. Yeah, I didn’t think it would want to come. I thought I was like, I could try. I don’t think it’s going to come.”

Clip 2: Demonstration

[The landfill robot gets stuck and struggles to get out.]
A girl walks towards the robot and stops when she
realizes that the robot is moving.
She then approaches a man standing next to the robot and
asks, "Whose trashcans are those?"
The man replies, "I have no idea, bro."
The girl walks a few steps forward and stops to
observe the robots again.
[The recycling robot bumps the landfill robot to get
it unstuck.]
The girl says, "Are they fighting? Hello."
A woman approaches from the back of the robot with trash
in her hand.
[The woman previously had interacted with the robots.]
The woman says, "That’s a trashcan," and then she looks
at the girl briefly.
Then, the woman turns to the robot and says, "I have some
trash here. Here. Can I put it in?"
[The landfill robot approaches the woman.]
The woman says, "Is that a yes?"
[The landfill robot approaches the woman more.]
The woman says, "Haha, it’s a yes."
She then put her trash into the trash barrel.
The woman says, "Thank you! Very nice service you are
doing. Thank you very much. Now stop hitting your friend."

In Clip 2, the girl first approached other bystanders to inquire about the ownership of the robots, which is consistent with the deployment theme in the interview analysis. Then, the girl kept observing the robots’ behavior and tried to decipher the meaning of their motions. However, she misunderstood the ”bump to rescue” behavior as robots attacking each other. When the woman approached later, the woman was clearly demonstrating the usage of the robots to the girl. She first pointed out that the robots were still trash cans. Then, she elaborately had a ”conversation” with the robot to show that the robot has intelligence. Throughout the interaction, the woman and the girl never spoke to each other directly, but the usage of the robots was demonstrated and shared. It is worth mentioning that the woman had interacted with the robot before this encounter, where she also had a brief conversation with the robot (she said, ”Want some trash? There, there’s some trash.”) Note that she does not wait for the robot’s confirmation as she did in this encounter. To demonstrate the robot and help the girl make sense of the robot, she modeled and narrated every single step of the interaction she felt was proper, and looked to the robot for confirmation.

A close analysis of the woman’s interview transcripts revealed indications of her demonstrative behavior. ”I thought what a good idea. So people can be putting their trash in because New Yorkers are, sometimes, they leave trash at the table… It’s really cute.” The lady supports her interpreted idea behind the deployments, which gives her incentive to promote the concept. She added, ”I knew exactly what I was supposed to do if I [had] trash [to] put it in”, reflecting her belief that her actions were appropriate. Her understanding of the robots’ affordances and belief in the concept provided her with both the rationale and the confidence to demonstrate the robots’ use to others.

Clip 3: Antagonism

A man sees the landfill robot and says,
"What the f**k?"
He then follows the robot.
Whenever some pedestrians
walk past him and the robots,
he will point at the robot and
speak to the pedestrians.
The man points at the robot and asks,
"Who the f**k is  doing this?"
The man is standing in place, constantly
looking around.
After one pedestrian says, "I love them,"
he walks toward the robot and
pulls the robot by the side
to make it fall to the ground.
He says, Not that sh*t.
He then walks away but, after a few steps,
stops to look back.

Clip 3 demonstrates a violent encounter with the robots in a public place. It is clear that the man was trying to figure out the ownership of the robot. He asked all the pedestrians he encountered about the trash can. While his voice was mostly obfuscated by the robot’s motor sound, it is reasonable to assume that all his inquiries were about the ownership of the robot. From his aggressive behavior and language, we assume that he associated the robots with public surveillance, which is a common concern among participants.

4.2.3. Results

Unlike the interview analysis, which largely shows what people are trying to make sense of and features a diverse range of themes, the video analysis captures how people perform sensemaking in action.

The video provides evidence of confusion that we also see in the interaction theme. As shown in Clip 1, when new users interact with the robots for the first time, their action tends to be exploratory and hesitant. We observed many instances of coaxing action, such as waving, holding trash, and whistling, where people tested out the robots’ affordances. Some people intentionally blocked the camera to see how the robot would respond to it, others tried to hold a conversation with the robots directly by inquiring if the robots wanted trash, despite the robots showing no capability of conversing. Once people initiated the interaction, they naturally assigned meaning to the robots’ subsequent motion, even though sometimes the hidden wizard did not recognize their requests. For example, people would interpret the robots’ wobbling motion as a ’yes’ to their question, even if the wobbling was merely caused by the bumpy surface. The girl in Clip 1 perceived the robot’s slow response as a rejection of her trash, even though the delay was rather caused by technical issues. Once the participants familiarized themselves with the robots, they were willing to demonstrate or even perform the usage of the robots to other people. As shown in Clip 2, the woman had interacted with the robots before, and she deliberately demonstrated the interaction for the girl.

The other theme that can be supported by visual evidence is deployment. To some pedestrians, when they first encountered the trash barrel robots, their first reaction was to look around to find out who the robots belonged to. As shown in Clip 3, when the man spotted the robots, he immediately looked around and asked passers-by who was in charge of the robots. The deployment of robots was also a common topic among serendipitous conversations triggered by the robots’ presence, where strangers gathered together to make sense of the purpose and owner of the robots.

Sentiment was also easily recognizable from video clips, where people who liked the robots were observed to hug and play with the robots, while people who were not fond of the robots would verbally or physically express disagreement. However, different from the interview analysis, the reasons for their sentiments could not be observed from the footage.

In summary, the video analysis provides visual evidence for some of the themes that emerged from the interview analysis. The footage shows people exploring, poking, probing, testing, and openly discussing the robots and their deployment in this specific place. For other themes, the video footage mostly captures the result of sensemaking processes, rather than their source. For instance, when a person decides to welcome the robot with a hug or kick it with their foot, they have already gone through the sensemaking process of why the robots were deployed here and their roles. Video analysis could not then capture their sensemaking process and what resources they based their reasoning upon.

5. Discussion

Our analysis of the interviews and the videos has shown that to make sense of the robots, people take a broad range of context factors into account. Among those are the deployers, i.e., the people behind the robot deployment in the specific space (cf. also (Cameron et al., 2023)). Concretely, in their evaluation of the robots and their experience with the robots, the participants’ understanding of who is operating it, who put it there, who profits from it, and who gets to see the video footage plays a crucial role. The insecurities arising need to be taken into account – preferably in the design of the robots, since the robots are probably encountered more readily than other possible signage in the environment. That is, designs of robots for the field should provide implicit answers to people’s questions that arise.

Beyond the concrete deployment, people were found to draw on a large range of resources, from a shared cultural background, which included (common realizations of) city policies, local recycling practices, and culture-specific symbols, via human nature, to personal common ground from experience with the robots themselves, in line with the categories of common ground described by Clark (Clark, 1996). With this, our findings go beyond the findings by Cameron et al. (Cameron et al., 2023), who raised the role that the deployer of a robot has on the trust within human-robot interactions; our work shows that it is not the deployer as a stand-alone entity that needs to be considered, but a far wider cultural and contextual context the interaction takes place in, of which the question of ”who is deploying the robot” is just one question.

At the same time, our results show that many of the processes identified to be relevant in one-on-one human-robot interactions are only relevant to some extent in our field scenario; for instance, while many were very excited about the robots and talked about them in emotional, anthropomorphizing terms (e.g. cute or sentient), many more considered them as a good idea or concept, thus not focusing on the robots as such, but as instantiations of a concept or ”reminders” of a particular practice. Several also just referred to them as trashcans that happen to move, and were discussing the value of trashcans moving around compared to static ones. In line with this view is also the lengthy discussion about how the robot deployment may make people lazy or destroy jobs – which in turn is in line with Gretzel & Murphy’s (Gretzel and Murphy, 2019) suggestion that robots, in comparison with other technologies, invite considerations about social justice and equity.

5.1. Public Sensemaking

5.1.1. Sources of Sensemaking

To make sense of the robots, people in public spaces draw on different resources. They look for information beyond the space, and they ask questions aloud of one another; they also experiment with the robot, talking and asking it questions even though it does not respond. We also see people watching one another interact with the robot and commenting on each other’s interactions. When they feel they understand the interactions, they take it upon themselves to perform demonstrations to help other people know how to interact.

5.1.2. Sensemaking of Behavior

From the moment that they glimpsed the robots through the corner of their eyes to the moment they threw trash into the robots’ barrels, people went through the sensemaking process to form an understanding of the robots and piece together the affordances of the robots.

5.1.3. Sensemaking of Deployment

While the behavior and interaction of the participants in the videos show how they are making sense of how to interact with the trash barrel robot, the interview data helps us to understand some of the questions they are trying to ask beyond the interaction. They are trying to map the specific instance of the robot they are seeing to the larger role or concept of robots that they have heard about. They are also trying to understand the implications that the robot they are seeing has on society. For example, they think about the implications for cleanliness and public health, of impacts on workers or vulnerable populations, or the implications of the surveillance and data collected by the robots.

Sensemaking of deployment usually happens before sensemaking of behavior, because the framing of the interaction is integral to the way they interpret it. As we observed in the video analysis, people’s interaction behavior varies drastically if they perceive it as a commercial prototype, as a part of public infrastructure, or as an art project. Without a clear understanding or assumption of the circumstances of the robots’ deployment, people may be hesitant to act. Thus, sensemaking of deployment is a precondition of sensemaking of behavior and is often ignored during the design process.

5.2. Implications for Design

What this study makes clear is that, when designing robots for public deployment, it is important to think about all of the other questions and issues that people encountering the environment will be trying to make sense of. This goes beyond the form, interface, controls, movements, behaviors, and interaction of the robot, which have been the traditional remit of HRI research and development. People in public spaces want to understand who is deploying the robots, and why, and that understanding changes whether and how they want to interact with the robot. In this section, we discuss some of the implications for design emerging from our analysis of the videos and interviews from our deployment.

5.2.1. Design for Sensemaking

Because this robot deployment was a study to understand how people make sense of robots in the absence of information, one of the most obvious things to do is to add the information that was missing. What this research exercise helps us do, however, is to understand specifically what kinds of information are needed to support the sensemaking processes that people will engage in. Signage, iconography, or messaging that helps to indicate what the trash barrel is for, do’s and don’ts for engaging the robot, who the robot belongs to, who services the robot, and where else these robots are being deployed would have helped answer some of the questions that people encountering the robots had.

By and large, people did not seem to have technical questions about the trash barrel robot: we did not find evidence that people were asking how the robot worked, or how fast the robot could move. A few people asked how long the batteries last; more asked how long the trash barrel would be in the square. They also did not ask questions about what would happen to the garbage or recycling–how often the barrels were emptied, who would empty them, and whether or how the recycling would be sorted. People did, however, have a lot of questions about the 360° camera on the robot–if the robot used that to navigate, whether it was recording, and who else could see the video. The complexity of possible responses to the question of what happens to the data leads us to feel that there should be icons to signal how footage is processed after it is captured, much as the recycling symbol indicates to people what happens to waste products after they are deposited in each can.

5.2.2. Design of Robot Behavior

Designers of the robot have some decisions to make about what they want to communicate about the robot through its behavior. For instance, a robot that is meant to be more incidental in the environment does not need to engage participants at all–it can circle a space automatically and merely draw attention to the affordance of trash disposal, or make the task of moving trash to the trash barrel a little easier. On the other hand, a robot that is actually intended to actively solicit trash would need to find potential users and approach them. A more imperative trash barrel might have fewer, more pointed movements, to assert social power and insinuate ”orders” to visitors in the square.

Pragmatically, the trash barrel robot has no mechanisms of enforcement, no way to ”make” people use the trash barrel or clean up after themselves, but the sensemaking perspective helps illustrate that indicating the intended purpose can help people know how to ”fall in line” if they want to.

5.2.3. Design of other contextual elements

Beyond designing the looks or the actions of the robot, it is also important to consider other elements in the environment that can aid in sensemaking. Because the interaction unfolds over time, people often learn about how to interact with the robot from watching one another. From that perspective, then, building in exemplar interaction moments–from a confederate, or from the cafe stand owners or janitors in the vicinity–can help people to make sense of the new technology and how to engage or not with it.

5.2.4. Design for the concept

Many people engaged not only in the intermediate interaction with the robots but also in the idea and concept behind it. They liked the idea that the robots’ behavior would push people to use the trash can more, and the fact that there were two robots encouraged recycling. When they were concerned about the robot, they were concerned about whether the designers were thinking about more vulnerable populations who share the public space, and thinking ahead to the people who might become lazy or have their jobs displaced. Designers of robots must consider how their designs are perceived by the public and what messages are being delivered. As a matter of fact, the first stage of sensemaking was not about the affordances of the robot, but the idea and concept behind it. People’s behavior towards the robot largely depends on the result of such a sensemaking process: If people decide that robots have a positive effect on reducing trash, they are likely to welcome and promote them. If people decide the robots are apparatus for surveillance, they will reject, avoid, or even antagonize them, as shown in Clip 3.

5.3. Interview Analysis and Video Vignettes for Sensemaking

In our analysis, we leverage both transcripts from post-interaction interviews and vignettes from the robot’s perspective to decipher the sensemaking process behind the interplay between users and robots. Overall, we took the approach where we analyzed transcripts and vignettes individually but linked different resources when possible to analyze the specific instance of interaction further.

Analyzing interview transcripts gives us insights into both the sensemaking of deployment and the sensemaking of behavior, where people commented on the concept behind the deployment and the assumptions they made about the robots’ functionality. Through thematic analysis, we can observe high-level trends in people’s comments. Subsequently, locating video footage within these trends can help us better understand the embodied behaviors of these concepts. For example, people who assume robots are pragmatic are less likely to wave the robot over than people who assume robots are imperative.

On the other hand, analyzing video vignettes provides us with a first-person perspective of how the interaction unfolds at a turn-taking level. It reinforces some of the themes being discussed in the thematic analysis and expands them with diverse, concrete visual examples. When possible, subsequently analyzing the interview transcripts of users captured in the video provides us with a better understanding of the situated context of the specific interaction from a personal level.

Overall, the thematic analysis provides a top-down approach to analyzing the high-level patterns that emerged during the sensemaking process. The video vignettes provide a bottom-up opportunity to analyze specific instances in fine detail.

5.4. Limitations

This research, which focused on one public plaza in one city, is premised on the contextually specific nature of the research. For technical reasons, the wizards did not have access to the robots’ onboard video and audio in real-time. Thus, some of the robots’ behaviors were not at the human performance level, especially when the participants talked to the robots. Also, since the wizards were not the same on different deployment days, the robots may show different ”personalities.” While this increases the diversity of robots’ interaction styles, the lack of consistency makes categorizing behaviors challenging. Lastly, even though we deployed the robots for two weeks, most participants were first-time users. Thus, the dataset provides little insight into long-term deployments of robots in public spaces.

In an ideal scenario, we would love to have both video and interview transcripts for every observed interaction. However, due to logistical difficulties, we are only able to interview a portion of the users who used our robot. In addition, the population who agreed to be interviewed were usually fond of the idea of our robots. People who did not like our robot would either leave or refuse to be interviewed. We are aware of the biases that exist in our interview dataset.

6. Conclusion

In this work, we analyze video data and interview transcripts from a public deployment of two trash barrel robots in \anonNew York City to better understand sensemaking activities people perform when they encounter robots in public spaces. We found that people had thoughts not only of the robots or the interactions the robots engaged in, but also the role or the idea of the robots, how other people would respond if they saw the robot, or what the implications on society were. They wanted to understand details about the deployment–are the robots a commercial product? An art project? public infrastructure?–because having that knowledge would change the way they would interact with the robot. Based on our data and analysis, we have provided implications for design that may be topics for future human-robot design researchers who are exploring robots for public space deployment.

Acknowledgements.

This research was conducted under Cornell Tech’s IRB Protocol #IRB0147271, with sponsorship from Tata Consultancy Services and the Jacobs Urban Tech Hub at Cornell Tech, as well as support from NSF #2423127 FRR: Social and Contextual Models of Interaction with Everyday Robots. We appreciate the wizards who played integral roles behind the scenes: Rei Lee, Saki Suzuki, Ricardo Gonzalez, Stacey Li, Maria Teresa Parreira, and Jorge Pardo Gaytan.

References

(1)
Babel et al. (2022) Franziska Babel, Johannes Kraus, and Martin Baumann. 2022. Findings from a qualitative field study with an autonomous robot in public: exploration of user reactions and conflicts. International Journal of Social Robotics 14, 7 (2022), 1625–1655.
Bozikovic (2022) Alex Bozikovic. 2022. The end of Sidewalk Labs. Architectural Record (Mar 2022). https://www.architecturalrecord.com/articles/15573-the-end-of-sidewalk-labs
Brown et al. (2024) Barry Brown, Fanjun Bu, Ilan Mandel, and Wendy Ju. 2024. Trash in Motion: Emergent Interactions with a Robotic Trashcan. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 1–17.
Bu et al. (2024) Fanjun Bu, Alexandra WD Bremers, Mark Colley, and Wendy Ju. 2024. Field Notes on Deploying Research Robots in Public Spaces. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems. 1–6.
Bu and Ju (2024) Fanjun Bu and Wendy Ju. 2024. SSUP-HRI: Social Signaling in Urban Public Human-Robot Interaction dataset. arXiv preprint arXiv:2403.10994 (2024).
Bu et al. (2023) Fanjun Bu, Ilan Mandel, Wen-Ying Lee, and Wendy Ju. 2023. Trash barrel robots in the city. In Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction. ACM/IEEE, Stockholm, Sweden, 875–877.
Cameron et al. (2023) David Cameron, Emily C Collins, Stevienna de Saille, Iveta Eimontaite, Alice Greenwood, and James Law. 2023. The Social Triad Model: Considering the Deployer in a Novel Approach to Trust in Human–Robot Interaction. International Journal of Social Robotics 0, 0 (2023), 1–14. https://doi.org/10.1007/s12369-023-01048-3
Clark (1996) Herbert H Clark. 1996. Using language. Cambridge University Press, Cambridge, UK.
Clark and Fischer (2023) Herbert H. Clark and Kerstin Fischer. 2023. Social robots as depictions of social agents. Behavioral and Brain Sciences 46 (2023), e21. https://doi.org/10.1017/S0140525X22000668
Cross (2006) Nigel Cross. 2006. Designerly ways of knowing. Springer, London, UK.
Dix (2002) Alan Dix. 2002. Beyond intention-pushing boundaries with incidental interaction. In Proceedings of Building Bridges: Interdisciplinary Context-Sensitive Computing, Glasgow University, Vol. 9. University of Glasgow, Glasgow, UK, 1–6. https://www.research.lancs.ac.uk/portal/en/publications/beyond-intention--pushing-boundaries-with-incidental-interaction(4c954c9a-0f00-4820-be4c-c92e41631179)/export.html
Fantasia et al. (2022) Valentina Fantasia, Ingar Brinck, and Christian Balkenius. 2022. Making sense with social robots: Extending the landscape of investigation in HRI. In AIC 2022: 8th International Workshop on Artificial Intelligence and Cognition. CEUR-WS, CEUR-WS, Örebro, Sweden, 8 pages.
Filion et al. (2023) Pierre Filion, Markus Moos, and Gary Sands. 2023. Urban neoliberalism, smart city, and Big Tech: The aborted Sidewalk Labs Toronto experiment. Journal of Urban Affairs 0, 0 (2023), 1–19. https://doi.org/10.1080/07352166.2022.2081171
Fischer (2016) Kerstin Fischer. 2016. Designing speech for a recipient. John Benjamins Publishing Company, Amsterdam, Netherlands. 1–337 pages.
Fischer et al. (2021) Kerstin Fischer, Lars Christian Jensen, and Nadine Zitzmann. 2021. In the same boat: The Influence of Sharing the Situational Context on a Speaker’s (a Robot’s) Persuasiveness. Interaction Studies 22, 3 (2021), 488–515.
Fischer et al. (2015) Kerstin Fischer, Stephen Yang, Brian Mok, Rohan Maheshwari, David Sirkin, and Wendy Ju. 2015. Initiating interactions and negotiating approach: a robotic trash can in the field. In 2015 AAAI Spring Symposium Series. Association for the Advancement of Artificial Intelligence, Stanford.
Forlizzi (2007) Jodi Forlizzi. 2007. How robotic products become social products: an ethnographic study of cleaning in the home. In Proceedings of the ACM/IEEE international conference on Human-robot interaction. ACM/IEEE, Arlington, USA, 129–136.
Gretzel and Murphy (2019) Ulrike Gretzel and Jamie Murphy. 2019. Making sense of robots: Consumer discourse on robots in tourism and hospitality service settings. In Robots, artificial intelligence, and service automation in travel, tourism and hospitality. Emerald Publishing Limited, Bingley, England, 93–104.
Hallgren (2012) Kevin A Hallgren. 2012. Computing inter-rater reliability for observational data: an overview and tutorial. Tutorials in quantitative methods for psychology 8, 1 (2012), 23.
Hoggenmueller et al. (2020) Marius Hoggenmueller, Jiahao Chen, and Luke Hespanhol. 2020. Emotional expressions of non-humanoid urban robots: the role of contextual aspects on interpretations. In Proceedings of the 9TH ACM International Symposium on Pervasive Displays. 87–95.
Jung and Hinds (2018) Malte Jung and Pamela Hinds. 2018. Robots in the wild: A time for more robust theories of human-robot interaction. , 5 pages.
Kahn et al. (2008) Peter H Kahn, Nathan G Freier, Takayuki Kanda, Hiroshi Ishiguro, Jolina H Ruckert, Rachel L Severson, and Shaun K Kane. 2008. Design patterns for sociality in human-robot interaction. In Proceedings of the 3rd ACM/IEEE international conference on Human robot interaction. 97–104.
Lee and Sabanović (2014) Hee Rin Lee and Selma Sabanović. 2014. Culturally variable preferences for robot design and use in South Korea, Turkey, and the United States. In Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction. ACM/IEEE, Bielefeld, Germany, 17–24.
Lee and Forlizzi (2009) Min Kyung Lee and Jodi Forlizzi. 2009. Designing adaptive robotic services. In Proceedings of IASDR’09. IASDR, Seoul, Korea, 1–10.
Lee et al. (2019) Wen-Ying Lee, Yoyo Tsung-Yu Hou, Cristina Zaga, and Malte Jung. 2019. Design for serendipitous interaction: Bubblebot-bringing people together with bubbles. In 2019 14th ACM/IEEE international conference on human-robot interaction (HRI). IEEE, 759–760.
Lee and Jung (2020) Wen-Ying Lee and Malte Jung. 2020. Ludic-hri: Designing playful experiences with robots. In Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction. 582–584.
Lucero (2015) Andrés Lucero. 2015. Using affinity diagrams to evaluate interactive prototypes. In Proceedings of INTERACT 2015: 15th IFIP TC 13 International Conference. Springer, Springer, Bamberg, Germany, 231–248.
Lupetti et al. (2021) Maria Luce Lupetti, Cristina Zaga, and Nazli Cila. 2021. Designerly ways of knowing in HRI: Broadening the scope of design-oriented HRI through the concept of intermediate-level knowledge. In Proceedings of the 2021 ACM/IEEE International Conference on Human-Robot Interaction. ACM/IEEE, online, 389–398.
Lyons et al. (2023) Joseph B Lyons, Izz aldin Hamdan, and Thy Q Vo. 2023. Explanations and trust: What happens to trust when a robot partner does something unexpected? Computers in Human Behavior 138 (2023), 107473.
Madsbjerg (2019) C. Madsbjerg. 2019. Sensemaking: What Makes Human Intelligence Essential in the Age of the Algorithm. Little, Brown Book Group Limited, London.
Moshkina et al. (2014) Lilia Moshkina, Susan Trickett, and J Gregory Trafton. 2014. Social engagement in public places: a tale of one robot. In Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction. ACMIEEE, Bielefeld, Germany, 382–389.
Papagni and Koeszegi (2020) Guglielmo Papagni and Sabine Koeszegi. 2020. Understandable and trustworthy explainable robots: A sensemaking perspective. Paladyn, Journal of Behavioral Robotics 12, 1 (2020), 13–30.
Pelikan et al. (2024) Hannah RM Pelikan, Stuart Reeves, and Marina N Cantarutti. 2024. Encountering Autonomous Robots on Public Streets. In Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction. 561–571.
Rudaz et al. (2023) Damien Rudaz, Karen Tatarian, Rebecca Stower, and Christian Licoppe. 2023. From inanimate object to agent: Impact of pre-beginnings on the emergence of greetings with a robot. ACM Transactions on Human-Robot Interaction 12, 3 (2023), 1–31.
Sabanovic et al. (2006) Selma Sabanovic, Marek P Michalowski, and Reid Simmons. 2006. Robots in the wild: Observing human-robot social interaction outside the lab. In 9th IEEE International Workshop on Advanced Motion Control, 2006. IEEE, IEEE, Istanbul, Turkey, 596–601.
Salvini (2018) Pericle Salvini. 2018. Urban robotics: Towards responsible innovations for our cities. Robotics and Autonomous Systems 100 (2018), 278–286.
Sauppé and Mutlu (2015) Allison Sauppé and Bilge Mutlu. 2015. The social impact of a robot co-worker in industrial settings. In Proceedings of the 33rd annual ACM conference on human factors in computing systems. ACM, Seoul, Korea, 3613–3622.
Siino and Hinds (2005) Rosanne M Siino and Pamela J Hinds. 2005. Robots, Gender & amp; Sensemaking: Sex Segregation’s Impact On Workers Making Sense Of a Mobile Autonomous Robot. In Proceedings of the 2005 IEEE international conference on robotics and automation. IEEE, IEEE, Barcelona, Spain, 2773–2778.
Sommers and Miller (2013) Roseanna Sommers and Franklin G Miller. 2013. Forgoing debriefing in deceptive research: Is it ever ethical? Ethics & Behavior 23, 2 (2013), 98–116.
Szafir and Szafir (2021) Daniel Szafir and Danielle Albers Szafir. 2021. Connecting human-robot interaction and data visualization. In Proceedings of the 2021 ACM/IEEE International Conference on Human-Robot Interaction. ACM/IEEE, online, 281–292.
Turner et al. (2023) John R Turner, Jeff Allen, Suliman Hawamdeh, and Gujjula Mastanamma. 2023. The multifaceted sensemaking theory: A systematic literature review and content analysis on sensemaking. Systems 11, 3 (2023), 145.
U.S. Department of Health and Human Services (2018) U.S. Department of Health and Human Services. 2018. Code of Federal Regulations, 45 CFR 46, Protection of Human Subjects. 45 CFR 46. https://www.ecfr.gov/cgi-bin/retrieveECFR?gp=&SID=83cd09e1c0f5c6937cd9d7513160fc3f&pitd=20180719&n=pt45.1.46&r=PART&ty=HTML
Weick et al. (2005) Karl E Weick, Kathleen M Sutcliffe, and David Obstfeld. 2005. Organizing and the process of sensemaking. Organization science 16, 4 (2005), 409–421.
Weinberg et al. (2023) David Weinberg, Healy Dwyer, Sarah E Fox, and Nikolas Martelaro. 2023. Sharing the Sidewalk: Observing Delivery Robot Interactions with Pedestrians during a Pilot in Pittsburgh, PA. Multimodal Technologies and Interaction 7, 5 (2023), 53.
Weiss et al. (2008) Astrid Weiss, Regina Bernhaupt, Manfred Tscheligi, Dirk Wollherr, Kolja Kuhnlenz, and Martin Buss. 2008. A methodological variation for acceptance evaluation of human-robot interaction in public places. In RO-MAN 2008-The 17th IEEE International Symposium on Robot and Human Interactive Communication. IEEE, IEEE, Munich, Germany, 713–718.
While et al. (2021) Aidan H While, Simon Marvin, and Mateja Kovacic. 2021. Urban robotic experimentation: San Francisco, Tokyo and Dubai. Urban Studies 58, 4 (2021), 769–786.
Yang et al. (2015) Stephen Yang, Brian Ka-Jun Mok, David Sirkin, Hillary Page Ive, Rohan Maheshwari, Kerstin Fischer, and Wendy Ju. 2015. Experiences developing socially acceptable interactions for a robotic trash barrel. In 2015 24th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN). IEEE, IEEE, Portland, USA, 277–284.
Zamfirescu-Pereira et al. (2021) J.D. Zamfirescu-Pereira, David Sirkin, David Goedicke, Ray LC, Natalie Friedman, Ilan Mandel, Nikolas Martelaro, and Wendy Ju. 2021. Fake It to Make It: Exploratory Prototyping in HRI. In Companion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction (Boulder, CO, USA) (HRI ’21 Companion). Association for Computing Machinery, New York, NY, USA, 19–28. https://doi.org/10.1145/3434074.3446909

Appendix A Appendix

A.1. Wizard Instructions

For every deployment, two members of the research group (including associated visiting researchers) teleoperated the robots during the deployment as wizards. All of our wizards have backgrounds in computer science and information science. The wizards were encouraged to keep the two robots close to each other during the deployment to contrast their different purposes. Again, different from the constrained instructions given to the wizards in (Yang et al., 2015), we took a more explorative approach and allowed for more dynamic control of the robot. We provided succinct guidance to the wizards, encouraged them to engage with users in the square in a natural manner, and gave them the freedom to decide how to operate the robot trash barrels based on the situate context.

A.2. Interview Questions

These questions were posed to people after they had interacted with a trash barrel robot, after the interviewer got verbal consent both to use the video of the interaction, and to interview them for the study.

•

What words come to your mind when you think of the trashbots?
•

What are things you noticed the trashbots doing that you liked or approved of? Why?
•

What are things you noticed the trashbots doing that you didn’t like or disapproved of? Why?
•

On a scale from 1 to 10, how do you feel toward the trashbots? (0 means hated it, 10 means loved it.)
•

Is there anything else you would like to share with us about trashbots?

A.3. Authorization and Consent

The study was conducted with written permission to conduct the experiment in that space from the business improvement district that manages the location, as well as with a certificate of insurance to cover damages that might inadvertently result from the deployment.

The protocol was approved under the Cornell University IRB#806008080; in this protocol, elements of informed consent are waived, based on the IRB finding that the research involves no more than minimal risk to the participants, that the research could not practicably be carried out without the alteration, and that alteration will not adversely affect the rights and welfare of participants (see (U.S. Department of Health and Human Services, 2018), §46.116(e)(2)). Where people were recorded actively interacting with the robot, we obtained consent post-interaction, and also asked for permission to use images and footage they are featured in. Consent was documented through recorded verbal assent based on (U.S. Department of Health and Human Services, 2018) §46.117(c)(1), as signed consent would be the only record linking the subject and the research, and the principal risk of this minimal-risk protocol would be potential harm resulting from a breach of confidentiality. Consistent with field research conducted in public spaces (see discussion in (Sommers and Miller, 2013)), we did not ask for consent from passersby who were only incidentally involved in the study, even though it could be argued that the use of the unseen ”wizard” operators constitutes deception, because those interactions were deemed to be trivial.

Making Sense of Robots in Public Spaces: A Study of Trash Barrel Robots