comment

Responsible development of autonomous

robotics in agriculture
Despite the potential contributions of autonomous robots to agricultural sustainability, social, legal and
ethical issues threaten adoption. We discuss how responsible innovation principles can be embedded into the
user-centred design of autonomous robots and identify areas for further empirical research.

David Christian Rose, Jessica Lyon, Auvikki de Boon, Marc Hanheide and Simon Pearson

A
dding to the list of environmental extrapolated from empirical research on Anticipation
challenges facing agriculture, smart farming technologies in general or With the objective of minimizing negative,
COVID-19 and the demographics the use of autonomous robots in other unintended outcomes8, ‘anticipation’
of age, migration and urbanization pose a workplaces. Here, we identify examples of involves identifying, predicting and
serious threat to the sustainability of farm responsible innovation principles being exploring the potential short- and long-
businesses and food security1. In particular, implemented and indicate where more term consequences of future innovation
farm businesses across the world are needs to be done. across society and is therefore essential
struggling to fill vacancies and provide safe for the responsible development of
working conditions for labourers. Responsible innovation in agriculture autonomous robots.
Autonomous robots could help address and beyond Very little empirical anticipatory work
these immediate challenges2. Whilst The most widely used framework for for autonomous robots in farming has
their physical manifestation comprises responsible innovation was proposed by included a variety of stakeholders in the
hardware, such as a vehicle combined Stilgoe and colleagues8 and involves four process, though a recent paper by Legun
with manipulators, their autonomy is key components: anticipating the impacts and Burch12 begins to describe a process of
derived from sophisticated algorithms of innovation; reflecting on one’s work co-design in the context of robotic apple
rooted in artificial intelligence. These and adapting accordingly (reflexivity); orchards in New Zealand. Empirical studies
algorithms fuse sensor data to enable including a wide range of stakeholders have otherwise been limited to the narrow
control and real-time decision support. in the design process; and responding use of foresight exercises in the form of
Autonomous robots can perform tasks to stakeholders’ concerns, ideas and technology use and acceptance surveys
collaboratively with humans (so-called knowledge by constructing appropriate and farmer13 or public opinion surveys14
co-bots) or on their own3. Apart from institutional structures. using online questionnaires and short
isolated on-farm examples, autonomous Guidance on responsible innovation interviews. Foresight is also used to elucidate
platforms with robotic mobility that fuse — provided by funders such as the future benefits and challenges associated
multiple technologies across a single fleet Engineering and Physical Sciences Research with combining a technology with other
(for example, crop forecasting, planting, Council9, InnovateUK10 and the European methods, such as the Delphi technique
harvesting and packing) are not yet Commission — encourages companies to (which relies on anonymous rounds of
fully implementable and face substantial be cognisant of their responsibility and voting)15. Other anticipatory processes
barriers. However, there is already adoption committed to responsible research and include ‘horizon scanning’ (scanning data
of static robotic milking technologies in innovation principles, by exploring the sources to detect early developments16) and
the dairy sector and in-field deployment of challenges that could arise from innovation ‘socio-literary techniques’ (using science
tractor-mounted robotic manipulators to and acting on their findings in a transparent, fiction as a tool to encourage dialogue
remove weeds and protect crops from pests inclusive and timely manner. Despite about technology futures17, possibly
and diseases2. frequent calls for companies to conduct through ‘Ag-Tech movie nights’18). A typical
We know, however, that the history a transparent and iterative process of methodology in robotics and human–
of agricultural innovation is littered with responsible innovation, there is a lack of robotic interaction are ‘Wizard of Oz’
failure and slow adoption, and there are commitment to, or reporting of, the steps studies19, where autonomy is ‘fake’; robots
legal, ethical and social concerns associated taken in technology development in the are usually remote-controlled, anticipating
with autonomous agriculture4,5. Potential agriculture industry. the abilities they may have once fully
challenges, opportunities and consequences In the following sections we discuss implemented. Video studies are also often
of autonomous agriculture (Fig. 1) are how the four key components mentioned employed20, where participants are presented
interlinked and depend on how technologies above can be operationalized to guide with recordings of robot behaviour and
are designed and implemented. Many technology development in agriculture11, assess it from a third-person perspective.
of these aspects have been discussed in outlining key research needs. The examples One further method to consider is
the burgeoning literature on the social referenced herein alongside the guidance backcasting, which involves building
and ethical impacts of digitalization from Stilgoe et al.8 and Eastwood et al.11 an (ideal) future scenario, and working
in agriculture6,7. Empirical research provide a good overview of techniques backwards to identify the steps needed to
remains limited for autonomous farming that can be used to apply responsible get to there. This is done in anticipatory
robotics; potential issues have largely been innovation principles. governance approaches, for example.
306 Nature Food | VOL 2 | May 2021 | 306–309 | www.nature.com/natfood
 comment

+ Reduced labour costs

+ Reduced input costs
Can drive productivity
+
in economy
+ Attracts younger workers
+ Helps plug labour gap + Prevents collapse
of unprofitable farms
+ Increase in skilled jobs
+ Standards and legislation
can improve safety + Lose dull, dangerous + Potential for improved
farm jobs – productivity
– Determining responsibility – Loss of jobs
difficult Lack of capital for farmers
+ Improves decision-making –
– Legislation often reactive – Workers unable to retrain to invest
Reliance on companies
– Difficult to regulate farm Data ownership and
– privacy concerns
Dangerous jobs displaced
– elsewhere –
for repairs and upgrades
environments

Regulation Data Employment Economics

Autonomous agriculture

Trust Health and safety Sustainability and environment Social and cultural

+ Robots do the dangerous jobs

+ Data can improve +
Lower greenhouse-gas
+ Improved farmer
transparency of production emissions via electric robots well-being and lifestyle
– Cyber security weakness
Robots/AI lack an emotive + Resource use efficiency + Consumers protected
Danger to workers or (water, chemicals) from price increases
– state, which can make them –
the public
hard to trust + May help to facilitate
agroecological systems + Changes in the structure
– of the farm industry
+ Perpetuates industrial design
–
Some farmers
+ Land sparing could benefit the –
‘left behind’
– environment
Loss of experiential
+ Potential for less soil –
knowledge
– compaction
– Loss of culture
– May help to facilitate more intensive –
Disruption to rural
monoculture communities
– Old machines wasted New stress created by
–
Artificial versus natural public tech
– perceptions

– Unsustainable sourcing of
material to build robots

Fig. 1 | Overview of challenges, opportunities and potential consequences of autonomous agriculture. Positive (+), negative (–) and uncertain (+/–)
consequences are indicated. Positive consequences denote opportunities to be harnessed, whereas negative consequences denote challenges to be overcome
concerning the operationalization, adoption and/or deployment of innovations (see refs. 4,5 for more detail).

A key area for future research will be to use included in the process of anticipation produced in that way. Including such a wide
different anticipatory methods with diverse should be those directly affected by the range of stakeholders will create a number
stakeholders specifically on the subject of adoption of robotics, including farmers, of practical challenges related to power
autonomous robots in agriculture. Those farm workers and consumers of food inequality (for example, farm managers

Nature Food | VOL 2 | May 2021 | 306–309 | www.nature.com/natfood 307

comment

versus farm workers) and language barriers safety aspects such as tactile and pressure co-developers; markets, customers and
(for example, migrant farm workers) and sensors, safe maximum speed, proximity end-users; regulators and standards
these will need to be managed sensitively by sensors, human detection cameras, and bodies; NGOs representing civil society
trained facilitators. emergency stop are described to ensure the stakeholders; and individual citizens likely
safety of human–robot collaboration. Other to be affected. Beyond the usual suspects, it
Reflexivity relevant international standards include: is important to engage with harder-to-reach
Reflexivity entails “holding a mirror up ISO 18497 (design principles for safety with stakeholders. Schillo and Robinson28
to one’s own activities, commitments and highly automated agricultural machines discuss the importance of engaging with
assumptions, being aware of the limits — operations of robots in-field are not historically marginalized groups. In the
of knowledge, and being mindful that a covered); ISO 17757 (for use of autonomous case of autonomous agriculture, this could
particular framing of an issue may not be machinery in mining); and ISO/SAE involve small farmers (who may be pushed
universally held”8. Constant analysis and DIS 21434, currently under development out of the industry by larger farmers with
critique of one’s work among peers is an (for cybersecurity in road vehicles). The more capacity to adopt and adapt), organic
embedded practice of rigorous science. agricultural industry can glean insights from farmers (whose farming strategy may be
However, scientists and engineers typically these standards, however there is a necessity more difficult to align with autonomous
carry out reflexivity and other responsible to further develop agriculture-specific robots focused on precision fertilization32),
innovation practices behind closed doors, in standards and codes of practice that account as well as farm workers (who could lose jobs
the lab, and do not recognize these processes for human–robot collaboration in flexible, as they are replaced by robots). Blok33 argues
as reflexivity in responsible innovation unstructured environments such as in the that stakeholder inclusion and participation
terms21. Opening these conversations up to field. Understanding how this might be can typically become reductive as it focuses
the public and acknowledging and listening done effectively, bringing together relevant on the cognitive approach to understanding
to other actors can improve the quality stakeholders, is an important future area the perspectives of stakeholders in a
of reflexivity. for research. self-serving ‘immunization strategy’, where
Reflexivity in the realm of autonomous the goal is to convince others, prevent
robots in agriculture has mostly come from Inclusion criticism and portray the company as having
the user-centred design process. Work Concepts of inclusion are frequently limited good intentions. We should ultimately
to date in this space has recognized that to the ‘consideration’ of how stakeholders ensure that we are undertaking substantive,
robotic systems interacting with humans may be impacted or react to innovation rather than tokenistic inclusion.
need to undergo an iterative development by a limited group of experts28. Genuine The involvement of stakeholders
approach22, bringing together subjective user inclusion should involve the participation should not be restricted to the exploration
experience with actual system logs. After of a full range of stakeholders. If we do not of consequences in terms of economic
including stakeholders and seeking their pursue methods for the substantive inclusion opportunity or technology acceptance,
information requirements and preferences of a full range of actors, not just the usual but include wider implications and
for autonomous robots through surveys23, suspects, and do not give due attention to society’s ‘grand challenges’. To date there
workshops24 and field experiments25–27, power inequalities between stakeholders are limited examples of this work: Pfeiffer
designers have altered prototypes and design throughout the participatory process, then et al.14 explored public opinion of digital
paths to ensure that the robots work for we risk reinforcing unequal participation farming technology through surveys and
the user. Yet, this is narrow reflexivity; it under the guise of inclusivity. It may appear spontaneous associations; Kester et al.13
involves developers tweaking design based that increased participation from the start is surveyed farmers’ views of the future of
on user feedback, rather than conducting a time-consuming and resource-intensive, but automation on topics such as perceived
fundamental analysis of the assumptions and user-centred design can prevent problems value, applications and expectations; and
values underlying the proposed solution or further down the line. Baxter et al.26 asked fruit pickers questions
questioning if agricultural robotics is really Within the development of autonomous regarding the impact of autonomous robots
the path we want to take as a society. We robots in agriculture, inclusion has on their job security.
rarely carry out a deeper form of reflexivity, mostly taken the form of consultation
possibly missing alternative solutions. and sometimes collaboration, involving Responsiveness
The development of and engagement with feedback from farmers and farm workers Identifying potential consequences,
best practice guidelines, codes of conduct on the technical side of robot development. reflecting on underlying assumptions,
and international standards is another Simulation experiments29,30 and field-based values and problem-solving processes, and
form of reflexivity that can guide industry workshops23 have allowed farmers and farm including stakeholders in the innovation
to conduct innovation in a responsible workers to test the usability of a technology. process can only lead to responsible
manner, although it is not always clear Researchers have used task scenarios, innovation if newly gained insights are
whether they continue to serve the purpose observations, and participant feedback acted on. Actors should be reactive to
of reflexivity once adopted. In Australia, a to feed into prototype development. The new knowledge and ensure development
code of practice for agricultural mobile field social sciences have developed a number is iterative. This could be in the form of
machinery with autonomous functions is of participatory methods that allow adapting R&D projects or early design
currently under development to help guide substantive inclusion, such as citizen juries prototypes based on feedback from
safe working procedures in the field; this and deliberative workshops, and a greater stakeholders. Other actions that result from
code of practice is intended to hold some selection of these should be brought to bear new information could include adjusting
legal weight. International standards for for inclusion surrounding autonomous business models, altering control or access
the use of autonomous robots such as ISO agriculture31. to software, amending workers contracts
10218 provide norms for worker safety when Stakeholders identified in the PAS and working conditions3, or refraining from
collaborating with robots in a structured, 440 Responsible Innovation framework developing a certain robot altogether if it is
industrial environment. In ISO 10218, developed for InnovateUK10 include not desired by society.

308 Nature Food | VOL 2 | May 2021 | 306–309 | www.nature.com/natfood

 comment

Responsiveness is also important within are turned into responsive action on the 10. PAS 440:2020 Responsible Innovation – Guide (British Standards
Institution, 2020).
institutional structures, which should ground. As highlighted in this paper, most 11. Eastwood, C., Klerkx, L., Ayre, M. & Dela Rue, B. J. Agric.
respond promptly to new information, in empirical work for the development of Environ. Ethics 32, 741–768 (2017).
policy, law and regulatory environment. autonomous agriculture has been focused 12. Legun, K. & Burch, K. J. Rural Stud. 82, 380–390 (2021).
13. Kester, C., Griepentrog, H. W., Hörner, R. & Tuncer, Z. A survey
Regulation can restrict innovation (for on the technical aspects of robot operation of future farm automation – a descriptive analysis of survey
example, genetically modified crops with some level of inclusion and reflexivity responses. In Precision Agriculture ‘13 (ed. Stafford J. V.) 785–792
in Europe), efficiency and competitive to ensure improvement of technical (Wageningen Academic Publishers, 2013).
14. Pfeiffer, J., Gabriel, A. & Gandorfer, M. Agric. Human Values 38,
advantage, however legal structures will be performance. Little published work has gone 107–128 (2020).
important to ensure protection for users of beyond this to use methods that allow for 15. Brier, D., Eastwood, C. R., Dela Rue, B. T. & Viehland, D. W. J.
autonomous robots and for clarifying the substantive inclusion and deeper reflexivity Agric. Environ. Ethics 33, 549–549 (2020).
16. Hamilton, G. et al. Horizon Scanning: Opportunities for New
liability framework. Hence, regulation can on the subject. Yet, if society decides that Technologies and Industries (AgriFutures, 2019).
act as both a barrier to and an enabler of autonomous robotics for farming is the way 17. Miller, C. A. & Bennett, I. Sci. Public Policy 35, 597–606 (2008).
adoption. Basu et al.5 describe the current to go, then practising responsible innovation 18. DiSalvo, C., Lodato, T., Fries, L., Schechter, B. & Barnwell, T.
CoDesign 7, 185–197 (2011).
legal frameworks, regulations and standards in their development is vitally important to
19. Riek, L. D. J. Hum.-Rob. Interact. 1, 119–136 (2012).
that are relevant to the development of prevent future controversy, implementation 20. Walters, M. L. et al. Adv. Rob. 25, 2233–2254 (2011).
autonomous robots in agriculture, as well delays and negative consequences. 21. Glerup, C., Davies, S. R. & Horst, M. J. Responsible Innov. 4,
as the gaps in areas such as data protection Ultimately, the success or failure of 319–336 (2017).
22. Weiss, A., Bernhaupt, R. & Tscheligi, M. In New Frontiers in
law, ethics of robot autonomy and artificial autonomous robots in agriculture will not Human–Robot Interaction (eds Dautenhahn, K. & Saunders, J.)
intelligence. Similarly to how the European rest on the limits of our technical enterprise, 89–110 (John Benjamins Publishing Company, 2011).
Union embedded ‘privacy by design’ into but on our ability to involve society, learn 23. Edet, U., Hawley, E. & Mann, D. D. Can. Biosyst. Eng. 60,
2.19–2.31 (2018).
its General Data Protection Regulation, from it and respond appropriately. ❐ 24. Relf-Eckstein, J. E., Ballantyne, A. T. & Phillips, P. W. B. NJAS
others are calling for ‘equality by design’ in Wageningen J. Life Sci. 90–91, 100307 (2019).
artificial intelligence regulation to safeguard David Christian Rose 1 ✉, Jessica Lyon1, 25. Adamides, G. et al. J. Field Rob. 34, 1407–1426 (2017).
26. Baxter, P., Cielniak. G., Hanheide, M., From, P. Safe
against bias and discrimination that may Auvikki de Boon1, Marc Hanheide2 and Human-Robot Interaction in Agriculture. In HRI ’18. Companion
inadvertently be engrained in technology Simon Pearson 3 of the 2018 ACM/IEEE International Conference on Human-Robot
and machine learning34. There are examples 1
School of Agriculture, Policy and Development, Interaction 59–60 (IEEE, 2018).
27. Paniflov, I. & Mann, D. D. Can. Biosyst. Eng. 60, 2.11–2.18 (2018).
of ‘technological redlining’ as well as University of Reading, Earley, Reading, UK. 2School
28. Schillo, R. S. & Robinson, R. M. Technol. Innov. Manag. Rev. 7,
technological limitations of measurement of Computer Science, University of Lincoln, Brayford 34–46 (2017).
such as unequal object detection or Way, Lincoln, UK. 3Lincoln Institute for Agri-Food 29. Meusel, C. et al. Comput. Electron. Agric. 148, 309–321 (2018).
30. Huuskonen, J. & Oksanen, T. IFAC-PapersOnLine 52,
lower quality heart rate measurement Technology, University of Lincoln, Riseholme
367–372 (2019).
for those with darker skin34. A lack of Campus, Lincoln, UK. 31. The Use of Public Engagement for Technological Innovation:
transparency with algorithms, machine ✉e-mail: [email protected] Literature Review and Case Studies (BEIS, 2021).
learning and artificial intelligence — the 32. Bronson, K. NJAS Wageningen J. Life Sci. 90–91, 100294 (2019).
33. Blok, V. In International Handbook on Responsible Innovation
‘black box problem’ — can lead to bias Published online: 20 May 2021 (eds von Schomberg, R. & Hankins, J.) Ch. 16 (Edward Elgar
and discrimination issues within machine https://doi.org/10.1038/s43016-021-00287-9 Publishing, 2019).
learning becoming further entrenched and 34. Amani, B. In Artificial Intelligence and the Law in Canada
References (eds Martin-Bariteau, F. & Scassa, T.) Ch. 11 (LexisNexis
replicated. Regulatory oversight of equality Canada, 2021).
1. COVID-19: Policies and Impact on Seasonal Agricultural Workers
by design34 is key to ensure that programmers (IOM-UN, 2020).
address any bias and discrimination that 2. Duckett, T., Pearson, S., Blackmore, S. & Grieve, B. Agricultural Acknowledgements
may be produced in algorithms, ultimately Robotics: The Future of Robotic Agriculture (UK-RAS, 2018). This paper was developed from the Robot Highways
3. Marinoudi, V., Sørensen, C. G., Pearson, S. & Bochtis, D. Biosyst. project funded by InnovateUK as part of the ISCF TFP
ensuring that technology treats users fairly. Eng. 184, 111–121 (2019). Science and Technology into Practice: Demonstration call
4. Sparrow, R. & Howard, M. Precis. Agric. 22, 818–833 (2021). (grant number 51367).
Conclusion 5. Basu, S., Omotubora, A., Beeson, M. & Fox, C. AI Soc. 35,
113–134 (2020).
Addressing the social, legal and ethical 6. Klerkx, L., Jakku, E. & Labarthe, P. NJAS Wageningen J. Life Sci.
Competing interests
implications of autonomous robots is 90–91, 100315 (2019).
The authors declare no competing interests.
arguably a greater challenge than the 7. Barrett, C. B. et al. Nat. Sustain. 3, 974–976 (2020).
Additional information
development of the technology itself. 8. Stilgoe, J., Owen, R. & Macnaghten, P. Res. Policy, 42, 1568–1580.
Peer review information Nature Food thanks Robert
9. Framework for Responsible Innovation. Engineering and Physical
More research is needed to ensure that Sciences Research Council https://epsrc.ukri.org/index.cfm/ Sparrow and Daniel Mason-D’Croz for their contribution
anticipation, reflexivity and inclusion efforts research/framework/ (2020). to the peer review of this work.

Nature Food | VOL 2 | May 2021 | 306–309 | www.nature.com/natfood 309