ed

Non-invasive Diagnosis of Nutrient Deficiencies in Winter Wheat

and Winter Rye using UAV-Based RGB Images

iew
Jinhui Yi1,∗, Gina Lopez3, Sofia Hadir3, Jan Weyler4, Lasse Klingbeil4, Marion
Deichmann5, Juergen Gall1,2, Sabine J. Seidel3
1
University of Bonn, Computer Vision Group, Institute of Computer Science,
Friedrich-Hirzebruch-Allee 8, 53115 Bonn, Germany
2
Lamarr Institute for Machine Learning and Artificial Intelligence, Germany

ev
3
University of Bonn, Crop Science Group, INRES, Katzenburgweg 5, 53115 Bonn, Germany
4
University of Bonn, Institute of Geodesy and Geoinformation, Nußallee 17, 53115 Bonn, Germany
5
University of Bonn, Plant Nutrition Group, INRES, Karlrobert-Kreiten-Strasse 13, 53115 Bonn,
Germany

r
Abstract
er
Better matching of the timing and amount of fertilizer inputs to plant requirements will
pe
improve nutrient use efficiency and crop yields and could reduce negative environmental
impacts. Deep learning can be a powerful digital tool for on-site real-time non-invasive
diagnosis of crop nutrient deficiencies. A drone-based RGB image dataset was generated
together with ground truthing data in winter wheat (2020) and in winter rye (2021) during
tillering and booting in the long-term fertilizer experiment (LTFE) Dikopshof. In this LTFE,
the crops are fertilized with the same amounts since decades. The selected treatments include
ot

full fertilization including manure (NPKCa+m+s), mineral fertilization (NPKCa), mineral

fertilization but no nitrogen (N) application (_PKCa), no phosphorous (P) application
(N_KCa), no potassium (K) application (NP_Ca), or no liming (Ca) (NPK_), as well
tn

as an unfertilized treatment. The image dataset consisting of more than 3600 UAV-based
RGB images was used to train and evaluate in total eight CNN-based and transformer-
based models as baselines within each crop-year and across the two crop-year combinations
aiming to detect the specific fertilizer treatments including the specific nutrient deficiencies.
rin

The field observations show a strong biomass decline in the case of N omission and no
fertilization though the effects are lower in the case of P, K, and lime omission. The mean
detection accuracy within one year was 75% (winter wheat) and 81% (winter rye) across
models and treatments. Hereby, the detection accuracy for winter wheat was highest for the
NPKCa+m+s (100%) and the unfertilized (96%) treatments as well as the _PKCa treatment
ep

(92%), whereas for treatments N_KCa and NPKCa the accuracy was lowest (about 50%).
The results were similar for winter rye. In the cross-year and cross-cereal species transfer
(training on winter wheat, application on winter rye, and vice versa) the mean accuracy
was about 18%. The results highlight the potential of deep learning as a digital tool for
Pr

decision-making in smart farming but also the difficulties of transferring models across years
1

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4549653
 ed
and crops.
Keywords: deep learning, non-invasive diagnostics, nitrogen; phosphorous; potassium;
liming, fertilization

iew
1. Introduction
Applying nutrients to agricultural systems is critical to maximizing crop yields while
minimizing the negative environmental impacts of fertilizers, such as nitrous oxide emissions

ev
or groundwater pollution. These goals can be achieved by timely calculating crop nutrient
demand, which allows for precise fertilization management. Traditionally, farmers estimate
the nitrogen (N) fertilizer demand by considering the soil mineral N content (Nmin ) at the
beginning of the season and the expected yield. Farmers may base their fertilization strategy

r
on regionally crop-specific aggregated Nmin values delivered by the official advisory system
or on their own soil Nmin analysis of a representative field [1]. In most European countries,
fertilizer recommendations for the plant macronutrients potassium (K) and phosphorous
er
(P), which are less mobile than N in the soil, are based on actual measurements of plant-
available P and K in the soil, expected yields, and on experimental data from long-term
field experiments (LTFEs) [2, 3]. Such LTFEs which provide key information regarding
pe
long-term trends in agroecosystems [4], are also great test beds for non-invasive nutrient
deficiency diagnosis under real field conditions [5].
Moreover, N uptake can be detected during the growth period to better match crop N
demand. Destructive methods to diagnose plant foliar N status are accurate but expensive
and time-consuming as they require tissue sampling and subsequent laboratory analysis.
Non-destructive methods are rapid and less expensive but usually less accurate. The chloro-
ot

phyll measurements frequently distinguished between plots with insufficient N and those
with adequate N [6]. Hand-held chlorophyll meters are convenient devices for rapid diag-
nostic results [7], but they require manual fieldwork. Unmanned aerial vehicles (UAV) and
tn

satellite imaging offer a non-invasive alternative. Non-destructive spectral sensing has been
established for estimating the plant N status and, thus, the N demand in later crop devel-
opment stages [8]. Since N fertilization affects several plant physiological parameters, such
as the canopy chlorophyll content, they are well suited for spectral detection of N deficiency
([9], [10]). Spectral sensors can be mounted on tractors or unmanned aerial vehicles (UAV)
rin

for optical, non-destructive detection of the plant canopy’s reflectance signature [11, 12].
Non-invasive sensing of the N status of plants grown in the field, e.g., for assessment
of N uptake or improved N fertilization, is well advanced ([11, 13]), however, there are
only a few studies on other nutrient deficiencies [12, 14, 15]. [12] used a hand-held Raman
ep

Corresponding author.
∗

Email addresses: [email protected] (Jinhui Yi1, ), [email protected] (Gina Lopez3 ),

[email protected] (Sofia Hadir3 ), [email protected] (Jan Weyler4 ),
Pr

[email protected] (Lasse Klingbeil4 ), [email protected] (Marion Deichmann5 ),

[email protected] (Juergen Gall1,2 ), [email protected] (Sabine J. Seidel3 )
2 August 5, 2023

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4549653
 ed
spectrometer for pre-symptomatic diagnostics of N, P, and K deficiencies in rice grown in
hydroponic conditions with induced N, P, and K deficiencies and salinity stresses. To avoid
working with datasets that are collected manually or that only contain images of single
plant leaves only [15–19], RGB images captured with a camera or a UAV can be applied.

iew
Combining RGB images with deep learning can be a powerful tool for the non-invasive
diagnosis of nutrient deficiencies because RGB images can capture visual cues related to the
nutrient status of plants, including changes in color, texture, and shape. An optimal field
dataset could be augmented to include variations in lighting, camera angle, and other factors
to improve the robustness of models. A model could then be trained on the datasets using
deep learning methods such as convolution neural networks (CNNs) [20–27], transformers

ev
[28–32] and transfer learning [19, 33]. The model would learn to identify patterns in the RGB
images that correspond to nutrient deficiencies, such as changes in leaf color or texture. Once
the model is trained, it could be used to classify unseen RGB images of plants of the same
or different species from the same or other years. This would allow farmers to quickly and

r
non-invasively diagnose nutrient deficiencies in different crops and years, enabling targeted
interventions to improve crop nutrition and yield. er
[14] reported state-of-the-art research in the domain of nutrient deficiency identification
in plants using various CNN-based architectures. The deficiency classification results ranged
from 40 to about 100% for the presented eleven studies on various crops. Most of the studies
were based on RGB images of plant organs such as leaves (or fruits in the case of tomato)
pe
captured using smartphones or digital cameras in greenhouses and the dataset is used for
both training and testing [15, 34–39].
Only three studies used images of real field data [5, 40, 41]. [5] used RGB camera im-
ages of sugar beets growing in one year on a LTFE with various deficiency treatments and
analyzed them with five convolutional neural networks (CNN) aiming to recognize nutrient
ot

deficiency symptoms [5]. While the best network could recognize the symptoms of nutrient
deficiencies with high accuracy if all development stages have been observed during training,
identifying nutrient deficiencies across crop development stages remained difficult. [41] ap-
plied a large number of sequential images acquired with a camera from oilseed rape canopies
tn

at different growth stages and three N, P, and K levels during a two-year field experiment.
The model performance in a cross-year test was 92-93%. [40] used RGB images captured by
a UAV to detect iron deficiency chlorosis in soybean in three conditions to test model ro-
bustness namely different soybean trials, field locations, and vegetative growth stages. The
rin

results showed that unsupervised pre-training of a convolutional autoencoder on unlabeled

aerial images improves the robustness.
Overall, deep learning methods hold great potential for improving agricultural practices
by enabling more precise and efficient diagnosis and allowing for targeted treatment of
ep

crops. To test on-site non-invasive nutrient status diagnosis, we collected RGB images of
cereal crops with a UAV in a field with nutrient deficiency plots that were established by
decades of nutrient omission at the LTFE Dikopshof [42]. In this study, (1) a high-quality
RGB image dataset named DND-Diko-WWWR for nutrient deficiencies classification in
winter wheat and winter rye in Dikopshof is introduced, (2) a baseline evaluation of both
Pr

Convolutional Neural Network (CNN)-based and Transformer-based architectures on the

3 August 5, 2023

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4549653
 ed
proposed dataset is presented; and (3) the difficulties in transferring models across crops
and years are highlighted.

iew
2. Material and Methods
2.1. Experimental Site Description and Fertilizer Treatments
The static long-term fertilizer field experiment was established at the Dikopshof farm in
1904 near Bonn, Germany (50° 48’ 21” N, 6° 59’ 9” E, altitude: 62 m a.s.l.). The climate
in the area is characterized by mild winters and summers, resulting in an average annual
temperature of 11.1 °C and a mean annual precipitation of 670 mm (1990-2020). The soil

ev
type is classified as a Haplic Luvisol derived from loess above the sand. The soil organic
carbon content of the silty loam (topsoil) and silty clay loam (below 30 cm soil depth) varies
from 0.7 to about 1.3%. The crop rotation over a five-year cycle at the Dikopshof LTFE
includes sugar beet (Beta vulgaris subsp. vulgaris), winter wheat (Triticum aestivum L.),

r
winter rye (Secale cereale L.), a fodder legume (mainly Persian clover, Trifolium resupinatum
L.), and oat/potato (Avena sativa L./Solanum tuberosum L., potato replaced oat in 1953)
er
[43].
The experiment comprises six fertilization treatments: NPKCa, _PKCa, N_KCa,
NP_Ca, NPK_, and unfertilized (UF). In this context, _ indicates the omission of the
corresponding nutrient and Ca stands for liming [44]. Cattle farmyard manure from cows is
pe
applied on sugar beet, potato, and winter rye plots after harvesting of the preceding crop at
an average rate of 60 t ha−1 year−1 (fresh matter, manure rotation mean since 1953, treat-
ments with “+ m”). Additionally, there are treatments without the application of manure.
Since 1953, supplemental mineral fertilizer (N, P, K) has been applied in the treatments de-
scribed as “+ s”. The supplemental mineral fertilizer provides the same amount of nutrients
ot

as in manure, but it is applied in the form of mineral fertilizer. There are also treatments
without the application of mineral fertilizer. All the combinations result in 24 treatments
in total.
The experiment is a non-randomized block design and comprises five parallel strips (A to
tn

E). In each of the strips, one of the crops of the rotation is grown. In total, the experiment
consists of five strips with 24 treatments per strip and thus 120 plots. The plot size is 15
× 18.5 m. For details on the LTFE (detailed description, maps, and long-term soil and
yield data), please refer to [45] and [42]. In this study, we focus on seven treatments,
rin

namely unfertilized (UF), _PKCa, N_KCa, NP_Ca, NPK_, NPKCa, NPKCa+m+s (fully
fertilized), see exemplarily strip B in Figure 1a with marked plots.

2.2. Crop Management, Crop Growth and Field Data Collection

ep

Winter wheat (Triticum aestivum, variety Boss) was grown in the season 2019/20 in strip
B, in the following referred to as WW 2020. Winter rye (Secale cereale, variety Trebiano
KWS) was grown in the season 2020/21, in the following referred to as WR 2021. Table 1
shows the dates of sowing, flowering, harvesting, and fertilization in each season. Farmyard
manure and mineral P and K fertilizer were applied end of October before plowing and
Pr

sowing of winter rye.

4 August 5, 2023

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4549653
 ed
Table 1: Crop management for winter wheat grown in 2019/20 (WW 2020) and for winter rye grown in
2020/21 (WR 2021) at the long-term fertilizer experiment Dikopshof.

Date WW 2020 WR 2021

iew
Sowing 25th November 2019 5th November 2020
Flowering 28th June 2020 4th June 2021
Harvesting 4th August 2020 30th July 2021
N fertilizer application 22nd April & 19th May 2020 20th May 2021

ev
In this study, we present the observed dry matter shoot biomass and soil analysis of one
representative date per crop to evaluate the effect of various fertilization treatments on crop
growth. Shoot biomass was collected from a 1 m2 area (four replicates per plot) on the same
day or the day after the UAV flights. The shoot fresh matter samples were oven-dried until

r
constant weight (105°C) and weighed to estimate dry matter shoot weight. Additionally,
grain and straw were collected at harvest and processed as described above.
Soil samples were collected using a Pürkhauer auger in four sampling points per treatment
er
and divided by layer (0-30 cm, 30-50cm and 50-100 cm). The samples were then pooled
per layer and per treatment and frozen. After thawing, the soil was analyzed for mineral
nitrogen content (Nmin ). The soil samples were prepared by drying and sieving. Plant-
pe
available phosphorus (Pcal ) and potassium (Kcal ) were determined using a calcium acetate-
lactate extract as described in [46]. The pH of the soil samples was determined using a CaCl
solution and a pH Meter Multi 3630 IDS from WTW and Sentix 940P electrode.

2.3. Drone Data Acquisition, Dataset Structure and Processing

UAV drone flights mounted with a Matric 600 and 100 MP camera (resolution 1 mm
ot

per pixel) were conducted on three dates during each main growth period in WW 2020
and WR 2021. The number of images and flight days are detailed in Table 2. The pre-
sented dataset Deep Nutrient Deficiency - Dikopshof - Winter Wheat and Winter Rye
tn

(DND-Diko-WWWR) consists of 3600 UAV-based RGB images captured on several dates

for winter wheat (spring and summer 2020) and winter rye (spring and summer 2021)
leading to two crop-year combinations generated at the LTFE Dikopshof (Table 2). The
dataset is accomplished by ground truthing crop data demonstrating the respective nutrient
deficiencies.
rin

All RGB images were captured by a camera (PhaseOne iXM-100) mounted on a UAV
(flight height: 21 m) and an ortho-photo, which is a single very large image generated from
all camera images of one flight date, was created. The ortho-photos were subdivided into
ep

RBG images with the size of 1100 × 1100 pixels as follows. For each plot center (core plot)
in Figure 1a, about 20 non-overlapping squares with coordinates (Figure 1b) were cut out
of the ortho-photos, and each square was further subdivided into four quadrants, namely
NW, NE, SW, SE (N for North, E for East, S for South and W for West) as shown in
Figure 1d). Each square with a coordinate was annotated with a distinct ID (Figure 1c),
Pr

which is consistent across different dates of data acquisition.

5 August 5, 2023

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4549653
 ed
Table 2: The number of images in the Deep Nutrient Deficiency for Winter Wheat and Winter Rye (DND-
Diko-WWWR) dataset captured for two crop-year combinations. “_” stands for the omission of the corre-
sponding nutrient (N: nitrogen, P: phosphorous, K: potassium, Ca: lime) and +m+s stands for additional
application of mineral fertilizer and farmyard manure. Dates 1, 2, 3 refer to 25th March, 22th April, 6th
May in 2020 for WW 2020, and 14th March, 26th March, 5/6th May 2021 for WR 2021, respectively.

iew
Crop
Class
WW 2020 WR 2021
unfertilized 264 264
_PKCa 240 240

ev
N_KCa 264 264
NP_Ca 264 264
NPK_ 252 252
NPKCa 264 264
NPKCa+m+s 252 252

r
Date WW 2020
er WR 2021
Date 1 600 600
Date 2 600 600
Date 3 600 600
pe
Total 1800 1800

The images were annotated with the sampling date, strip and treatment (unfertilized,
_PKCa, N_KCa, NP_Ca, NPK_, NPKCa, NPKCa+m+s) as well as a random number and
the quadrant (file name: date_strip_treatment_RandomNumber_quadrant). For example,
ot

in WW 2020, there are 150 IDs. Each ID is divided into 4 quadrants over 3 dates, resulting
in 150 × 4 × 3 = 1800 images (Table 2). Example images are shown in Fig. 2.
tn

2.4. Baseline Model Training

The dataset was used to systematically evaluate how accurately recent Convolutional
Neural Networks (CNNs) like [24] and Vision Transformers (ViTs) like [30] recognize each fer-
tilizer treatment (unfertilized, _PKCa, N_KCa, NP_Ca, NPK_, NPKCa, NPKCa+m+s)
and can be transferred across development stages, crops and years. For that, the architec-
rin

tures were (1) trained and tested based on the dataset of winter wheat 2020, (2) trained and
tested based on the dataset of winter rye 2021, and (3) transfer learning was conducted, i.e.,
from WW2020 to WR2021 and from WR2021 to WW2020 for a cross-species and cross-year
validation.
ep

2.4.1. Convolutional Network Architectures

CNNs are based on the convolutional operation, which is a mathematical operation that
applies a filter to an input image to produce a feature map. This operation is combined
with other non-linear functions and pooling layers and repeated multiple times. Each layer
Pr

in the network learns more complex features from the input.

6 August 5, 2023

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4549653
 ed
iew
(a) Field

r ev
er
(b) Field with squares (RGB images)
pe
ot
tn

(c) Squares with IDs and coordinates (d) Four quadrants of each square

Figure 1: Aerial views of the long-term fertilizer experiment (LTFE) Dikopshof in 2020 and scheme of data
splitting.
rin

In this work, several CNN architectures are evaluated with the proposed dataset including
AlexNet [22], VGG [23], ResNet [24], DenseNet [25], ConvNeXt [26] and EfficientNet [27].
AlexNet [22] has been a pioneering network for image classification and it is composed of
five convolutional layers, three pooling layers, and three fully connected layers. VGG [23]
ep

is a simple yet powerful architecture that uses small filters in each layer, making it deeper
than AlexNet. ResNet [24] has become the standard architecture for many tasks including
image classification. It uses skip connections to overcome the vanishing gradient problem,
which occurs when gradients become too small to affect the weights during backpropagation.
Pr

DenseNet [25] is a variant of ResNet, where each layer is connected to every other layer in

7 August 5, 2023

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4549653
 ed
iew
ev
(a) Winter Wheat 2020 (WW2020)

r
er
pe
ot

(b) Winter Rye 2021 (WR2021)

tn

Figure 2: Example images. Column 1-7: unfertilized, _PKCa, N_KCa, NP_Ca, NPK_, NPKCa,
NPKCa+m+s; row 1-3: at tillering (mid March), tillering/ stem elongation (end of March/April), booting
(mid of May).
rin

a feed-forward manner. ConvNeXt [26] is another variant of ResNet, but its design is
inspired by Vision Transformers. EfficientNet [27] is a resource-efficient network and utilizes
a compound scaling method that carefully balances the depth, width, and resolution of the
network.
ep

2.4.2. Transformer Architectures

ViTs use the self-attention mechanism, which allows the model to attend to different parts
of the input image while processing it. The self-attention mechanism was first introduced in
Pr

the Transformer architecture [28], which was primarily used for natural language processing.

8 August 5, 2023

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4549653
 ed
Training set (4) Finetune Pre-trained Model

unfertilized

iew
(3) Data Augmentation _PKCa

Feature Extractor

Feature Extractor
(2) Dataset Splitting N_KCa

FC
FC
NP_Ca
NPK_
NPKCa
(1) Data Acquisition NPKCa+m+s
& Labelling
Prediction: NPK_

Test set (5) Inference

ev
Figure 3: The proposed evaluation setup for nutrient deficiency detection in winter wheat and winter rye.

ViTs adapt this mechanism to images by dividing the input image into patches, which

r
are then fed into the transformer network. The architecture of a ViT consists of multiple
transformer blocks, and the output of the final block is fed into a classification head. Typical
er
ViT models include ViT [29] and Swin Transformer [30]. ViT is the original ViT model,
and it uses a simple transformer architecture with a pre-trained token embedding. Swin
Transformer is a recent architecture that uses a hierarchical transformer, where the input of
each swin transformer block is divided into multiple non-overlapping windows, and the self-
pe
attention mechanism is applied within each window, where each window will be shifted in
consecutive blocks. Swin Transformer V2 [47] is a variation that can be trained on images
with higher input sizes (up to 1536 x 1536) and can be scaled up to 3 billion parameters
by introducing techniques of post-normalization, scaled cosine attention, and log-spaced
continuous position bias. In this paper, Swin Transformer and its variation V2 are evaluated.
ot

2.4.3. Domain Adaptation

Domain adaptation methods aim to adapt a model trained on a source domain with
abundant labeled data to perform well on a target domain without annotated data. The
tn

challenge arises because there is a significant distribution shift of features between the two
domains despite the same feature space, leading to a dramatic drop in performance when
directly applying the source-only-trained model on the target domain. To bridge this gap,
domain adaptation methods try to learn domain-invariant representations to improve the
rin

generalization and performance of the model in the target domain. In this work, we have
applied different baseline methods of domain adaptation [48–55] for cross-species and cross-
year transfer.
ep

2.4.4. Model Training Protocol

The protocol for evaluating the architectures for nutrient deficiency detection is illus-
trated in Figure 3. Using the training set, we apply standard data augmentation and train
the network. The trained network is then applied to the test set and the prediction accuracy
is measured. We report the results when the networks are randomly initialized, i.e., trained
Pr

9 August 5, 2023

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4549653
 ed
from scratch, and when the networks have been pre-trained on a different dataset. In this
case, the pre-trained network is used for initialization.
If not specifically pointed out, the default hyper-parameters in our experiments are as
follows: The original image was resized to 896×896 and normalized with a mean value

iew
of [0.485, 0.456, 0.406] and a standard deviation of [0.229, 0.224, 0.225] calculated from
ImageNet [56]. For data augmentation, the images were randomly flipped horizontally.
The purpose of data augmentation is to increase the variability of the training images in a
dataset, which helps the trained model to generalize better to images that were obtained
from different scenarios.
If the networks were trained from scratch, the parameters of the neural networks were

ev
randomly initialized using the Xavier algorithm [57], otherwise, the models are pre-trained
on ImageNet. Each model is trained for 50 epochs with a batch size of 4. Stochastic
gradient descent (SGD) was used with an initial learning rate of 10−3 , where the momentum
and weight decay were set as 0.9 and 10−2 , respectively. The learning rate was reduced by a

r
factor of 0.1 when no improvement was observed for three epochs, and the training process
stopped early when the learning rate was smaller than 10−7 .
er
By default, all images of WW2020 or WR2021 are used for evaluation where 75% of
the data from each class was chosen as the training set and the rest as the test set. The
sampling is performed based on random IDs (Figure 1). To maintain the class balance, 75%
of the IDs are sampled from each class (each ID is associated with one class, and each class
pe
has around 22 IDs) for training and the rest for testing.

2.4.5. Pre-training
A network can either be trained from scratch or pre-trained on a large-scale dataset
like ImageNet [56]. To validate whether pre-training the models on agricultural datasets
ot

to customize an agriculture-oriented prior knowledge would help, we initialize the models

with weights learned from related datasets, including DND-Diko-B (Barley), DND-Diko-SB
(Sugar Beet) [5], and DND-CKA-B (Barley). DND-Diko-SB and DND-Diko-B have been
collected as DND-Diko-WWWR at the long-term fertilizer experiment (LFTE) Dikopshof,
tn

and contain 5648 and 5210 RGB images, respectively. While the images in the DND-Diko-
WWWR dataset were captured by a UAV, the images in DND-Diko-SB and DND-Diko-B
were captured by mobile phones. The DND-CKA-B dataset contains 9300 RGB images
and was collected at Campus Klein-Altendorf (CKA) near Bonn with mobile phones. It
rin

contains six nutrient treatments, including N deficiency (_PKCa), P deficiency (N_KCa), K

deficiency (NP_Ca), B deficiency, S deficiency and NPKCa, where the nutrient treatment of
_PKCa, N_KCa, NP_Ca and NPKCa are the same as in the DND-Diko-WWWR dataset.
Figure 4 shows example images of the agricultural datasets for pre-training, as well as
vegetation images of the ImageNet dataset. We also evaluate the impact of pre-training on
ep

WR2021 and finetuning on WW2020.

2.4.6. Evaluation
For the ground-truth data, a one-way analysis of variance (ANOVA) was conducted with
Pr

the crops’ shoot biomass and plant height to compare the means of different treatments.
10 August 5, 2023

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4549653
 ed
iew
ev
(a) WR2021 (b) DND-Diko-SB (c) DND-Diko-B

r
er
pe

(d) DND-CKA-B (e) ImageNet-Greenhouse1 (f) ImageNet-Greenhouse2

Figure 4: Example images of: [a-d] related agricultural datasets; [e-f] vegetation images in ImageNet.
ot

Multiple comparisons between treatments were performed using Tukey’s test, and means
with the same letter are considered not significantly different (Tukey test, P>0.05). For the
tn

networks, the top-1 accuracy metric on the test set was reported, which denotes whether
the predicted category with the highest confidence matches the ground truth category.

3. Results
rin

In this section, the experimental results for analyzing the ground-truth data and for eval-
uating the networks are provided. First, the soil nutrient analysis results and the crop growth
variables are presented in Section 3.1. In Section 3.2, various CNN-based and transformer-
based models are evaluated. The influence of pre-training is further explored in Section 3.3.
ep

In Section 3.4, the challenge of transferability across years and crop species is highlighted
for encouraging further research.

3.1. Winter Wheat and Winter Rye Growth and Soil Nutrient Status
The growth period 2019/20 was warmer compared with the growth period of 2020/21.
Pr

The rainfall during the period 2019/20 was scarce, even in the months when it is usually
11 August 5, 2023

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4549653
 ed
Table 3: Soil pH, soil mineral nitrogen content (Nmin ), and plant-available phosphorus (Pcal ) and potas-
sium (Kcal ) in three soil layers during the winter wheat (May 19th 2020) and winter rye (May 25th 2021)
campaigns.

iew
WW2020
Treatment Layer NPKCa+m+s NPKCa _PKCa N_KCa NP_Ca NPK_ UF
pH 0-30 6.6 6.1 6.7 6.1 6.1 5.6 6.1
30-60 6.6 6.2 6.7 5.9 6.2 5.5 6
60-90 7.1 6.9 6.9 6.7 6.6 6.3 6.2
Nmin (kg ha−1 ) 0-30 6.7 3.6 2.6 3.9 2.3 6.5 2.9
30-60 1.2 3.5 1 3.2 0.8 2.6 2.3
60-90 0.7 1.9 0.3 1.5 0.3 1.5 1.8
Kcal (mg kg−1 ) 0-30 283 114 126 110 31 125 38

ev
30-60 202 79 82 89 29 75 39
60-90 72 57 44 41 46 47 42
Pcal (mg kg−1 ) 0-30 205 75 76 15 74 46 31
30-60 140 46 36 12 46 44 22
60-90 11 11 9 4 8 12 10
WR2021

r
Treatment Layer NPKCa+m+s NPKCa _PKCa N_KCa NP_Ca NPK_ UF
pH 0-30 6.4 6.7 6.6 6.4 5.3 5.5
30-60 6.5 6.7er 6.7 6.5 6.6 5.5 5.7
60-90 6.8 7.0 6.7 6.7 6.5 6.1 5.9
Nmin (kg ha−1 ) 0-30 10.4 5.8 1.8 4.9 4.3 3.9
30-60 4.7 3.1 3.0 4.3 3.2 2.9 2.8
60-90 1.6 2.1 3.1 2.9 1.9 3.1 2.3
Kcal (mg kg−1 ) 0-30 376 141 178 104 97 40
pe
30-60 251 88 112 66 32 64 43
60-90 105 44 50 41 39 44 42
Pcal (mg kg−1 ) 0-30 226 116 115 27 63 23
30-60 112 64 71 21 60 38 12
60-90 18 9 14 15 18 10 8

concentrated (July). In the growth period 2020/21, amounts of rainfall were low in May and
ot

June but high in July, and mean temperatures were lower as compared to 2019/20. The soil
nutrient status was clearly affected by the fertilizer treatments, as shown in Table 3.
The biomass and plant height were assessed during the early-flowering stage and are
tn

reported in Table 4. For both periods and crops (WW2020 and WR2021), the NPKCa+m+s
treatment consistently showed the highest biomass and plant height, while the unfertilized
and the _PKCa treatments exhibited the lowest values. In WW2020, the NPKCa, N_KCa,
NP_Ca, and NPK_ treatments displayed similar levels of shoot biomass and plant height,
while the _PKCa treatment had notably lower values for these parameters. Moving to WR
rin

2021, both NP_Ca and NPK_ treatments developed comparable values, ranking as the
second-highest in shoot biomass. They were followed by NPKCa and N_KCa treatments,
which had relatively lower values. Finally, the _PKCa and unfertilized treatments achieved
the significantly lowest biomass levels among all the treatments.
ep

3.2. Evaluation of Networks for Nutrient Deficiency Detection

CNN-based and transformer-based architectures were analyzed and compared in terms of
accuracy, number of parameters, and floating point operations (FLOPs). For each network
Pr

architecture, we performed five experimental trials with the same hyper-parameter configu-

12 August 5, 2023

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4549653
 ed
Table 4: Dry matter shoot biomass (t ha−1 ) and plant height (cm) (mean ± standard deviation) of winter
wheat harvested in 2020 (WW2020) and of winter rye harvested in 2021 (WR2021). Different letters indicate
significant differences (Anova, α = 0.05, Tukey-test).

WW2020

iew
Sampling Date Treatment Dry biomass (t ha−1 ) Plant height (cm)
NPKCa+m+s 5.11 ± 0.89 a 54 ± 4.3 a
NPKCa 4.40 ± 0.80 ab 46 ± 1.4 b
_PKCa 3.51 ± 0.54 b 37 ± 0.9 c
19-May-20 N_KCa 4.83 ± 0.42 ab 46 ± 3.4 b
NP_Ca 4.38 ± 0.81 ab 47 ± 1.1 b

ev
NPK_ 4.91 ± 0.87 ab 49 ± 2.8 ab
unfertilized 1.91 ± 0.22 c 35 ± 2.8 c
WR2021
Sampling Date Treatment Dry biomass (t ha−1 ) Plant height (cm)

r
NPKCa+m+s 8.53 ± 0.58 a 133 ± 4.8 a
NPKCa 5.09 ± 0.43 b 109 ± 1.7 b
_PKCa 2.43 ± 0.40
er c 85 ± 7.2 c
25-May-21 N_KCa 5.69 ± 0.60 b 110 ± 3.2 b
NP_Ca 5.58 ± 0.61 ab 113 ± 1.0 b
NPK_ 6.16 ± 0.11 ab 115 ± 3.5 b
unfertilized 1.49 ± 0.19 c 80 ± 2.7 c
pe

ration where the only difference was the random seed for training. We report the average
over five trials in Table 5 for WW2020 and in Table 6 for WR2021. For WW2020, Swin
Transformer (V2) and ResNet-50 outperform the other models by a large margin, as shown
in Table 5. The best accuracy is achieved by Swin Transformer V2 with an accuracy of
ot

83.5%. ResNet-50 achieves 80.8%, but it requires much fewer parameters and FLOPs. For
WR2021, DenseNet-161 and Swin Transformer V2 perform best with an accuracy of 87.0%
and 86.3%, respectively.
tn

To understand how well each nutrient deficiency was classified, the per-class accuracy is
reported in Table 5 for WW2020 and in Table 6 for WR2021. The class of unfertilized and
NPKCa+m+s were the easiest nutrient deficiencies (or fertilizer treatments) to be recog-
nized, which is reasonable as the crop growth and height is lowest for unfertilized and highest
rin

for NPKCa+m+s over all growth stages (Table 4). On the contrary, P deficiency (N_KCa)
and mineral fertilization at a rather low level (NPKCa) are the most difficult to be recog-
nized. The accuracy of Swin Transformer V2 was higher for all treatments except for the
N omission treatment _PKCa for WW2020 as compared to ResNet-50, and was similar or
slightly worse for all treatments for WR2021 as compared to DenseNet-161. The confusion
ep

matrices of Swin Transformer V2 for WW2020 and WR2021 are shown in Figure 5.
3.3. Pre-training
We analyzed the impact of pre-training in Table 7. As expected, pre-training on any
dataset performs better than training from scratch. However, pre-training on nutrient de-
Pr

ficiency detection datasets brings no benefits compared to pre-training on ImageNet since

13 August 5, 2023

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4549653
 ed
Table 5: Results for different network architectures on test data of WW2020. The networks were trained on
the training data of WW2020. The reported accuracy refers to the top-1 accuracy metric on the test set.
PM denotes the number of parameters and UF denoted unfertilized.

iew
Model PM FLOPs UF _PKCa N_KCa NP_Ca NPK_ NPKCa NPKCa+m+s AVG
(M) (G) (%) (%) (%) (%) (%) (%) (%) (%)
random - - 14.3 14.3 14.3 14.3 14.3 14.3 14.3 14.3
AlexNet 61.1 11.5 94 65 33 68 45 31 98 61.5
VGG-16 138.4 245.7 96 77 49 76 73 44 100 72.9
ResNet-50 25.6 66.3 93 90 60 86 78 62 100 80.8
DenseNet-161 28.7 126.0 93 92 74 86 78 35 100 78.8
ConvNeXt 50.2 139.3 94 77 61 61 77 46 100 72.9

ev
EfficientNet 21.5 46.6 97 87 53 86 67 35 100 74.2
Swin Transformer 49.6 140.3 96 87 50 78 87 72 100 80.6
Swin Transformer V2 49.7 142.6 94 82 72 88 85 67 100 83.5

Table 6: Results for different network architectures on test data of WR2021. The networks were trained on

r
the training data of WW2021. The reported accuracy refers to the top-1 accuracy metric on the test set.
PM denotes the number of parameters and UF denoted unfertilized.

Model PM
(M)
FLOPs
(G)
UF
(%) (%)
er
_PKCa N_KCa
(%)
NP_Ca
(%)
NPK_
(%)
NPKCa
(%)
NPKCa+m+s AVG
(%) (%)
random - - 14.3 14.3 14.3 14.3 14.3 14.3 14.3 14.3
AlexNet 61.1 11.5 88 75 49 43 54 50 94 64.7
pe
VGG-16 138.4 245.7 90 83 62 72 92 69 100 80.6
ResNet-50 25.6 66.3 89 78 68 78 90 69 100 81.2
DenseNet-161 28.7 126.0 92 88 79 83 100 71 100 87.0
ConvNeXt 50.2 139.3 89 70 62 79 100 79 100 82.3
EfficientNet 21.5 46.6 90 87 60 79 98 61 100 81.2
Swin Transformer 49.6 140.3 92 85 58 81 98 78 100 83.8
Swin Transformer V2 49.7 142.6 92 85 74 86 97 75 100 86.3
ot

the datasets are still small. It is interesting to see that the model pre-trained from the
most related dataset WR2021 performs worst due to the smallest amount of images. Over-
all the accuracy increased as the number of images in the pre-trained datasets increased.
tn

For example, enlarging the scale of DND-CKA-B from 5200 images to 9300 images results
in a marginal improvement of accuracy from 61.0% to 75.4%, and the model pre-trained
with ImageNet with more than 1.28 million images performs best (81.2%) due to its large
amount of images despite not being related to nutrient deficiency detection. When the scale
rin

of datasets is the same, e.g. DND-Diko-B and DND-CKA-B with 5200 images, the former
has a superior result to the latter because it was collected under similar conditions (seven
nutrient treatments in the field at Dikopshof), while the latter was collected in mini plots
at another experimental site with only four shared classes (_PKCa, N_KCa, NP_Ca and
NPKCa).
ep

3.4. Transfer Across Years and Crop Species

We also investigated how well the networks generalize across year and crop cultivar. To
this end, we trained the networks on the training set of WW2020 and tested on the test
Pr

set of WR2021 (WW2020 → WR2021) and, vice versa, the networks were trained on the
14 August 5, 2023

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4549653
 ed
iew
ev
(a) WW2020 (b) WR2021

Figure 5: Confusion matrices of SwinTransformer V2 on (a) WW2020 and (b) WR2021 (x-axis: predicted
treatment, y-axis: real treatment.

r
Table 7: Results for different pre-training datasets using ResNet50 on WW2020.

Dataset
From Scratch
WR2021
-
1.8
-
7
er
#Images (k) #Class #Shared Class
-
7
Device

UAV
-
Location
-
Field
Year Accuracy (%)
-
2021
54.9
62.8
DND-Diko-SB (Sugar Beet) 5.6 7 7 Mobile Phone Field 2019 63.5
DND-Diko-B (Barley) 5.2 7 7 Mobile Phone Field 2021 71.6
pe
DND-CKA-B (Barley) 5.2 6 4 Mobile Phone outside Mini-Plots 2022 61.0
DND-CKA-B (Barley) 9.3 6 4 Mobile Phone outside Mini-Plots 2022 75.4
ImageNet 1281 1000 0 - - - 81.2

training set of WR2021 and tested on the test set of WW2020 (WR2021 → WW2020). The
results are reported in Table 8) and Table 9, respectively. As shown in Table 8, the average
ot

transferred accuracy without domain adaptation on WR2021 was around 20% while the
per-nutrient accuracy varied from 0% to 63%. For all networks, the accuracy is much lower
compared to training and testing on WR2021. In particular, NPKCa is not recognized at all.
tn

The same holds for WR2021 → WW2020 reported in Table 9. In this setup, unfertilized,
_PKCa, N_KCa and NPKCa are not recognized.
To bridge differences between training and test data, domain adaptation approaches,
which use an annotated training dataset and transfer the learned knowledge to an unseen
dataset that is not annotated, can be applied [5]. We thus investigated several methods for
rin

domain adaptation [48], including DAN [49], DANN [50], ADDA [51], CDAN [52], BSP [53],
AFN [54] and ERM [55]. Although DenseNet-161 has the highest transferred accuracy
without domain adaptation, we use ResNet-50 for evaluation since it performs only slightly
worse but has fewer parameters and FLOPs, as reported in Tables 5 6. As shown in Table 8
ep

and Table 9, the approaches improve the accuracy of ResNet-50. BSP [53] achieves the best
results in both transfer settings. The best accuracy with domain adaptation is 31.2% for
WW2020 → WR2021 and 28.6% for WR2021 → WW2020.
Pr

15 August 5, 2023

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4549653
 ed
Table 8: Results across year and crop. The networks are trained on the training set of WW2020 and tested
on the test set of WR2021 (WW2020 → WR2021). In the upper part, results without domain adaptation
are reported. In brackets, we report the results for training on the training set of WR2021 and testing on

iew
the test set of WR2021. In the lower part, results for ResNet-50 with various domain adaptation methods
are reported.

WW2020 → WR2021
Model
unfertilized _PKCa N_KCa NP_Ca NPK_ NPKCa NPKCa+m+s AVG
AlexNet 61 (88) 50 (75) 0 (49) 0 (43) 18 (54) 0 (50) 10 (94) 19.4 (64.7)
w/o domain adapt.

VGG-16 8 (90) 61 (83) 19 (62) 0 (72) 15 (92) 0 (69) 32 (100) 19.2 (80.6)

ev
ResNet-50 57 (89) 20 (78) 44 (68) 11 (78) 7 (90) 0 (69) 20 (100) 23.3 (81.2)
DenseNet-161 38 (92) 32 (88) 22 (79) 33 (83) 30 (100) 0 (71) 25 (100) 25.4 (87.0)
ConvNeXt 43 (89) 25 (70) 15 (62) 17 (79) 18 (100) 0 (79) 52 (100) 23.7 (82.3)
EfficientNet 68 (90) 58 (87) 0 (60) 8 (79) 0 (98) 0 (61) 33 (100) 23.5 (81.2)
Swin Transformer 24 (92) 13 (85) 21 (58) 10 (81) 3 (98) 10 (78) 62 (100) 19.9 (83.8)
Swin Transformer V2 36 (92) 0 (85) 22 (74) 31 (86) 0 (97) 0 (75) 63 (100) 21.8 (86.3)

r
DAN 88 27 22 0 0 0 33 24.6
w domain adapt.

DANN 31 53 13 40 17 28 42 30.4
ADDA 41 31 5 er 30 7 36 45 28.0
CDAN 29 38 10 40 52 18 33 30.8
BSP 32 30 7 33 49 30 36 31.2
AFN 8 2 40 13 23 24 33 20.5
ERM 72 32 0 3 42 0 18 23.3
pe

Table 9: Results across year and crop. The networks are trained on the training set of WR2021 and tested
on the test set of WW2020 (WR2021 → WW2020). In the upper part, results without domain adaptation
are reported. In brackets, we report the results for training on the training set of WW2020 and testing on
the test set of WW2020. In the lower part, results for ResNet-50 with various domain adaptation methods
ot

are reported.

WR2021 → WW2020
tn

Model
unfertilized _PKCa N_KCa NP_Ca NPK_ NPKCa NPKCa+m+s AVG
AlexNet 10 (94) 0 (65) 0 (33) 4 (68) 15 (45) 0 (31) 98 (98) 16.7 (61.5)
w/o domain adapt.

VGG-16 21 (96) 0 (77) 1 (49) 26 (76) 17 (73) 4 (44) 55 (100) 17.3 (72.9)
ResNet-50 0 (93) 0 (90) 0 (60) 14 (86) 2 (78) 7 (62) 93 (100) 15.4 (80.8)
DenseNet-161 0 (93) 0 (92) 0 (74) 28 (86) 18 (78) 0 (35) 90 (100) 18.2 (78.8)
rin

ConvNeXt 1 (94) 0 (77) 0 (61) 11 (61) 15 (77) 0 (46) 93 (100) 15.8 (72.9)
EfficientNet 1 (97) 0 (87) 0 (53) 4 (86) 13 (67) 0 (35) 98 (100) 15.2 (74.2)
Swin Transformer 7 (96) 2 (87) 1 (50) 61 (78) 32 (87) 0 (72) 2 (100) 15.2 (80.6)
Swin Transformer V2 0 (94) 0 (82) 7 (72) 57 (88) 33 (85) 1 (67) 17 (100) 16.5 (83.5)
DAN (ResNet-50) 68 7 36 0 0 0 58 24.4
w domain adapt.

DANN (ResNet-50) 4 10 0 43 7 38 63 23.3

ep

ADDA (ResNet-50) 0 52 7 61 30 4 38 26.5

CDAN (ResNet-50) 24 8 21 35 8 6 50 21.6
BSP (ResNet-50) 43 33 19 36 18 11 40 28.6
AFN (ResNet-50) 47 0 26 10 8 0 33 18.2
ERM (ResNet-50) 1 0 0 47 54 0 65 24.1
Pr

16 August 5, 2023

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4549653
 ed
4. Discussion
4.1. Accuracy of Recognition of Fertilizer Level
The observed shoot biomass and plant height differed significantly between NPKCa+m+s

iew
(highest values) and treatments _PKCa and unfertilized (lowest values) but not between
NPKCa, N_PKCa, NP_KCa, and NPK_. The rather limited effects of P, K, and lime
omission on observed shoot variables explain the performance of the network architectures:
The detection accuracy was highest for NPKCa+m+s and _PKCa and unfertilized treat-
ments. P deficiency (N_KCa) and NPKCa are the most difficult to be recognized, which is
similar to the results reported in [5]. This is in line with [45] who reported that the effect of

ev
fertilizer omission on mean yield loss due to P omission (N_KCa) was low for winter wheat
and winter rye (7 - 8%) in a LTFE (mean from 1989 to 2018).
[14] reported deficiency classification accuracies for various crops ranging from 40 to
99.8%. With 75% (winter wheat) and 81% (winter rye), the mean detection accuracy is in

r
the range of [15] with an accuracy of 80 to 94% in basil plants (hydroponic system) and
[5] with an accuracy of 63% (mean value, maximum: 98,4%) for field-grown sugar beet, all
with training and testing on same crop-year combination. Our cross-year cross-cereal species
er
model performance was about 18%. [40] tested the model robustness in different soybean
trials, field locations, and vegetative growth stages. In general, correlations between the
model-predicted scores and the visual scores varied widely (R2 of 0 to 88) depending on the
pe
model and training/test setup. With 92-93%, the cross-year model performance presented
in [41] was much higher than our cross-year cross-crop model performance.

4.2. Network Choice and Pre-training

In contrast to images taken from the ground as in [5], images captured by a UAV contain
ot

less fine details. As fine details are captured by high frequencies and low frequencies focus
more on global structures in an image [58], transformer-based architectures perform better in
this task because they are more like low-pass filters. In comparison, CNN-based architectures
are high-pass filters where fine details of an input image can be captured [59]. However,
tn

transformer-based architectures are usually more resource-demanding, which is not suitable

for resource-limited platforms.
Pre-training from large-scale datasets like ImageNet is crucial for classifying nutrient
deficiencies as acquiring data in the field is very difficult and training data will always be
rin

a limiting factor. But as the overall performance increases with pre-training architectures
with more images, larger agricultural datasets might be beneficial to facilitate downstream
agricultural tasks including nutrient deficiency detection.
ep

5. Conclusions
Nutrient deficiencies impose significant limitations on global crop production, and even
commonly occurring deficiencies like K and P deficiencies are currently challenging to be
accurately diagnosed at early stages using non-invasive methods. In this paper, we intro-
Pr

duced a novel dataset comprising RGB images captured by a UAV from winter wheat and
17 August 5, 2023

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4549653
 ed
winter rye subjected to seven distinct nutrient treatments, offering a unique benchmark un-
der field conditions. We evaluated the performance of CNN-based and Transformer-based
architectures and highlighted the difficulties of generalizing across years and crops. Overall,
transferring the learned knowledge of networks across years and/or crops remains challeng-

iew
ing, and developing domain adaptation methods for this task on the DND-Diko-WWWR
dataset is an interesting direction for further research. Another future research direction
is the investigation of whether large-scale agricultural datasets provide a better source for
pre-training than commonly used image classification datasets.

ev
Funding
The presented study has partially been funded by the Deutsche Forschungsgemein-
schaft (DFG, German Research Foundation) under Germany’s Excellence Strategy—EXC
2070–390732324 (PhenoRob), by the German Federal Ministry of Education and Research

r
(BMBF) in the framework of the funding measure ‘Soil as a Sustainable Resource for the
Bioeconomy—BonaRes’, project BonaRes (Module A): BonaRes Center for Soil Research,
er
subproject ‘Sustainable Subsoil Management— Soil3’ (grant 031B0151A), and the Federal
Ministry of Food and Agriculture (BMEL) based on a decision of the Parliament of the
Federal Republic of Germany via the Federal Office for Agriculture and Food (BLE) (grant
number 2822ABS010).
pe
Acknowledgements
The authors thank all technicians and other assisting staff members (especially Dr. Hu-
bert Hüging and Gunther Krauss) who have been running the LTFE experiment for decades.
Moreover, the authors thank the BonaRes Data Center for the financial support to prepare
ot

a part of the data.

tn

References
[1] P. Heinemann, U. Schmidhalter, Simplifying residual nitrogen (nmin) sampling strategies and crop
response, European Journal of Agronomy 130 (2021) 126369.
[2] U. Buczko, M. van Laak, B. Eichler-Löbermann, W. Gans, I. Merbach, K. Panten, E. Peiter, T. Reitz,
H. Spiegel, S. von Tucher, Re-evaluation of the yield response to phosphorus fertilization based on
rin

meta-analyses of long-term field experiments, Ambio 47 (2018) 50–61.

[3] R. O. Kuchenbuch, U. Buczko, Re-visiting potassium-and phosphate-fertilizer responses in field ex-
periments and soil-test interpretations by means of data mining, Journal of Plant Nutrition and Soil
Science 174 (2) (2011) 171–185.
[4] A. Berti, A. Dalla Marta, M. Mazzoncini, F. Tei, An overview on long-term agro-ecosystem experiments:
ep

Present situation and future potential, European Journal of Agronomy 77 (2016) 236–241.
[5] J. Yi, L. Krusenbaum, P. Unger, H. Hüging, S. J. Seidel, G. Schaaf, J. Gall, Deep learning for non-
invasive diagnosis of nutrient deficiencies in sugar beet using rgb images, Sensors 20 (20) (2020) 5893.
[6] J. Schröder, J. Neeteson, O. Oenema, P. Struik, Does the crop or the soil indicate how to save nitrogen
in maize production?: Reviewing the state of the art, Field Crops Research 66 (2) (2000) 151–164.
[7] C. Ravier, M. Quemada, M.-H. Jeuffroy, Use of a chlorophyll meter to assess nitrogen nutrition index
Pr

during the growth cycle in winter wheat, Field Crops Research 214 (2017) 73–82.

18 August 5, 2023

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4549653
 ed
[8] Y. Cai, K. Guan, E. Nafziger, G. Chowdhary, B. Peng, Z. Jin, S. Wang, S. Wang, Detecting in-season
crop nitrogen stress of corn for field trials using uav-and cubesat-based multispectral sensing, IEEE
Journal of Selected Topics in Applied Earth Observations and Remote Sensing 12 (12) (2019) 5153–
5166.

iew
[9] G. Fitzgerald, D. Rodriguez, G. O’Leary, Measuring and predicting canopy nitrogen nutrition in wheat
using a spectral index—the canopy chlorophyll content index (ccci), Field crops research 116 (3) (2010)
318–324.
[10] L. Prey, U. Schmidhalter, Sensitivity of vegetation indices for estimating vegetative n status in winter
wheat, Sensors 19 (17) (2019) 3712.
[11] M. Ali, A. Al-Ani, D. Eamus, D. K. Tan, Leaf nitrogen determination using non-destructive techniques–
a review, Journal of Plant Nutrition 40 (7) (2017) 928–953.
[12] L. Sanchez, A. Ermolenkov, S. Biswas, E. M. Septiningsih, D. Kurouski, Raman spectroscopy enables

ev
non-invasive and confirmatory diagnostics of salinity stresses, nitrogen, phosphorus, and potassium
deficiencies in rice, Frontiers in Plant Science 11 (2020) 573321.
[13] G. Ruan, U. Schmidhalter, F. Yuan, D. Cammarano, X. Liu, Y. Tian, Y. Zhu, W. Cao, Q. Cao,
Exploring the transferability of wheat nitrogen status estimation with multisource data and evolutionary
algorithm-deep learning (ea-dl) framework, European Journal of Agronomy 143 (2023) 126727.

r
[14] M. Sharma, K. Nath, R. K. Sharma, C. J. Kumar, A. Chaudhary, Ensemble averaging of transfer
learning models for identification of nutritional deficiency in rice plant, Electronics 11 (1) (2022) 148.
er
[15] Z. Gul, S. Bora, Exploiting pre-trained convolutional neural networks for the detection of nutrient
deficiencies in hydroponic basil, Sensors 23 (12) (2023) 5407.
[16] S. Sladojevic, M. Arsenovic, A. Anderla, D. Culibrk, D. Stefanovic, Deep neural networks based recog-
nition of plant diseases by leaf image classification, Computational Intelligence and Neuroscience 2016
(2016).
pe
[17] L. Hallau, M. Neumann, B. Klatt, B. Kleinhenz, T. Klein, C. Kuhn, M. Röhrig, C. Bauckhage, K. Ker-
sting, A.-K. Mahlein, U. Steiner, E.-C. Oerke, Automated identification of sugar beet diseases using
smartphones, Plant Pathology 67 (2) (2018) 399–410.
[18] A. Milioto, P. Lottes, C. Stachniss, Real-time blob-wise sugar beets vs weeds classification for monitor-
ing fields using convolutional neural networks, ISPRS Annals of the Photogrammetry, Remote Sensing
and Spatial Information Sciences 4 (2017) 41.
ot

[19] H. K. Suh, J. Ijsselmuiden, J. W. Hofstee, E. J. van Henten, Transfer learning for the classification of
sugar beet and volunteer potato under field conditions, Biosystems Engineering 174 (2018) 50–65.
[20] A. Voulodimos, N. Doulamis, A. Doulamis, E. Protopapadakis, Deep learning for computer vision: A
brief review, Computational Intelligence and Neuroscience (2018).
tn

[21] Y. Guo, Y. Liu, A. Oerlemans, S. Lao, S. Wu, M. S. Lew, Deep learning for visual understanding: A
review, Neurocomputing 187 (2016) 27–48.
[22] A. Krizhevsky, I. Sutskever, G. E. Hinton, Imagenet classification with deep convolutional neural net-
works, in: Advances in Neural Information Processing Systems, 2012, pp. 1097–1105.
[23] K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition, arXiv
preprint arXiv:1409.1556 (2014).
rin

[24] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the
IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778.
[25] G. Huang, Z. Liu, L. Van Der Maaten, K. Q. Weinberger, Densely connected convolutional networks,
in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp.
4700–4708.
ep

[26] Z. Liu, H. Mao, C.-Y. Wu, C. Feichtenhofer, T. Darrell, S. Xie, A convnet for the 2020s, in: Proceedings
of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 11976–11986.
[27] M. Tan, Q. Le, Efficientnet: Rethinking model scaling for convolutional neural networks, in: Interna-
tional conference on machine learning, PMLR, 2019, pp. 6105–6114.
[28] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, I. Polosukhin,
Pr

Attention is all you need, Advances in neural information processing systems 30 (2017).

19 August 5, 2023

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4549653
 ed
[29] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani,
M. Minderer, G. Heigold, S. Gelly, et al., An image is worth 16x16 words: Transformers for image
recognition at scale, in: Proceedings of the International Conference on Learning Representations,
2021.

iew
[30] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, B. Guo, Swin transformer: Hierarchical
vision transformer using shifted windows, in: Proceedings of the IEEE/CVF international conference
on computer vision, 2021, pp. 10012–10022.
[31] Y. Cong, M. Y. Yang, B. Rosenhahn, Reltr: Relation transformer for scene graph generation, IEEE
Transactions on Pattern Analysis and Machine Intelligence (2023).
[32] Y. Cong, J. Yi, B. Rosenhahn, M. Y. Yang, Ssgvs: Semantic scene graph-to-video synthesis, in: Pro-
ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2023,
pp. 2554–2564.

ev
[33] S. J. Pan, Q. Yang, A survey on transfer learning, IEEE Transactions on knowledge and data engineering
22 (10) (2010) 1345–1359.
[34] A. Patel, P. Swaminarayan, M. Patel, Identification of nutrition’s deficiency in plant and prediction of
nutrition requirement using image processing, in: Proceedings of the Second International Conference
on Information Management and Machine Intelligence: ICIMMI 2020, Springer, 2021, pp. 463–469.

r
[35] T.-T. Tran, J.-W. Choi, T.-T. H. Le, J.-W. Kim, A comparative study of deep cnn in forecasting and
classifying the macronutrient deficiencies on development of tomato plant, Applied Sciences 9 (8) (2019)
er
1601.
[36] P. Jahagirdar, S. V. Budihal, Framework to detect npk deficiency in maize plants using cnn, in: Progress
in Advanced Computing and Intelligent Engineering: Proceedings of ICACIE 2019, Volume 2, Springer,
2021, pp. 366–376.
[37] P. K. Sethy, N. K. Barpanda, A. K. Rath, S. K. Behera, Nitrogen deficiency prediction of rice crop
pe
based on convolutional neural network, Journal of Ambient Intelligence and Humanized Computing 11
(2020) 5703–5711.
[38] L. A. Wulandhari, A. A. S. Gunawan, A. Qurania, P. Harsani, T. F. Tarawan, R. F. Hermawan, Plant
nutrient deficiency detection using deep convolutional neural network, ICIC Express Lett 13 (10) (2019)
971–977.
[39] Z. Xu, X. Guo, A. Zhu, X. He, X. Zhao, Y. Han, R. Subedi, Using deep convolutional neural networks
ot

for image-based diagnosis of nutrient deficiencies in rice, Computational Intelligence and Neuroscience
2020 (2020).
[40] J. Li, C. Oswald, G. L. Graef, Y. Shi, Improving model robustness for soybean iron deficiency chloro-
sis rating by unsupervised pre-training on unmanned aircraft system derived images, Computers and
tn

Electronics in Agriculture 175 (2020) 105557.

[41] A. Abdalla, H. Cen, L. Wan, K. Mehmood, Y. He, Nutrient status diagnosis of infield oilseed rape
via deep learning-enabled dynamic model, IEEE Transactions on Industrial Informatics 17 (6) (2020)
4379–4389.
[42] V. Rueda-Ayala, H. E. Ahrends, S. Siebert, T. Gaiser, H. Hüging, F. Ewert, Impact of nutrient supply
on the expression of genetic improvements of cereals and row crops–a case study using data from a
rin

long-term fertilization experiment in germany, European Journal of Agronomy 96 (2018) 34–46.

[43] S. Seidel, T. Gaiser, H. Ahrends, H. Hüging, S. Siebert, S. Bauke, M. Gocke, M. Koch, K. Schweitzer,
G. Schaaf, et al., Crop response to p fertilizer omission under a changing climate-experimental and
modeling results over 115 years of a long-term fertilizer experiment, Field Crops Research 268 (2021)
108174.
ep

[44] V. Rueda-Ayala, H. E. Ahrends, S. Siebert, T. Gaiser, H. Hüging, F. Ewert, Impact of nutrient supply
on the expression of genetic improvements of cereals and row crops–a case study using data from a
long-term fertilization experiment in germany, European Journal of Agronomy 96 (2018) 34–46.
[45] S. Seidel, T. Gaiser, H. Ahrends, H. Hueging, S. Siebert, S. Bauke, M. Gocke, M. Koch, K. Schweitzer,
G. Schaaf, et al., Crop response to p fertilizer omission under a changing climate-experimental and
Pr

modeling results over 115 years of a long-term fertilizer experiment, Field Crops Research 268 (2021)

20 August 5, 2023

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4549653
 ed
108174.
[46] H. Schüller, Die cal-methode, eine neue methode zur bestimmung des pflanzenverfügbaren phosphates
in böden, Zeitschrift für Pflanzenernährung und Bodenkunde 123 (1) (1969) 48–63.
[47] Z. Liu, H. Hu, Y. Lin, Z. Yao, Z. Xie, Y. Wei, J. Ning, Y. Cao, Z. Zhang, L. Dong, et al., Swin

iew
transformer v2: Scaling up capacity and resolution, in: Proceedings of the IEEE/CVF conference on
computer vision and pattern recognition, 2022, pp. 12009–12019.
[48] S. J. Pan, Q. Yang, A survey on transfer learning, IEEE Transactions on knowledge and data engineering
22 (10) (2009) 1345–1359.
[49] M. Long, Y. Cao, J. Wang, M. Jordan, Learning transferable features with deep adaptation networks,
in: International conference on machine learning, PMLR, 2015, pp. 97–105.
[50] Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, V. Lempit-
sky, Domain-adversarial training of neural networks, The journal of machine learning research 17 (1)

ev
(2016) 2096–2030.
[51] E. Tzeng, J. Hoffman, K. Saenko, T. Darrell, Adversarial discriminative domain adaptation, in: Pro-
ceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 7167–7176.
[52] M. Long, Z. Cao, J. Wang, M. I. Jordan, Conditional adversarial domain adaptation, Advances in
neural information processing systems 31 (2018).

r
[53] X. Chen, S. Wang, M. Long, J. Wang, Transferability vs. discriminability: Batch spectral penalization
for adversarial domain adaptation, in: International conference on machine learning, PMLR, 2019, pp.
er
1081–1090.
[54] R. Xu, G. Li, J. Yang, L. Lin, Larger norm more transferable: An adaptive feature norm approach
for unsupervised domain adaptation, in: Proceedings of the IEEE/CVF international conference on
computer vision, 2019, pp. 1426–1435.
[55] R. Vedantam, D. Lopez-Paz, D. J. Schwab, An empirical investigation of domain generalization with
pe
empirical risk minimizers, Advances in Neural Information Processing Systems 34 (2021) 28131–28143.
[56] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla,
M. Bernstein, et al., Imagenet large scale visual recognition challenge, International journal of computer
vision 115 (2015) 211–252.
[57] X. Glorot, Y. Bengio, Understanding the difficulty of training deep feedforward neural networks, in:
Proceedings of the thirteenth international conference on artificial intelligence and statistics, JMLR
ot

Workshop and Conference Proceedings, 2010, pp. 249–256.

[58] Z. Pan, J. Cai, B. Zhuang, Fast vision transformers with hilo attention, Advances in Neural Information
Processing Systems 35 (2022) 14541–14554.
[59] N. Park, S. Kim, How do vision transformers work?, in: Proceedings of the International Conference
tn

on Learning Representations, 2022.

rin
ep
Pr

21 August 5, 2023

This preprint research paper has not been peer reviewed. Electronic copy available at: https://ssrn.com/abstract=4549653