Automatic Smart Contract Generation Through LLMs:

When The Stochastic Parrot Fails⋆

Fadi Barbàra1,* , Emanuele A. Napoli2 , Valentina Gatteschi2 and Claudio Schifanella1
1
Department of Computer Science, University of Turin, Turin, Italy Via Pessinetto 12, 10149, Turin, Italy
2
Department of Computer and Control Engineering, Politecnico di Torino, Turin, Italy, Corso Duca degli Abruzzi 24, 10129, Turin,
Italy

Abstract
Blockchain and Artificial Intelligence (AI) have revolutionized the technology landscape, evolving predominantly
along parallel trajectories. This research investigates the feasibility of integrating these two domains, specifically
examining the potential of the latest advancements in Large Language Models (LLMs) to assist non-experts in
the development of production-ready Solidity smart contracts. Utilizing a lease agreement as a case study, we
employ GPT-4 to translate the document into smart contract code. To ensure consistency, we design several
distinct prompts with analogous objectives, each employed multiple times.
Our evaluation methodology encompasses both automated analysis and expert manual review. The findings
indicate a clear limitation: the current iteration of GPT-4 is incapable of generating production-ready smart
contracts, primarily due to undetected coding flaws and discrepancies between the prompts and the generated
code. This study underscores the challenges and limitations inherent in leveraging LLMs for the autonomous
generation of complex, real-world applicable smart contracts.

Keywords
Blockchain, Smart Contract, AI, LLM, GPT-4, propt engineering,

1. Introduction
In recent years, remarkable advancements in artificial intelligence (AI) have paralleled the evolution of
blockchain technology, with Large Language Models (LLMs) emerging as a significant development.
These models, including LLaMa 1 & 2 by Meta [1, 2], PaLM by Google/Alphabet [3], or Mistral by
MistralAI [4], have revolutionized problem-solving across diverse domains. Through processing and
generating human-like text, they have fundamentally altered our approach to everyday challenges,
demonstrating an advanced capacity to understand context, produce coherent responses, and adapt to
various tasks. Their versatility has rendered them indispensable across industries such as healthcare,
finance, and software engineering [5, 6, 7], signaling a new era of AI-driven innovation and problem-
solving.
Another disruptive technology that has taken hold in recent years is blockchain. In summary,
blockchain represents a decentralized database, shared by all peers in the network, in which so-called
transactions are kept track of. Each group of transactions, or “block”, is linked to the previous one
cryptographically to form a blockchain. Blockchain technology eliminates the need for intermediaries
while still guaranteeing the validity and integrity of the data, ensuring that it cannot be modified.
The fact that the database is distributed ensures transparency among participants and reduces the
risk of fraud or tampering. This type of technology lends itself well to areas such as supply chain
or notarization of documents where it is essential to maintain a copy of data that is chronologically
ordered and cannot be modified.
The power of blockchain extends to the concept of smart contracts (SC), which are self-executing
programs with terms written directly into code. Smart contracts serve as digital counterparts to legal
6th Distributed Ledger Technology Workshop, May 24-25, 2024, Turin, Italy
*
Corresponding author.
$ [email protected] (F. Barbàra); [email protected] (E. A. Napoli); [email protected] (V. Gatteschi);
[email protected] (C. Schifanella)
€ https://fadibarbara.it (F. Barbàra); insert (E. A. Napoli); https://staff.polito.it/valentina.gatteschi (V. Gatteschi); insert
(C. Schifanella)
© 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).

CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings

1
Fadi Barbàra et al. CEUR Workshop Proceedings 1–24

agreements, leveraging properties of non-repudiability and trustless execution. By embodying these

characteristics, smart contracts offer a secure and transparent mechanism for automating and enforcing
contractual obligations across various domains. However, their development traditionally requires
a deep understanding of blockchain technology and development expertise, posing challenges for
non-technical individuals.
Over the years, interest in blockchain has grown because of the above-mentioned properties of
transparency, reliability, non-tampering and programmability, attracting the attention not only of
people with programming skills but also of people with no special knowledge in this area. In this sense,
LLMs lend themselves well to narrowing the knowledge gap of non-experts who aim to harness the
potential of blockchain. In this paper, as a preliminary step for developing a comprehensive tool for
automatic smart contract generation, we try to establish which is the best prompt to feed to LLMs
to produce the best smart contract. In order to do so, we developed a pipeline that automatically
generates 𝑠 smart contracts starting from a legal contract or textual description. To properly assess
the effectiveness of the pipeline, we introduce a benchmarking suite tailored to our specific objectives.
This suite incorporates multifaceted metrics, such as compilability, vulnerability detection, and the
presence of comments, among others. To the best of our knowledge, this is the first work proposing
a framework for prompt engineering applied to the generation of smart contracts. We also present
an expert evaluation of the produced smart contract. By contrasting these two evaluation methods
we see how major flaws in the code produced by the LLM go undetected by automatic vulnerability
detection tools. In the interest of promoting transparency and facilitating reproducibility, we have made
all pertinent data publicly available.
Specifically, the corpus of 68 smart contracts under examination is accessible via our dedicated
GitHub repository1 . This repository also contains the scripts employed in our analysis pipeline and
the procedures for automatic testing. Additionally, the outcomes of our automatic testing process are
meticulously documented in a spreadsheet2 .
Finally, the lease agreement we used as case study can be found in Appendix A while the prompts
used for the LLM are in Appendix B.
In summary, the following are the contributions of our current work:
1. we propose a set of metrics to evaluate the quality of the generated smart contracts in order to
evaluate the effectiveness of proposed tool;
2. we perform a double smart contract review (automatic and manual) to evaluate different prompts,
adhering to the CO-STAR [8](Context, Objective, Style, Tone, Audience, Response) methodology
in order to determine the most effective approach to producing high-quality smart contracts;
3. we assess the consistency, stability and flaws of the LLM’s response across multiple calls using
the same prompt;
4. we freely release a usable dataset of LLM-produced code that can be used for other studies.
The paper is so composed. In Section 2 we present the related works. We explain how our testing has
been done in Section 3 and we analyze the results in Section 4. We perform a discussion of the results
in Section 5. Finally in Section 6 we present conclusions and future works.

2. Related Works
The advent of Large Language Models creates the need to get the best response from them by asking
the right question. This art is called prompt engineering and includes all those practices that aim
to advise how to communicate clearly with the LLM and get the best quality response in terms of
precision, context, and data. Prompt engineering has already been used in a wide range of context
such as education [9, 10], academic writing [11], healthcare [12, 13], automotive [14], and insolvency
1
GitHub link: https://github.com/BChain4all/pipeline/tree/main/results-feb-24
2
Spreadsheet file containing the outcomes of the proposed research: https://docs.google.com/spreadsheets/d/
1zPk6Ucb8n2UAaExGa1F18ODV_s2CYQV6/edit?usp=sharing&ouid=101965872120892214476&rtpof=true&sd=true

2
Fadi Barbàra et al. CEUR Workshop Proceedings 1–24

and bankruptcy law [15]. In the domain of Natural Language Processing, a spectrum of strategies
has been employed [16]. Nevertheless, any papers investigated the best strategies and framework in
terms of prompt engineering so that the LLM response is the most accurate possible. Wang et al. [10]
investigated the best strategies used by undergraduates students in prompting ChatGPT to effective
complete tasks by evaluating the information quality obtained by the LLM. Ratnayake et al. [16] propose
the PERFECT prompting framework, which entails specifying the Purpose (the primary goal or objective
of the prompt), Element (an additional element to consider), Role (the role or perspective the model
should adopt), Format (the desired response format), Examples (including examples or case studies),
Conditions (conditions or constraints the answer should adhere to), and Timeframe (historical context
or future predictions). To address the complexities inherent in constructing a prompt another method
adopted is the so-called CO-STAR [8]. This method involves providing COntext, defining the Style for
consistency, articulating the Target for focus, defining the Tone for sentiment alignment, identifying the
Audience for personalization, and specifying the Response for format clarity. Through this structured
approach, the aim is to improve the effectiveness and relevance of the language model output within
the defined constraints. In their recent work, Bao et al. [17] introduce a novel prompt technique tailored
for multi-document summarization. The Chain-of-Event (CoE) technique comprises four components:
task description, process summary reasoning, an exemplar of summary reasoning, and multi-document
input. The effectiveness of the proposed method is evaluated against both zero-shot and few-shot
prompting methods, assessing its comparative efficacy. On the other hand, Arawjo et al. [18] introduce
ChainForge, an open-source visual programming environment tailored for prompt engineering and
LLM sensemaking, thereby facilitating users in navigating and comprehending LLM outputs. Chen
et al. [19] present a novel approach called AutoPrompt-based Prompt Tuning (APT), which enhances
continuous prompts with a gradient-guided automatic search mechanism aimed at creating optimal
discrete templates and identifying trigger tokens.
Several papers evaluate LLMs regarding the generation of code from a text description, also called
text-to-code task. Nguyen et al. [20] present a framework aimed at generating API testing scripts
employing ChatGPT. Their approach involves designing a prompt template that incorporates the test
case scenario, data generation rules, pertinent information extracted from API documentation, the
expected result format, and, if available, anticipated execution feedback. Leveraging the few-shot
technique, they utilize previous logs and the model’s responses. Kwon et al. [21] explore the potential of
ChatGPT to enhance the quality of Ansible scripts within the domain of edge cloud systems. Through
the evaluation of ChatGPT’s code recommendation proficiency on 48 code revision instances extracted
from 25 Ansible project GitHub repositories by three independent raters, the study verifies the model’s
capability to detect and comprehend Ansible scripts. However, the authors highlight that ChatGPT’s
efficacy is substantially influenced by the query formulation. In their investigation, Liu et at. [22]
conduct experiments utilizing the CodeXGlue dataset to assess the text-to-code and code-to-code
generation capabilities of ChatGPT. Employing the chain-of-thought strategy, the authors implement
multi-step optimization techniques guided by feedback from ChatGPT. This approach results in a
notable enhancement in performance levels, underscoring the efficacy of their methodology.
The ”one-shot“, ”few-shot“, and the PERFECT framework proposed by [16] are unsuitable for our goal
since they involves providing the LLM with illustrative examples. Indeed, in our case, the availability of
an existing smart contract example would eliminate the need to generate the smart contract from the
provided textual description or legal agreement unnecessary. However, the techniques mentioned above
are more suitable for enhancing or debugging pre-existing contracts rather than generate new ones
from legal documents. Therefore, we decided to use the CO-START method which does not require
providing a code example in its formulation.
As far as text-to-code is concerned, there is a lack of prompt engineering methods and frameworks,
or at least the frameworks proposed in the literature are not tested on code generation.

3
Fadi Barbàra et al. CEUR Workshop Proceedings 1–24

Legal agreement
Or Smart Contract Vulnerability
Text Description Prompting AI Exploitation Generation Detection

Slither
gpt-4

Figure 1: The adopted pipeline

3. Experimental Methodology and Evaluation

Our research experiment integrates the generation and evaluation of smart contracts from prompts
composed of a legal agreement or textual descriptions using Large Language Models, specifically
OpenAI’s GPT-4 model, with a focus on security and performance metrics. As depicted in Figure 1, our
experiment consists of a pipeline split into three key steps:
1. Selection of a legal agreement or textual description as input (Section 3.1)
2. Embedding the selected text into a set of prompts, tailored for the LLM to generate preliminary
smart contract code (Section 3.2)
3. Analysis of the resulting source code with a vulnerability detection tool to assess security integrity
(Section 3.3)
In this section we present these steps and in Section 4 we present an evaluation of the results gathered.

3.1. Legal Agreement Analysis for Smart Contract Conversion

We specifically focus on a legal agreement 3 detailing the lease terms between a landlord and a potential
tenant. This document is featured in Appendix A and has been chosen not only for its relevance but
also because its length falls within the optimal context window of the LLM. This characteristic ensures
that the model can fully comprehend and process the entire agreement without the need for segmenting
or summarizing the document, which is crucial for maintaining the legal integrity of the converted
smart contract.
This Standard Lease Agreement outlines the rental terms between David Miller (Landlord) and
Richard Garcia (Tenant) for a residential property located at 7000 NW 27th Ave, Miami, Florida. The
lease runs for a fixed term from March 1, 2022, to March 1, 2023, with the Tenant agreeing to pay 850
USD monthly. A security deposit of 1,700 USD is required, which is refundable under certain conditions
post-lease termination. A late fee of 200 USD for rent payments made after the 10th day past due are
expected.

3.2. Prompt Formulation and LLM Selection

Based on the findings from Chang et al. [23] and our analysis, we have selected OpenAI’s
gpt-4-0125-preview model for its superior performance in tasks related to natural sciences, engi-
neering, and semantic understanding. This selection is critical for accurately translating complex legal
agreements into secure smart contracts.
The employment of “one-shot” or “few-shot” (see Section 2) prompting methods did not align with
our project goals, as these techniques hinge on providing the model with specific examples to direct

3
https://www.lawdepot.com/

4
Fadi Barbàra et al. CEUR Workshop Proceedings 1–24

Metric Evaluation Criteria/Weight

Compilability 1 if compiles, 0 if does not
Vulnerability Assessment -1 (high risk), -0.5, -0.25 (low risk)
Solidity Version 1 if ≥ 0.8.0, 0 if ≤ 0.8.0
Number of Contracts Generated 0.25 per contract
Number of Functions 0.15 per function
Presence of Comments 1 if present, 0 if absent
Number of Initialized Parameters 0.5 per parameter
Table 1
Summary of Smart Contract Evaluation Metrics

its outputs toward a desired result. For this reason, the strategy advocated by Ratnayake et al. [16],
which also involves supplying the LLM with illustrative instances, was considered unsuitable for our
needs. In our particular case, the presence of an existing smart contract example would eliminate the
necessity for initiating prompts to the model, implying that such prompting techniques are better suited
for enhancing or debugging pre-existing contracts rather than creating new ones from legal documents.
Given the inadequacy of “few-shot” or “one-shot” prompting for our objectives, we adopted the
CO-STAR framework for prompt construction, which includes Context, Objective, Style, Tone, Audience,
and Response components. This approach ensures that the prompts are comprehensively designed to
elicit high-quality smart contract code from the LLM [8].

3.3. Security Analysis Tools and Evaluation Metrics

For ensuring the security of the generated smart contracts, we incorporated Slither [24], a static analysis
framework for Solidity contracts. This tool is selected for its efficiency in identifying a broad spectrum
of security vulnerabilities and its fast execution time, which is paramount for processing multiple
iterations within our pipeline.
The reliability, security, and effectiveness of the generated smart contracts are assessed starting from
the metrics defined by Tonelli et al. [25]. These metrics include compilability, vulnerability assessment,
Solidity version compatibility, and structural aspects such as the number of contracts, functions, and
the presence of comments and initialized parameters, see Table 1 for the exact values. A Weighted Total
Score, aggregating these metrics by multiplying the compilability result with the sum of all the other
values, serves as a comprehensive measure to evaluate the smart contracts’ quality.
These methodologies and metrics ensure a thorough evaluation of the smart contracts generated
through our pipeline, providing insights into both their technical robustness and security posture. On
the other hand, the tools can not assess functional correctness. for this reason we also performed a
exper evaluation as explained in Section 4

4. Comprehensive Evaluation
In the following we present the results of the automatic analysis (Section 4.1 and the expert evaluations
(Sections 4.2 to 4.4)

4.1. Weighted Total Score Evaluation

As delineated in Section 3.3, we carried out an assessment of the results derived from the autonomous
generation of smart contracts, utilizing appropriate metrics (Table 1). The results are available as a
spreadsheet file4 . Of the entire 68 smart contracts under examination, merely four of them could not be
successfully compiled. The aforementioned derivative is rather promising as it indicates the proficient

4
The spreadsheet file containing our results can be found here: https://docs.google.com/spreadsheets/d/
1zPk6Ucb8n2UAaExGa1F18ODV_s2CYQV6/edit?usp=sharing&ouid=101965872120892214476&rtpof=true&sd=true

5
Fadi Barbàra et al. CEUR Workshop Proceedings 1–24

capabilities of the LLM in generating a majority of the code that could be compiled with no difficulties
(94.1%).
Additionally, the vulnerability evaluation carried out employing the automated tool, Slither, proved
to be promising. In essence, a majority of the smart contracts revealed only low-impact vulnerabilities.
A minor segment of the smart contracts displayed moderate vulnerabilities, on the other hand, only 2 of
the smart contracts exhibited any critical vulnerability. These results can be seen in the vuln_count
column of the spreadsheet file.
A Weighted Total Score (total_score column of the spreadsheet file) was formulated for the
purpose of this evaluation (Section 3.3 to see how it is computed). A worrying observation made in this
regard was the apparent lack of correlation between the prompt and the consequential value of the total
score. Interestingly, all the resultant values tend to cluster around 4 to 6, yet there are sporadic peaks
at values of 10 and 12, predominantly in prompt 𝑃 𝑅3. Based on the metrics employed, this scenario
signifies a confluence of a very low presence of vulnerabilities, combined with a high function count,
and elevated comment and variable initialization and assignment rates. This outcome does not align
with the anticipate results as per the CO-STAR techniques, whereby a higher role accompanied by a
specific constraint in the generation is expected to yield optimized and superior code, a finding we were
unable to corroborate.

4.2. Functions and Functionalities

In this section we explore the integration of smart contract functionalities within generated smart
contracts. From a purview of the functional elements present in the lease agreement (Appendix A)
and encoded within the smart contracts generated by the LLM, we expected that the generated smart
contract included four key functionalities. The preliminary functionality we perceived as indispensable
relates to ensuring the efficient execution of lease payments. This would entail the establishment of a
function capable of facilitating the seamless transition of payment from the tenant to the landlord in an
automatic fashion.
Moreover, an effective smart contract must be equipped with a feature conducive to efficient security
deposit management. This illustrates a functionality analogous to the rent payment operation; however,
in stark contrast, the designated funds are anticipated to be retained within the confines of the tenant’s
account rather than facilitated towards the landlord. Thus, effectively enabling the smart contract to
function in the capacity of a trustless escrow.
Additionally, any robust leasing agreement would necessitate the presence of a meticulously crafted
lease management system. We anticipated two distinct functions to serve the purpose: the initial being
for lease termination, and the latter concerning activation. In our comprehensive analysis, we assumed
that these simplistic yet efficient functions would be optimally suited to undertake checks on the status
of the smart contract. In doing so, it would render the decision, whether to engage or disengage the
lease, by modifying a select variable accordingly.
Another anticipated functionality pertained to the comprehensive management of late fees. We
envisioned a single function that systematically checks for the projected payment timeline for the rent.
If a delay on the tenant’s behalf is ascertained, the function would promptly generate the updated rent
payment and corresponding amount reflective of the late fee incurred. The function geared towards
managing late fees could be integrated within the ambit of rent management or it could potentially
function independently.
Lastly, we conceived the potential inclusion of a function dedicated to coin management. Given that
the agreed rental payment is 850 USD as per the lease agreement specifications, we expected the smart
contract to comprehend if an equivalent payment of the said amount was due in Ethers. The function
would then process the conversion of 850 USD into Ethers or alternatively, manage a scenario wherein
said payment should be made in a stablecoin. In the first instance, the coin management feature would
arrange potential swaps between Ether and a landlord-specified stablecoin. In the second instance, it
would facilitate potential transfers between different stablecoins under certain conditions. This feature
could potentially extend to manage late fee and security deposits in the same way.

6
Fadi Barbàra et al. CEUR Workshop Proceedings 1–24

In the following we present the analysis of the actual functions we obtained by the smart contracts
coding the functionalities we just described. All smart contracts are available in our GitHub5 .

4.2.1. The payRent Function

The payRent function, found ubiquitously across all 68 smart contracts analyzed, is primarily re-
sponsible for controlling the rent payment operation. These contracts present two main versions of
this function. The first variant receives the month as an input parameter, predominantly exploited to
verify whether the payment for the current month has been executed. If not, it proceeds to update the
corresponding variable to indicate the receipt of payment.
The alternative version of this function notably lacks any parameters. Consequently, the sole validity
check conducted within the function is to ensure that a rent payment arrives within the designated
leaseStart and leaseEnd timeframe.
Both versions verify that the rent amount corresponds with the incoming message value (msg.value
in solidity). This function, as is explained in Section 5.5, confirms this message value in Ethers, even if it
is highly implausible for that amount (850 Ethers) to be made in that currency.
An anticipated feature within the payRent function would be the automated forwarding of the
payment to the landlord. However, we observed that the bulk of the payRent functions do not contain
the code lines necessary for this operation. This may lead to instances where the coins either remain
unspent in the smart contract or require manual collection by the landlord via another function trigger.
It’s crucial to note that the latter scenario is rarely encountered, while the majority of payRent
function implementations fail to forward any coins to the landlord. We discerned this to be a form of
logic error undetectable by current vulnerability detection systems (see Section 5.2).
The behavior mentioned above could potentially be attributable to a comment found within the smart
contract, procured using prompt 𝑃 𝑅1 during the second test, which alluded to the fact that the entire
smart contract was an imitation of the landlord’s balance for simplification purposes. This comment
suggested that the smart contract may be the landlord himself. However, given that actual landlords are
real individuals, inhibiting their ability to withdraw coins is tantamount to the coins being immobilized
(freezed) within the smart contract.
The payRent functions across different smart contracts exhibited minor variations concerning their
modifiers. All of these functions are designated as external and payable. Some versions bear the
onlyTenant modifier, which limits the function trigger to the tenant exclusively. They also apply the
isLeaseActive modifier, which verifies an active lease status.
Contrarily, the alternative payRent function variant integrates a require directive within the
function to check the message sender or the current lease status. Nonetheless, a significant percentage
of smart contracts opt for the modifier version, which was deemed the preferred option in our analysis
owing to the adaptability of these modifiers for application in other contract functions.

4.2.2. The paySecurityDeposit Function

The paySecurityDeposit function, which facilitates the process of tenants paying the security
deposit, serves as the focal point of this exposition.
Surprisingly, this function is absent in the majority of Solidity contracts we examined. This particular
omission signifies a critical error by the LLM, as the function’s absence renders it unfeasible for the tenant
to make payments towards their respective security deposit. This particular anomaly is particularly
baffling, considering all smart contracts ubiquitously include the securityDeposit variable.
The paySecurityDeposit function, when implemented, exhibits uniformity across all smart con-
tracts. Noteworthy is that the function does not incorporate any variables and is characterized as
external and payable and can only be activated by the tenant. It commonly requires the message
value to align with the pre-established securityDeposit variable.

5
GitHub link: https://github.com/BChain4all/pipeline/tree/main/results-feb-24

7
Fadi Barbàra et al. CEUR Workshop Proceedings 1–24

Despite the expectation that this function should be executed prior to the initiation of the lease
term, as explicitly stated within the lease agreement, several iterations of the paySecurityDeposit
function conspicuously lack embedded controls for this timeline regulation.
Additionally, multiple versions of the paySecurityDeposit function include a comment acknowl-
edging the distinct security requirements that should be managed by the smart contract developer.
This recognition implies a crucial consideration for potential code enhancement, but implementation
remains sorely lacking.
Another point of contention is the common misinterpretation of the security deposit fund flow
dynamics. Ideally, the security deposit funds should be retained within the smart contract as a form of
escrow, but it is alarming to note that certain paySecurityDeposit functions facilitate an automatic
transfer of these funds to the landlord, a feature we categorize as programmatically erroneous. This
necessary escrow provision is designed to ensure that the security deposit is appropriately sheltered
from premature landlord access until the conclusion of the lease tenure.

4.2.3. The refundSecurityDeposit Function

The refundSecurityDeposit function, a ubiquitous feature in all smart contracts, introduces a
nuanced perspective compared to the paySecurityDeposit function.
The function exists in two distinct versions. The first one does not necessitate any parameter and
is generally executed by the landlord. Additionally, it mandates that the lease is no longer operative.
Both requisite conditions are substantiated through assigned modifiers. Pertinent checks concern-
ing the admission of the initial security deposit are incorporated within the function provided the
paySecurityDeposit function is a present aspect of the smart contract. However, in its absence, no
substantial controls are executed beyond the ones established by the modifiers.
The alternate version of the refundSecurityDeposit function requires a parameter, typically
referred to as deductions. These deductions are consequently deducted from the securityDeposit,
establishing a refund proportional to the tenant.
In-depth analysis reveals how almost all iterations of the refundSecurityDeposit function au-
tonomously transfer the refund amount, either equivalent to the security deposit itself or the resultant
difference between the security deposit and the deduction, to the tenant. These versions invariably
circumvent any transfers to the landlord, hence the funds associated with the security deposit typically
remain freezed within the smart contract given the absence of auxiliary functions to extricate them.

4.2.4. Management of Late Fees

Pertaining to the lease agreement, referenced in Appendix A, a penalty fee of 200 USD is levied against
the tenant who fails to timely settle rent payments, specifically beyond fifteen days after the fifth of
the month. The handling of said late fee necessitates the involvement of a designated function or
incorporation within the payRent function.
The examination of 68 smart contract revealed that, in the majority of them, this late fee provision
has been neglected. In the few instances where late fees have been administrated within the smart
contract, we have identified two primary methods of implementation.
One approach relies on the security deposit (refer to Sections 4.2.2 and 4.2.3 for an explanation of
what it is and how it is managed) and offers a lateFee function that deducts the late fee from the
security deposit. This function has been adapted such that only the landlord can collect the late fee.
Generally, it overlooks checking the sufficiency of funds within the security deposit. Evidently, this
method of late fee management falls short of expectations. Theoretically, a tenant could accrue a total
of USD 1, 800.00 or more in late fees (for nine or more occurrences). This equivalency is more than
USD 1, 700.00, exceeding the value that should be withdrawn from the security deposit.
We propose that a superior method would be to internalize the management of late fees within the
payRent function. Consequently, rent totals would increase relative to a designated date via the block
timestamp. A limited number of payRent functions were discovered to effectively manage late fees in

8
Fadi Barbàra et al. CEUR Workshop Proceedings 1–24

this manner. It is noteworthy that these functions lack the automatic transfer of rent to the landlord and
therefore, even when late fees are duly administered, the respective amount fails to reach the landlord.
This situation is elaborated on further in Section 4.2.1.

4.2.5. Lease start management

The deployment of a smart contract on a blockchain does not necessarily implicate that the lease
agreement commences immediately upon deployment. A more intuitive approach is to deploy the
smart contract before the lease’s start date, principally considering that preliminary security deposits
stipulated in the lease agreement need to be addressed before the lease’s activation. As discussed in
Section 4.2.2, these payments are essential components of the smart contract’s function. Consequently,
two predominant strategies can be identified to manage this aspect.
In the first approach, the LLM integrates the variables leaseStart (indicating the start time of
the lease) and leaseActive (representing the activation status of the lease) within the contract’s
constructor. In such instances, the commencement of the lease is considered instantaneous and auto-
matic upon deployment, as the leaseActive variable is set to true inside the constructor. Resultantly,
the entire lease contract is activated. It is worth noting that in this scenario, those contracts featuring a
paySecurityDeposit function should not be obliged to verify leaseActive, as it is inherently set
to true at the contract’s start and deployment phase. These contracts should solely verify leaseStart.
This autonomous lease commencement management is the most prevalent method.
Conversely, the second approach offers a more manual control over lease commencement. In these
cases, smart contracts typically feature a specialised function, startLease, that can solely be triggered
by the landlord after verifying the leaseStart variable. This customarily sets leaseActive to true,
thereby activating the lease agreement. In such instances, the paySecurityDeposit function can
be triggered prior to the lease activation, thus ensuring adherence to the terms stipulated in the lease
agreement.
In cases where the startLease function can assign the active status to the current lease state,
the paySecurityDeposit function may verify the lease activity status insted of a the configuration
derived from the current timestamp set by the block, merely referencing the defined leaseStart
variable. This ensures a more meticulous verification procedure.

4.2.6. Management of Lease Termination

Unlike lease commencement, lease termination within smart contracts incorporates an assigned function.
However, this function often exhibits notable deficiencies. Predominantly, it can only be activated by
the landlord (onlyLandlord modifier) during the period of active lease (isLeaseActive modifier),
but it doesn’t account for specifics such as lease termination dates or other relevant variables. As such,
to the landlord is given the capacity to terminate the lease at will, in accordance with the stipulations of
smart contracts generated by the LLM. This makes it challenging to forestall unwarranted terminations
initiated by landlords. We note that, while it is relatively easy for experts to introduce legal prerequisites,
translating these requirements into practical components of a smart contract proves more complex.
An apt solution may involve invoking a specific oracle prior to lease termination. Although it has
been observed that the GPT-4 LLM probably lacks currently the capacity to devise separate contracts to
manage such scenarios (since it never created them nor suggested to the developer to do so in any of
the 68 calls, see Section 4.1), it is pertinent to note that it has not been explicitly tasked to do so. Equally,
numerous smart contracts’ comments have highlighted the need for completion of such functions. This
is yet another manifestation of the flaw observed earlier in the payRent function (Section 4.2.1), which
resists detection through automated tools.
In closing, we discerned that the lease termination function demonstrates remarkable consistency
across different smart contracts, with negligible variation witnessed in its formulation.

9
Fadi Barbàra et al. CEUR Workshop Proceedings 1–24

4.3. Parameters and Initialization

In the formulation of our analysis, we elected to scrutinize the initialization and assignment of variables.
The initialization of a variable is ascribed when a variable obtains a value beyond basic initialization.
For instance, the uint256 public constant rentAmount variable is denoted as initialized, while
uint256 public constant rent amount = 850 is identified as both initialized and assigned.
We explicitly directed the language model to assign values to the variables in prompts 𝑃 𝑅4, 𝑃 𝑅5,
𝑃 𝑅9, 𝑃 𝑅10, and 𝑃 𝑅14 to 𝑃 𝑅17. Our expectation stipulated that, within these prompts, every
conceivable variable must be assigned. The practical implication of this assertion ensures that these
parameters are not passed as input during deployment of the smart contract.
Upon interpreting the lease agreement, we identified multiple variables that could be assigned. We
postulated these must be set prior to deploying the smart contract:
• The names and addresses of both the landlord and tenant
• The address of the property for lease
• The lease commencement and conclusion dates
• The monthly rental charge
• The security deposit
• The late fee
There typically exist two distinct mechanisms for assigning variables in a smart contract in Solidity.
The primary method simply assigns a value to an initialized variable within the contract. Alternatively,
the value of the variable can be assigned within the constructor. Consequently, even if variable
assignment transpires during contract deployment, user input is not necessary during the deployment
stage.
In our evaluation of the smart contracts, we discovered an unexpectedly weak correlation between
our expressly requested variable assignment and the actual resultant assignment. For instance, within
prompt 𝑃 𝑅4, only one of the four calls includes the assignment of the landlord and tenant names and
addresses. In contrast, prompt 𝑃 𝑅5 never assigns the landlord and tenant’s names and addresses in
any contract, notwithstanding our explicit request.
An intriguing variable assignment feature is the consistent use of Ether as a currency, despite our
lease agreement exclusively utilizing USD currency. This utilization occurs throughout all 68 smart
contracts, evidenced by a 200 USD late fee listed in the agreement always being equated as 200 Ether in
the smart contract. This detail is explored further in Section 5.5, which notes the absence of a function
to manage currency conversion in any smart contract.
Finally, any lease start and end times that are assigned adopt UTC times in seconds. The lease
commencement date is March 1st, 2022, while the lease conclusion date is March 1st, 2023. The UTC
time used is always accurate6 .

4.4. Smart contract comments

Our examination extended to the comments embedded within the smart contracts, notwithstanding
that our prompts did not explicitly mandate their inclusion. The variability in comments within the
smart contracts under review was not significantly compelling.
The comments, ostensibly terse, offered a correspondingly precise explanation of the functionality
being executed. They were employed strategically to demarcate distinct sections of the smart contracts,
distinguishing among function and variable definitions, events, and modifiers.
However, an argument for the substantive relevance of these comments raises some doubt. Despite
their accurate representation of the function’s purpose, the comments, for the most part, revert to an
6
The bash command date --date=’@1646092800’ (the set time in the smart contracts for the starting date) resulted in
Wed Mar 1 01:00:00 AM CET 2022. This time corresponds to the midnight in CET, the timezone in which the LLM was
called. We had anticipated the midnight according to a USA timezone, considering the lease agreement clearly pertains to an
apartment located in Florida, USA, and the OpenAI servers are also located in the USA. Same result has been obtained for the
termination date.

10
Fadi Barbàra et al. CEUR Workshop Proceedings 1–24

oversimplified explanation that could be deemed inconsequential. The core reasoning behind this style
of commentary remains uncertain.
One interpretation is that this reflects an inherent simplicity of the functions under scrutiny, such
that any developer, irrespective of their expertise level, might comprehend the functionality readily. An
alternative supposition posits a restriction in the capability of the large language model in fabricating
comprehensive comments.
It is important to note that the LLM’s training draws from Solidity code available publicly on GitHub,
thus implying a possibility that the observed comment trend might echo a broader developer trend. This
trend may suggest a prevalent underutilization of comments as a tool for meaningful and expressive
communication within the developer community.

5. Discussion
The following discussion pertains to the revelations gathered from the automated and expert evaluations
carried out during this study and presented in Section 4.

5.1. Absence of correlation between prompts and results

We applied the CO-STAR technique, anticipating an observable connection between the ranking of the
software developer (referring to varying roles) used in the prompt, and the quality of the resulting smart
contract. We expected that the code engendered by an imaginary master’s student would be inferior
to the contract generated by the junior developer, and subsequently the senior Solidity developer (see
Appendix B to read all the prompts used throughout the study). Additionally, we formulated different
objectives manifested in the prompts, envisioning that objectives with explicit precision (for instance,
the version of Solidity to use, the demand for fully articulated function logic, the overt call for variable
assignment), would mirror the directive more than prompts lacking these attributes. These requests
and expectations are in line with our ultimate goal to fashion a prompt, or a series of prompts, which
can be employed by non-professionals in the field to autonomously generate production-ready smart
contracts.
As highlighted before, our principal conclusion is the absence of any correlation between the assigned
ranking and the quality of the resultant smart contract. This is discernible both in the automated
evaluation deploying quantitative parameters and metrics and in the manual expert evaluation. Similarly,
we found no correlation between the unambiguous objectives introduced in the prompts and the actual
realization of said objectives. Refer e.g. to Sections 4.2.2 and 4.3 where we note multiple instances of
missing variable assignments and a critical insufficiency of requisite functions.

5.2. Non-detectable Flaws

Another alarming and prevalent issue we discovered during the smart contract analysis was the
production of logical bugs by the LLM. These bugs are effectively undetectable through automated
vulnerability detection tools. For instance, as presented in Section 4.2.1, from the perspective of the
LLM, the smart contract itself personifies the landlord. Consequently, funds are stuck inside the smart
contract and the (real) landlord can not withdraw them. This conclusion was reached only after a
thorough manual review of all contracts. As a result, non-experts in the field would be unlikely to
discover such a flaw.
The subsequent issue of concern pertains to the absence of a genuine comprehension of the inherent
complexities of a legal contract. For instance, despite the fact that the termination of a contract can only
be effected by the landlord and only while the contract is active, a plain variable alteration bereft of any
additional control can potentially engender vulnerabilities which include unwarranted interference by
the landlord himself, who can then terminate the contract prematurely.

11
Fadi Barbàra et al. CEUR Workshop Proceedings 1–24

5.3. Incomplete Functions and Variable Assignment

Another observation was the tendency for the LLM to neglect to complete functions. Despite the clarity
of instructions from the legal agreement, the generated smart contracts seldom possess fully developed
functions. Strikingly, this pattern does not correlate with whether or not the prompt explicitly requests
fully defined function logic.
A related issue pertains to the assignment of variables: only a few variables are assigned. Furthermore,
numerical variables (e.g. the amount of the rent) are generally perceived as fixed parameters and
therefore assigned, whereas string parameters (e.g. the physical address of the rental property) are
seldom utilized as a whole and lack assignment.

5.4. Limited Variations of Function Representation

Another commonality we discerned across all functions in every smart contract was the limited variabil-
ity for each function. Most functions manifested in one of two versions, with a third version accounting
for the absence of that function altogether. It is challenging to predict which function will be included
or excluded. This further underscores the lack of correlation between the LLM prompts and the ensuing
results.

5.5. Failures in Currency Understanding

A peculiar error we observed in the LLM pertained to the management of currency. All values in
the lease agreement were expressed in USD, while each smart contract expressed every variable in
Ethers. The currency discrepancy clearly affects the sustainability of the rent and further exemplifies
how unsuitable the smart contracts generated by the LLM are for non-professional users in terms of
production readiness.

6. Conclusions
This research embarked on an exploratory journey to examine the potential integration between two
disruptive technologies, Blockchain and AI, with a focus on the application of LLMs for aiding non-
experts in crafting production-ready Solidity smart contracts. Through a methodical investigation
involving the translation of a lease agreement into smart contract code by GPT-4, supported by 17
meticulously designed prompts that generated 68 smart contracts (4 per prompt), our study provided a
thorough assessment of the capabilities of current LLM technology in this context.
The dual-faceted evaluation methodology, integrating automated analysis and expert manual review,
has revealed significant insights into the practical utility of GPT-4 within the blockchain development
sphere. The conclusions drawn from this investigation are unequivocal: the present version of GPT-4
falls short of the necessary standards for generating production-ready smart contracts. This shortfall is
attributed predominantly to the presence of undetected logic flaws and the misalignment between the
provided prompts and the resulting code.
These findings not only highlight the existing gaps within the capabilities of LLMs like GPT-4 but
also accentuate the critical challenges associated with the autonomous creation of smart contracts that
are applicable in real-world scenarios.
Building upon our findings, our future endeavors aim to incorporate alternative prompting techniques.
Our exploration thus far suggests that zero-shot prompting offers suboptimal results; consequently,
we intend to incorporate more promising strategies such as few-shot and chain of thought prompting
techniques in our methodology.
In theory, we could consider training or fine-tuning the LLM specifically to enhance its comprehension
of smart contracts. This potential adaptation can potentially serve to augment its overall performance
in this context.

12
Fadi Barbàra et al. CEUR Workshop Proceedings 1–24

Acknowledgments
The work discussed in this paper has been supported by the B4A - Blockchain for All project
(https://www.blockchain4all.it/), ref. no. 20225MN5K3. This project has been funded with support
from the Ministry of Education, University and Research. This document reflects the views only of the
authors, and the Ministry of Education, University and Research cannot be held responsible for any use
which may be made of the information contained therein.

References
[1] H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal,
E. Hambro, F. Azhar, et al., Llama: Open and efficient foundation language models, 2023.
[2] H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhar-
gava, S. Bhosale, et al., Llama 2: Open foundation and fine-tuned chat models, 2023.
[3] A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung,
C. Sutton, S. Gehrmann, et al., Palm: Scaling language modeling with pathways, Journal of
Machine Learning Research 24 (2023) 1–113.
[4] A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. d. l. Casas, F. Bressand,
G. Lengyel, G. Lample, L. Saulnier, et al., Mistral 7b, 2023.
[5] Y. Li, S. Wang, H. Ding, H. Chen, Large language models in finance: A survey, 2023, p.
374 – 382. URL: https://www.scopus.com/inward/record.uri?eid=2-s2.0-85179853539&doi=10.
1145%2f3604237.3626869&partnerID=40&md5=ee2b728c8b9fe0013974da302d3afc8a. doi:10.1145/
3604237.3626869, cited by: 0; All Open Access, Green Open Access, Hybrid Gold Open Access.
[6] J. Li, A. Dada, B. Puladi, J. Kleesiek, J. Egger, Chatgpt in healthcare: A taxonomy and systematic
review, Computer Methods and Programs in Biomedicine 245 (2024). doi:10.1016/j.cmpb.
2024.108013.
[7] L. Belzner, T. Gabor, M. Wirsing, Large language model assisted software engineering: Prospects,
challenges, and a case study, Lecture Notes in Computer Science (including subseries Lecture
Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 14380 LNCS (2024) 355 – 374.
doi:10.1007/978-3-031-46002-9_23, cited by: 2.
[8] G. D. Science, S. G. AI Division, Prompt engineering playbook, ???? URL: https:
//www.developer.tech.gov.sg/products/collections/data-science-and-artificial-intelligence/
playbooks/prompt-engineering-playbook-beta-v3.pdf.
[9] U. Lee, H. Jung, Y. Jeon, Y. Sohn, W. Hwang, J. Moon, H. Kim, Few-shot is enough: exploring
chatgpt prompt engineering method for automatic question generation in english education,
Education and Information Technologies (2023) 1–33.
[10] M. Wang, M. Wang, X. Xu, L. Yang, D. Cai, M. Yin, Unleashing chatgpt’s power: A case study on
optimizing information retrieval in flipped classrooms via prompt engineering, IEEE Transactions
on Learning Technologies 17 (2024) 629–641. doi:10.1109/TLT.2023.3324714.
[11] L. Giray, Prompt engineering with chatgpt: A guide for academic writers, Annals of Biomedical
Engineering (2023) 1–5.
[12] D. Grabb, The impact of prompt engineering in large language model performance: a psychiatric
example, Journal of Medical Artificial Intelligence 6 (2023).
[13] L. Wang, X. Chen, X. Deng, H. Wen, M. You, W. Liu, Q. Li, J. Li, Prompt engineering in consistency
and reliability with the evidence-based guideline for llms, npj Digital Medicine 7 (2024) 41.
[14] X. Li, E. Liu, T. Shen, J. Huang, F.-Y. Wang, Chatgpt-based scenario engineer: A new framework
on scenario generation for trajectory prediction, IEEE Transactions on Intelligent Vehicles (2024)
1–10. doi:10.1109/TIV.2024.3363232.
[15] M. Ribary, P. Krause, M. Orban, E. Vaccari, T. Wood, Prompt engineering and provision of context
in domain specific use of gpt, in: Legal Knowledge and Information Systems, IOS Press, 2023, pp.
305–310.

13
Fadi Barbàra et al. CEUR Workshop Proceedings 1–24

[16] H. Ratnayake, C. Wang, A prompting framework to enhance language model output, in: Aus-
tralasian Joint Conference on Artificial Intelligence, Springer, 2023, pp. 66–81.
[17] S. Bao, T. Li, B. Cao, Chain-of-event prompting for multi-document summarization by large
language models, International Journal of Web Information Systems (2024).
[18] I. Arawjo, P. Vaithilingam, M. Wattenberg, E. Glassman, Chainforge: An open-source visual
programming environment for prompt engineering, in: Adjunct Proceedings of the 36th Annual
ACM Symposium on User Interface Software and Technology, UIST ’23 Adjunct, Association for
Computing Machinery, New York, NY, USA, 2023. URL: https://doi.org/10.1145/3586182.3616660.
doi:10.1145/3586182.3616660.
[19] Y. Chen, G. Yang, D. Wang, D. Li, Eliciting knowledge from language models with automati-
cally generated continuous prompts, Expert Systems with Applications 239 (2024) 122327. URL:
https://www.sciencedirect.com/science/article/pii/S0957417423028294. doi:https://doi.org/
10.1016/j.eswa.2023.122327.
[20] C. Nguyen, H. Bui, V. Nguyen, T. Nguyen, An approach to generating api test scripts using gpt, in:
Proceedings of the 12th International Symposium on Information and Communication Technology,
SOICT ’23, Association for Computing Machinery, New York, NY, USA, 2023, p. 501–509. URL:
https://doi.org/10.1145/3628797.3628947. doi:10.1145/3628797.3628947.
[21] S. Kwon, S. Lee, T. Kim, D. Ryu, J. Baik, Exploring the feasibility of chatgpt for improving the quality
of ansible scripts in edge-cloud infrastructures through code recommendation, in: International
Conference on Web Engineering, Springer, 2023, pp. 75–83.
[22] C. Liu, X. Bao, H. Zhang, N. Zhang, H. Hu, X. Zhang, M. Yan, Improving chatgpt prompt for code
generation, arXiv preprint arXiv:2305.08360 (2023).
[23] Y. Chang, X. Wang, J. Wang, Y. Wu, L. Yang, K. Zhu, H. Chen, X. Yi, C. Wang, Y. Wang, et al., A
survey on evaluation of large language models, ACM Transactions on Intelligent Systems and
Technology (2023).
[24] J. Feist, G. Grieco, A. Groce, Slither: A static analysis framework for smart contracts, in:
2019 IEEE/ACM 2nd International Workshop on Emerging Trends in Software Engineering for
Blockchain (WETSEB), IEEE, 2019. URL: http://dx.doi.org/10.1109/WETSEB.2019.00008. doi:10.
1109/wetseb.2019.00008.
[25] R. Tonelli, G. A. Pierro, M. Ortu, G. Destefanis, Smart contracts software metrics: A first study,
PLOS ONE 18 (2023) 1–31. URL: https://doi.org/10.1371/journal.pone.0281043.

A. Lease Agreement
STANDARD LEASE AGREEMENT
This Agreement, dated , 20. , by and between an individual known as David Miller of 324
W Gore St, Orlando, Florida, 32806, hereinafter known as the "Landlord",
AND
An individual known as Richard Garcia, hereinafter known as the "Tenant(s)", agree to the
following:
OCCUPANT(S): The Premises is to be occupied strictly as a residential dwelling with only
the Tenant(s) mentioned above as the Occupant(s).
OFFER TO RENT: The Landlord hereby rents to the Tenant(s), subject to the following
terms and conditions of this Agreement, an apartment with the address of 7000 NW 27th
Ave, Miami, Florida, 33147 consisting of 1 bathroom(s) and 2 bedroom(s) hereinafter known
as the "Premises". The Landlord may also use the address for notices sent to the Tenant(s).
PURPOSE: The Tenant(s) and any Occupant(s) may only use the Premises as a residential
dwelling. It may not be used for storage, manufacturing of any type of food or product,
professional service(s), or for any commercial use unless otherwise stated in this Agreement.

14
Fadi Barbàra et al. CEUR Workshop Proceedings 1–24

FURNISHINGS: The Premises is furnished with the following:

Bedroom Set(s), Dining Room Set(s), Kitchen Set (Including Pots, Pans, Glasses, Mugs,
Dishes, etc.), and all other furnishings to be provided by the Tenant(s). Any damage to
the Landlord’s furnishings shall be the liability of the Tenant(s), reasonable wear-and-tear
excepted, to be billed directly or less the Security Deposit.
APPLIANCES: The Landlord shall not provide any appliances in the Premises.
LEASE TERM: This Agreement shall be a fixed-period arrangement beginning on March 1
2022 and ending on March 1 2023 with the Tenant(s) being required to move-out at the end
of the Lease Term if a new Lease Agreement is not authorized. Hereinafter known as the
"Lease Term".
RENT: Tenant(s) shall pay the Landlord in equal monthly installments of $850.00 (USD)
hereinafter known as the "Rent". The Rent will be due on the Fifth (5th) of every month
and be paid by sending payment to the Landlord’s aforementioned mailing address.
NON-SUFFICIENT FUNDS (NSF CHECKS): If the Tenant(s) attempts to pay the rent with
a check that is not honored or an electronic transaction (ACH) due to insufficient funds
(NSF) there
shall be no fee (USD).
LATE FEE: If rent is not paid on the due date, there shall be a late fee assessed by the
Landlord in the amount of $200.00 (USD) per occurrence for each month payment that is
late after the 10th Day rent is due.
FIRST (1ST) MONTH’S RENT: First (1st) month’s rent shall be due by the Tenant(s) upon
the execution of this Agreement.
PRE-PAY MENT: The Landlord shall not require any pre-payment of rent by the Tenant(s).
PRORATION PERIOD: The Tenant(s) will not move into the Premises before the start of
the Lease Term.
SECURITY DEPOSIT: A Security Deposit in the amount of $1,700.00 (USD) shall be paid
by the Tenant(s) to the Landlord, at the execution of this Agreement for the faithful
performance of all the terms and conditions. The Security Deposit is to be returned by the
Landlord to the Tenant(s) within 15 days after this Agreement has terminated if there are
no deductions. If there are deductions to the Security Deposit the amount shall be returned
within 30 days from termination of this Agreement. This Security Deposit shall not be
credited towards rent unless the Landlord gives their written consent.
POSSESSION: Tenant(s) has examined the condition of the Premises and by taking posses-
sion acknowledges that they have accepted the Premises in good order and in its current
condition except as herein otherwise stated. Failure of the Landlord to deliver possession of
the Premises at the start of the Lease Term to the Tenant(s) shall terminate this Agreement
at the option of the Tenant(s). Furthermore, under such failure to deliver possession by
the Landlord, and if the Tenant(s) cancels this Agreement, the Security Deposit (if any)
shall be returned to the Tenant(s) along with any other pre-paid rent, fees, including if the
Tenant(s) paid a fee during the application process before the execution of this Agreement.
ACCESS: Upon the beginning of the Proration Period or the start of the Lease Term,
whichever is earlier, the Landlord agrees to give access to the Tenant(s) in the form of
keys, fobs, cards, or any type of keyless security entry as needed to enter the common
areas and the Premises. Duplicate copies of the access provided may only be authorized
under the consent of the Landlord and, if any replacements are needed, the Landlord may
provide them for a fee. At the end of this Agreement all access provided to the Tenant(s)
shall be returned to the Landlord or a fee will be charged to the Tenant(s) or the fee will be
subtracted from the Security Deposit.

15
Fadi Barbàra et al. CEUR Workshop Proceedings 1–24

MOVE-IN INSPECTION: Before, at the time of the Tenant(s) accepting possession, or

shortly thereafter, the Landlord and Tenant(s) shall perform an inspection documenting
the present condition of all appliances, fixtures, furniture, and any existing damage within
the Premises.
SUBLETTING: The Tenant(s) shall not have the right to sub-let the Premises or any part
thereof without the prior written consent of the Landlord. If consent is granted by the
Landlord, the Tenant(s) will be responsible for all actions and liabilities of the Sublessee
including but not limited to: damage to the Premises, non-payment of rent, and any eviction
process (In the event of an eviction the Tenant(s) shall be responsible for all court filing
fee(s), representation, and any other fee(s) associated with removing the Sublessee). The
consent by the Landlord to one sub-let shall not be deemed to be consent to any subsequent
subletting.
ABANDONMENT: If the Tenant(s) vacates or abandons the property for a time-period
that is the minimum set by State law or seven (7) days, whichever is less, the Landlord
shall have the right to terminate this Agreement immediately and remove all belongings
including any personal property off of the Premises. If the Tenant(s) vacates or abandons
the property, the Landlord shall immediately have the right to terminate this Agreement.
ASSIGNMENT: Tenant(s) shall not assign this Lease without the prior written consent of
the Landlord. The consent by the Landlord to one assignment shall not be deemed to be
consent to any subsequent assignment.
PARKING: The Landlord shall not provide parking to the Tenant(s).
RIGHT OF ENTRY: The Landlord shall have the right to enter the Premises during normal
working hours by providing notice in accordance with the minimum State requirement
in order for inspection, make necessary repairs, alterations or improvements, to supply
services as agreed or for any reasonable purpose. The Landlord may exhibit the Premises
to prospective purchasers, mortgagees, or lessees upon reasonable notice.
SALE OF PROPERTY: If the Premises is sold, the Tenant(s) is to be notified of the new
Owner, and if there is anew Manager, their contact details for repairs and maintenance shall
be forwarded. If the Premises is conveyed to another party, the new owner shall not have
the right to terminate this Agreement and it shall continue under the terms and conditions
agreed upon by the Landlord and Tenant(s).
UTILITIES: The Landlord agrees to pay for the following utilities and services:
Electricity, Internet, Natural Gas, Water, with all other utilities and services to be the
responsibility of the Tenant(s).
MAINTENANCE, REPAIRS, OR ALTERATIONS: The Tenant(s) shall, at their own expense
and at all times, maintain premises in a clean and sanitary manner, and shall surrender
the same at termination hereof, in as good condition as received, normal wear and tear
excepted. The Tenant(s) may not make any alterations to the leased premises without
the consent in writing of the Landlord. The Landlord shall be responsible for repairs to
the interior and exterior of the building. If the Premises includes a washer, dryer, freezer,
dehumidifier unit and/or air conditioning unit, the Landlord makes no warranty as to the
repair or replacement of units if one or all shall fail to operate. The Landlord will place
fresh batteries in all battery-operated smoke detectors when the Tenant(s) moves into
the premises. After the initial placement of the fresh batteries it is the responsibility of
the Tenant(s) to replace batteries when needed. A monthly "cursory" inspection may be
required for all fire extinguishers to make sure they are fully charged.
EARLY TERMINATION: The Tenant(s) may not be able to cancel this Agreement unless
the Tenant is a victim of Domestic Violence, in such case, the Tenant may be able to cancel
in accordance with any local, state, or federal laws.

16
Fadi Barbàra et al. CEUR Workshop Proceedings 1–24

PETS: The Tenant(s) shall not be allowed to have pets on the Premises or common areas
except those that are necessary for individuals with disabilities.
NOISE/WASTE: The Tenant(s) agrees not to commit waste on the premises, maintain, or
permit to be maintained, a nuisance thereon, or use, or permit the premises to be used, in
an unlawful manner. The Tenant(s) further agrees to abide by any and all local, county,
and State noise ordinances.
GUESTS: There shall be no other persons living on the Premises other than the Tenant(s)
and any Occupant(s). Guests of the Tenant(s) are allowed for periods not lasting for more
than forty-eight hours unless otherwise approved by the Landlord.
SMOKING POLICY: Smoking on the Premises is prohibited on the entire property, including
individual units, common areas, every building and adjoining properties.
COMPLIANCE WITH LAW: The Tenant(s) agrees that during the term of the Agreement,
to promptly comply with any present and future laws, ordinances, orders, rules, regulations,
and requirements of the Federal, State, County, City, and Municipal government or any of
their departments, bureaus, boards, commissions and officials thereof with respect to the
premises, or the use or occupancy thereof, whether said compliance shall be ordered or
directed to or against the Tenant(s), the Landlord, or both.
DEFAULT: If the Tenant(s) fails to comply with any of the financial or material provisions
of this Agreement, or of any present rules and regulations or any that may be hereafter
prescribed by the Landlord, or materially fails to comply with any duties imposed on
the Tenant(s) by statute or State laws, within the time period after delivery of written
notice by the Landlord specifying the non- compliance and indicating the intention of the
Landlord to terminate the Agreement by reason thereof, the Landlord may terminate this
Agreement. If the Tenant(s) fails to pay rent when due and the default continues for the
time-period specified in the written notice thereafter, the Landlord may, at their option,
declare the entire balance (compiling all months applicable to this Agreement) of rent
payable hereunder to be immediately due and payable and may exercise any and all rights
and remedies available to the Landlord at law or in equity and may immediately terminate
this Agreement.
The Tenant(s) will be in default if: (a) Tenant(s) does not pay rent or other amounts that are
owed in accordance with respective State laws; (b) Tenant(s), their guests, or the Occupant(s)
violate this Agreement, rules, or fire, safety, health, or criminal laws, regardless of whether
arrest or conviction occurs; (c) Tenant(s) abandons the Premises; (d) Tenant(s) gives incorrect
or false information in the rental application; (e) Tenant(s), or any Occupant(s) is arrested,
convicted, or given deferred adjudication for a criminal offense involving actual or potential
physical harm to a person, or involving possession, manufacture, or delivery of a controlled
substance, marijuana, or drug paraphernalia under state statute; (f) any illegal drugs or
paraphernalia are found in the Premises or on the person of the Tenant(s), guests, or
Occupant(s) while on the Premises and/or; (g) as otherwise allowed by law.
MULTIPLE TENANT(S) OR OCCUPANT(S): Each individual that is considered a Tenant(s)
is jointly and individually liable for all of this Agreement’s obligations, including but not
limited to rent monies. If any Tenant(s), guest, or Occupant(s) violates this Agreement, the
Tenant(s) is considered to have violated this Agreement. Landlord’s requests and notices
to the Tenant(s) or any of the Occupant(s) of legal age constitutes notice to the Tenant(s).
Notices and requests from the Tenant(s) or any one of the Occupant(s) (including repair
requests and entry permissions) constitutes notice from the Tenant(s). In eviction suits, the
Tenant(s) is considered the agent of the Premise for the service of process.
DISPUTES: If a dispute arises during or after the term of this Agreement between the
Landlord and Tenant(s), they shall agree to hold negotiations amongst themselves, in "good

17
Fadi Barbàra et al. CEUR Workshop Proceedings 1–24

faith", before any litigation.

SEVERABILITY: If any provision of this Agreement or the application thereof shall, for
any reason and to any extent, be invalid or unenforceable, neither the remainder of this
Agreement nor the application of the provision to other persons, entities or circumstances
shall be affected thereby, but instead shall be enforced to the maximum extent permitted
by law.
SURRENDER OF PREMISES: The Tenant(s) has surrendered the Premises when (a) the move-
out date has passed and no one is living in the Premise within the Landlord’s reasonable
judgment; or (b) Access to the Premise have been turned in to Landlord — whichever comes
first. Upon the expiration of the term hereof, the Tenant(s) shall surrender the Premise in
better or equal condition as it were at the commencement of this Agreement, reasonable
use, wear and tear thereof, and damages by the elements excepted.
RETALIATION: The Landlord is prohibited from making any type of retaliatory acts against
the Tenant(s) including but not limited to restricting access to the Premises, decreasing or
cancelling services or utilities, failure to repair appliances or fixtures, or any other type of
act that could be considered unjustified.
WAIVER: A Waiver by the Landlord for a breach of any covenant or duty by the Tenant(s),
under this Agreement is not a waiver for a breach of any other covenant or duty by the
Tenant(s), or of any subsequent breach of the same covenant or duty. No provision of this
Agreement shall be considered waived unless such a waiver shall be expressed in writing
as a formal amendment to this Agreement and executed by the Tenant(s) and Landlord.
EQUAL HOUSING: If the Tenant(s) possess(es) any mental or physical impairment, the
Landlord shall provide reasonable modifications to the Premises unless the modifications
would be too difficult or expensive for the Landlord to provide. Any impairment of the
Tenant(s) is/are encouraged to be provided and presented to the Landlord in writing in
order to seek the most appropriate route for providing the modifications to the Premises.
HAZARDOUS MATERIALS: The Tenant(s) agrees to not possess any type of personal
property that could be considered a fire hazard such as a substance having flammable or
explosive characteristics on the Premises. Items that are prohibited to be brought into the
Premises, other than for everyday cooking or the need of an appliance, includes but is not
limited to gas (compressed), gasoline, fuel, propane, kerosene, motor oil, fireworks, or any
other related content in the form of a liquid, solid, or gas.
WATERBEDS: The Tenant(s) is not permitted to furnish the Premises with waterbeds.
INDEMNIFICATION: The Landlord shall not be liable for any damage or injury to the
Tenant(s), or any other person, or to any property, occurring on the Premises, or any
part thereof, or in common areas thereof, and the Tenant(s) agrees to hold the Landlord
harmless from any claims or damages unless caused solely by the Landlord’s negligence. It
is recommended that renter’s insurance be purchased at the Tenant(s)’s expense.
COVENANTS: The covenants and conditions herein contained shall apply to and bind the
heirs, legal representatives, and assigns of the parties hereto, and all covenants are to be
construed as
conditions of this Agreement.
NOTICES: Any notice to be sent by the Landlord or the Tenant(s) to each other shall use
the following mailing addresses:
Landlord’s/Agent’s Mailing Address
David Miller 324 W Gore St, Orlando, Florida, 32806
Tenant(s)’s Mailing Address

18
Fadi Barbàra et al. CEUR Workshop Proceedings 1–24

Richard Garcia 7000 NW 27th Ave, Miami, Florida, 33147

AGENT/MANAGER: The Landlord does not have an Agent or Manager and all contact in
regards to any repair, maintenance, or complaint must go through the Landlord through
the following contact information:
Landlord’s Phone Number: (185) 546-3277 Email: [email protected].
PREMISES DEEMED UNINHABITABLE: If the Property is deemed uninhabitable due to
damage beyond reasonable repair the Tenant(s) will be able to terminate this Agreement by
written notice to the Landlord. If said damage was due to the negligence of the Tenant(s),
the Tenant(s) shall be liable to the Landlord for all repairs and for the loss of income due to
restoring the Premises back to a livable condition in addition to any other losses that can
be proved by the Landlord.
ACCESS BY LANDLORD: The Landlord must provide at least twelve (12) hours notice to
the Tenant(s) before entering the Premises for any non-emergency reason.
RADON GAS: Radon is a naturally occurring radioactive gas that, when it has accumulated
in a building in sufficient quantities, may present health risks to persons who are exposed
to it over time. Levels of radon that exceed federal and state guidelines have been found
in buildings in Florida. Additional information regarding radon and radon testing may be
obtained from your county health department.
SECURITY DEPOSIT DISCLOSURE: YOUR LEASE REQUIRES PAYMENT OF CERTAIN
DEPOSITS. THE LANDLORD MAY TRANSFER ADVANCE RENTS TO THE LANDLORD’S
ACCOUNT AS THEY ARE DUE AND WITHOUT NOTICE. WHEN YOU MOVE OUT, YOU
MUST GIVE THE LANDLORD YOUR NEW ADDRESS SO THAT THE LANDLORD CAN
SEND YOU NOTICES REGARDING YOUR DEPOSIT. THE LANDLORD MUST MAIL YOU
NOTICE, WITHIN 30 DAYS AFTER YOU MOVE OUT, OF THE LANDLORD’S INTENT TO
IMPOSE A CLAIM AGAINST THE DEPOSIT.
IF YOU DO NOT REPLY TO THE LANDLORD STATING YOUR OBJECTION TO THE CLAIM
WITHIN 15 DAYS AFTER RECEIPT OF THE LANDLORD’S NOTICE, THE LANDLORD
WILL COLLECT THE CLAIM AND MUST MAIL YOU THE REMAINING DEPOSIT, IF
ANY.
IF THE LANDLORD FAILS TO TIMELY MAIL YOU NOTICE, THE LANDLORD MUST RE-
TURN THE DEPOSIT BUT MAY LATER FILE A LAWSUIT AGAINST YOU FOR DAMAGES.
IF YOU FAIL TO TIMELY OBJECT TO A CLAIM, THE LANDLORD MAY COLLECT FROM
THE DEPOSIT, BUT YOU MAY LATER FILE A LAWSUIT CLAIMING A. REFUND.
‘YOU SHOULD ATTEMPT TO INFORMALLY RESOLVE ANY DISPUTE BEFORE FILING A
LAWSUIT. GENERALLY, THE PARTY IN WHOSE FAVOR A JUDGMENT IS RENDERED
WILL BE AWARDED COSTS AND ATTORNEY FEES PAYABLE BY THE LOSING PARTY.
THIS DISCLOSURE IS BASIC. PLEASE REFER TO PART IT OF CHAPTER 83, FLORIDA
STATUTES, TO DETERMINE YOUR LEGAL RIGHTS AND OBLIGATIONS.
SERVICEMEMBERS CIVIL RELIEF ACT: In the event the Tenant(s) is or hereafter becomes,
a member of the United States Armed Forces on extended active duty and hereafter the
Tenant(s) receives permanent change of station (PCS) orders to depart from the area where
the Premises are located, or is relieved from active duty, retires or separates from the
military, is ordered into military housing, or receives deployment orders, then in any of
these events, the Tenant may terminate this lease upon giving thirty (30) days written
notice to the Landlord. The Tenant shall also provide to the Landlord a copy of the official
orders or a letter signed by the Tenant’s commanding officer, reflecting the change which
warrants termination under this clause. The Tenant will pay prorated rent for any days
which he/she occupies the dwelling past the beginning of the rental period.

19
Fadi Barbàra et al. CEUR Workshop Proceedings 1–24

The damage/security deposit will be promptly returned to Tenant, provided there are no
damages to the Premises.
LEAD PAINT: The Premises was not constructed before 1978 and therefore does not contain
lead- based paint.
GOVERNING LAW: This Agreement is to be governed under the laws located in the State
of Florida.
ADDITIONAL TERMS AND CONDITIONS: There are no further terms or conditions that
will be added to this Agreement other than any attachments or addendums attached.
ENTIRE AGREEMENT: This Agreement contains all the terms agreed to by the parties
relating to its subject matter including any attachments or addendums. This Agreement
replaces all previous discussions, understandings, and oral agreements. The Landlord and
Tenant(s) agree to the terms and conditions and shall be bound until the end of the Lease
Term.
The parties have agreed and executed this agreement on , 20 .
LANDLORD(S) SIGNATURE
Landlord’s Signature
TENANT(S) SIGNATURE
Tenant’s Signature

B. Prompts
B.1. Prompt PR1
You are a Masters student in Computer Science. Write a smart contract in Solidity based on the following
legal agreement
<legal agreement >

B.2. Prompt PR2

You are a Masters student in Computer Science. Write a smart contract in Solidity based on the following
legal agreement
<legal agreement >
Be creative and use your own style. You will deliver the smart contract to your professor.

B.3. Prompt PR3

You are a Masters student in Computer Science. Write a smart contract in Solidity based on the following
legal agreement
<legal agreement >
Be professional and use a formal style. You will deliver the smart contract to your professor.

B.4. Prompt PR4

You are a Masters student in Computer Science. Write a smart contract in Solidity based on the following
legal agreement
<legal agreement >
The software requirements are:

• target blockchain: Ethereum

20
Fadi Barbàra et al. CEUR Workshop Proceedings 1–24

• Solidity pragma >0.8

• fully defined function logic
• fully defined function logic
• assign value of available parameters
• ready to deploy
• compilable
• secure

Be professional and use a formal style. You will deliver the smart contract to your professor.

B.5. Prompt PR5

You are a Masters student in Computer Science. Write a smart contract in Solidity that reflects payments,
conditions, constraint, and termination of the following legal agreement
<legal agreement >
The software requirements are:

• target blockchain: Ethereum

• Solidity pragma >0.8
• fully defined function logic
• fully defined function logic
• assign value of available parameters
• ready to deploy
• compilable
• secure

Be professional and use a formal style. You will deliver the smart contract to your professor.

B.6. Prompt PR6

You are a junior developer. Write a smart contract in Solidity based on the following legal agreement
<legal agreement >

B.7. Prompt PR7

You are a junior developer. Write a smart contract in Solidity based on the following legal agreement
<legal agreement >
Be creative and use your own style. You will deliver the smart contract to your supervisor.

B.8. Prompt PR8

You are a junior developer. Write a smart contract in Solidity based on the following legal agreement
<legal agreement >
Be professional and use a formal style. You will deliver the smart contract to your supervisor.

B.9. Prompt PR9

You are a junior developer. Write a smart contract in Solidity based on the following legal agreement
<legal agreement >
The software requirements are:

• target blockchain: Ethereum

21
Fadi Barbàra et al. CEUR Workshop Proceedings 1–24

• Solidity pragma >0.8

• fully defined function logic
• fully defined function logic
• assign value of available parameters
• ready to deploy
• compilable
• secure

.
Be professional and use a formal style. You will deliver the smart contract to your supervisor.

B.10. Prompt PR10

You are a junior developer. Write a smart contract in Solidity that reflects payments, conditions,
constraint, and termination of the following legal agreement
<legal agreement >
The software requirements are:

• target blockchain: Ethereum

• Solidity pragma >0.8
• fully defined function logic
• fully defined function logic
• assign value of available parameters
• ready to deploy
• compilable
• secure

Be professional and use a formal style. You will deliver the smart contract to your supervisor.

B.11. Prompt PR11

You are a senior Solidity developer. Write a smart contract in Solidity based on the following legal
agreement
<legal agreement >

B.12. Prompt PR12

You are a senior Solidity developer. Write a smart contract in Solidity based on the following legal
agreement
<legal agreement >
Be creative and use your own style. You will deliver the smart contract to another senior developer.

B.13. Prompt PR13

You are a senior Solidity developer. Write a smart contract in Solidity based on the following legal
agreement
<legal agreement >
Be professional and use a formal style. You will deliver the smart contract to another senior developer.

22
Fadi Barbàra et al. CEUR Workshop Proceedings 1–24

B.14. Prompt PR14

You are a senior Solidity developer. Write a smart contract in Solidity based on the following legal
agreement
<legal agreement >
The software requirements are:
• target blockchain: Ethereum
• Solidity pragma >0.8
• fully defined function logic
• fully defined function logic
• assign value of available parameters
• ready to deploy
• compilable
• secure
Be professional and use a formal style. You will deliver the smart contract to another senior developer.

B.15. Prompt PR15

You are a senior Solidity developer. Write a smart contract in Solidity that reflects payments, conditions,
constraint, and termination of the following legal agreement
<legal agreement >
The software requirements are:
• target blockchain: Ethereum
• Solidity pragma >0.8
• fully defined function logic
• fully defined function logic
• assign value of available parameters
• ready to deploy
• compilable
• secure
Be professional and use a formal style. You will deliver the smart contract to another senior developer.

B.16. Prompt PR16

You are a senior Solidity developer. A company will pay you $500,000 for the completion of the project.
Write a smart contract in Solidity that reflects payments, conditions, constraint, and termination of the
following legal agreement
<legal agreement >
The software requirements are:
• target blockchain: Ethereum
• Solidity pragma >0.8
• fully defined function logic
• fully defined function logic
• assign value of available parameters
• ready to deploy
• compilable
• secure
Be professional and use a formal style. You will deliver the smart contract to another senior developer.

23
Fadi Barbàra et al. CEUR Workshop Proceedings 1–24

B.17. Prompt PR17

You are a senior Solidity developer. A company will pay you $500,000 for the completion of the project.
Write a smart contract in Solidity that reflects payments, conditions, constraint, and termination of the
following legal agreement
<legal agreement >
The software requirements are:

• target blockchain: Ethereum

• Solidity pragma >0.8
• fully defined function logic
• fully defined function logic
• assign value of available parameters
• ready to deploy
• compilable
• secure

Be professional and use a formal style. You will deliver the smart contract to your company’s legal
department.

24