Skip to content

Commit 3c0de66

Browse files
authored
Merge pull request #583 from stratika/feat/idea-xml-generation
Automated generation of IDE profile files from tornado command
2 parents 2054d43 + e7fcbee commit 3c0de66

File tree

9 files changed

+218
-59
lines changed

9 files changed

+218
-59
lines changed

.build/.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# Ignore everything in this directory
2+
# except this file
3+
!.gitignore

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
.idea/
22
.unittestingStatus
3+
.build/*.xml
34
beehive-spirv-toolkit/
45
bin/__pycache__/
56
bin/bin

docs/source/faq.rst

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,16 +18,16 @@ API is under **Apache 2**, and hence it can be freely used in any application.
1818
3. How can I use it?
1919
--------------------
2020

21-
In Linux and Mac OSx, TornadoVM can be installed by the `installer <https://github.com/beehive-lab/tornadovm-installer>`__.
21+
In Linux and Mac OSx, TornadoVM can be installed by the `installer <https://github.com/beehive-lab/TornadoVM/blob/master/bin/tornadovm-installer>`__.
2222
Alternatively, TornadoVM can be configured either manually (:ref:`installation`) or by using docker images (:ref:`docker`).
2323

2424
List of compatible JDKs
2525
^^^^^^^^^^^^^^^^^^^^^^^
2626

2727
TornadoVM can be currently executed with the following configurations:
2828

29-
- TornadoVM with GraalVM (JDK 21): see the installation guide: :ref:`installation_graalvm`.
30-
- TornadoVM with JDK21 (e.g. OpenJDK 21, Red Hat Mandrel 21, Amazon Corretto 21, Azul Zulu JDK 21): see the installation guide: :ref:`installation_jdk17plus`.
29+
- TornadoVM with GraalVM (JDK 21).
30+
- TornadoVM with JDK21 (e.g. OpenJDK 21, Red Hat Mandrel 21, Amazon Corretto 21, Azul Zulu JDK 21).
3131

3232
Windows
3333
~~~~~~~~~~

docs/source/ide-integration.rst

Lines changed: 13 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -152,41 +152,19 @@ b. In the **Modules** tab:
152152

153153
- Ensure that the *Language level* of every module matches the project level (e.g. Java 21).
154154

155-
Configuring the TornadoVM Utilities
156-
***********************************
155+
Configuring IntelliJ for TornadoVM
156+
**********************************
157157

158-
1. Configuring the TornadoVM Maven Build Utility
159-
================================================
158+
1. Initializing the IntelliJ Project Files
159+
==========================================
160160

161-
a. Navigate to the Maven configuration
162-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
161+
To initialize IDE project files for building and running TornadoVM from IntelliJ, you must have first built TornadoVM and loaded the file with the environment variables (``setvars.sh``, ``setvars.cmd``), as explained in the :ref:`installation`.
163162

164-
Go to **Run > Edit Configurations > Maven > _internal_TornadoVM_Maven-cleanAndinstall_**
163+
Then you can execute the command to generate the IDE project files based on your built TornadoVM instance (i.e., with the JAVA_HOME and the backends), as follows:
165164

166-
b. Set up the build profiles
167-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
168-
In the **Profiles** field, list the profiles you want to build, separated by spaces.
169-
170-
**Examples**:
171-
172-
- To build with the *graal-jdk-21* and *opencl-backend* profiles, type:
173-
174-
.. code:: text
175-
176-
graal-jdk-21 opencl-backend
177-
178-
- To build with all backends, type:
179-
180-
.. code:: text
181-
182-
graal-jdk-21 opencl-backend ptx-backend spirv-backend
183-
184-
c. Check available profiles
185-
~~~~~~~~~~~~~~~~~~~~~~~~~~~
186-
187-
You can find the available profiles in the right-hand vertical bar in IntelliJ under **Maven > Profiles**.
165+
.. code:: bash
188166
189-
**Important:** Even though profiles are listed in the Maven pane, you must explicitly configure them in the **_internal_TornadoVM_Maven-cleanAndinstall_** utility. The enablement/disablement of profiles in the Maven pane does not always reflect in this utility.
167+
$ tornado --intellijinit
190168
191169
2. Configuring the TornadoVM Python Build Utility
192170
=================================================
@@ -204,17 +182,13 @@ In the **Use specified interpreter** field, select a valid Python interpreter in
204182
c. Update environment variables for selected backends
205183
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
206184

207-
In the **Environmental variables** section, locate the `selected_backends` field. Update the list of backends you want to use, separated by commas.
208-
209-
**Examples**:
210-
211-
- To use all backends, set the value to:
185+
The **Environmental variables** section has been populated based on your built TornadoVM instance.
186+
If you change ``JAVA_HOME`` or built with different backends, you will need to run the ``tornado --intellijinit`` command.
212187

213-
.. code:: text
214-
215-
opencl-backend,ptx-backend,spirv-backend
188+
d. Build TornadoVM from IntelliJ
189+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
216190

217-
Apply to save your settings and run the build by clicking **Run TornadoVM-Build**.
191+
Run a new build by clicking **Run TornadoVM-Build**.
218192

219193
Configuring Applications to Debug/Run
220194
*************************************

.build/_internal_TornadoVM_Maven-cleanAndinstall.run.xml renamed to scripts/templates/intellij-settings/ideinit/maven_template.xml

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
12
<component name="ProjectRunConfigurationManager">
23
<configuration default="false" name="_internal_TornadoVM_Maven-cleanAndinstall" type="MavenRunConfiguration" factoryName="Maven">
34
<MavenSettings>
@@ -8,7 +9,7 @@
89
<option name="emulateTerminal" value="true" />
910
<option name="failureBehavior" value="NOT_SET" />
1011
<option name="localRepository" value="" />
11-
<option name="mavenHome" value="$PROJECT_DIR$/etc/dependencies/apache-maven-3.9.3" />
12+
<option name="mavenHome" value= "[@@MAVEN_DIRECTORY@@]" />
1213
<option name="nonRecursive" value="false" />
1314
<option name="outputLevel" value="INFO" />
1415
<option name="printErrorStackTraces" value="false" />
@@ -36,15 +37,15 @@
3637
<option name="pomFileName" />
3738
<option name="profilesMap">
3839
<map>
39-
<entry key="graal-jdk-21" value="true" />
40-
<entry key="opencl-backend" value="true" />
40+
<entry key="[@@JAVA_PROFILE@@]" value="true" />
41+
[@@BACKEND_PROFILES@@]
4142
</map>
4243
</option>
4344
<option name="projectsCmdOptionValues">
4445
<list />
4546
</option>
4647
<option name="resolveToWorkspace" value="false" />
47-
<option name="workingDirPath" value="$PROJECT_DIR$" />
48+
<option name="workingDirPath" value="[@@PROJECT_DIR@@]" />
4849
</MavenRunnerParameters>
4950
</option>
5051
</MavenSettings>

.build/TornadoVM-Build.run.xml renamed to scripts/templates/intellij-settings/ideinit/tornadovm_build_template.xml

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
12
<component name="ProjectRunConfigurationManager">
23
<configuration default="false" name="TornadoVM-Build" type="PythonConfigurationType"
34
factoryName="Python">
@@ -7,17 +8,18 @@
78
<envs>
89
<env name="PYTHONUNBUFFERED" value="1"/>
910
<env name="BACKEND" value="opencl"/>
10-
<env name="TORNADO_SDK" value="$PROJECT_DIR$/bin/sdk"/>
11-
<env name="selected_backends" value="opencl-backend"/>
11+
<env name="TORNADO_SDK" value="[@@TORNADO_SDK@@]"/>
12+
<env name="JAVA_HOME" value="[@@JAVA_HOME@@]" />
13+
<env name="selected_backends" value="[@@BACKEND_PROFILES@@]"/>
1214
</envs>
13-
<option name="SDK_HOME" value=""/>
14-
<option name="SDK_NAME" value="Python 3.9 (tornado)"/>
15-
<option name="WORKING_DIRECTORY" value="$PROJECT_DIR$"/>
15+
<option name="SDK_HOME" value="[@@PYTHON_SDK_HOME@@]"/>
16+
<option name="SDK_NAME" value="[@@PYTHON_SDK_NAME@@]"/>
17+
<option name="WORKING_DIRECTORY" value="[@@PROJECT_DIR@@]"/>
1618
<option name="IS_MODULE_SDK" value="false"/>
1719
<option name="ADD_CONTENT_ROOTS" value="true"/>
1820
<option name="ADD_SOURCE_ROOTS" value="true"/>
1921
<EXTENSION ID="PythonCoverageRunConfigurationExtension" runner="coverage.py"/>
20-
<option name="SCRIPT_NAME" value="$PROJECT_DIR$/bin/post_installation.py"/>
22+
<option name="SCRIPT_NAME" value="[@@POST_INSTALLATION_SCRIPT_DIRECTORY@@]"/>
2123
<option name="PARAMETERS" value=""/>
2224
<option name="SHOW_COMMAND_LINE" value="false"/>
2325
<option name="EMULATE_TERMINAL" value="false"/>

.build/TornadoVM-Tests.run.xml renamed to scripts/templates/intellij-settings/ideinit/tornadovm_tests_template.xml

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
12
<component name="ProjectRunConfigurationManager">
23
<configuration default="false" name="TornadoVM-Tests" type="PythonConfigurationType" factoryName="Python">
34
<module name="tornado-annotation" />
@@ -7,18 +8,18 @@
78
<envs>
89
<env name="BACKEND" value="opencl" />
910
<env name="PYTHONUNBUFFERED" value="1" />
10-
<env name="selected_backends" value="opencl-backend" />
11-
<env name="TORNADO_SDK" value="$PROJECT_DIR$/bin/sdk" />
12-
<env name="JAVA_HOME" value="$PROJECT_DIR$/etc/dependencies/TornadoVM-graal-jdk-21/graalvm-community-openjdk-21.0.1+12.1" />
11+
<env name="selected_backends" value="[@@BACKEND_PROFILES@@]"/>
12+
<env name="TORNADO_SDK" value="[@@TORNADO_SDK@@]" />
13+
<env name="JAVA_HOME" value="[@@JAVA_HOME@@]" />
1314
</envs>
14-
<option name="SDK_HOME" value="" />
15-
<option name="SDK_NAME" value="Python 3.10" />
16-
<option name="WORKING_DIRECTORY" value="$PROJECT_DIR$" />
15+
<option name="SDK_HOME" value="[@@PYTHON_SDK_HOME@@]"/>
16+
<option name="SDK_NAME" value="[@@PYTHON_SDK_NAME@@]"/>
17+
<option name="WORKING_DIRECTORY" value="[@@PROJECT_DIR@@]" />
1718
<option name="IS_MODULE_SDK" value="false" />
1819
<option name="ADD_CONTENT_ROOTS" value="true" />
1920
<option name="ADD_SOURCE_ROOTS" value="true" />
2021
<EXTENSION ID="PythonCoverageRunConfigurationExtension" runner="coverage.py" />
21-
<option name="SCRIPT_NAME" value="$PROJECT_DIR$/bin/sdk/bin/tornado-test" />
22+
<option name="SCRIPT_NAME" value="[@@TORNADO_TEST_DIR@@]" />
2223
<option name="PARAMETERS" value="--ea --verbose --quickPass" />
2324
<option name="SHOW_COMMAND_LINE" value="false" />
2425
<option name="EMULATE_TERMINAL" value="false" />
Lines changed: 171 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,171 @@
1+
#!/usr/bin/env python3
2+
3+
#
4+
# Copyright (c) 2024, APT Group, Department of Computer Science,
5+
# The University of Manchester.
6+
#
7+
# Licensed under the Apache License, Version 2.0 (the "License");
8+
# you may not use this file except in compliance with the License.
9+
# You may obtain a copy of the License at
10+
#
11+
# http://www.apache.org/licenses/LICENSE-2.0
12+
#
13+
# Unless required by applicable law or agreed to in writing, software
14+
# distributed under the License is distributed on an "AS IS" BASIS,
15+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
16+
# See the License for the specific language governing permissions and
17+
# limitations under the License.
18+
#
19+
20+
import os
21+
import subprocess
22+
import sys
23+
import xml.etree.ElementTree as ET
24+
from pathlib import Path
25+
26+
def generate_backend_profiles_as_xml_entries(BACKEND_PROFILES):
27+
xml_entries = []
28+
for backend in BACKEND_PROFILES:
29+
xml_entry_line = f'<entry key="{backend}" value="true" />'
30+
xml_entries.append(xml_entry_line)
31+
32+
return "\n".join(xml_entries)
33+
34+
def read_template(template_path):
35+
with open(template_path, 'r') as file:
36+
template_content = file.read()
37+
return template_content
38+
39+
def write_generated_template(xml_content, xml_file_path):
40+
with open(xml_file_path, "w") as file:
41+
file.write(xml_content)
42+
43+
def define_and_get_internal_maven_content(xml_templates_directory, project_directory, java_home, backend_profiles):
44+
maven_directory = os.path.join(project_directory, "etc", "dependencies", "apache-maven-3.9.3")
45+
xml_internal_maven_build_content_directory = os.path.join(xml_templates_directory, "maven_template.xml")
46+
xml_internal_maven_build_content = read_template(xml_internal_maven_build_content_directory)
47+
48+
if "graal" in java_home:
49+
java_profile = "graal-jdk-21"
50+
else:
51+
java_profile = "jdk21"
52+
53+
xml_backend_profiles = generate_backend_profiles_as_xml_entries(backend_profiles)
54+
55+
xml_internal_maven_build_content = xml_internal_maven_build_content.replace("[@@MAVEN_DIRECTORY@@]", maven_directory)
56+
xml_internal_maven_build_content = xml_internal_maven_build_content.replace("[@@JAVA_PROFILE@@]", java_profile)
57+
xml_internal_maven_build_content = xml_internal_maven_build_content.replace("[@@BACKEND_PROFILES@@]", xml_backend_profiles)
58+
xml_internal_maven_build_content = xml_internal_maven_build_content.replace("[@@PROJECT_DIR@@]", project_directory)
59+
60+
return xml_internal_maven_build_content
61+
62+
def define_and_get_tornadovm_build_content(xml_templates_directory, project_directory, tornado_sdk, java_home, python_home, backend_profiles):
63+
post_installation_script_directory = os.path.join(project_directory, "bin", "post_installation.py")
64+
xml_TornadoVM_build_content_directory = os.path.join(xml_templates_directory, "tornadovm_build_template.xml")
65+
xml_TornadoVM_build_content = read_template(xml_TornadoVM_build_content_directory)
66+
67+
python_version = subprocess.check_output([str(python_home), "--version"]).decode().strip()
68+
python_version_name = python_version.split()[1]
69+
python_sdk_home = str(python_home)
70+
python_sdk_name = f"Python {python_version_name}"
71+
72+
xml_TornadoVM_build_content = xml_TornadoVM_build_content.replace("[@@TORNADO_SDK@@]", tornado_sdk)
73+
xml_TornadoVM_build_content = xml_TornadoVM_build_content.replace("[@@JAVA_HOME@@]", java_home)
74+
xml_TornadoVM_build_content = xml_TornadoVM_build_content.replace("[@@BACKEND_PROFILES@@]", backend_profiles)
75+
xml_TornadoVM_build_content = xml_TornadoVM_build_content.replace("[@@PYTHON_SDK_HOME@@]", python_sdk_home)
76+
xml_TornadoVM_build_content = xml_TornadoVM_build_content.replace("[@@PYTHON_SDK_NAME@@]", python_sdk_name)
77+
xml_TornadoVM_build_content = xml_TornadoVM_build_content.replace("[@@PROJECT_DIR@@]", project_directory)
78+
xml_TornadoVM_build_content = xml_TornadoVM_build_content.replace("[@@POST_INSTALLATION_SCRIPT_DIRECTORY@@]", post_installation_script_directory)
79+
80+
return xml_TornadoVM_build_content
81+
82+
def define_and_get_tornadovm_tests_content(xml_templates_directory, project_directory, tornado_sdk, java_home, python_home, backend_profiles):
83+
test_script_directory = os.path.join(tornado_sdk, "bin", "tornado-test")
84+
xml_TornadoVM_tests_content_directory = os.path.join(xml_templates_directory, "tornadovm_tests_template.xml")
85+
xml_TornadoVM_tests_content = read_template(xml_TornadoVM_tests_content_directory)
86+
87+
python_version = subprocess.check_output([str(python_home), "--version"]).decode().strip()
88+
python_version_name = python_version.split()[1]
89+
python_sdk_home = str(python_home)
90+
python_sdk_name = f"Python {python_version_name}"
91+
92+
xml_TornadoVM_tests_content = xml_TornadoVM_tests_content.replace("[@@BACKEND_PROFILES@@]", backend_profiles)
93+
xml_TornadoVM_tests_content = xml_TornadoVM_tests_content.replace("[@@TORNADO_SDK@@]", tornado_sdk)
94+
xml_TornadoVM_tests_content = xml_TornadoVM_tests_content.replace("[@@JAVA_HOME@@]", java_home)
95+
xml_TornadoVM_tests_content = xml_TornadoVM_tests_content.replace("[@@PYTHON_SDK_HOME@@]", python_sdk_home)
96+
xml_TornadoVM_tests_content = xml_TornadoVM_tests_content.replace("[@@PYTHON_SDK_NAME@@]", python_sdk_name)
97+
xml_TornadoVM_tests_content = xml_TornadoVM_tests_content.replace("[@@PROJECT_DIR@@]", project_directory)
98+
xml_TornadoVM_tests_content = xml_TornadoVM_tests_content.replace("[@@TORNADO_TEST_DIR@@]", test_script_directory)
99+
100+
return xml_TornadoVM_tests_content
101+
102+
def generate_internal_maven_build_xml(xml_templates_directory, project_directory, java_home, backend_profiles):
103+
xml_internal_content = define_and_get_internal_maven_content(xml_templates_directory, project_directory, java_home, backend_profiles)
104+
xml_build_directory = os.path.join(project_directory, ".build", "_internal_TornadoVM_Maven-cleanAndinstall.run.xml")
105+
print("Generating " + xml_build_directory)
106+
write_generated_template(xml_internal_content, xml_build_directory)
107+
108+
def generate_tornadovm_build_xml(xml_templates_directory, project_directory, tornado_sdk, java_home, python_home, backend_profiles):
109+
xml_build_content = define_and_get_tornadovm_build_content(xml_templates_directory, project_directory, tornado_sdk, java_home, python_home, backend_profiles)
110+
xml_build_directory = os.path.join(project_directory, ".build", "TornadoVM-Build.run.xml")
111+
print("Generating " + xml_build_directory)
112+
write_generated_template(xml_build_content, xml_build_directory)
113+
114+
def generate_tornadovm_tests_xml(xml_templates_directory, project_directory, tornado_sdk, java_home, python_home, backend_profiles):
115+
xml_tests_content = define_and_get_tornadovm_tests_content(xml_templates_directory, project_directory, tornado_sdk, java_home, python_home, backend_profiles)
116+
xml_tests_directory = os.path.join(project_directory, ".build", "TornadoVM-Tests.run.xml")
117+
print("Generating " + xml_tests_directory)
118+
write_generated_template(xml_tests_content, xml_tests_directory)
119+
120+
def generate_xml_files(xml_templates_directory, project_directory, tornado_sdk, java_home, python_home, backends):
121+
backends_separated_comma = ",".join(backends)
122+
generate_internal_maven_build_xml(xml_templates_directory, project_directory, java_home, backends)
123+
generate_tornadovm_build_xml(xml_templates_directory, project_directory, tornado_sdk, java_home, python_home, backends_separated_comma)
124+
generate_tornadovm_tests_xml(xml_templates_directory, project_directory, tornado_sdk, java_home, python_home, backends_separated_comma)
125+
print("IntelIj Files Generated ............... [ok]")
126+
127+
def cleanup_build_directory(build_directory_string):
128+
tornado_root_directory = Path(build_directory_string)
129+
if tornado_root_directory.exists() and tornado_root_directory.is_dir():
130+
for file in tornado_root_directory.iterdir():
131+
if file.name == ".gitignore":
132+
continue
133+
if file.is_file():
134+
file.unlink()
135+
136+
def tornadovm_ide_init(tornado_sdk, java_home, backends):
137+
"""
138+
Function to generate IDEA xml files for building and running TornadoVM.
139+
"""
140+
141+
if tornado_sdk == None:
142+
print("Cannot initiate ide. TORNADO_SDK is not defined")
143+
sys.exit(0)
144+
145+
if java_home == None:
146+
print("Cannot initiate ide. JAVA_HOME is not defined")
147+
sys.exit(0)
148+
149+
if backends == None:
150+
print("Cannot initiate ide. Backends are not defined")
151+
sys.exit(0)
152+
153+
if os.name == 'nt':
154+
# Multiple paths of python executables may be listed, the first is selected as default
155+
python_home = subprocess.check_output(["where", "python"]).decode().strip().split("\r\n")
156+
python_home = Path(python_home[0])
157+
else:
158+
python_home = subprocess.check_output(["which", "python3"]).decode().strip()
159+
160+
if python_home == None:
161+
print("python is not found in the PATH.")
162+
sys.exit(0)
163+
164+
tornado_sdk_path = Path(tornado_sdk)
165+
tornado_root = tornado_sdk_path.parents[1]
166+
project_directory = str(tornado_root)
167+
168+
cleanup_build_directory(os.path.join(project_directory, ".build"))
169+
xml_templates_directory = os.path.join(project_directory, "scripts", "templates", "intellij-settings", "ideinit")
170+
171+
generate_xml_files(xml_templates_directory, project_directory, tornado_sdk, java_home, python_home, backends)

tornado-assembly/src/bin/tornado

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@ import re
2424
import shlex
2525
import subprocess
2626
import sys
27+
import idea_xml_utils as ideaUtils
2728
from pathlib import Path
2829

2930
# ########################################################
@@ -394,6 +395,10 @@ class TornadoVMRunnerTool():
394395
print(javaFlags)
395396
sys.exit(0)
396397

398+
if (args.intellijinit):
399+
ideaUtils.tornadovm_ide_init(os.environ['TORNADO_SDK'], self.java_home, self.listOfBackends)
400+
sys.exit(0)
401+
397402
if (args.showDevices):
398403
command = javaFlags + "uk.ac.manchester.tornado.drivers.TornadoDeviceQuery verbose"
399404
os.system(command)
@@ -467,6 +472,7 @@ def parseArguments():
467472
parser.add_argument("application", nargs="?")
468473
parser.add_argument("--truffle", action="store", dest="truffle_language", default=None,
469474
help="Enable Truffle languages through TornadoVM. Example: --truffle python|r|js")
475+
parser.add_argument("--intellijinit", action="store_true", dest="intellijinit", default=False, help="Generate internal xml files for IntelliJ IDE")
470476
parser.add_argument("param1", nargs="?")
471477
parser.add_argument("param2", nargs="?")
472478
parser.add_argument("param3", nargs="?")

0 commit comments

Comments
 (0)