Scribd
Upload
116 views7 pages

Python in High Performance Computing

Uploaded by

Yongki Andita Aiman
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as ODT, PDF, TXT or read online on Scribd
116 views7 pages

Python in High Performance Computing

Uploaded by

Yongki Andita Aiman
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as ODT, PDF, TXT or read online on Scribd
You are on page 1/ 7

Python in High Performance Computing

Partnership for Advanced Computing in Europe (PRACE)

Week 1: Introduction and performance analysis


1.1. Welcome to Python in High Performance Computing
During the following weeks we will discuss various topics related to high-performance computing and
Python.

• Analysing performance of Python programs


• Performing numerical computations efficiently with Python
• Using Python with compiled code
• Parallel programming with Python

Dr. Jussi Enkovaara and Dr. Martti Louhivuori have over ten years of experience in working with
Python in some of the biggest supercomputers in the world, you may follow us during the course:
Please introduce yourself in the comments section below, tell also why you are attending the course!

This is course is brought to you by The Partnership for Advanced Computing in Europe (PRACE)
PRACE is an international non-profit association with its seat in Brussels. The PRACE Research
Infrastructure provides a persistent world-class high performance computing service for scientists and
researchers from academia and industry in Europe. The computer systems and their operations
accessible through PRACE are provided by 5 PRACE members (BSC representing Spain, CINECA
representing Italy, CSCS representing Switzerland, GCS representing Germany and GENCI
representing France). The Implementation Phase of PRACE receives funding from the EU’s Horizon
2020 Research and Innovation Programme (2014-2020) under grant agreements 730913 and 823767.
For more information, see www.prace-ri.eu.”

© CSC - IT Center for Science Ltd.

https://training.prace-ri.eu/
https://prace-ri.eu/
1.2. Prerequisities and structure of the course

In order to succeed in this course, you need to know few things beforehand.
First, this course is aimed at people who already know how to program in Python. We will not be
teaching Python programming as such. However, that said, you do not need to be an expert in Python
programming either.
We expect you to be familiar (and comfortable) with:
• Python syntax
• Basic built-in datastructures (lists, tuples and dictionaries)
• Control structures (if-else, while, for)
• How to write and use functions and modules
• File I/O
You can test if you have the prerequisite level of skills and knowledge by going through the quiz in the
next step.
Second, previous knowledge of high-performance computing is not needed, we will discuss the
relevant topics as we proceed.
Third, the course duration is four weeks. Each week we discuss a different topic related to Python
performance. The main aspects of each topic is covered in articles and videos, but most importantly,
these are followed by hands-on exercises on the topic. At least half of your time should be devoted to
the exercises, since getting your hands dirty and doing it yourself is simply the best way to learn
programming.
We provide simple instructions for setting up a virtual machine with the programming environment for
the course in your own laptop/workstation. The virtual machine is based on Linux, and in order to carry
out the exercises some familiarity with the Linux command line is useful.

FutureLearn platform and social learning


If you have not used the FutureLearn platform previously we suggest that you have a look on the
guidelines.
Social learning is important aspect of the course. That means that in addition to educators you can learn
also from your fellow learners. Therefore we encourage you to leave comments and have a
conversation in the comments sections at the end of the steps. Please ask if there is something you do
not understand, and answer to questions of others if you know the answer. You will find that both
asking and answering questions helps to consolidate not only your but also other people’s knowledge,
and it can be also great fun!
Please remember that because of the scale of the course and volume of comments, educators will not be
able to respond to each and every one of them. If you notice a comment that you think should be
addressed, then like it. Educators will try to reply to the most liked comments.

1.4. Setting up the programming environment


Software
In order to carry out the hands-on programming exercises, you need a working Python environment
that contains a selection of Python packages for scientific and high-performance computing.

The following software will be used in this course:


• Python 3 interpreter
• NumPy
• Numexpr
• Matplotlib
• Cython
• Cffi
• Mpi4py
• (optionally) iPython and Scipy

You have two options:


• Install the software to your own Linux machine
• Use a virtual machine image we have prepared that has all the required software pre-installed

Option 1: Install software to your own Linux machine


In a Ubuntu based Linux distribution, the installation can usually be done (admin rights needed) with
the following commands:
sudo apt install python3-numpy python3-numexpr python3-matplotlib cython3
python3-cffi python3-mpi4py python3-scipy ipython3
For other distributions, please refer to their documentation on how to install the required software.

Option 2: Use the Virtual Machine prepared for this course


1. Download the Virtual Machine image (By the way, we are using the same server where Linux was
first released to the world in 1991.) Size of download is about 1.9 GB
2. Install the VirtualBox software
• Download the installation file for your operating system (Windows, Mac, and Linux are all
supported): https://www.virtualbox.org/
• When installing the software you might be asked to install some device drivers (Oracle
Corporation Universal Serial Bus, Network Adapters, Network Service). Install them as well. If
you are asked, choose to install all the components of the VirtualBox software.
• After installation you will be notified that the installation is complete and you can already start
Oracle VM VirtualBox in that step by clicking finish.
• In some machines virtualization hardware extension may need to be enabled in BIOS. Some
instructions on how to achieve this can be found here.
3. Start VirtualBox
4. Import the Virtual Machine image
• In the File menu choose Import appliance
• Select the HPCPython.ova image you downloaded in step 1.
• Default values for memory and number of CPUs should be OK, but you can also increase them
a bit depending on your system

5. You should now see HPC Python image listed in VirtualBox. You can start it either by double
clicking it or via the Start button.
6. The system should now boot up. Once you are greeted with a login screen, log in with the following
credentials:
• username: Monty Python
• password: hpc1python
7. Hands-on exercises are carried out in the command line terminal which you can open from the
launcher panel on the left.

• There are several standard text editors (gedit, nano, emacs, vim) available, if you are not
familiar with any of these we recommend starting with gedit.

Exercise material
In addition to installing the required software (see above), you need to download also some material for
the hands-on exercises.
Exercise material is hosted on GitHub at: https://github.com/csc-training/hpc-python

Download
You have three options for downloading the material to your own Linux system or to the Virtual
Machine:
1. Recommended approach: Fork the GitHub repository and clone your fork
• You need to have a GitHub user account for this option
• Go to course repository in GitHub and Sign in
• Next, Fork the repository via the button in the top right corner.
• After forking the repository, open the Terminal and clone your fork with the command (using
your own GitHub username):
git clone https://github.com/my-github-username/hpc-python.git
An easy way to get the URL for cloning, is to copy it from the green Clone or download button
on the Github page of your fork.
• No further usage of git is required in the course, but if you are familiar with it we strongly
recommend committing often and pushing your work back to your own GitHub.
2. Clone the repository directly
• Open the Terminal and clone the repository directly with the command:
git clone https://github.com/csc-training/hpc-python.git
• However, with this option, any changes you make will only be available locally and can not be
pushed back to Github
3. (Not recommended): If needed, you can also download all the material via from the Clone or
download button as a Zip-file. However, this option means you will loose all the benefits of version
control.

Overall structure
Skeleton code snippets and model solutions to hands-on exercises are in separate directories for each
exercise. The exercises are organised in subdirectories (mpi/, numpy/ etc.) for each topic under the
main directory (hpc-python/).
The file README.md contains a list of all exercises (with links), which is also shown as the default
page on Github, and can be an easy way to navigate to an exercise.
Each exercise has also a README.md file that contains the instructions for the exercise and a solution/
directory that contains a model solution. Additional files (skeleton code, input data etc.) may also be
present.

Test the software installation


Go into the test subdirectory and execute the test set:
pythonuser@ubuntu:~/hpc-python$ cd test
pythonuser@ubuntu:~/hpc-python/test$ python3 test.py
If everything is fine, you should see Test set passed printed on the screen.
Please comment if you have any problems setting up the environment or downloading the exercises.
Git Tutorial: https://coderefinery.github.io/git-intro/

© CC-BY-NC-SA 4.0 by CSC - IT Center for Science

You might also like