ENGG1003

Python Applications on
Jupyter Notebook
Outline
• Introducing Jupyter Notebook

• Using Jupyter Notebook on VM

• Using Notebook on Google Colab

• Application Examples and Demonstrations

2
Open Platform
for All
• https://jupyter.org/
• "Free software, open standards, and web services
for interactive computing across all programming
languages"
• "Project Jupyter is a non-profit, open-source
project, born out of the IPython Project in 2014 as
it evolved to support interactive data science and
scientific computing across all programming
languages"
• Both Python and R are supported
• Jupyter Notebook is a "classic" tool in data science

3
Jupyter Notebook
• Web-based notebook interface front-end

• Supports Read-Evaluate-Print-Loop (REPL) interactivity

(like Python IDLE running in a web-browser)

• Supports both code, text, math formula, graph plots and

rich media

• A notebook is kept as a ".ipynb" document

4
Jupyter Kernel
• Kernel is the power-house back-end, supporting the operations
of Jupyter Notebook

• Usually, Python kernel is supported by default

• Program code are sent from the Notebook to the kernel

• Data processing results are delivered from the kernel to the

Notebook, showing on a web-browser
5
 First Taste of Jupyter Notebook
https://jupyter.org/try-jupyter/notebooks/?path=notebooks/Intro.ipynb

6
Setup and Run Jupyter
Notebook
• Download and install on your own computer

• Run on provided VM
• https://labs.azure.com/virtualmachines

• Run on Google Colaboratory

• https://colab.research.google.com

• Run on DataCamp Workspace

• https://app.datacamp.com/workspace
7
Data Science Lab on Cloud
https://colab.research.google.com
• Google Colab – web-based computing environment
through browser

Terminal Google Colab

- Free and accessible from CU & HK
- Just need a Google account
Python
Browser
IDLE
- Web interface, NO setup needed
- Store work on Google Drive
- Interactive

8
Data Science Workspace on Cloud
https://app.datacamp.com/workspace
• DataCamp Workspace – web-based computing
environment through browser

Terminal DataCamp Workspace

- Free and accessible from CU & HK
- Just need a DataCamp account
Python
Browser
IDLE
- Web interface, NO setup needed
- Interactive

9
Web-based Data
Processing with Jupyter
Notebook
• Retrieve and process online data on the fly

• Share notebooks for online cooperation and

presentation

• Intuitive and interactive data presentation and

visualization

10
Python IDLE or Jupyter
Notebook?
• If you are building and distributing standalone
Python applications, you may work on Python IDLE
• If you are building and sharing Notebooks that can
be used on web-browsers, use Jupyter Notebook
• For the course project, you shall use Excel and/or
Jupyter Notebook
• You may perform some data analysis on Orange and
paste those results and figures into Jupyter Notebook !
11
Examples Do NOT Work ?!
• If you try the examples with Python IDLE on your own computer,
they may not work!
• Moreover, Python IDLE cannot handle Notebooks!

• Beginner's own Python IDLE installation may not have libraries and
packages, such as pandas, matplotlib, numpy, scipy, scikit-learn,
pytorch, tensorflow, etc.

• So, please try on the provided VM or Google Colab

• https://labs.azure.com/virtualmachines
• https://colab.research.google.com
12
 Reading Data from the
Web (URL)
• Get data from the Internet on the fly; not local files!
• Tabular format: Comma Separated Values (CSV)
• Use package pandas:

import pandas as pd
header = {'User-Agent': 'Mozilla/5.0'}
ambulance_service_indicators = pd.read_csv("http://www.hkfsd.gov.hk/" +
"datagovhk/datasets/Ambulance_Service_Indicators_eng.csv",
storage_options=header)
print( ambulance_service_indicators.describe() )

Source: http://www.hkfsd.gov.hk/datagovhk/datasets/Ambulance_Service_Indicators_eng.csv 13
Reading Data from the
Web (URL)
• Use package ssl first, then pandas:
• Sacrificing security as a temporary measure:
# On Azure Labs VM Win10 or Python 3.10, there may be ERROR due to failure in
Problem resolved
verifying the web-site digital certificates on"Hongkong
issued by MS Azure Post
Labs VM by
e-Cert
SSL CA 3 - 17", here is a workaround:
import ssl technically post-installing HK Post CA Certificates !
ssl._create_default_https_context = ssl._create_unverified_context

import pandas as pd
header = {'User-Agent': 'Mozilla/5.0'}
ambulance_service_indicators = pd.read_csv("http://www.hkfsd.gov.hk/" +
"datagovhk/datasets/Ambulance_Service_Indicators_eng.csv",
storage_options=header)
print( ambulance_service_indicators.describe() )

14
Ambulance
Service
Indicators -
Data from
HK Fire
Services
Departme
nt
15
Demonstration: Plot Data
in a CSV
import pandas as pd

header = {'User-Agent': 'Mozilla/5.0'}

ambulance_service_indicators = pd.read_csv("http://www.hkfsd.gov.hk/" +

"datagovhk/datasets/Ambulance_Service_Indicators_eng.csv", storage_options=header)

print(ambulance_service_indicators.keys()) # keys (field names) is a list of string

print(ambulance_service_indicators[

["Ambulance Service Indicators", "no. of emergency calls"] ] )

print("Mean calls per ambulance:",

ambulance_service_indicators["calls per ambulance"].mean())

import matplotlib.pyplot as plt

plt.plot(ambulance_service_indicators["Ambulance Service Indicators"],

ambulance_service_indicators["no. of emergency calls"])

plt.ylim( bottom = 0 ) # avoid BROKEN y-axis

plt.grid()

plt.show()

16
Using Jupyter Notebook on
VM
• Start Jupyter Notebook by double-clicking on the
provided desktop icon
• There will be a "Command Prompt" window appearing
• Jupyter Server and Python Kernel is running there!
• Do NOT touch it or close it!

17
Using Jupyter Notebook on
VM
• A browser window will then pop-up
• This is your workspace ( http://localhost:8888 )
• It looks like Explorer/ Finder window in your browser
• When you click [Quit], Jupyter will shutdown and the
URL ( http://localhost:8888 ) will stop working

18
Get Notebooks on VM
• Have example Notebooks ready in two ways:

• Login Blackboard in your VM, download from course

• Login Blackboard on your computer, download from course,

THEN copy-and-paste to your VM over Remote Desktop

• On your VM, SAVE to Documents folder

19
Click to Open Notebook or
New

20
Notebook in Another
Browser Tab

21
Notebook Cell Type

Markdown cell

Markdown cell

Code cell

Raw (Plaintext cell)

Code cell

22
 Markdown
• Our old friend: HyperText Markup Language (HTML)
• Full featured
• Markup using <tags> for styling and formatting content

• "Markdown is a lightweight markup language for creating formatted

text using a plain-text editor."

# Heading #
**Bold** _italic_
Horizontal rule:
---
[Hyperlink](http://localhost:8888)
Quote from: Wikipedia contributors, "Markdown," Wikipedia, The Free Encyclopedia,
23
https://en.wikipedia.org/w/index.php?title=Markdown&oldid=1076766851 (accessed March 22, 2022).
Code Cells
• Edit and Run Python code in cells
• The Python Kernel keeps track of the running states
in a Notebook
• Store a value in a variable, like:
x=7
• Subsequent cells can make use of x:
y=x+8
• print( ) is assumed in many cases, showing immediate
results

24
Running a Notebook
• Run All
• Run cells one-by-one
• Save / auto-save (file-type: .ipynb)
• Save and Checkpoint for version-control
• File  Download as
• Export to different document formats
• Edit  Insert Image

25
Using Notebook on Google
Colab
• Google Account is required

• It is basically free, but without service guarantee

• You may use Google Colab on your computer

OR mobile devices
OR on VM !
• Some regions may not be able to use Google services
• Try over VPN or VM !
26
Google Colab [DigLitCUHK]
Shared
Ambulance Services Stat
Example
Save you own copy!

https://drive.google.com/file/d/1kukGM4uGWkAHrJvk
mkGtAUE7rah6lfdS/view?usp=sharing

27
Manage Colab Notebooks
on Google Drive

Share!

Drag-and-drop / Upload / Download

files to/from your device or VM
28
Application Example and
Demonstrations
• More Jupyter Notebook examples are available on
Blackboard
• Download and save NotebookExamples.zip to
Documents
• Right-click on the ZIP file  Extract All…
• Fire up Jupyter Notebook, enter Documents folder
• When you get lost, browse http://localhost:8888

29
Summary
• Interactive computing, data science and visual
presentation with Jupyter Notebook

• Markdown and Code cells in Notebook

• Using Notebook (.ipynb) on VM and Google Colab

30