Distributed Computing
Models
1
Computing Paradigm Shift
2
Mainframe Computing
• Jobs
• Batches
• Processing
• To carry out some mundane and
routine jobs such as payroll,
accounts, inventory thus sparing
employees from tedious jobs.
• It was available in one location,
and anyone who needs it must
go to computer center for
availing it.
3
Personal Computing
• Desktop computing - personal
computer small enough to fit
conveniently in an individual
workspace.
• Providing computers to each
employee on their desktop or
workspace.
• Decentralized computing.
• Less expensive, easy to upgrade and
less accessories needed.
• Information sharing with other users
is a tedious process.
4
Network Computing
• Networked computers - Local
Area network (LAN) achieved
this.
• In the networked computing
model- a relatively powerful
computer- server is loaded with
all software needed
• Each user is provided with a
connected- terminal to access
and work
5
Internet Computing
• Network computing such as LAN
connected users within an office
or institutions.
• Internet computing - connect
organizations located in different
geographical locations.
6
Utility Computing
• Conventional Internet hosting services have the capability to quickly arrange for the rental of
individual servers, for example to provision a bank of web servers to accommodate a sudden
surge in traffic to a web site.
• “Utility computing” usually envisions some form of virtualization so that the amount of storage or
computing power available is considerably larger than that of a single time-sharing computer.
Multiple servers are used on the “back end” to make this possible.
• These might be a dedicated computer cluster specifically built for the purpose of being rented
out, or even an under-utilized supercomputer.
• The technique of running a single calculation on multiple computers is known as distributed
computing.
7
Cluster Computing
• A computer cluster is a group of linked computers,
working together closely thus in many respects forming a
single computer.
• The components of a cluster are connected to each other
through fast local area networks.
• Clusters are mainly used for load balancing and providing
high availability.
• Requirements for such a computing increasing fast.
• More data to process.
• More compute intensive algorithms available.
8
Cluster Computing
Benefits Drawbacks
• High Availability. • Problem in Finding Fault.
• Reducing cost. • The machines in a cluster are
dedicated to work as a single unit.
• Manageability.
• The computers in the cluster are
normally contained in a single
location.
9
Grid Computing
• Computing power available within an
enterprise is not sufficient to carry out
the computing task.
• Data required for the processing is
generated at various geographical
locations.
• GC requires the use of software that
can divide and farm out pieces of a
program as one large system image to
several thousand computers.
10
Grid Computing
Benefits Drawbacks
• Enables applications to be easily • Proprietary approach should be
scaled eliminated
• Better utilization of underused • There is a single point of failure
resources if one unit on the grid degrades
• Parallelization of processing • No pay as you go
11
Grid Computing vs Cluster
Computing
• Cluster is homogenous.
• The cluster computers all have the same hardware and OS.
• The computers in the cluster are normally contained in a single
location
• Grids are heterogeneous.
• Run different operating systems and have different hardware.
• Grids are inherently distributed by its nature over a LAN, metropolitan
or WAN.
12
Cloud Computing
• Cloud Computing refers to both the applications
delivered as services over the Internet and the
hardware and systems software in the
datacenters that provide those services.
• The datacenter hardware and software is what
we will call a Cloud.
13
Cloud Computing
Benefits Drawbacks
• Disaster recovery • Constant Internet Connection
• Increased Scalability • High Speed Internet Required
• Faster Deployment • Data Stored is not secure
• Metered Service
• Highly Automated
14
Grid Vs Cluster Vs Cloud Computing
Properties Cloud Cluster Grid
On-demand self- Yes No No
Service
Broad network Yes Yes Yes
access
Resource pooling Yes Yes Yes
Rapid elasticity Yes No No
Measured service Yes No Yes
15
Multi/Many Core Systems
16
17
Processor Trends So Far..
• Smarter Brain
–(e.g. x386 x486 Pentium P2 P3 P4)
• Larger Memory
–Larger caches, DRAM, Disk
• Smaller Head
–Fewer chips (integrate more things onto a chip)
• More Power Consumption
–few Watts 120+ Watts!
• More Complex
–1Billion Transistors; design + verification complexity
18
Processor-Memory
Performance Gap
1000 CPU
µProc
60%/yr.
“Moore’s Law”
Performance
100
10
DRAM
DRAM
7%/yr.
1
1980
1982
1983
1985
1987
1990
1992
1994
1995
1997
1999
1981
1984
1986
1988
1991
1993
1996
1998
2000
1989
From D. Patterson, CS252, Spring 1998 ©UCB
19
Major Problem Today: End of the Road!
• Can’t increase Power Consumption (~100W!)
• Can’t increase Design Complexity
• Can’t increase Verification Requirements
No further improvements to a Processor?!
20
Obvious Answer: Use Multiple Brains
• If single brain can’t be improved, use
multiple brains!
• Put multiple simple CPUs on a single chip
Multi-Core!
21
The First Multi-Core: IBM Power 4 Processor
22
Definitions
• A Multi-Core processor combines two or more independent cores
into a single package composed of a single integrated circuit (IC),
called a die, or more dies packaged together.
A dual-core processor contains two cores, and a quad-core processor
contains four cores.
• Typically the term Many-Core is sometimes used to describe multi-
core architectures with an especially high number of cores (tens or
hundreds).
23
Multi core
• Single Chip
• Multiple distinct processing Engine
• E.g.) Shared-cache Dual Core Architecture
Core Core
0 1
CPU CPU
L1 L1
Cache Cache
L2
Cache
Multicore.. 24
Multi core
• Cores in a multi-core device may share a single coherent cache at
the highest on- device cache level (e.g. L2 for the Intel Core 2) or
may have separate caches (e.g. current AMD dual-core processors).
• The processors also share the same interconnect to the rest of the
system.
• Each "core" independently implements optimizations such as
superscalar execution, pipelining, and multithreading.
• A system with n cores is effective when it is presented with n or
more threads concurrently.
25
Advantages of Multi-core
• The proximity of multiple CPU cores on the same die allows the
cache coherency circuitry to operate at a much higher clock rate
than is possible if the signals have to travel off-chip.
• Combining equivalent CPUs on a single die significantly improves
the performance of cache snoop (alternative: Bus snooping)
operations.
• This means that signals between different CPUs travel shorter
distances, and therefore those signals degrade less. These higher
quality signals allow more data to be sent in a given time period
since individual signals can be shorter and do not need to be
repeated as often.
26
Advantages of Multi-core(cont..)
• The largest boost in performance will likely be noticed in improved
response time while running CPU-intensive processes, like antivirus
scans, ripping/ burning media (requiring file conversion), or searching
for folders.
• For example, if the automatic virus scan initiates while a movie is
being watched, the application running the movie is far less likely to
be starved of processor power, as the antivirus program will be
assigned to a different processor core than the one running the movie
playback.
27
