Node js internal

NodeJS Internal
Source:
1. What is NodeJS?
Node.js = Runtime Environment + JavaScript Library
RECAP
2. Features of Node.js
Following are some of the important features that make Node.js the first choice of so ware
architects.
Asynchronous and Event Driven
Node.js Tutorial - Tutorialspoint•
NodeJS - Server Side JS•
Node js•
Node.js Internals: An introduction to Node’s runtime and architecture•
Don't Block the Event Loop (or the Worker Pool) | Node.js•
Node.js is a server-side platform built on Google Chrome's JavaScript Engine (V8 Engine).•
Node.js was developed by Ryan Dahl in 2009 and its latest version is v0.10.36.•
Node.js is an open source, cross-platform runtime environment for developing server-side and
networking applications.
•
Node.js applications are written in JavaScript, and can be run within the Node.js runtime on
OS X, Microso Windows, and Linux
•
Node.js also provides a rich library of various JavaScript modules which simplifies the
development of web applications using Node.js to a great extent.
•
Built on Google Chrome's JavaScript Engine (V8 Engine)•
Single-thread, event oriented•
Event-Driven with Non-Blocking I/O•
Use a Event Loop for Non-Blocking I/O•
Can handle thousands of concurrent connections with minimal overhead (CPU/Memory)•
40% JavaScript and 60% C++•
Exported from www.nuclino.com 1/13

Very Fast
Single Threaded but Highly Scalable
No Buﬀering
License
3. When to use NodeJS?
Following are the areas where Node.js is proving itself as a perfect technology partner.
4. When not to use NodeJS?
5. Architecture
5.1 Traditional Server
All APIs of Node.js library are asynchronous, that is, non-blocking.•
It essentially means a Node.js based server never waits for an API to return data.
The server moves to the next API a er calling it and a notification mechanism of Events of
Node.js helps the server to get a response from the previous API call.
•
Being built on Google Chrome's V8 JavaScript Engine, Node.js library is very fast in code
execution.
•
Node.js uses a single threaded model with event looping.•
Event mechanism helps the server to respond in a non-blocking way and makes the server
highly scalable as opposed to traditional servers which create limited threads to handle
requests.
•
Node.js uses a single threaded program and the same program can provide service to a much
larger number of requests than traditional servers like Apache HTTP Server.
•
Node.js applications never buﬀer any data.•
These applications simply output the data in chunks.•
Node.js is released under the MIT license.•
I/O bound Applications•
Data Streaming Applications•
Data Intensive Real-time Applications (DIRT)•
Single Page Applications•
It is not advisable to use Node.js for CPU intensive applications•

Thesis
5.2 NodeJS Architecture
5.2.1 Event Loop
New thread per request•
The thread is blocked during IO operations•
IO is expensive•
Thread per connection is memory expensive•
Node is single-thread•
Follow an event-driven functional programming model•
Errors in the main thread kills server•

5.2.2 Node’s runtime architecture
Node’s runtime is designed as a set of layers where the user code sits on top and each layer
uses the API provided by the layer below.

Layer Purpose
User code Javascript application code that is written by the programmers.
NodeJS API A list of methods that Node offers, which can be used in the user code
Bindings and
C++ add-ons
When reading about Node, you see that V8 is written in C++, Libuv is written in C and so on.
Basically, all the modules are either written in C or C++, since these languages are so good and fast
in dealing with low-level tasks and using OS API.
But how is it possible that the Javascript code on the upper layers can use code that is written in
other languages?
V8 V8 is the library that provides Node.js with a Javascript engine.
It is a JIT (Just-in-time) compiler. Converts Javascript code into machine code of given OS.
Libuv Libuv is a library written in C that is used to abstract non-blocking I/O operations. It offers the
following features:
5.3 Don't Block the Event Loop (or the Worker Pool)
Node.js runs JavaScript code in the Event Loop (initialization and callbacks), and offers a
Worker Pool to handle expensive tasks like file I/O.
The secret to the scalability of Node.js is that it uses a small number of threads to handle many
clients.
This is what bindings do.•
They act as the glue between the two layers, so Node can smoothly use low level code written
in C or C++.
•
The event loop•
Asynchronous TCP and UDP sockets•
Asynchronous DNS resolution•
Asynchronous file and file system operations•
Thread pool•
Child processes•
High resolution clock•
Threading and synchronisation primitives•
Polling•
Streaming•
Pipes•

Here's a good rule of thumb for keeping your Node.js server speedy:
5.3.1 Why should we avoid blocking the Event Loop and the Worker Pool?
Node.js uses a small number of threads to handle many clients. In Node.js there are two types
of threads:
If a thread is taking a long time to execute a callback (Event Loop) or a task (Worker), we call it
"blocked".
While a thread is blocked working on behalf of one client, it cannot handle requests from any
other clients. This provides 2 motivations for blocking neither the Event Loop nor the Worker
Pool:
5.3.2 A quick review of Node
Node.js uses the Event-Driven Architecture: it has an Event Loop for orchestration and a
Worker Pool for expensive tasks.
(1) What code runs on the Event Loop?
If Node.js can make do with fewer threads, then it can spend more of your system's time and
memory working on clients rather than on paying space and time overheads for threads
(memory, context-switching).
•
But because Node.js has only a few threads, you must structure your application to use them
wisely.
•
Node.js is fast when the work associated with each client at any given time is "small".•
Libuv's Threadpool default size is 4.•
Refer: Thread pool work scheduling — libuv documentation•
One Event Loop (aka the main loop, main thread, event thread, etc.)•
A pool of k Workers in a Worker Pool (aka the threadpool).•
Performance: If you regularly perform heavyweight activity on either type of thread,
the throughput (requests/second) of your server will suﬀer.
•
Security: If it is possible that for certain input one of your threads might block, a malicious
client could submit this "evil input", make your threads block, and keep them from working on
other clients. This would be a Denial of Service attack.
•

(2) What code runs on the Worker Pool?
These are the Node.js module APIs that make use of this Worker Pool:
(3) How does Node.js decide what code to run next?
Abstractly, the Event Loop and the Worker Pool maintain queues for pending events and
pending tasks, respectively.
5.3.3 Don't block the Event Loop
The Event Loop notices each new client connection and orchestrates the generation of a
response. All incoming requests and outgoing responses pass through the Event Loop.
You should make sure you never block the Event Loop. In other words, each of your JavaScript
callbacks should complete quickly.
A good way to ensure this is to reason about the "computational complexity" of your
callbacks. If your callback takes a constant number of steps no matter what its arguments are,
then you'll always give every pending client a fair turn.
Event Loop executes the JavaScript callbacks registered for events, and is also responsible for
fulfilling non-blocking asynchronous requests like network I/O.
•
The Worker Pool of Node.js is implemented in libuv.•
Node.js uses the Worker Pool to handle "expensive" tasks, this includes I/O for which an
operating system does not provide a non-blocking version, as well as particularly CPU-
intensive tasks.
•
I/O-intensive•
DNS: dns.lookup() , dns.lookupService() .•
File System: All file system APIs except fs.FSWatcher() and those that are explicitly
synchronous use libuv's threadpool.
•
CPU-intensive•
Crypto: crypto.pbkdf2() , crypto.scrypt() , crypto.randomBytes() , crypto.randomFill() , crypto.generateK
eyPair( .
•
Zlib: All zlib APIs except those that are explicitly synchronous use libuv's threadpool.•
The Event Loop does not actually maintain a queue.•
Instead, it has a collection of file descriptors that it asks the operating system to monitor,
using a mechanism like epoll (Linux), kqueue (OSX), event ports (Solaris),
or IOCP (Windows).
•
These file descriptors correspond to network sockets, any files it is watching, and so on.•
In contrast, the Worker Pool uses a real queue whose entries are tasks to be processed.•

(1) Blocking the Event Loop: REDOS
A regular expression (regexp) matches an input string against a pattern.
One common way to block the Event Loop disastrously is by using a "vulnerable" regular
expression.
•
Avoiding vulnerable regular expressions.•
We usually think of a regexp match as requiring a single pass through the input string --
- O(n) time where n is the length of the input string. In many cases, a single pass is indeed
all it takes.
•
Unfortunately, in some cases the regexp match might require an exponential number of
trips through the input string --- O(2^n) time.
•
Here is an example vulnerable regexp exposing its server to REDOS:•

(2) Blocking the Event Loop: Node.js core modules
Several Node.js core modules have synchronous expensive APIs, including:
These APIs are expensive, because they involve significant computation (encryption,
compression), require I/O (file I/O), or potentially both (child process).
In a server, you should not use the following synchronous APIs from these modules:
(3) Blocking the Event Loop: JSON DOS
Encryption•
Compression•
File system•
Child process•
Encryption:•
crypto.randomBytes (synchronous version)•
crypto.randomFillSync•
crypto.pbkdf2Sync•
You should also be careful about providing large input to the encryption and decryption
routines.
•
Compression:•
zlib.inflateSync•
zlib.deflateSync•
File system:•
Do not use the synchronous file system APIs. For example, if the file you access is in
a distributed file system like NFS, access times can vary widely.
•
Child process:•
child_process.spawnSync•
child_process.execSync•
child_process.execFileSync•

JSON.parse and JSON.stringify are other potentially expensive operations. While these
are O(n) in the length of the input, for large n they can take surprisingly long.
(4) Complex calculations without blocking the Event Loop
Suppose you want to do complex calculations in JavaScript without blocking the Event Loop.
You have two options:
(4.1) How to offload?
You have 2 options for a destination Worker Pool to which to offload work.
(4.2) Downside of offloading
The downside of the offloading approach is that it incurs overhead in the form
of communication costs.
(4.3) Some suggestions for offloading
You may wish to distinguish between CPU-intensive and I/O-intensive tasks because they have
markedly different characteristics.
Partitioning•
Offloading: Move the work off of the Event Loop onto a Worker Pool.•
You can use the built-in Node.js Worker Pool by developing a C++ addon.•
You can create and manage your own Worker Pool dedicated to computation rather than the
Node.js I/O-themed Worker Pool.
The most straightforward ways to do this is using Child Process or Cluster.
•
Only the Event Loop is allowed to see the "namespace" (JavaScript state) of your application.•
From a Worker, you cannot manipulate a JavaScript object in the Event Loop's namespace.
Instead, you have to serialize and deserialize any objects you wish to share.
Then the Worker can operate on its own copy of these object(s) and return the modified object
(or a "patch") to the Event Loop.
•

If you rely on only one Worker Pool, e.g. the Node.js Worker Pool, then the differing
characteristics of CPU-bound and I/O-bound work may harm your application's performance.
(4.4) Offloading: conclusions
Node.js excels for I/O-bound work, but for expensive computation it might not be the best
option.
5.3.4 Don't block the Worker Pool
CPU-intensive task•
A CPU-intensive task only makes progress when its Worker is scheduled, and the Worker
must be scheduled onto one of your machine's logical cores.
•
If you have 4 logical cores and 5 Workers, one of these Workers cannot make progress.•
As a result, you are paying overhead (memory and scheduling costs) for this Worker and
getting no return for it.
•
I/O-intensive tasks•
I/O-intensive tasks involve querying an external service provider (DNS, file system, etc.)
and waiting for its response.
•
While a Worker with an I/O-intensive task is waiting for its response, it has nothing else to
do and can be de-scheduled by the operating system, giving another Worker a chance to
submit their request.
•
Thus, I/O-intensive tasks will be making progress even while the associated thread is not
running.
•
External service providers like databases and file systems have been highly optimized to
handle many pending requests concurrently.
•
For simple tasks, like iterating over the elements of an arbitrarily long array, partitioning
might be a good option.
•
If your computation is more complex, offloading is a better approach: the communication
costs, i.e. the overhead of passing serialized objects between the Event Loop and the Worker
Pool, are offset by the benefit of using multiple cores.
•

(1) Minimizing the variation in Task times
(2) Task partitioning
Each Worker completes its current Task before proceeding to the next one on the Worker Pool
queue.
•
So there will be variation in the cost of the Tasks required to handle your clients' requests.•
Some Tasks can be completed quickly (e.g. reading short or cached files, or producing a
small number of random bytes)
•
Others will take longer (e.g reading larger or uncached files, or generating more random
bytes).
•
Your goal should be to minimize the variation in Task times, and you should use Task
partitioning to accomplish this.
•
Each relatively long Task eﬀectively decreases the size of the Worker Pool by one until it is
completed.
This is undesirable because, up to a point, the more Workers in the Worker Pool, the greater
the Worker Pool throughput (tasks/second) and thus the greater the server throughput (client
requests/second).
One client with a relatively expensive Task will decrease the throughput of the Worker Pool, in
turn decreasing the throughput of the server.
•
To avoid this, you should try to minimize variation in the length of Tasks you submit to the
Worker Pool.
•
Variation example: Long-running file system reads•
Suppose your server must read files in order to handle some client requests.•
A er consulting the Node.js File system APIs, you opted to use fs.readFile() for simplicity.
However, fs.readFile() is (currently) not partitioned: it submits a single fs.read() Task
spanning the entire file.
If you read shorter files for some users and longer files for others, fs.readFile() may
introduce significant variation in Task lengths, to the detriment of Worker Pool throughput.
•
Tasks with variable time costs can harm the throughput of the Worker Pool.
To minimize variation in Task times, as far as possible you should partition each Task into
comparable-cost sub-Tasks.
•

Note that the number of sub-Tasks completed is not a useful metric for the throughput of the
Worker Pool.
Instead, concern yourself with the number of Tasks completed.
(3) Worker Pool: conclusions
6. Useful Modules
To continue the fs.readFile() example, you should instead use ReadStream (automatically
partitioned).
•
Whether you use only the Node.js Worker Pool or maintain separate Worker Pool(s), you
should optimize the Task throughput of your Pool(s).
•
To do this, minimize the variation in Task times by using Task partitioning.•
Express - Expree.js, or simply Express, is a web application framework for Node.js. To make
things simpler. For example: syntax, DB connections
•
Jade - HTML template•
Socket.IO - To create real-time apps•
Nodemon - To monitor NodeJS and push change automatically•
CoﬀeeScript - For easier JavaScript development•

Node js internal

Recommended

More Related Content

What's hot (18)

Similar to Node js internal (20)

More from Chinh Ngo Nguyen (6)

Recently uploaded (20)

Node js internal