Learning a Node stream

            2012-01-28
     kumatch / Yosuke Kumakura
• Yosuke Kumakura (kumatch)
• @kumatch
•     Feedtailor inc.
• Video game fun
Agenda

• Summary
• Usage
• Case study
• Stream classes
Stream summary


• An abstract interface Node's I/O
• Controls data stream
• Readable / Writable stream
Case study


How to copy files on Node ?
var fs = require('fs');

var input = './oriainal.jpg';
var output = './copy.jpg';

var data = fs.readFileSync(input);
fs.writeFileSync(output, data);

console.log('copied.');
var fs = require('fs');

var input = './original.jpg';
var output = './copy.jpg';

fs.readFile(input, function (err, data) {
    if (err) throw err;

      fs.writeFile(output, data, function (err) {
         if (err) throw err;

            console.log('copied.');
      });
});
var fs = require('fs');
var readStream = fs.createReadStream('./original.jpg');
var writeStream = fs.createWriteStream('./copy.jpg');
readStream.resume();
readStream.on('data', function (buffer) {
    writeStream.write(buffer);
});
readStream.on('end', function () {
    writeStream.end();
});
writeStream.on('close', function () {
    console.log('copied');
});
var fs = require('fs');

var readStream = fs.createReadStream('./original.jpg');
var writeStream = fs.createWriteStream('./copy.jpg');

readStream.pipe(writeStream);

writeStream.on('close', function () {
    console.log('copied');
});
Stream usage


• Stream is EventEmmiter
• and has some methods.
Stream usage /
    Readable stream
• Methods
 • resume
 • pause
 • destroy
Stream usage /
    Readable stream
• Events
 • data
 • end
 • close
 • error
Stream usage /
    Writable stream
• Methods
 • write
 • end
 • destroy
Stream usage /
    Writable stream
• Events
 • drain
 • close
 • error
var fs = require('fs');
var readStream = fs.createReadStream('./original.jpg');
var writeStream = fs.createWriteStream('./copy.jpg');
readStream.resume();
readStream.on('data', function (buffer) {
    writeStream.write(buffer);
});
readStream.on('end', function () {
    writeStream.end();
});
writeStream.on('close', function () {
    console.log('copied');
});
Stream usage /
         Stream pipe()

                pipe
Source stream          Destination stream
  (readable)               (writable)
Stream usage /
        Stream pipe()

• destination.write() if source on ‘data’.
• source.pause() if destination buffer is full.
• source.resume() if destination on ‘drain’.
Stream usage /
        Stream pipe()

• [optional]
  Keeps the destination stream open.
   • Do not destination.end().
var fs = require('fs');

var readStream = fs.createReadStream('./original.jpg');
var writeStream = fs.createWriteStream('./copy.jpg');

readStream.pipe(writeStream);

writeStream.on('close', function () {
    console.log('copied');
});
Case study 2


Digest SHA1 hash of a file
Stream classes


• Filesystem (fs)
 • readStream (Readable)
 • writeStream (Writable)
Stream classes


• Net
 • net.Socket (Readable/Writable)
Stream classes


• HTTP
 • http.ServerRequest (Readable)
 • http.ServerResponse (Writable)
Stream classes

• Zlib
 • all classes (Readable/Writable)
   • Gzip/Gunzip
   • Deflate/Inflate
   • DeflateRaw/InflateRaw
Stream classes

fs.Readable


        pipe   zlib.Gzip


                     pipe   http.response
References
• Node manual & documentation
• by Jxck
 • http://d.hatena.ne.jp/Jxck/20111204
• A future in stream /
    Streams2 (Github issue)
 • https://github.com/joyent/node/pull/1681

More Related Content

PDF
Streams in node js
PDF
Node.js streams talk
PPTX
Thinking in Sequences - Streams in Node.js & IO.js
PDF
PDF
Updating materialized views and caches using kafka
ODP
Gsummit apis-2013
PPT
Linux50commands
PDF
Replicating application data into materialized views
Streams in node js
Node.js streams talk
Thinking in Sequences - Streams in Node.js & IO.js
Updating materialized views and caches using kafka
Gsummit apis-2013
Linux50commands
Replicating application data into materialized views

What's hot (20)

PDF
Rustでパケットと戯れる
PDF
Gsummit apis-2012
PPTX
HTTP::Parser::XS - writing a fast & secure XS module
PDF
Asynchronous IO in Rust - Enrico Risa - Codemotion Rome 2017
PDF
Mastering the MongoDB Shell
PDF
Libral - a systems management API for Linux
PDF
How do i Meet MongoDB
PPTX
Comets notes
PDF
Reverse on go
PPTX
Writing data analysis pipeline as ruby gem
PDF
WebCamp 2016: PHP.Алексей Петров.PHP at Scale: System Architect Toolbox
PDF
The Popper Experimentation Protocol and CLI tool
PDF
Hypertable Nosql
PDF
In-core compression: how to shrink your database size in several times
PDF
PHP Streams: Lucky Dip
PPTX
Spark Gotchas and Lessons Learned
PDF
Linux cheat sheet
PDF
Rethinkdb
ODP
Rethink db with Python
PDF
CRONtab Tutorial
Rustでパケットと戯れる
Gsummit apis-2012
HTTP::Parser::XS - writing a fast & secure XS module
Asynchronous IO in Rust - Enrico Risa - Codemotion Rome 2017
Mastering the MongoDB Shell
Libral - a systems management API for Linux
How do i Meet MongoDB
Comets notes
Reverse on go
Writing data analysis pipeline as ruby gem
WebCamp 2016: PHP.Алексей Петров.PHP at Scale: System Architect Toolbox
The Popper Experimentation Protocol and CLI tool
Hypertable Nosql
In-core compression: how to shrink your database size in several times
PHP Streams: Lucky Dip
Spark Gotchas and Lessons Learned
Linux cheat sheet
Rethinkdb
Rethink db with Python
CRONtab Tutorial
Ad

Viewers also liked (11)

PPTX
Node.js File system & Streams
PPTX
Node.js Socket.IO
KEY
OSCON 2011 - Node.js Tutorial
PPTX
Introduction to Node.js
PPTX
OpenStreetMap in 3D using Python
PDF
Visualisation of Complex 3D City Models on Mobile Webbrowsers Using Cloud-bas...
PPTX
Introduction to Node.js
PDF
Node Foundation Membership Overview 20160907
PPTX
Best Practices You Must Apply to Secure Your APIs - Scott Morrison, SVP & Dis...
PDF
GeoBeer July 3rd, 2013
PPTX
3d mit Python (PythonCamp)
Node.js File system & Streams
Node.js Socket.IO
OSCON 2011 - Node.js Tutorial
Introduction to Node.js
OpenStreetMap in 3D using Python
Visualisation of Complex 3D City Models on Mobile Webbrowsers Using Cloud-bas...
Introduction to Node.js
Node Foundation Membership Overview 20160907
Best Practices You Must Apply to Secure Your APIs - Scott Morrison, SVP & Dis...
GeoBeer July 3rd, 2013
3d mit Python (PythonCamp)
Ad

Similar to Learning a node stream (20)

PPTX
File handling in C
PDF
PDF
IO Streams, Files and Directories
PDF
Shell scripting
PPT
C-Programming Chapter 5 File-handling-C.ppt
PPSX
File mangement
PDF
Module 03 File Handling in C
PDF
Streams in Node.js
PPTX
Data file handling
PDF
Functional Programming with Streams in node.js
KEY
Node.js - A practical introduction (v2)
KEY
Introduction to NodeJS with LOLCats
PPT
file
PPTX
Managing console i/o operation,working with files
PPTX
Managing,working with files
PPT
How to do file-handling - in C language
PDF
Using Node.js to Build Great Streaming Services - HTML5 Dev Conf
PPTX
File handle in PROGRAMMable extensible interpreted .pptx
PDF
C_and_C++_notes.pdf
PDF
FunctionalJS - May 2014 - Streams
File handling in C
IO Streams, Files and Directories
Shell scripting
C-Programming Chapter 5 File-handling-C.ppt
File mangement
Module 03 File Handling in C
Streams in Node.js
Data file handling
Functional Programming with Streams in node.js
Node.js - A practical introduction (v2)
Introduction to NodeJS with LOLCats
file
Managing console i/o operation,working with files
Managing,working with files
How to do file-handling - in C language
Using Node.js to Build Great Streaming Services - HTML5 Dev Conf
File handle in PROGRAMMable extensible interpreted .pptx
C_and_C++_notes.pdf
FunctionalJS - May 2014 - Streams

More from kumatch kumatch (7)

PDF
AngularJSからReactに移ったケースの話
PDF
Node platforms
PDF
Nodeを稼働させる
PDF
Node.js patterns of module export / require
PDF
Node.js Error & Debug Leveling
PDF
[Node] Multiprocessing and runs continuously
PDF
PHPカンファレンス関西2011/スマートフォン時代のWebシステム
AngularJSからReactに移ったケースの話
Node platforms
Nodeを稼働させる
Node.js patterns of module export / require
Node.js Error & Debug Leveling
[Node] Multiprocessing and runs continuously
PHPカンファレンス関西2011/スマートフォン時代のWebシステム

Recently uploaded (20)

PDF
Decision Optimization - From Theory to Practice
PDF
giants, standing on the shoulders of - by Daniel Stenberg
PDF
EIS-Webinar-Regulated-Industries-2025-08.pdf
PDF
LMS bot: enhanced learning management systems for improved student learning e...
PDF
Connector Corner: Transform Unstructured Documents with Agentic Automation
PDF
Build Real-Time ML Apps with Python, Feast & NoSQL
PPTX
Presentation - Principles of Instructional Design.pptx
PDF
Altius execution marketplace concept.pdf
PDF
Co-training pseudo-labeling for text classification with support vector machi...
PDF
A hybrid framework for wild animal classification using fine-tuned DenseNet12...
PDF
CEH Module 2 Footprinting CEH V13, concepts
PDF
Transform-Your-Streaming-Platform-with-AI-Driven-Quality-Engineering.pdf
PDF
Electrocardiogram sequences data analytics and classification using unsupervi...
PDF
Data Virtualization in Action: Scaling APIs and Apps with FME
PDF
Human Computer Interaction Miterm Lesson
PDF
Transform-Your-Factory-with-AI-Driven-Quality-Engineering.pdf
PDF
A symptom-driven medical diagnosis support model based on machine learning te...
PDF
Transform-Your-Supply-Chain-with-AI-Driven-Quality-Engineering.pdf
PDF
Examining Bias in AI Generated News Content.pdf
PPTX
Internet of Everything -Basic concepts details
Decision Optimization - From Theory to Practice
giants, standing on the shoulders of - by Daniel Stenberg
EIS-Webinar-Regulated-Industries-2025-08.pdf
LMS bot: enhanced learning management systems for improved student learning e...
Connector Corner: Transform Unstructured Documents with Agentic Automation
Build Real-Time ML Apps with Python, Feast & NoSQL
Presentation - Principles of Instructional Design.pptx
Altius execution marketplace concept.pdf
Co-training pseudo-labeling for text classification with support vector machi...
A hybrid framework for wild animal classification using fine-tuned DenseNet12...
CEH Module 2 Footprinting CEH V13, concepts
Transform-Your-Streaming-Platform-with-AI-Driven-Quality-Engineering.pdf
Electrocardiogram sequences data analytics and classification using unsupervi...
Data Virtualization in Action: Scaling APIs and Apps with FME
Human Computer Interaction Miterm Lesson
Transform-Your-Factory-with-AI-Driven-Quality-Engineering.pdf
A symptom-driven medical diagnosis support model based on machine learning te...
Transform-Your-Supply-Chain-with-AI-Driven-Quality-Engineering.pdf
Examining Bias in AI Generated News Content.pdf
Internet of Everything -Basic concepts details

Learning a node stream

  • 1. Learning a Node stream 2012-01-28 kumatch / Yosuke Kumakura
  • 2. • Yosuke Kumakura (kumatch) • @kumatch • Feedtailor inc. • Video game fun
  • 3. Agenda • Summary • Usage • Case study • Stream classes
  • 4. Stream summary • An abstract interface Node's I/O • Controls data stream • Readable / Writable stream
  • 5. Case study How to copy files on Node ?
  • 6. var fs = require('fs'); var input = './oriainal.jpg'; var output = './copy.jpg'; var data = fs.readFileSync(input); fs.writeFileSync(output, data); console.log('copied.');
  • 7. var fs = require('fs'); var input = './original.jpg'; var output = './copy.jpg'; fs.readFile(input, function (err, data) { if (err) throw err; fs.writeFile(output, data, function (err) { if (err) throw err; console.log('copied.'); }); });
  • 8. var fs = require('fs'); var readStream = fs.createReadStream('./original.jpg'); var writeStream = fs.createWriteStream('./copy.jpg'); readStream.resume(); readStream.on('data', function (buffer) { writeStream.write(buffer); }); readStream.on('end', function () { writeStream.end(); }); writeStream.on('close', function () { console.log('copied'); });
  • 9. var fs = require('fs'); var readStream = fs.createReadStream('./original.jpg'); var writeStream = fs.createWriteStream('./copy.jpg'); readStream.pipe(writeStream); writeStream.on('close', function () { console.log('copied'); });
  • 10. Stream usage • Stream is EventEmmiter • and has some methods.
  • 11. Stream usage / Readable stream • Methods • resume • pause • destroy
  • 12. Stream usage / Readable stream • Events • data • end • close • error
  • 13. Stream usage / Writable stream • Methods • write • end • destroy
  • 14. Stream usage / Writable stream • Events • drain • close • error
  • 15. var fs = require('fs'); var readStream = fs.createReadStream('./original.jpg'); var writeStream = fs.createWriteStream('./copy.jpg'); readStream.resume(); readStream.on('data', function (buffer) { writeStream.write(buffer); }); readStream.on('end', function () { writeStream.end(); }); writeStream.on('close', function () { console.log('copied'); });
  • 16. Stream usage / Stream pipe() pipe Source stream Destination stream (readable) (writable)
  • 17. Stream usage / Stream pipe() • destination.write() if source on ‘data’. • source.pause() if destination buffer is full. • source.resume() if destination on ‘drain’.
  • 18. Stream usage / Stream pipe() • [optional] Keeps the destination stream open. • Do not destination.end().
  • 19. var fs = require('fs'); var readStream = fs.createReadStream('./original.jpg'); var writeStream = fs.createWriteStream('./copy.jpg'); readStream.pipe(writeStream); writeStream.on('close', function () { console.log('copied'); });
  • 20. Case study 2 Digest SHA1 hash of a file
  • 21. Stream classes • Filesystem (fs) • readStream (Readable) • writeStream (Writable)
  • 22. Stream classes • Net • net.Socket (Readable/Writable)
  • 23. Stream classes • HTTP • http.ServerRequest (Readable) • http.ServerResponse (Writable)
  • 24. Stream classes • Zlib • all classes (Readable/Writable) • Gzip/Gunzip • Deflate/Inflate • DeflateRaw/InflateRaw
  • 25. Stream classes fs.Readable pipe zlib.Gzip pipe http.response
  • 26. References • Node manual & documentation • by Jxck • http://d.hatena.ne.jp/Jxck/20111204 • A future in stream / Streams2 (Github issue) • https://github.com/joyent/node/pull/1681