In Node.js, streams and buffers are used to handle data efficiently, especially when dealing with large files, network requests, or real-time data.
What is a Buffer?
A Buffer is a temporary memory area used to store binary data directly in memory.
👉 Node.js works with raw data (like files, videos, images), so buffers help manage that data before processing.
Example
const buffer = Buffer.from("Hello");
console.log(buffer); // <Buffer 48 65 6c 6c 6f>
console.log(buffer.toString()); // Hello
Key Points
- Fixed size (allocated in memory)
- Stores binary data (not just strings)
- Used when data is small and available at once
What is a Stream?
A Stream is a way to handle data piece by piece (chunks) instead of loading everything into memory at once.
👉 Best for handling large data efficiently.
Types of Streams in Node.js
- Readable Stream → read data (e.g., file read)
- Writable Stream → write data (e.g., file write)
- Duplex Stream → read + write (e.g., sockets)
- Transform Stream → modify data while streaming (e.g., compression)
Example (Readable Stream)
const fs = require("fs");
const stream = fs.createReadStream("largeFile.txt");
stream.on("data", (chunk) => {
console.log("Chunk received:", chunk.toString());
});
👉 File is read in chunks → memory efficient
Example (Pipe - Best Practice)
const fs = require("fs");
fs.createReadStream("input.txt")
.pipe(fs.createWriteStream("output.txt"));
👉 Data flows directly → no need to store full file in memory
Buffer vs Stream (Simple Difference)
| Feature | Buffer | Stream |
|---|---|---|
| Data Handling | Whole data at once | Chunk by chunk |
| Memory Usage | High (for large data) | Low |
| Use Case | Small data | Large data / real-time data |
| Speed | Slower for large files | Faster and efficient |
Real-world Example
- Buffer → reading a small JSON file
- Stream → uploading a video or downloading a large file
Key Takeaway
- Buffer stores data temporarily in memory
- Stream processes data in chunks
👉 For large-scale applications, streams are preferred because they are more memory-efficient and scalable.
