The Need for Asynchronous Iteration

You're likely familiar with synchronous iterators and the for...of loop, which allow you to traverse collections like arrays, strings, and maps. For example:

const numbers = [1, 2, 3];
for (const num of numbers) {
    console.log(num); // 1, 2, 3
}
        

This works perfectly when all data is available immediately. However, imagine you're reading a large file line by line, fetching paginated data from an API, or processing real-time events from a WebSocket. These operations don't provide all data at once; they provide data over time. A traditional for...of loop can't naturally "await" the next item in such a stream.

This is precisely the problem Async Iterators solve, allowing us to use a syntax familiar to synchronous iteration, but with the added power of waiting for each item.

Introducing Async Iterators and for...await...of

An object is an Async Iterable if it implements a method accessible via Symbol.asyncIterator. This method must return an Async Iterator, which is an object with a next() method that returns a Promise resolving to an object of the form { value: any, done: boolean }.

The magic happens with the for...await...of loop. This new loop construct is specifically designed to work with async iterables, pausing execution to await the next value from the iterator's next() method.

The Symbol.asyncIterator Protocol

Similar to Symbol.iterator for synchronous iteration, an async iterable must define a method at the Symbol.asyncIterator key. This method should return an iterator object. The iterator object, in turn, must have a next() method. The key difference is that the next() method for an async iterator must return a Promise that resolves to the { value, done } object.

interface AsyncIteratorResult {
  value: any;
  done: boolean;
}

interface AsyncIterator {
  next(): Promise<AsyncIteratorResult>;
}

interface AsyncIterable {
  [Symbol.asyncIterator](): AsyncIterator;
}
        

Creating Your Own Async Iterator (Manual Way)

Let's create a simple async iterator that simulates fetching data with a delay. We'll fetch numbers from 1 to 5, with a 1-second delay between each.

class AsyncNumberGenerator {
    constructor(limit) {
        this.limit = limit;
        this.current = 0;
    }

    // This makes the class an Async Iterable
    [Symbol.asyncIterator]() {
        return {
            current: this.current,
            limit: this.limit,
            // The next() method returns a Promise
            async next() {
                // Simulate an asynchronous operation (e.g., network request, file read)
                await new Promise(resolve => setTimeout(resolve, 1000)); // 1 second delay

                if (this.current < this.limit) {
                    this.current++;
                    return { value: this.current, done: false };
                } else {
                    return { value: undefined, done: true };
                }
            }
        };
    }
}

// Now, let's consume it using for...await...of
async function consumeAsyncNumbers() {
    console.log("Starting async iteration...");
    const generator = new AsyncNumberGenerator(3); // Will generate numbers 1, 2, 3

    for await (const num of generator) {
        console.log(`Received: ${num} at ${new Date().toLocaleTimeString()}`);
    }
    console.log("Async iteration finished.");
}

consumeAsyncNumbers();
/*
Output (with ~1-second delays):
Starting async iteration...
Received: 1 at ...
Received: 2 at ...
Received: 3 at ...
Async iteration finished.
*/
        

Notice how the for...await...of loop gracefully handles the asynchronous nature of the next() method, waiting for each promise to resolve before moving to the next iteration.

The Easier Way: Async Generator Functions (async function*)

Manually implementing the Symbol.asyncIterator and next() method can be verbose. Thankfully, JavaScript provides async generator functions (async function*), which combine the power of async/await with the simplicity of generator functions.

An async function* automatically becomes an async iterable. Inside, you can use await to pause execution and wait for promises to resolve, and yield to produce values for the iterator. Each yield automatically returns a promise-wrapped { value, done: false }, and the function's completion returns a promise-wrapped { value: undefined, done: true }.

Example with Async Generator Function

Let's recreate our number generator using an async generator:

async function* asyncNumberGenerator(limit) {
    for (let i = 1; i <= limit; i++) {
        await new Promise(resolve => setTimeout(resolve, 1000)); // Simulate async delay
        yield i; // Yield the value
    }
}

async function consumeAsyncGenerator() {
    console.log("Starting async generator consumption...");
    for await (const num of asyncNumberGenerator(3)) {
        console.log(`Received from generator: ${num} at ${new Date().toLocaleTimeString()}`);
    }
    console.log("Async generator consumption finished.");
}

consumeAsyncGenerator();
/*
Output (with ~1-second delays):
Starting async generator consumption...
Received from generator: 1 at ...
Received from generator: 2 at ...
Received from generator: 3 at ...
Async generator consumption finished.
*/
        

As you can see, the async generator function provides a much cleaner and more intuitive syntax for creating async iterators. It encapsulates the asynchronous logic and iteration state beautifully.

Practical Use Cases for Async Iterators

• Reading large files line by line: Instead of loading an entire file into memory, an async iterator can yield lines as they are read from the disk, ideal for processing massive datasets without exhausting memory.
• Paginating API responses: An async iterator can abstract away the pagination logic of an API, yielding items from subsequent pages as if they were part of a continuous stream, fetching the next page only when needed.
• Processing real-time data streams: Whether from WebSockets, Server-Sent Events (SSE), or other stream-based APIs, async iterators can provide a structured way to consume continuous event data.
• Database cursor iteration: Some database drivers might expose cursors as async iterables, allowing you to fetch records one by one without loading the entire result set into memory.

Error Handling

Error handling in for...await...of loops works just like synchronous for...of loops, using standard try...catch blocks. If a promise returned by the next() method (or thrown by an async generator) rejects, the error will be caught by the catch block.

async function* unreliableGenerator() {
    yield 1;
    await new Promise(resolve => setTimeout(resolve, 500));
    // Simulate an error
    throw new Error("Oops! Something went wrong during iteration.");
    yield 2; // This will never be reached
}

async function consumeUnreliableGenerator() {
    console.log("Consuming unreliable generator...");
    try {
        for await (const item of unreliableGenerator()) {
            console.log(`Received: ${item}`);
        }
    } catch (error) {
        console.error(`Caught an error: ${error.message}`);
    }
    console.log("Unreliable generator consumption finished.");
}

consumeUnreliableGenerator();
/*
Output:
Consuming unreliable generator...
Received: 1
Caught an error: Oops! Something went wrong during iteration.
Unreliable generator consumption finished.
*/
        

Conclusion

Async Iterators, especially when combined with async generator functions, provide a powerful and elegant solution for consuming streams of asynchronous data in JavaScript. They bridge the gap between synchronous iteration patterns and the unpredictable nature of asynchronous operations, leading to more readable, maintainable, and efficient code when dealing with data over time. Embracing this pattern will significantly enhance your ability to build robust and responsive applications that handle complex data flows with ease.