Mastering Lazy Loading and Pagination in AG-Grid with React

Handling large datasets efficiently is a critical aspect of building performant and user-friendly web applications. Displaying thousands, or even millions, of rows in a grid without proper optimization can lead to slow loading times, unresponsive UIs, and a poor user experience. This is where lazy loading and pagination come into play, and AG-Grid, combined with React, provides a powerful solution through its Infinite Row Model.

In this installment of our AG-Grid React series, we'll dive deep into implementing lazy loading and pagination, ensuring your grid remains snappy and responsive, regardless of your data volume.

Why Lazy Loading and Pagination?

Imagine loading 100,000 records all at once into a client-side grid. The browser would struggle, memory consumption would soar, and the initial render would take ages. Lazy loading and pagination address these challenges by:

• Improving Performance: Only fetch and render the data that is currently visible or immediately needed, significantly reducing initial load times and memory footprint.
• Enhancing User Experience: Provide a smooth scrolling and interaction experience, even with massive datasets. Users don't have to wait for all data to load.
• Optimizing Network Usage: Reduce the amount of data transferred over the network, making your application faster, especially on slower connections.

Introducing AG-Grid's Infinite Row Model

AG-Grid offers several row models, each suited for different use cases. For server-side data and lazy loading, the Infinite Row Model is your go-to choice. This model is designed to fetch blocks of data from the server on demand, as the user scrolls or navigates through pages. It works seamlessly with pagination, filtering, and sorting, all handled on the server side.

Key characteristics of the Infinite Row Model:

• Data is fetched in blocks (or pages) as needed.
• Only the required data is held in memory.
• Provides a virtually infinite scroll or paginated experience.
• Requires a server-side component to handle data requests.

Setting Up the Grid for Infinite Row Model

To enable the Infinite Row Model, you need to configure your AG-Grid component with a few essential props.

1. Enable Infinite Row Model

Set the `rowModelType` prop to 'infinite'.

2. Enable Pagination (Optional, but common)

Set `pagination` to true. You can also define `paginationPageSize` to control the number of rows per page.

3. Define `cacheBlockSize`

This is crucial. `cacheBlockSize` determines how many rows are requested from your server in each data block. It directly influences how many rows are displayed per page when pagination is enabled. A common value is 100.

4. Implement the `datasource`

The `datasource` is an object that AG-Grid uses to interact with your data source. It must implement the `getRows` method, which is the heart of lazy loading.

Here's a basic React component structure:

import React, { useState, useEffect, useRef, useCallback } from 'react';
import { AgGridReact } from 'ag-grid-react';
import 'ag-grid-community/styles/ag-grid.css';
import 'ag-grid-community/styles/ag-theme-alpine.css';

const MyLazyLoadedGrid = () => {
    const gridRef = useRef();
    const [columnDefs] = useState([
        { field: 'id', headerName: 'ID' },
        { field: 'name', headerName: 'Name' },
        { field: 'email', headerName: 'Email' },
        { field: 'country', headerName: 'Country' }
    ]);

    const defaultColDef = {
        flex: 1,
        minWidth: 100,
        sortable: true,
        filter: true,
        resizable: true,
    };

    // Simulate a backend API call
    const fetchDataFromServer = useCallback(async (startRow, endRow, sortModel, filterModel) => {
        console.log('Requesting rows:', startRow, 'to', endRow);
        // In a real app, you'd make an actual API call here
        // e.g., axios.post('/api/data', { startRow, endRow, sortModel, filterModel })
        // For demonstration, we'll generate mock data
        await new Promise(resolve => setTimeout(resolve, 500)); // Simulate network latency

        const allData = Array.from({ length: 100000 }, (_, i) => ({
            id: i + 1,
            name: `User ${i + 1}`,
            email: `user${i + 1}@example.com`,
            country: `Country ${(i % 10) + 1}`,
        }));

        // Apply server-side filtering (simplified example - a real backend would do this efficiently)
        let filteredData = allData;
        if (filterModel && Object.keys(filterModel).length > 0) {
            Object.keys(filterModel).forEach(key => {
                const filter = filterModel[key];
                if (filter.filterType === 'text' && filter.type === 'contains') {
                    filteredData = filteredData.filter(item =>
                        item[key].toLowerCase().includes(filter.filter.toLowerCase())
                    );
                }
            });
        }

        // Apply server-side sorting (simplified example)
        let sortedData = [...filteredData];
        if (sortModel && sortModel.length > 0) {
            sortModel.forEach(sort => {
                const { colId, sort: sortDirection } = sort;
                sortedData.sort((a, b) => {
                    const valA = a[colId];
                    const valB = b[colId];
                    if (valA < valB) return sortDirection === 'asc' ? -1 : 1;
                    if (valA > valB) return sortDirection === 'asc' ? 1 : -1;
                    return 0;
                });
            });
        }

        const totalRows = sortedData.length;
        const rowsThisBlock = sortedData.slice(startRow, endRow);

        return {
            rows: rowsThisBlock,
            totalRows: totalRows,
        };
    }, []);

    const onGridReady = useCallback((params) => {
        const infiniteDatasource = {
            getRows: async (params) => {
                const { startRow, endRow, successCallback, failCallback, sortModel, filterModel } = params;

                try {
                    const { rows, totalRows } = await fetchDataFromServer(
                        startRow,
                        endRow,
                        sortModel,
                        filterModel
                    );
                    successCallback(rows, totalRows);
                } catch (error) {
                    console.error('Error fetching data:', error);
                    failCallback();
                }
            },
        };

        params.api.setDatasource(infiniteDatasource);
    }, [fetchDataFromServer]);

    return (
        <div className="ag-theme-alpine" style={{ height: 600, width: '100%' }}>
            <AgGridReact
                ref={gridRef}
                columnDefs={columnDefs}
                defaultColDef={defaultColDef}
                rowModelType={'infinite'}
                pagination={true}
                paginationPageSize={100} // Corresponds to cacheBlockSize in this setup
                cacheBlockSize={100} // Important for infinite row model
                onGridReady={onGridReady}
                // Optional: For server-side sorting and filtering, ensure default sort/filter is not client-side
                // suppressServerSideSorting={false} // Default is false for infinite row model with infinite row model
                // suppressServerSideFiltering={false} // Default is false for infinite row model with infinite row model
            />
        </div>
    );
};

export default MyLazyLoadedGrid;

Understanding the `getRows` Method

The `getRows` method within your `datasource` object is the cornerstone of the Infinite Row Model. AG-Grid calls this method whenever it needs more data, providing an object with crucial parameters:

• startRow: The starting row index for the requested block of data.
• endRow: The ending row index for the requested block of data.
• sortModel: An array of objects describing the current sort state (e.g., [{ colId: 'field', sort: 'asc' }]).
• filterModel: An object describing the current filter state (e.g., { 'field': { filterType: 'text', type: 'contains', filter: 'value' } }).
• successCallback(rowsThisPage, lastRow): A function you must call with the fetched rows and the total number of rows available on the server.
• failCallback(): A function to call if fetching data fails.

Your `getRows` implementation should:

1. Make an asynchronous call to your backend API, passing `startRow`, `endRow`, `sortModel`, and `filterModel`.
2. Receive the data from the backend.
3. Call `successCallback` with the slice of data requested and the total count of rows available (after filtering/sorting).
4. Call `failCallback` if an error occurs.

Important: The `lastRow` parameter in `successCallback` tells AG-Grid the total number of rows. If you don't know the exact count (e.g., truly infinite scrolling where you only know if there's *more* data), you can pass `null` or `undefined` and AG-Grid will continue to request blocks until `rowsThisPage.length < cacheBlockSize` in the last block.

Server-Side Sorting and Filtering

One of the significant advantages of the Infinite Row Model is that it naturally pushes sorting and filtering responsibilities to the server. When a user sorts a column or applies a filter, AG-Grid will:

1. Clear its internal cache.
2. Call `getRows` again, but this time, the `sortModel` and `filterModel` parameters will contain the new state.

Your backend should then apply these sort and filter criteria to the full dataset before slicing out the requested block of data and returning the filtered/sorted total count. The example `fetchDataFromServer` function above provides a simplified illustration of this.

Pagination with Infinite Row Model

When `pagination={true}` is set alongside `rowModelType={'infinite'}`, AG-Grid automatically displays pagination controls. The `paginationPageSize` property directly corresponds to the size of the data blocks fetched by the `getRows` method, which is controlled by `cacheBlockSize`.

Clicking on pagination buttons or changing the page size will trigger AG-Grid to make new `getRows` calls with updated `startRow` and `endRow` parameters.

Advanced Considerations and Best Practices

• Loading Indicators: Implement visual loading indicators (e.g., a spinner) while data is being fetched. AG-Grid provides lifecycle methods like `onBodyScroll` or checking `loading` state to help with this.
• Error Handling: Always include robust error handling in your `getRows` method and display user-friendly messages.
• Debouncing/Throttling: For rapid-fire filtering or sorting changes, consider debouncing your backend requests to avoid overwhelming the server.
• Cache Management: AG-Grid offers props like `maxBlocksInCache` and `cacheOverflowSize` to fine-tune how many data blocks the grid keeps in memory, balancing memory usage and performance.
• Server Performance: Ensure your backend API is optimized to handle queries with `startRow`, `endRow`, sort, and filter parameters efficiently, especially for very large datasets.

Conclusion

Implementing lazy loading and pagination with AG-Grid's Infinite Row Model in your React applications is a powerful way to deliver a high-performance and scalable data grid experience. By offloading data management to the server and fetching data in manageable blocks, you ensure your application remains fast, responsive, and capable of handling even the most extensive datasets.

Experiment with the `cacheBlockSize` and `paginationPageSize` to find the optimal balance for your application's specific needs and network conditions. Happy coding!