Q67- How do you paginate large datasets in MongoDB?

When you have large collections, fetching all records at once is inefficient. Pagination helps you load data in chunks — improving performance and user experience.

---

✅ 1. Basic Pagination with skip() and limit()

// Example: Get page 3 with 10 items per page
const page = 3;
const limit = 10;
const skip = (page - 1) * limit;

const users = await db.collection("users")
  .find({})
  .skip(skip)
  .limit(limit)
  .toArray();

Pros:

• Simple to implement.

Cons:

• Inefficient for large datasets (MongoDB still scans skipped docs).
• Slow when page is high (e.g., page 1000+).

---

✅ 2. Pagination with Range Queries (Using _id or Indexed Field)

Instead of skip(), use a filter with range to get the next set efficiently.

// Suppose you store the last fetched _id
const lastId = "6718e4d3abf9d9e0c31fa122";

const users = await db.collection("users")
  .find({ _id: { $gt: ObjectId(lastId) } })
  .limit(10)
  .toArray();

Pros:

• Very fast, uses indexes efficiently.
• Ideal for infinite scrolling or APIs.

Cons:

• Works best when sorting by a unique, sequential field like _id or createdAt.

---

✅ 3. Pagination with Aggregation Pipelines

You can combine pagination with filtering and sorting inside an aggregation:

const page = 2;
const limit = 5;

const users = await db.collection("users").aggregate([
  { $match: { active: true } },
  { $sort: { createdAt: -1 } },
  { $skip: (page - 1) * limit },
  { $limit: limit },
]).toArray();

Useful when: You need filtered, computed, or joined data before pagination.

---

✅ 4. Cursor-based Pagination (for APIs and React apps)

Use cursors to handle “Next” and “Previous” efficiently.

// Client stores last timestamp or _id
const lastCreatedAt = "2025-10-23T10:00:00Z";

const results = await db.collection("posts")
  .find({ createdAt: { $lt: new Date(lastCreatedAt) } })
  .sort({ createdAt: -1 })
  .limit(10)
  .toArray();

Pros:

• Efficient and stateless.
• No performance drop as the dataset grows.
• Perfect for infinite scroll UIs.

---

✅ 5. Using Mongoose for Simplicity

const page = 1, limit = 10;
const users = await User.find()
  .sort({ createdAt: -1 })
  .skip((page - 1) * limit)
  .limit(limit);

Or use a plugin like mongoose-paginate-v2 for automatic pagination metadata:

const result = await User.paginate({}, { page, limit });
console.log(result.docs, result.totalPages);

---

✅ Summary

Approach	When to Use	Pros	Cons
skip() + limit()	Small datasets	Simple	Slow for large data
Range / _id based	Large datasets	Fast & scalable	One-direction only
Aggregation	Filtered results	Flexible	Slightly complex
Cursor-based	APIs & infinite scroll	Efficient	Needs client tracking

---

In short:
For large MongoDB collections, avoid .skip() for deep pages — instead, use range-based or cursor pagination with indexed fields like _id or createdAt to ensure scalability and fast responses.