Q21- Node.js: An upstream API has 300ms latency - how would you optimize overall response time in your Node.js app?

When your Node.js app depends on a slow upstream API, you can’t make that API faster — but you can optimize how your app handles it to make the response much snappier for users.
Here’s how 👇

---

🧠 1. Add Caching to Avoid Unnecessary Calls

If the data from the upstream API doesn’t change frequently, cache it using Redis or in-memory cache.

import redis from "redis";
const client = redis.createClient();

app.get("/data", async (req, res) => {
  const cached = await client.get("api:data");
  if (cached) {
    return res.json(JSON.parse(cached)); // ⚡ Instant response
  }

  const response = await fetch("https://upstream.com/api");
  const data = await response.json();

  await client.setEx("api:data", 60, JSON.stringify(data)); // cache for 1 min
  res.json(data);
});

✅ Result: Your users won’t wait 300 ms on every request.

---

🧠 2. Make Requests in Parallel (If Multiple APIs)

If you need data from multiple slow APIs, don’t call them sequentially.

❌ Bad:

const a = await fetch(A);
const b = await fetch(B);

✅ Better:

const [a, b] = await Promise.all([fetch(A), fetch(B)]);

👉 This saves a lot of time when there are multiple dependencies.

---

🧠 3. Use Background Refresh / Stale-While-Revalidate Pattern

Serve cached data instantly, but refresh it in the background so users get fast responses and your cache stays fresh.

app.get("/data", async (req, res) => {
  const cached = await client.get("api:data");
  if (cached) {
    res.json(JSON.parse(cached)); // serve old data immediately

    // refresh in background
    fetch("https://upstream.com/api")
      .then(r => r.json())
      .then(data => client.setEx("api:data", 60, JSON.stringify(data)));
  } else {
    const response = await fetch("https://upstream.com/api");
    const data = await response.json();
    await client.setEx("api:data", 60, JSON.stringify(data));
    res.json(data);
  }
});

✅ User sees data instantly, even if upstream is slow.

---

🧠 4. Implement Compression

Compressing large payloads can reduce network time significantly.

import compression from "compression";
app.use(compression());

✅ Especially useful if the upstream API returns large JSON data.

---

🧠 5. Avoid Repeated Calls — Deduplicate Requests

If multiple clients request the same resource at once, share the in-flight promise instead of making multiple upstream calls.

let inFlightPromise = null;

app.get("/data", (req, res) => {
  if (inFlightPromise) {
    return inFlightPromise.then(data => res.json(data));
  }

  inFlightPromise = fetch("https://upstream.com/api").then(r => r.json());
  inFlightPromise.finally(() => (inFlightPromise = null));

  inFlightPromise.then(data => res.json(data));
});

✅ Prevents upstream overload and saves time.

---

🧠 6. Optimize Network (Keep-Alive & Connection Pooling)

Use persistent connections to avoid handshake delays.

import https from "https";

const agent = new https.Agent({ keepAlive: true });
fetch("https://upstream.com/api", { agent });

✅ Reduces extra milliseconds from repeated connection setups.

---

✅ In short:

• Cache responses to avoid repeated slow API calls.
• Fetch multiple APIs in parallel.
• Use stale-while-revalidate for instant responses.
• Compress data and use keep-alive to save network time.
• Deduplicate in-flight requests.

👉 With these, even if the upstream takes 300 ms, your users can see responses in a few ms.