HTTP2 download of parallel requests is slower than HTTP1.1

[stanislav-halyn]

Version

v20.17.0

Platform

MacOS Sonoma 14.5, 2,3 GHz Quad-Core Intel Core i7

Subsystem

No response

What steps will reproduce the bug?

Define an http2 and http1.1 clients and fetch some data in parallel.

You can use the script below. If you run it using the following command, it will send 25 parallel requests using HTTP1.1

node client.js

or you can run this command and it will send 25 parallel requests using HTTP2:

node client.js http2

client.js

import http2 from 'node:http2';
import https from 'node:https';

const isHttp2 = process.argv[2] === 'http2';

const MB = 1024 * 1024;
const WINDOW_SIZE = 32 * MB;

const url = new URL('https://nodejs.org/dist/v17.1.0/node-v17.1.0-x64.msi');

const client = http2.connect(url, {
  settings: {
    initialWindowSize: WINDOW_SIZE,
  },
});

client.on('connect', () => {
  client.setLocalWindowSize(WINDOW_SIZE);
});

const fetchHttp1 = (id) => {
  return new Promise((resolve) => {
    const req = https.request({
        host: url.host,
        path: url.pathname,
        headers: {
          connection: 'keep-alive',
        },
      }, (res) => {
        res.on('data', () => {});
        res.on('end', () => {
          console.log(`Complete request with id: ${id}`);
          resolve();
        });
      },
    );

    req.end();
  });
};

const fetchHttp2 = (id) => {
  return new Promise((resolve) => {
    const req = client.request({
      ':path': url.pathname,
    });
    req.on('data', () => {});
    req.on('end', () => {
      console.log(`Complete request with id: ${id}`);
      resolve();
    });
    req.end();
  });
};

const main = async () => {
  console.log(`Starting requests using HTTP${isHttp2 ? '2' : '1.1'} protocol.`);

  const startTime = Date.now();

  await Promise.all(
    Array.from({ length: 25 })
      .fill(null)
      .map((_, index) => isHttp2 ? fetchHttp2(index) : fetchHttp1(index)),
  );

  console.log(`Requests complete. Completion time: ${(Date.now() - startTime) / 1000}s`);
};

main();

How often does it reproduce? Is there a required condition?

It's not quite stable, because sometimes http2 has the same performance or even better performance.

What is the expected behavior? Why is that the expected behavior?

I expect http2 to be faster or at least have the same performance as http1.1.

What do you see instead?

Instead, I often see that http1.1 requests are faster. Most of the times http2 took 25s to complete while http1.1 took 20s.

There's a problem, though, because those results are not consistent when using the URL I provided in the example.

But, the results are consistent with some internal APIs I'm working with. First, I thought there might be a problem with our internal APIs, but making 25 parallel requests to our internal APIs using curl has consistently the same performance on both http 1.1 and http2.

I ran curl with the following command:

curl --config ./config.txt --parallel --parallel-max 25 --http2

where config.txt looked like this:

url = "our-internal-api"
output = "/dev/null"
// repeated 25 times

Additional information

I was thinking the reason for such slowness would be TCP stall because http2 uses only 1 tcp connection, so I checked in wireshark how many tcp connections creates node vs curl when using http2, and both created only 1 tcp connection.

Also, I noticed node receives first 10 http2 requests randomly, but then sequentially, while http1 requests are all received randomly.

http2:

---

http1:

[RedYetiDev]

It's not quite stable, because sometimes http2 has the same performance or even better performance.

Are you sure it's not the server that is being requested? Have you tried to reproduce with another http2 client?

But, the results are consistent with some internal APIs I'm working with. First, I thought there might be a problem with our internal APIs, but making 25 parallel requests to our internal APIs using curl has consistently the same performance on both http 1.1 and http2.

How different were the speeds compared?

CC @nodejs/http2

[stanislav-halyn]

@RedYetiDev

Are you sure it's not the server that is being requested? Have you tried to reproduce with another http2 client?

Yeah, I'm sure it's not the server that is being requested. I have tried curl and a rust server to make the same requests, and only on nodejs the results for http1.1 and http2 were inconsistent

How different were the speeds compared?

Here's the results of the runs I just completed on our internal APIs (25 parallel requests at the same time using the script I provided for node and for curl)

Agent	http2 time(s)	http1.1 time(s)
node	5.37	1.45
curl	1.5	1.37

---

Also, I was able to get different results even when fetching https://nodejs.org/dist/v17.1.0/node-v17.1.0-x64.msi, but it's not alway consistent.

[stanislav-halyn]

I just ran node profiler for the script I provided fetching our internal API, and the results are weird...

Here's the summary for http2

 [Summary]:
   ticks  total  nonlib   name
     22    4.3%    4.3%  JavaScript
      0    0.0%    0.0%  C++
      5    1.0%    1.0%  GC
      0    0.0%          Shared libraries
    484   95.5%          Unaccounted

and for http1.1

 [Summary]:
   ticks  total  nonlib   name
     29    4.7%    4.7%  JavaScript
      0    0.0%    0.0%  C++
      1    0.2%    0.2%  GC
      0    0.0%          Shared libraries
    588   95.3%          Unaccounted

http2 has less total ticks, but more GC ticks. I wonder if it could be somehow related to GC

[mcollina]

I guess something very odd is happening within http/2. I don't think the problem here is GC - it's 100% true that HTTP/2 consumes more resources than HTTP/1.1 but the difference should be lower.

It seems that curl can progress all HTTP/2 streams in parallel, while we can't, and after a bit it becomes sequential. I wonder if @jasnell has some ideas.

[stanislav-halyn]

@mcollina I did some testing today and found some new insights.

This does not happen for our internal endpoints with little data, those are processed faster with http2.

When fetching a heavy endpoint with lots of data(json with lots of text, most of the text is contained in one particular field), http1.1 seems to be much faster. Besides, when fetching heavy endpoint, http1.1 receives smaller chunks on data event whereas http2 larger ones(~1400 bytes vs ~12000 bytes).

What's interesting: in cases when http1.1 is slower than http2, it receives larger chunks on data event whereas http2 smaller ones.