The Wonders of Partial Requests: How Range Requests Can Improve Your Life ⚡

Udit Kumar's Avatar

Udit Kumar / February 08, 2022

7 min read • ––– views

Banner Image

Have you ever wondered how to make your web app faster? Or how to reduce the amount of data that your users have to download? If so, then you should definitely read this blog. In this blog, we will take a look at the wonders of partial requests and how they can improve your life.

What are Partial Requests (AKA Range Requests)?

Partial requests, also known as range requests, are a way for a client to ask a server for only a specific part of a resource, rather than the entire thing. This can be especially useful when dealing with large files, as it allows the client to just request the parts it needs, rather than downloading the whole file.

But partial requests aren't just for saving bandwidth - they can also be used to improve user experience. Imagine you're trying to stream a movie, but your internet connection is a little spotty. With partial requests, the client can just request the parts of the movie that it hasn't received yet, rather than starting the whole thing over from the beginning.

Where Can I Find Partial Requests in the Wild?

You might be wondering, "Okay, partial requests sound great, but where can I actually use them in my daily life?" Well, you might be surprised to learn that partial requests are used all over the place! Here are a few examples of platforms and technologies that make use of partial requests:

  • Web browsers: Most modern web browsers support partial requests, which means you can use them whenever you're downloading a file from the internet.
  • Streaming services: Services like Netflix and YouTube use partial requests to allow users to stream videos without having to download the entire file first.
  • CDN providers: Content delivery networks (CDNs) like Cloudflare use partial requests to allow clients to retrieve only the parts of a file that they need, improving performance and reducing bandwidth usage.

So as you can see, partial requests are a common feature of many technologies that you probably use on a daily basis. Next time you're streaming a video or downloading a file, take a moment to appreciate the power of partial requests at work!

How to Use Partial Requests?

Using partial requests is actually pretty simple - all you need to do is send an HTTP Range header along with your request. The Range header specifies the part of the resource you want to retrieve, using one of the following formats:

Single part ranges

We can request a single range from a resource. For example, if we wanted to request the first 100 bytes of a file, we would send the following header:

http
Range: bytes=0-99

You can also use the following shorthand:

  • bytes=n-m: retrieves the nth to mth byte of the resource
  • bytes=n-: retrieves all bytes starting from the nth byte of the resource
  • bytes=-n: retrieves the last n bytes of the resource

Multiple part ranges

We can also request multiple ranges from a resource at once. For example, if we wanted to request the first 100 bytes and the last 100 bytes of a file, we would send the following header:

http
Range: bytes=0-99,-100

The server responses with the 206 status code and a Content-Type: multipart/byteranges; boundary=<boundary> header. The response body will contain the requested ranges, separated by the boundary string and each chunk have its own Content-Range and Content-Type header.

http
HTTP/1.1 206 Partial Content
Content-Type: multipart/byteranges; boundary=3123k12312m3k
Content-Length: 200

--3123k12312m3k

Content-Type: text/plain
Content-Range: bytes 0-99/1000

<first 100 bytes of the file>

--3123k12312m3k

Content-Type: text/plain
Content-Range: bytes 900-999/1000

<last 100 bytes of the file>

--3123k12312m3k--

If you'll observe, the Content-Length header is set to the total length of the response body, which is the sum of the length of each range. This is because the response body is a single entity, and the Content-Length header is supposed to represent the length of the response body.

How to Implement Partial Requests

Okay, let's get started on building a server that supports partial requests! Here's how we can do it using Node.js and the Express framework.

First, let's define a function for generating boundary strings:

js
const genBoundary = () => {
  // generate a random boundary string using hex characters
  return Math.random().toString(16).slice(2);
};

Next up, let's define a middleware function to parse the Range header and set the req.ranges property:

js
const asyncHandler = (fn) => (req, res, next) =>
  Promise.resolve(fn(req, res, next)).catch(next);

const parseRanges = async (req, res, next) => {
  // get the Range header
  const range = req.headers.range;

  const filename = req.query.filename;
  if (!filename) {
    res.status(400).send('Missing filename');
    return;
  }

  // creating a path to the file
  const filepath = path.join(__dirname, 'assets', filename);

  // get the file size (in bytes)
  let filesize = await fs.promises.stat(filepath);

  filesize = filesize.size;

  res.file = { filepath, filesize };

  // if the Range header is not set, skip this middleware
  if (!range) {
    return next();
  }

  // get all ranges
  const parts = range.replace(/bytes=/, '').split(',');

  // parse all ranges and set the req.ranges property
  req.ranges = parts.map((part) => {
    const [start, end] = part.split('-');
    return {
      start: start ? parseInt(start, 10) : 0,
      end: end ? parseInt(end, 10) : filesize - 1
    };
  });

  // call the next middleware
  next();
};

In the above code, we are using the asyncHandler function to wrap the parseRanges middleware function. This is because we are using await inside the parseRanges function, and we need to wrap it in a try/catch block to handle errors. So, to avoid having to write the try/catch block every time we use the parseRanges middleware, we can use the asyncHandler function to wrap it.

In parseRanges, we are first checking if the Range header is set. If it's not set, we just call the next middleware and return. If it is set, we parse the header and set the req.ranges and res.file properties.

Now, let's define main route handler that will send the requested ranges:

js
app.get('/download', asyncHandler(parseRanges), (req, res) => {
  const { ranges } = req;
  const { filepath, filesize } = res.file;

  // check if the file exists
  if (!fs.existsSync(filepath)) {
    res.status(404).send('File not found');
    return;
  }

  if (!ranges) {
    // no range header, send the whole file
    res.sendFile(filepath);
    return;
  }

  // check if the request is for a single range or multiple ranges
  if (ranges.length === 1) {
    // single range, send the requested part of the file
    const { start, end } = ranges[0];
    const file = fs.createReadStream(filepath, { start, end });
    res.writeHead(206, {
      'Content-Range': `bytes ${start}-${end}/${filesize}`,
      'Accept-Ranges': 'bytes',
      'Content-Length': end - start + 1
    });
    file.pipe(res);
  } else {
    // multiple ranges, send a multipart response
    const boundary = genBoundary();
    res.writeHead(206, {
      'Content-Type': `multipart/byteranges; boundary=${boundary}`,
      'Accept-Ranges': 'bytes'
    });
    ranges.forEach(({ start, end }) => {
      res.write(
        `--${boundary}\nContent-Type: text/plain\nContent-Range: bytes ${start}-${end}/${filesize}\n\n`
      );
      const file = fs.createReadStream(filepath, { start, end });
      file.pipe(res, { end: false });
      res.write('\n');
    });
    res.end(`--${boundary}--`);
  }
});

In the above code, we are first checking if the res.ranges property is set. If it's not set, we send the whole file. If it is set, we check if it's a single range or multiple ranges. If it's a single range, we send the requested part of the file. If it's multiple ranges, we send a multipart response.

Trying via shell

Now, let's try to download a file using the curl command:

bash
curl -i -H "Range: bytes=0-99" http://localhost:3000/download?filename=sample.txt

The above command will download the first 100 bytes of the file. You can change the range to download different parts of the file.

Conclusion

In this article, we learned what partial requests are and how they work. We also learned how to implement partial requests in Node.js using the Express framework.

If you have any questions or comments, feel free to reach out to me on Twitter or LinkedIn. I'd love to hear from you!

• Author Github Profile