@jialin.huang
FRONT-ENDBACK-ENDNETWORK, HTTPOS, COMPUTERCLOUD, AWS, Docker
To live is to risk it all Otherwise you are just an inert chunk of randomly assembled molecules drifting wherever the Universe blows you

© 2024 jialin00.com

Original content since 2022

back
RSS

HTTP 1.1/2/3

In this page

  • HTTP/1.0 to HTTP/1.1
  • HTTP/1.1 to HTTP/2
  • HTTP/2 to HTTP/3, and understanding how to reach 0 RTT
  • HTTP Negotiation vs. HTTP Upgrade Header

Terminologies

RTT Concept

1 RTT represents a round trip. Below are all 1 RTT.

  1. TCP three-way handshake: you might think it would be 1.5 RTT, but the third step only involves sending a packet without expecting a return, so it's omitted.
  1. Another example is when I use HTTP 1.1 to make a request with a header to upgrade to HTTP/2: 1 RTT.
    1. The client: Send an HTTP 1.1 request to the server.
    1. The server responds with an HTTP/2 formatted response with a 101 Switching Protocols status.

TCP Handshake

TCP operates at the Transport layer, which is below the Network layer, also known as IP.

Regardless of the HTTP version, TCP always requires a 3-way handshake to establish a connection.

  1. C → S: SYN (Synchronize) Packet (Step 1)
  1. S → C: SYN-ACK (Synchronize-Acknowledge) Packet (Step 2)
  1. C →S: ACK (Acknowledge) Packet (Step 3)

OSI Wrapping: Alice & Bob

alice: segment() → packet(segment()) → frame(packet(segment()))

bob: frame(packet(segment())) → packet(segment()) → segment()

If Alice sends to Bob:

  • Alice’s Application Data: The application generates data.
  • Alice’s HTTP Message: Headers and message body.
  • Alice’s HTTP/2 Frames: The data is fragmented (note: this frame is different from the OSI Frame). The latter is the Frame at the physical layer, which is the final wrapping stage.
  • Alice's TCP Segments: Further divided into Segments, with sequence numbers, acknowledgments, and control information added.
  • Alice's IP Packets: Segments are wrapped into Packets, adding IP addresses, routing information, and some metadata.
  • Alice's Physical Frames: Finally, Packets are wrapped into Frames.

    (Wrap: Frame → Segment → Packet → Frame, getting bigger)

Bob Unpacks:

  • Bob’s Physical Frames

    (Unwrap: Frame → Packet → Segment → Frame, getting smaller)

  • Bob’s IP Packets: The Frame is unwrapped, revealing the IP address, routing information, and metadata.
  • Bob’s TCP Segments: Extract sequence numbers, acknowledgments, and control information.
  • Bob’s HTTP/2 Frames: Segments are reassembled into Frames.
  • Bob's HTTP Message: HTTP/2 frames are reconstructed into the original information, including headers and the message body.
  • Bob's Application Data: Delivered to the application for further processing.

HTTP

  • Client/Server Model
  • Is Stateless: Each transaction is independent and unrelated to other transactions, but cookies and sessions can give the impression of statefulness.
  • Application Layer: 99.9% of the time, HTTP uses TCP as its transport layer protocol.
    • Exception: HTTP/3 is based on QUIC, which is built on UDP.

HTTPS Handshake

  1. TCP Three-Way Handshake (the so-called TCP handshake)
  1. SSL/TLS Handshake (HTTPS), HTTP/2 NEED TLS.

    When the server does not support HTTP/2, it will ignore the client's HTTP/2 support request and respond using HTTP/1.1 to indicate the protocol. (Just in case)

    1. TLS 1.2 consumes 2 RTTs
      1. Client sends ClientHello, server responds with ServerHello, Certificate, ServerKeyExchange, ServerHelloDone.
      1. Client sends ClientKeyExchange, ChangeCipherSpec, Finished, server responds with ChangeCipherSpec, Finished.
    1. TLS 1.3 consumes 1 RTT
      1. Client sends ClientHello (including the shared key), server responds with ServerHello, ..., Finished.
  1. Start Conversation 😄😄😄

HTTP 1.1: What’s the better than 1.0?

  1. Persisted TCP Connection: HTTP 1.1 introduced the Keep Alive header, allowing a single connection to be established with the server, meaning that requests and responses can be exchanged without repeatedly performing the TCP handshake as in HTTP 1.0.

    While HTTP is generally stateless, HTTP 1.1 may seem stateful due to the persistent connection. However, by the strict definitions of stateful/stateless, the focus is mainly on sessions and cookies. This also applies to HTTP/2.

    All protocols are always stateful because TCP is stateful.

    While the "keep-alive" mechanism doesn't store application-specific state between requests (as it's still fundamentally a stateless protocol), it does introduce some persistence at the connection level, improving the efficiency of handling multiple requests within a single connection. This is in contrast to the default behavior of HTTP/1.0, where each request typically involved a separate connection.

  1. Streaming Chunked: For example, when an image loads in pieces.
  1. Pipelining: In HTTP 1.0, each request must wait for the previous request to finish. In 1.1, multiple requests can be sent out, but they must return in order. This only partially resolves the head-of-line blocking (HOLB) issue from 1.0.

    (Note: This feature in 1.1 is disabled by default and wasn't widely adopted).
  1. Caching:
    1. 1.0 - If-Modified-Since and Expires
    1. 1.1 - Cache-Control, ETag, and Vary
  1. Host Header: Not required in 1.0.
  1. Virtual Hosting: Due to the lack of a Host header in 1.0, virtual hosting wasn’t possible.
  1. Content Negotiation: 1.1 supports Accept and Content-Type headers.
  1. Range Requests: HTTP 1.1 introduced standardized support for range requests.
    1. request
      GET /resource HTTP/1.1
      Range: bytes=0-499
    1. response
      HTTP/1.1 206 Partial Content
      Content-Range: bytes 0-499/1234

Does HTTP/1.1 have connection limits?

https://queue.acm.org/detail.cfm?id=2555617

Modern browsers allow up to six parallel connections per origin

Modern browsers allow up to six parallel connections per origin.

Browsers Always Use HTTP, Right?

Yes, but HTTP is not limited to use in browsers.

For example, gRPC is a remote procedure call system built on HTTP/2, but it cannot be directly called in a browser. Instead, libraries like gRPC-Web help convert gRPC to a web-friendly format.

What is gRPC?
Remote Procedure Call, with "g" standing for Google. It offers better performance and more efficient data transmission. Of course, it's closer to the underlying system, making it harder to read. However, because it's harder to understand, it also has higher security.

HTTP/2’s Advantages

  1. Header Compression (HPACK algorithm)
    • Previously, we had compression for the body of requests/responses only using Content-Encoding: gzip. Before HTTP/2, header compression was not possible.
    • Static Table: A fixed table that contains common headers and values frequently used in HTTP. Instead of sending the actual header names and values, you can refer to these pre-defined entries.
      Index  Header Name      Header Value
      1      :authority
      2      :method          GET
      3      :method          POST
      4      :path            /
      5      :path            /index.html
      ...
    • Dynamic Table: Stores recently used headers.
      Index  Header Name      Header Value
      62     user-agent       Mozilla/5.0 ...
      63     cookie           session=abc123
      64     custom-header    some-value
      It does not store key-value headers directly. Instead, it uses a combination of the static and dynamic tables to reduce redundant header data transmission, achieving compression.
  1. Multiplexing (HTTP/2 introduces this concept with just one connection)
    1. In HTTP/1.0, you must wait for the previous response to complete before sending the next request ⇒ FIFO (First In First Out)
    1. HTTP/1.1 allows sending multiple requests without waiting for previous responses, but responses must still come back in order. If earlier responses are slow, subsequent responses may be delayed, and many HTTP clients do not use this pipelining feature extensively.
    1. HTTP/2 allows concurrent responses: It addresses the limitation of response ordering in HTTP/1.1, where responses are not guaranteed to be in sequence.

      A single message (request or response) consists of multiple frames (the smallest unit in HTTP/2) and is transmitted using multiple streams.

      https://stackoverflow.com/questions/34478967/what-is-the-difference-between-http-1-1-pipelining-and-http-2-multiplexing

      BUT, it resolves issues at the application layer, but TCP's HOLB (Head-of-Line Blocking) remains!

      TCP's characteristic is that lost packets are retransmitted, and HTTP cannot see TCP operations and must wait. Other unaffected streams continue normally.

  1. Server Push: Checks related resources in the HTML and pushes them along with the main response, similar to link preload.
    In fact, while support for server push as an HTTP protocol feature is new, many Web applications are already using it, just under a different name: inlining.

    However, fundamentally, it's different. Given that inlining is already in use, server push is not as revolutionary.

  1. SPDY: Developed by Google in 2009 as an alternative to HTTP. In 2015, HTTP/2 was standardized based on SPDY by the IETF.
    The spirit of SPDY lives on in HTTP/2. Think of HTTP/2 as the successor to SPDY.
  1. Secure by Default
  1. Protocol Negotiation during TLS (NPN/ALPN):

    NPN & ALPN are TLS extensions, think of them as TLS add-ons.

    • NPN (Next Protocol Negotiation) was an extension of SPDY and a precursor to ALPN.

      SPDY leads to HTTP/2, NPN leads to ALPN, which is easier to remember.

    • ALPN (Application-Layer Protocol Negotiation) is the extension used by HTTP/2.

    Used to decide which application layer protocol to use after establishing the TLS connection, e.g., HTTP 1.0 or HTTP/2.

Regardless of Whether the Browser Uses HTTP/1.1 or HTTP/2, Does the Server Respond Based on the Headers?

  • GET /news HTTP/1.1
  • GET /news HTTP/2.0

No. The server's response is not dependent on the HTTP version in the request but rather on the headers of the request:

connection: Upgrade
upgrade: HTTP/2.0

Can HTTP/2.0 Handle Multiple Requests at Once,
and Will the Responses Be Guaranteed in Order?

In practice, yes, but the mechanism itself does not guarantee it. Stream identifiers are used to ensure that the logical order of requests and responses is maintained.

Overall, HTTP/2's multiplexing allows for simultaneous transmission of multiple requests and responses, optimizing performance and resource utilization. While it does not guarantee the order of individual frames, the use of stream identifiers and prioritization ensures the integrity and logical order of the requests and responses within the HTTP/2 communication.

TCP Segments and HTTP/2 Frames: How Do They Relate?

Three requests: abc, split into frames: a1, a2, b1, b2, b3, b4, b5, c1.
Can b1-b5 be scattered across different segments?

Yes, this is an efficient way of transmission.

Frames: In HTTP/2, frames are the smallest unit of data transmission. Each frame belongs to a specific stream, and a stream corresponds to an individual request.

Streams are logical divisions, while frames are physical entities transmitted.

https://stackoverflow.com/questions/39739334/in-http-2-whats-the-relationship-between-req-resp-frame-and-tcp-packet

There is no physical HTTP/2 stream just like there is no physical TCP stream - it's a higher level concept/abstraction that is handled by the layer in question, which reassembles individual packets or frames into a stream, which makes it infinitely easier to deal with.

During transmission, what actually moves are the frames. The receiving end reassembles the information based on the stream identifiers in each frame.

In simple terms: stream = request, and it includes many frames.

Once at the stream level, streams can be scattered across different TCP segments:

  1. HTTP/2's multiplexing allows frames from different streams to interleave during transmission.
  1. TCP segments are based on packet size and network conditions, not directly related to HTTP/2 frames or streams.

TCP segments can contain many pieces of data. Think of a TCP segment as an outermost package. For efficiency, TCP might combine and split orders (which correspond to different streams).

So:

A single TCP segment might contain frames a2, b1, c1 (from three different streams).

The next segment might include frames b2 and b3.

A subsequent segment might contain frames a1, b4, b5.

If HTTPS is Not Needed and Only HTTP is Required,
Is TLS Negotiation Still Necessary?

When using HTTP (unencrypted), TLS negotiation is not needed. HTTP does not use TLS/SSL encryption, so there is no TLS handshake process in this case.

However, with HTTP/2.0, whether or not to use TLS is not entirely up to you. Nearly all modern browsers only support HTTP/2 over TLS. In other words, HTTP/2 will necessarily involve TLS, and you should handle both the TLS handshake and the negotiation for HTTP/2 in one go.

Although the standard itself does not require usage of encryption, all major client implementations (Firefox, Chrome, Safari, Opera, IE, Edge) have stated that they will only support HTTP/2 over TLS, which makes encryption de facto mandatory.

Does Negotiating HTTP/2 Add an Extra RTT?

To determine if HTTP/2 is used, there are two methods: HTTPS negotiation and the HTTP Upgrade header.

  1. HTTPS negotiation (aka ALPN, Application-Layer Protocol Negotiation)
    • ALPN with TLS 1.3: Total of 2 RTTs
    • ALPN with TLS 1.2: Total of 3 RTTs
    HTTPS negotiationtotal RTTTCPTLS
    ALPN w/ TLS 1.32 RTT =TCP1 RTT of TLS 1.3
    ALPN w/ TLS 1.23 RTT =TCP2 RTT of TLS 1.2
  1. HTTP Upgrade Header (only applicable outside of browsers)
    1. In practice, modern browsers support HTTP/2 only over TLS, which means they use the ALPN mechanism during the TLS handshake to agree on the HTTP version.
      Although the standard itself does not require usage of encryption, all major client implementations (Firefox, Chrome, Safari, Opera, IE, Edge) have stated that they will only support HTTP/2 over TLS, which makes encryption de facto mandatory.
    1. Thus, browsers only support h2, not h2c.

      https://stackoverflow.com/questions/37322430/browser-wont-upgrade-to-h2-http-2-although-upgrade-headers-are-sent

    1. If a browser includes Upgrade: h2, it typically shows I can rather than I want.

    Upgrade: h2c can be used in scenarios where encryption is not needed but HTTP/2 upgrade is required. The client sends an HTTP/1.1 request with the Upgrade header, and regardless of success or failure, it results in 1 RTT.

    • Success: Server responds with 101 Switching Protocols
    • Failure: Server responds with a standard HTTP/1.1 response, indicating the upgrade was not successful.

RTT Scenarios for HTTP Upgrade and HTTPS Negotiation

TCP Handshake: Consumes 1 RTT

2.0: HTTP Upgrade Headertotal RTTTCPTLSRequest/Response
⭕ HTTP upgrade + TLS 1.33 RTT =TCP1 RTT of TLS 1.31 RTT of HTTP upgrade
⭕ HTTP upgrade w/ TLS 1.24 RTT =TCP2 RTT of TLS 1.21
HTTP upgrade failed ❌
(HTTP
S 1.1 wth TLS 1.3)
3 RTT =TCP1 RTT of TLS 1.31 RTT of HTTP req/res
HTTP upgrade failed ❌
(HTTP
S 1.1 wth TLS 1.2)
4 RTT =TCP2 RTT of TLS 1.21
HTTP upgrade failed ❌
(HTTP 1.1)
2 RTT=TCPno TLS1
2.0: HTTPS negotiationtotal RTTTCPTLSRequest/Response
ALPN w/ TLS 1.2 (a)4RTT =TCP2 RTT of TLS 1.21
ALPN w/ TLS 1.3 (b)3 RTT =TCP1 RTT of TLS 1.31
ALPN w/ TLS 1.3 (e)2RTT =TCP0: Session Resumption1
HTTP 3.0QUIC including TCPRequest/Response
HTTP 3.0 QUIC (c)2x11
HTTP 3.0 QUIC (f)1x0: Session Resumption1

To answer the original question: negotiating HTTP/2 does not add an extra RTT.

  1. The HTTP negotiation is part of the TLS handshake process.
  1. Even if using the HTTP Upgrade header, regardless of success or failure, it results in only 1 RTT.There is no additional RTT for failure scenarios where the client needs to resend the request.

HTTP 3.0

aka HTTP QUIC

Based on UDP, and it addresses the shortcomings of UDP, such as:

  • TCP-like congestion control
  • loss recovery
  • Uses TLS 1.3

Advantages over HTTP/2.0:

HTTP/3 is almost identical to HTTP/2, except that HTTP/3 integrates TLS within QUIC

  • Reduce connection establishment latency
    • Improved congestion control
  • Multiplexing without head-of-line blocking
  • Forward error correction
  • Connection migration (e.g., during Wi-Fi or 4G switching, no need to reconnect)

HTTP/3.0 is by default encrypted with TLS. How it achieve 0-RTT?

0-RTT Definition: During the encryption process, requests are sent simultaneously, which means that during the TLS handshake, requests can be sent together.

Additionally, TLS handshake is incorporated within QUIC, eliminating an extra step.

https://www.smashingmagazine.com/2021/08/http3-performance-improvements-part2/

Note: 0-RTT does not mean completely eliminating RTT; it refers to the feature of Session Resumption in TLS that saves steps. So, in HTTP/2, session resumption still requires a separate TCP handshake.

For HTTP/3, since QUIC incorporates TCP, the remaining 2 steps are directly reduced to one step.

Session resumption enables reducing it to one RTT directly.

Resources

https://blog.apnic.net/2023/09/25/why-http-3-is-eating-the-world/

https://medium.com/@chester.yw.chu/http-3-傳輸協議-quic-簡介-5f8806d6c8cd

https://engineering.cred.club/head-of-line-hol-blocking-in-http-1-and-http-2-50b24e9e3372

https://blog.csdn.net/m0_49147102/article/details/134480975

https://dl.acm.org/doi/pdf/10.1145/2542661

https://www.cnblogs.com/flydean/p/15419443.html

EOF