summaryrefslogtreecommitdiff
path: root/public/http-notes.md
blob: be1a0dd8fb4ee09224ae5f679953531e19c5d593 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
Notes on subtleties of HTTP implementation
==========================================
---
date: "2016-09-30"
---

I may add to this as time goes on, but I've written up some notes on
subtleties HTTP/1.1 message syntax as specified in RFC 2730.

## Why the absolute-form is used for proxy requests

[RFC7230§5.3.2][] says that a (non-CONNECT) request to an HTTP
proxy should look like

    GET http://authority/path HTTP/1.1

rather than the usual

    GET /path HTTP/1.1
    Host: authority

And doesn't give a hint as to why the message syntax is different
here.

[A blog post by Parsia Hakimian][why-absform] claims that the reason
is that it's a legacy behavior inherited from HTTP/1.0, which had
proxies, but not the Host header field.  Which is mostly true.  But we
can also realize that the usual syntax does not allow specifying a URI
scheme, which means that we cannot specify a transport.  Sure, the
only two HTTP transports we might expect to use today are TCP (scheme:
http) and TLS (scheme: https), and TLS requires we use a CONNECT
request to the proxy, meaning that the only option left is a TCP
transport; but that is no reason to avoid building generality into the
protocol.

## On taking short-cuts based on early header field values

[RFC7230§3.2.2][] says:

>     The order in which header fields with differing field names are
>     received is not significant.  However, it is good practice to send
>     header fields that contain control data first, such as Host on
>     requests and Date on responses, so that implementations can decide
>     when not to handle a message as early as possible.

Which is great!  We can make an optimization!

This is only a valid optimization for deciding to *not handle* a
message.  You cannot use it to decide to route to a backend early
based on this.  Part of the reason is that [§5.4][RFC7230§5.4] tells
us we must inspect the entire header field set to know if we need to
respond with a 400 status code:

>     A server MUST respond with a 400 (Bad Request) status code to any
>     HTTP/1.1 request message that lacks a Host header field and to any
>     request message that contains more than one Host header field or a
>     Host header field with an invalid field-value.

However, if I decide not to handle a request based on the Host header
field, the correct thing to do is to send a 404 status code.  Which
implies that I have parsed the remainder of the header field set to
validate the message syntax.  We need to parse the entire field-set to
know if we need to send a 400 or a 404.  Did this just kill the
possibility of using the optimization?

Well, there are a number of "A server MUST respond with a XXX code if"
rules that can all be triggered on the same request.  So we get to
choose which to use.  And fortunately for optimizing implementations,
[§3.2.5][RFC7230§3.2.5] gave us:

>     A server that receives a            ...           set of fields,
>     larger than it wishes to process MUST respond with an appropriate 4xx
>     (Client Error) status code.

Since the header field set is longer than we want to process (since we
want to short-cut processing), we are free to respond with whichever
4XX status code we like!

## On normalizing target URIs

An implementer is tempted to normalize URIs all over the place, just
for safety and sanitation.  After all,
[RFC3986§6.1][] says it's safe!

Unfortunately, most URI normalization implementations will normalize an
empty path to "/".  Which is not always safe; [RFC7230§2.7.3][], which
defines this "equivalence", actually says:

>                                             When not being used in
>     absolute form as the request target of an OPTIONS request, an empty
>     path component is equivalent to an absolute path of "/", so the
>     normal form is to provide a path of "/" instead.

Which means we can't use the usual normalization implementation if we
are making an OPTIONS request!

Why is that?  Well, if we turn to [§5.3.4][RFC7230§5.3.4], we find the
answer.  One of the special cases for when the request target is not a
URI, is that we may use "\*" as the target for an OPTIONS request to
request information about the origin server itself, rather than a
resource on that server.

However, as discussed above, the target in a request to a proxy must
be an absolute URI (and [§5.3.2][RFC7230§5.3.2] says that the origin
server must also understand this syntax).  So, we must define a way to
map "\*" to an absolute URI.

Naively, one might be tempted to use "/\*" as the path.  But that
would make it impossible to have a resource actually named "/\*".  So,
we must define a special case in the URI syntax that doesn't obstruct
a real path.

If we didn't have this special case in the URI normalization rules,
and we handled the "/" path as the same as empty in the OPTIONS
handler of the last proxy server, then it would be impossible to
request OPTIONS for the "/" resources, as it would get translated into
"\*" and treated as OPTIONS for the entire server.

[RFC3986§6.1]: https://tools.ietf.org/html/rfc3986#section-6.1
[RFC7230§2.7.3]: https://tools.ietf.org/html/rfc7230#section-2.7.3
[RFC7230§3.2.2]: https://tools.ietf.org/html/rfc7230#section-3.2.2
[RFC7230§3.2.5]: https://tools.ietf.org/html/rfc7230#section-3.2.5
[RFC7230§5.3.2]: https://tools.ietf.org/html/rfc7230#section-5.3.2
[RFC7230§5.3.4]: https://tools.ietf.org/html/rfc7230#section-5.3.4
[RFC7230§5.4]: https://tools.ietf.org/html/rfc7230#section-5.4
[why-absform]: https://parsiya.net/blog/2016-07-28-thick-client-proxying---part-6-how-https-proxies-work/#3-1-1-why-not-use-the-host-header