HTTP Caching
*복습 자료라서 뻔한 내용은 생략
Summary.
1. The HTTP cache stores a response associated with a request, and reuses the stored response for subsequent request
+ There are several advantages to reusability. First, since there is no need to deliver the request to the origin server, then the closer the client and cache are, the faster the response will be. The most-typical example is when the browser itself stores a cache for browser requests
+ Also, when a response is reusable, the origin server does not need to process the request — so it does not need to parse and route the request, restore the session based on the cookie, query the DB for results, or render the template engine. That reduces the load on the server
2. there are two main types of caches: private caches and shared caches
+ A private cache is a cache tied to a specific client — typically a browser cache. Since the stored response is not shared with other clients, a private cache can store a personalized response for that user
+ Personalized contents are usually controlled by cookies, but the presence of a cookie does not always indicate that it is private, and thus a cookie alone does not make the response private.
Note that if the response has an Authorization header, it cannot be stored in the private cache (or a shared cache, unless public is specified (예시는 없나?)
Cache-Control: private+ The shared cache is located between the client and the server and can store responses that can be shared among users. And shared caches can be further sub-classified into proxy caches and managed caches
+ Proxy Cache: In addition to the function of access control, some proxies implement caching to reduce traffic out of the network. This is usually not managed by the service developer, so it must be controlled by appropriate HTTP headers and so on. However, in recent years, as HTTPS has become more common and client/server communication has become encrypted, proxy caches in the path can only tunnel a response and can't behave as a cache, in many cases. So in that scenario, there is no need to worry about outdated proxy cache implementations that cannot even see the response
// Kitchen Sink Headers 라고 불리며, outdated proxy cache implementation을 해결(우회?)하기 위해 사용됐던 헤더
Cache-Control: no-store, no-cache, max-age=0, must-revalidate, proxy-revalidate+ Managed Cache: Managed caches are explicitly deployed by service developers to offload the origin server and to deliver content efficiently. Examples include reverse proxies, CDNs, and service workers in combination with the Cache API. You can control its behavior not only through the Cache-Control header, but also through your own configuration files and dashboards. For example, the HTTP Caching specification essentially does not define a way to explicitly delete a cache — but with a managed cache, the stored response can be deleted at any time through dashboard operations, API calls, restarts, and so on. That allows for a more proactive caching strategy. It is also possible to ignore the standard HTTP Caching spec protocols in favor of explicit manipulation. For example, the following can be specified to opt-out of a private cache or proxy cache, while using your own strategy to cache only in a managed cache.
// (다음에) 이게 default인가?
Cache-Control: no-store+ That means if a managed cache intentionally ignores a no-store directive, there is no need to perceive it as being "non-compliant" with the standard. What you should do is, avoid using kitchen-sink headers, but carefully read the documentation of whatever managed-cache mechanism you're using, and ensure you're controlling the cache properly in the ways provided by the mechanism you've chosen to us. Note that some CDNs provide their own headers that are effective only for that CDN (for example, Surrogate-Control). Currently, work is underway to define a CDN-Cache-Control header to standardize those
3. HTTP is designed to cache as much as possible, so even if no Cache-Control is given, responses will get stored and reused if certain conditions are met. This is called heuristic caching
+ It is heuristically known that content which has not been updated for a full year will not be updated for some time after that. Therefore, the client stores this response (despite the lack of max-age) and reuses it for a while. How long to reuse is up to the implementation, but the specification recommends about 10% (in this case 0.1 year) of the time after storing
+ Heuristic caching is a workaround that came in being before Cache-Control support became widely, and basically all responses should explicitly specify a Cache-Control header
4. Stored HTTP responses have two states: fresh and stale. The fresh state usually indicates that the response is still valid and can be reused, while the stale state means that the cached response has already expired
+ The criterion for determining when a response is fresh and when it is stale is age. In HTTP, age is the time elapsed since the response was generated. This is similar to the TTL in other caching mechanisms
Date: Tue, 22 Feb 2022 22:22:22 GMT
Cache-Control: max-age=604800+ If that response is stored in a private cache, it will be available for reuse in response to client requests for one week after it is stored. If the shared cache saves it, it is necessary to inform the client of the time elapsed from when it was stored to the shared cache until it is reused by the client. If the response has been stored in the shared cache for one day and then reused by the client, then the following response will be sent from the shared cache to the client
Date: Tue, 22 Feb 2022 22:22:22 GMT
Cache-Control: max-age=604800
Age: 86400+ The client which receives that response will find it to be fresh for the remaining 604800-86400 seconds; that is, for 518400 seconds more
5. Expires는 과거 Spec이고 Max-Age가 더 우선됨 (Expires는 System Clock 조작, 등, 허점이 많았음)
6. The content of responses are not always the same even if they have the same URL. Especially when content negotiation is performed, the response from the server can depend on the values of the Accept, Accept-Language, and Accept-Encoding request headers
+ For example, for English content returned with an Accept-Language: en header and cached, it is undesirable to then reuse that cached response for requests that have an Accept-Language: ja request header. In this case, you can cause the responses to be cached separately — based on language — by adding "Accept-Language" to the value of the Vary header
Vary: Accept-Language
+ User-Agent는 워낙 다양하므로 Vary 대상이 되면 비효율적
+ For applications that employ cookies to prevent others from reusing cached personalized content, you should specify Cache-Control: private instead of specifying a cookie for Vary
7. Stale responses are not immediately discarded. HTTP has a mechanism to transform a stale response to a fresh one by asking the origin server. This is called validation, or sometimes, revalidation
+ Validation is done by using a conditional request that includes an If-Modified-Since or If-None-Match request header
GET /index.html HTTP/1.1
Host: example.com
Accept: text/html
If-Modified-Since: Tue, 22 Feb 2022 22:00:00 GMT// No body -> small size
HTTP/1.1 304 Not Modified
Content-Type: text/html
Date: Tue, 22 Feb 2022 22:23:22 GMT
Last-Modified: Tue, 22 Feb 2022 22:00:00 GMT
Cache-Control: max-age=3600+ Upon receiving that response, the client reverts the stored stale response back to being fresh and can reuse it during the remaining 1 hour
8. The server can obtain the modification time from the operating-system file system, which is relatively easy to do for the case of serving static files. However, there are some problems; for example, the time format is complex and difficult to parse, and distributed servers have difficulty synchronizing file-update times. To solve such problems, the ETag response header was standardized as an alternative
+ The value of theETag response header is an arbitrary value generated by the server. There are no restrictions on how the server must generate the value, so servers are free to set the value based on whatever means they choose — such as a hash of the body contents or a version number
Date: Tue, 22 Feb 2022 22:22:22 GMT
ETag: "deadbeef"
Cache-Control: max-age=3600GET /index.html HTTP/1.1
Host: example.com
Accept: text/html
If-None-Match: "deadbeef"+ The server will return 304 Not Modified if the value of the ETag header it determines for the requested resource is the same as the If-None-Match value in the request. But if the server determines the requested resource should now have a different ETag value, the server will instead respond with a 200 OK and the latest version of the resource
+ When evaluating how to use ETag and Last-Modified, consider the following: During cache revalidation, if both ETag and Last-Modified are present, ETag takes precedence. Therefore, if you are only considering caching, you may think that Last-Modified is unnecessary. However, Last-Modified is not just useful for caching; instead, it is a standard HTTP header that is also used by content-management (CMS) systems to display the last-modified time, by crawlers to adjust crawl frequency, and for other various purposes. So considering the overall HTTP ecosystem, it is preferable to provide both ETag and Last-Modified
9. If you do not want a response to be reused, but instead want to always fetch the latest content from the server, you can use the no-cache directive to force validation
+ By adding Cache-Control: no-cache to the response along with Last-Modified and ETag — as shown below — the client will receive a 200 OK response if the requested resource has been updated, or will otherwise receive a 304 Not Modified response if the requested resource has not been updated
+ It is often stated that the combination of max-age=0 and must-revalidate has the same meaning as no-cache
Cache-Control: max-age=0, must-revalidate+ max-age=0 means that the response is immediately stale, and must-revalidate means that it must not be reused without revalidation once it is stale — so in combination, the semantics seem to be the same as no-cache
+ However, that usage of max-age=0 is a remnant of the fact that many implementations prior to HTTP/1.1 were unable to handle the no-cache directive — and so to deal with that limitation, max-age=0 was used as a workaround. But now that HTTP/1.1-conformant servers are widely deployed, there's no reason to ever use that max-age=0-and-must-revalidate combination — you should instead just use no-cache
10. The no-cache directive does not prevent storage of responses, but instead prevents reuse of responses without revalidation. If you don't want a response stored in any cache, use no-store
Cache-Control: no-store- Don't want the response stored by anyone other than the specific client, for privacy reasons.
- Want to provide up-to-date information always.
- Don't know what could happen with in outdated implementations
+ not most appropriate for the most of time
+ private 해야하면 no-store 여도 private 추가할 것
+ no-store 써도 이전에 저장된 것은 못지우므로 reuse risk가 있음. 따라서 매번 revaliadate 하는 no-cache가 더 안전
+ As a workaround for outdated implementations that ignore no-store, you may see kitchen-sink headers such as the following being used
+ It is recommended to use no-cache as an alternative for dealing with such outdated implementations, and it is not a problem if no-cache is given from the beginning, since the server will always receive the request
Cache-Control: no-store, no-cache, max-age=0, must-revalidate, proxy-revalidate
11. The reload and force reload actions are common examples of validation performed from the browser side
+ A simplified view of the HTTP request sent during browser a reload looks as follows (*The requests from Chrome, Edge, and Firefox look very much like the above; the requests from Safari will look a bit different)
GET / HTTP/1.1
Host: example.com
Cache-Control: max-age=0
If-None-Match: "deadbeef"
If-Modified-Since: Tue, 22 Feb 2022 20:20:20 GMT// 위 브라우저 새로고침 누를 때와 같은 효과인 javascript
// Note: "reload" is not the right mode for a normal reload; "no-cache" is
fetch("/", { cache: "no-cache" });+ Browsers use max-age=0 during reloads for backward-compatibility reasons — because many outdated implementations prior to HTTP/1.1 did not understand no-cache. But no-cache is fine now in this use case, and force reload is an additional way to bypass cached responses
+ The HTTP Request during a browser force reload looks as follows:
GET / HTTP/1.1
Host: example.com
Pragma: no-cache
Cache-Control: no-cache// Note: "reload" — rather than "no-cache" — is the right mode for a "force reload"
fetch("/", { cache: "reload" });
12. Content that never changes should be given a long max-age by using cache busting — that is, by including a version number, hash value, etc., in the request URL. However, when the user reloads, a revalidation request is sent even though the server knows that the content is immutable. To prevent that, the immutable directive can be used to explicitly indicate that revalidation is not required because the content never changes
+ That prevents unnecessary revalidation during reloads
Cache-Control: max-age=31536000, immutable
13. You may want to overwrite that response once it expired on the server, but there is nothing the server can do once the response is stored — since no more requests reach the server due to caching
+ it should be assumed that any stored response will remain for its max-age period unless the user manually performs a reload, force reload, or clear-history action
+ Caching reduces access to the server, which means that the server loses control on that URL. If the server does not want to lose control of a URL — for example, in the case of resource that is frequently updated — you should add no-cache so that the server will always receive requests and send the intended responses
HTTP/1.1 200 OK
Content-Type: text/html
Content-Length: 1024
Cache-Control: max-age=31536000
14. if multiple identical requests arrive at a shared cache at the same time, the intermediate cache will forward a single request on behalf of itself to the origin, which can then reuse the result for all clients. This is called request collapse
+ Request collapse occurs when requests are arriving at the same time, so even if max-age=0 or no-cache is given in the response, it will be reused
+ If the response is personalized to a particular user and you do not want it to be shared in collapse, you should add the private directive
15. the default behavior for caching (that is, for a response without Cache-Control) is not simply "don't cache" but implicit caching according to so-called "heuristic caching". To avoid that heuristic caching, it's preferable to explicitly give all responses a default Cache-Control header
+ To ensure that by default the latest versions of resources will always be transferred, it's common practice to make the default Cache-Control value include no-cache
+ if the service implements cookies or other login methods, and the content is personalized for each user, private must be given too, to prevent sharing with other users
Cache-Control: no-cache, private
16. The resources that work best with caching are static immutable files whose contents never change. And for resources that do change, it is a common best practice to change the URL each time the content changes, so that the URL unit can be cached for a longer period of time
+ Making a response cacheable over a long period of time by changing the URL when the content changes is called cache busting
+ In modern web development, JavaScript and CSS resources are frequently updated as development progresses. Also, if the versions of JavaScript and CSS resources that a client uses are out of sync, the display will break
<script src="bundle.js"></script>
<link rel="stylesheet" href="build.css" />
<body>
hello
</body>+ Since the cache distinguishes resources from one another based on their URLs, the cache will not be reused again if the URL changes when a resources is updated
# version in filename
bundle.v123.js
# version in query
bundle.js?v=123
# hash in filename
bundle.YsAIAAAA-QG4G6kCMAMBAAAAAAAoK.js
# hash in query
bundle.js?v=YsAIAAAA-QG4G6kCMAMBAAAAAAAoK<script src="bundle.v123.js"></script>
<link rel="stylesheet" href="build.v123.css" />
<body>
hello
</body>+ QPACK is a standard for compressing HTTP header fields, with tables of commonly-used field values defined. If you select one of those numbered options, you can compress values in 1 byte when transferred via HTTP3
36 cache-control max-age=0
37 cache-control max-age=2592000
38 cache-control max-age=604800
39 cache-control no-cache
40 cache-control no-store
41 cache-control public, max-age=31536000
16. Don't forget to set the If-Modified-Since and ETag headers, so that you don't have to re-transmit a resource when reloading. It's easy to generate those headers for pre-built static files
+ In addition, immutable can be added to prevent validation on reload.
# bundle.v123.js
200 OK HTTP/1.1
Content-Type: application/javascript
Content-Length: 1024
Cache-Control: public, max-age=31536000, immutable
If-Modified-Since: Tue, 22 Feb 2022 20:20:20 GMT
ETag: YsAIAAAA-QG4G6kCMAMBAAAAAAAoK
17. 그 외 use case
+ https://developer.mozilla.org/en-US/docs/Web/HTTP/Caching#main_resources
18. With the method described in previous sections, subresources can be cached for a long time by using cache busting, but main resources (which are usually HTML documents) can't be
+ Caching main resources is difficult because, using just standard directives from the HTTP Caching specification, there's no way to actively delete cache contents when content is updated on the server. However, it is possible by deploying a managed cache such as a CDN or service worker
+ For example, a CDN that allows cache purging via an API or dashboard operation would allow for a more aggressive caching strategy by storing the main resource and explicitly purging the relevant cache only when an update occurs on the server. A service worker could do the same if it could delete the contents in the Cache API when an update occurs on the server
Additional.
1. (다음에) Header를 통해서 거쳐가는 Proxy Node에 Response를 Caching 하게끔 명령을 해두는 Standard인 셈인가? 어떻게 revalidate 하지?
2. (다음에) 이해 못해서 다음에 공부할 것
+ On the other hand, if a TLS bridge proxy decrypts all communications in a person-in-the-middle manner by installing a certificate from a CA managed by the organization on the PC, and performs access control, etc. — it is possible to see the contents of the response and cache it. However, since CT (certificate transparency) has become widespread in recent years, and some browsers only allow certificates issued with an SCT (signed certificate timestamp), this method requires the application of an enterprise policy. In such a controlled environment, there is no need to worry about the proxy cache being "out of date and not updated"
+ For example, Varnish uses VCL-based logic to handle cache storage, while service workers in combination with the Cache API allow you to create that logic in JavaScript
3. (다음에) 헤더로 Cache Control 명령을 보내도 이걸 실제로 적용하는 건 어떻게 구현돼있는지? 프레임워크에서 알아서 해결?
4. The public directive should only be used if there is a need to store the response when the Authorization header is set. It is not required otherwise, because a response will be stored in the shared cache as long as max-age is given
5. (다음에) Cache Tutorial https://www.mnot.net/cache_docs/
6. (다음에) CDN 운용하는 법 알기