Switch to a simpler progress bar library. I've been having trouble tracking
down a panic that seemed to affect the prior code; the new code is much
easier and uses far fewer async channels and things.
If we have some really tiny files, it's not worth compressing them. Among other things,
this will work around a bug in go-zopfli for 0- or 1-byte files.
During the 1.3.1 update, a change was made to stop using http.Handle (and ServeMux
underneath), in order to have manual control over the http.Server object.
Unfortunately, it was overlooked that nothing was doing routing / multiplexing, so
when using multiple packfiles only one handler (picked arbitrarily due to map)
would actually be active on any given invocation.
Correct this by adding an explicit handler. We don't use ServeMux so as to avoid
bringing in any of its more complex behaviours like path cleaning etc.
This option allows a configurable delay after receiving SIGTERM (or
SIGINT) but before the HTTP server stops accepting new connections. It
is quite useful for distributed systems where callers are only notified
asynchronously (e.g. via service discovery) that a service is being shut
down; it prevents the shut down from occurring prior to callers
processing the notification.
This required some minor refactoring to allow the Shutdown() method on
http.Server to be accessed.
After some experimentation, I found that the sendfile(2) support did not
really save any time compared to just write(2) from an already
memory-mapped file.
After some reading, I think open/sendfile is supposed to be slightly
more efficient than open/mmap/write — but if we already did the mmap
step, then it doesn't save us much.
Moreover, the code to support sendfile(2) is a bit icky, and also forces
us to close the HTTP connection after serving a file.
Sometimes we might be asked to serve up a zero-length input file,
typically from some machine-generated CSS etc. We make some very
rudimentary guess about the content-type the caller wanted and skip the
mmap(2) call.
Since packing can be quite slow, it is nice to display progress to the
caller. We use the excellent github.com/vbauerster/mpb library to do so,
and add a bit of colour with github.com/logrusru/aurora.
Finally, augment the inspector and packer with a summary printer that
displays the overall file size/count, and compression ratio for each
compression type.
We now support single part ranges, as per:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests#Single_part_ranges
Multi-part ranges are not implemented because there have been far too
many bugs in this area.
It interacts with compression by selecting a byte range from the
compressed stream. Since the compressed stream is fixed, the results are
consistent.
If a .svg file doesn't have a “<?xml…” prolog, then it would be detected as plain
text. Allow extension-based override of the content-type in this case.
Having done a bit more research on exactly how to use sendfile()
effectively with Go's own event loop, we can now leave the HTTP socket
in non-blocking mode and correctly integrate with the event loop to wait
for the socket to be writeable once more.
Rather than setting "Cache-Control: no-store", use "no-cache". The
difference is that "no-store" tells the browser that it must always
completely re-fetch the resource; whereas "no-cache" means that the
browser can write the resource to disk but must re-validate it (e.g.
using Etags) before re-using it.
Laurence Withers
Laurence Withers