cmd/htpacker: swap out progress bar library

Switch to a simpler progress bar library. I've been having trouble tracking down a panic that seemed to affect the prior code; the new code is much easier and uses far fewer async channels and things.
cmd/packserver: updated modules
2024-08-24 09:54:25 +01:00 · 2024-07-18 14:58:38 +01:00 · 2024-07-18 14:57:49 +01:00 · 2024-07-18 14:54:50 +01:00 · 2024-07-18 14:52:12 +01:00 · 2024-05-03 17:14:12 +01:00
17 changed files with 904 additions and 229 deletions
--- a/README.md
+++ b/README.md
@ -1,5 +1,7 @@
 # HTTP resource pack server

+[![GoDoc](https://img.shields.io/static/v1?label=godoc&message=reference&color=blue)](https://pkg.go.dev/src.lwithers.me.uk/go/htpack)
+
 A common scenario is that you have a set of static resources that you want to
 serve up quickly via HTTP (for example: stylesheets, WASM).

@ -7,7 +9,7 @@ This package provides a `net/http`-compatible `http.Handler` to do so, with
 support for:
 - compression
  - gzip
-  - brotli, if you have the external compression binary available at pack time
+  - brotli
  - does not yet support Transfer-Encoding, only Accept-Encoding/Content-Encoding
 - etags
 - ranges
@ -28,3 +30,42 @@ else will be ignored.
 The interaction between range handling and compression also seems a little
 ill-defined; as we have pre-compressed data, however, we can consistently
 serve the exact same byte data for compressed files.
+
+## Angular-style single-page application handling
+
+If you wish to support an angular.js-style single page application, in which
+a Javascript application uses the browser's history API to create a set of
+virtual paths ("routes"), it is necessary to somehow intercept HTTP 404 errors
+being returned from the handler and instead return an HTTP 200 with an HTML
+document.
+
+This can be achieved with a number of methods.
+
+The simplest method is to tell `packserver` itself which resource to use
+instead of returning an HTTP 404. Use the command line argument
+`--fallback-404 /index.html` (or whichever named resource). The filename must
+match a packed resource, so it will be preceded with a `/`. It must exist in
+all packfiles being served.
+
+If you have an nginx instance reverse proxying in front of `htpack`, then you
+can use a couple of extra directives. This is very flexible as it lets you
+override the resource for different routes. For example:
+
+	# prevent page loaded at "http://server.example/my-application" from
+	# requesting resources at "/*" when it should request them at
+	# "/my-application/*" instead
+	location = /my-application {
+		return 308 /my-application/;
+	}
+
+	location /my-application/ {
+		proxy_to http://htpack-addr:8080/;
+		proxy_intercept_errors on;
+		error_page 404 =200 /my-application/;
+	}
+
+If you are using the handler as a library, then you may call
+`handler.SetNotFound(filename)` to select a resource to return (with HTTP 200)
+if a request is made for a resource that is not found. The filename must match
+a packed resource, so it will be preceded with a `/` (for example it may be
+`"/index.html"`).
--- a/cmd/htpacker/content_type_glob.go
+++ b/cmd/htpacker/content_type_glob.go
@ -0,0 +1,105 @@
+package main
+
+import (
+	"fmt"
+	"path/filepath"
+	"strings"
+
+	"src.lwithers.me.uk/go/htpack/cmd/htpacker/packer"
+)
+
+type ctGlobEntry struct {
+	pattern, contentType string
+	pathComponents       int
+}
+
+type ctGlobList []ctGlobEntry
+
+func parseGlobs(flags []string) (ctGlobList, error) {
+	var ctGlobs ctGlobList
+	for _, flag := range flags {
+		// split pattern:content-type
+		pos := strings.LastIndexByte(flag, ':')
+		if pos == -1 {
+			return nil, &parseGlobError{
+				Value: flag,
+				Err:   "must be pattern:content-type",
+			}
+		}
+		pattern, ct := flag[:pos], flag[pos+1:]
+
+		// patterns starting with "/" must match the entire directory
+		// prefix; otherwise, an arbitrary number of path components are
+		// allowed prior to the prefix
+		var pathComponents int
+		if strings.HasPrefix(pattern, "/") {
+			pathComponents = -1
+			pattern = strings.TrimPrefix(pattern, "/")
+		} else {
+			pathComponents = 1 + strings.Count(pattern, "/")
+		}
+
+		// test that the pattern's syntax is valid
+		if _, err := filepath.Match(pattern, "test"); err != nil {
+			return nil, &parseGlobError{
+				Value: flag,
+				Err:   err.Error(),
+			}
+		}
+
+		ctGlobs = append(ctGlobs, ctGlobEntry{
+			pattern:        pattern,
+			contentType:    ct,
+			pathComponents: pathComponents,
+		})
+	}
+
+	return ctGlobs, nil
+}
+
+// ApplyContentTypes will scan the list of files to pack, matching by filename,
+// and on match will apply the given content type.
+func (ctGlobs ctGlobList) ApplyContentTypes(ftp packer.FilesToPack) {
+	for name := range ftp {
+		for _, entry := range ctGlobs {
+			testName := trimPathComponents(name, entry.pathComponents)
+			matched, _ := filepath.Match(entry.pattern, testName)
+			if matched {
+				f := ftp[name]
+				f.ContentType = entry.contentType
+				ftp[name] = f
+				break
+			}
+		}
+	}
+}
+
+func trimPathComponents(name string, components int) string {
+	name = strings.TrimPrefix(name, "/") // FilesToPack keys = absolute path
+
+	// if we are matching the full prefix, don't otherwise manipulate the
+	// name
+	if components < 0 {
+		return name
+	}
+
+	// otherwise, trim the number of components remaining in the path so
+	// that we are only matching the trailing path components from the
+	// FilesToPack key
+	parts := 1 + strings.Count(name, "/")
+	for ; parts > components; parts-- {
+		pos := strings.IndexByte(name, '/')
+		name = name[pos+1:]
+	}
+	return name
+}
+
+// parseGlobError is returned from parseGlobs on error.
+type parseGlobError struct {
+	Value string
+	Err   string
+}
+
+func (pge *parseGlobError) Error() string {
+	return fmt.Sprintf("--content-type entry %q: %s", pge.Value, pge.Err)
+}
--- a/cmd/htpacker/content_type_glob_test.go
+++ b/cmd/htpacker/content_type_glob_test.go
@ -0,0 +1,109 @@
+package main
+
+import (
+	"testing"
+
+	"src.lwithers.me.uk/go/htpack/cmd/htpacker/packer"
+)
+
+func TestParseGlobs(t *testing.T) {
+	ctGlobs, err := parseGlobs([]string{
+		"*.foo:text/html",
+		"*.bar:text/plain",
+		"baz/qux/*.js:application/javascript",
+		"/abs/file:image/png",
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	check := func(pos int, pattern, contentType string, pathComponents int) {
+		if pos >= len(ctGlobs) {
+			t.Errorf("entry %d not present", pos)
+			return
+		}
+
+		if pattern != ctGlobs[pos].pattern {
+			t.Errorf("entry %d: expected pattern %q but got %q",
+				pos, pattern, ctGlobs[pos].pattern)
+		}
+
+		if contentType != ctGlobs[pos].contentType {
+			t.Errorf("entry %d: expected content type %q but got %q",
+				pos, contentType, ctGlobs[pos].contentType)
+		}
+
+		if pathComponents != ctGlobs[pos].pathComponents {
+			t.Errorf("entry %d: expected num. path components %d but got %d",
+				pos, pathComponents, ctGlobs[pos].pathComponents)
+		}
+	}
+
+	check(0, "*.foo", "text/html", 1)
+	check(1, "*.bar", "text/plain", 1)
+	check(2, "baz/qux/*.js", "application/javascript", 3)
+	check(3, "abs/file", "image/png", -1)
+}
+
+func TestParseGlobsErrSep(t *testing.T) {
+	const badValue = "hello/dave.js" // missing ":" separator
+	_, err := parseGlobs([]string{badValue})
+	switch err := err.(type) {
+	case *parseGlobError:
+		if err.Value != badValue {
+			t.Errorf("expected value %q but got %q", badValue, err.Value)
+		}
+	case nil:
+		t.Fatal("expected error")
+	default:
+		t.Errorf("unexpected error type %T (value %v)", err, err)
+	}
+}
+
+func TestParseGlobsErrPattern(t *testing.T) {
+	const badValue = "[-z]:foo/bar" // malformed character class
+	_, err := parseGlobs([]string{badValue})
+	switch err := err.(type) {
+	case *parseGlobError:
+		if err.Value != badValue {
+			t.Errorf("expected value %q but got %q", badValue, err.Value)
+		}
+	case nil:
+		t.Fatal("expected error")
+	default:
+		t.Errorf("unexpected error type %T (value %v)", err, err)
+	}
+}
+
+func TestApplyContentTypes(t *testing.T) {
+	// XXX: we program our _expectation_ of content-type into the Filename field
+	ftp := packer.FilesToPack{
+		"foo.txt":     packer.FileToPack{Filename: "text/plain"},
+		"baz/foo.txt": packer.FileToPack{Filename: "text/plain"},
+
+		"baz/qux.png":     packer.FileToPack{Filename: "image/png"},
+		"foo/qux.png":     packer.FileToPack{},
+		"foo/baz/qux.png": packer.FileToPack{Filename: "image/png"},
+
+		"bar.jpeg":         packer.FileToPack{},
+		"foo/baz/bar.jpeg": packer.FileToPack{},
+		"baz/bar.jpeg":     packer.FileToPack{Filename: "image/jpeg"},
+	}
+
+	ctGlobs, err := parseGlobs([]string{
+		"*.txt:text/plain",         // should match anywhere
+		"baz/qux.png:image/png",    // won't match /foo/qux.png
+		"/baz/bar.jpeg:image/jpeg", // exact prefix match
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	ctGlobs.ApplyContentTypes(ftp)
+	for k, v := range ftp {
+		if v.Filename != v.ContentType {
+			t.Errorf("filename %q: expected content type %q but got %q",
+				k, v.Filename, v.ContentType)
+		}
+	}
+}
--- a/cmd/htpacker/go.mod
+++ b/cmd/htpacker/go.mod
@ -1,15 +1,24 @@
 module src.lwithers.me.uk/go/htpack/cmd/htpacker

-go 1.12
+go 1.22

 require (
-	github.com/andybalholm/brotli v1.0.0
+	github.com/andybalholm/brotli v1.1.0
 	github.com/foobaz/go-zopfli v0.0.0-20140122214029-7432051485e2
-	github.com/kr/pretty v0.1.0 // indirect
-	github.com/spf13/cobra v0.0.5
-	golang.org/x/sys v0.0.0-20200113162924-86b910548bc1
-	gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 // indirect
-	gopkg.in/yaml.v2 v2.2.2
-	src.lwithers.me.uk/go/htpack v1.1.5
+	github.com/gosuri/uiprogress v0.0.1
+	github.com/spf13/cobra v1.8.1
+	golang.org/x/sys v0.22.0
+	gopkg.in/yaml.v2 v2.4.0
+	src.lwithers.me.uk/go/htpack v1.3.3
 	src.lwithers.me.uk/go/writefile v1.0.1
 )
+
+require (
+	github.com/gogo/protobuf v1.3.2 // indirect
+	github.com/gosuri/uilive v0.0.4 // indirect
+	github.com/inconshreveable/mousetrap v1.1.0 // indirect
+	github.com/kr/pretty v0.1.0 // indirect
+	github.com/mattn/go-isatty v0.0.20 // indirect
+	github.com/spf13/pflag v1.0.5 // indirect
+	gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 // indirect
+)
--- a/cmd/htpacker/go.sum
+++ b/cmd/htpacker/go.sum
@ -1,57 +1,68 @@
-github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
-github.com/andybalholm/brotli v1.0.0 h1:7UCwP93aiSfvWpapti8g88vVVGp2qqtGyePsSuDafo4=
-github.com/andybalholm/brotli v1.0.0/go.mod h1:loMXtMfwqflxFJPmdbJO0a3KNoPuLBgiu3qAvBg8x/Y=
-github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6/go.mod h1:grANhF5doyWs3UAsr3K4I6qtAmlQcZDesFNEHPZAzj8=
-github.com/coreos/etcd v3.3.10+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=
-github.com/coreos/go-etcd v2.0.0+incompatible/go.mod h1:Jez6KQU2B/sWsbdaef3ED8NzMklzPG4d5KIOhIy30Tk=
-github.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
-github.com/cpuguy83/go-md2man v1.0.10/go.mod h1:SmD6nW6nTyfqj6ABTjUi3V3JVMnlJmwcJI5acqYI6dE=
-github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/andybalholm/brotli v1.1.0 h1:eLKJA0d02Lf0mVpIDgYnqXcUn0GqVmEFny3VuID1U3M=
+github.com/andybalholm/brotli v1.1.0/go.mod h1:sms7XGricyQI9K10gOSf56VKKWS4oLer58Q+mhRPtnY=
+github.com/cpuguy83/go-md2man/v2 v2.0.4/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
 github.com/foobaz/go-zopfli v0.0.0-20140122214029-7432051485e2 h1:VA6jElpcJ+wkwEBufbnVkSBCA2TEnxdRppjRT5Kvh0A=
 github.com/foobaz/go-zopfli v0.0.0-20140122214029-7432051485e2/go.mod h1:Yi95+RbwKz7uGndSuUhoq7LJKh8qH8DT9fnL4ewU30k=
-github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
-github.com/gogo/protobuf v1.2.1 h1:/s5zKNz0uPFCZ5hddgPdo2TK2TVrUNMn0OOX8/aZMTE=
-github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4=
-github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ=
-github.com/inconshreveable/mousetrap v1.0.0 h1:Z8tu5sraLXCXIcARxBp/8cbvlwVa7Z1NHg9XEKhtSvM=
-github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
-github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q=
+github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
+github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
+github.com/gosuri/uilive v0.0.4 h1:hUEBpQDj8D8jXgtCdBu7sWsy5sbW/5GhuO8KBwJ2jyY=
+github.com/gosuri/uilive v0.0.4/go.mod h1:V/epo5LjjlDE5RJUcqx8dbw+zc93y5Ya3yg8tfZ74VI=
+github.com/gosuri/uiprogress v0.0.1 h1:0kpv/XY/qTmFWl/SkaJykZXrBBzwwadmW8fRb7RJSxw=
+github.com/gosuri/uiprogress v0.0.1/go.mod h1:C1RTYn4Sc7iEyf6j8ft5dyoZ4212h8G1ol9QQluh5+0=
+github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
+github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
+github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
 github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
 github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=
 github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
 github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
 github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
 github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
-github.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ=
-github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
-github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
-github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic=
-github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
-github.com/russross/blackfriday v1.5.2/go.mod h1:JO/DiYxRf+HjHt06OyowR9PTA263kcR/rfWxYHBV53g=
-github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ=
-github.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=
-github.com/spf13/cobra v0.0.5 h1:f0B+LkLX6DtmRH1isoNA9VTtNUK9K8xYd28JNNfOv/s=
-github.com/spf13/cobra v0.0.5/go.mod h1:3K3wKZymM7VvHMDS9+Akkh4K60UwM26emMESw8tLCHU=
-github.com/spf13/jwalterweatherman v1.0.0/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo=
-github.com/spf13/pflag v1.0.3 h1:zPAT6CGy6wXeQ7NtTnaTerfKOsV6V6F8agHXFiazDkg=
-github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
-github.com/spf13/viper v1.3.2/go.mod h1:ZiWeW+zYFKm7srdB9IoDzzZXaJaI5eL9QjNiN/DMA2s=
-github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
-github.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8/go.mod h1:VFNgLljTbGfSG7qAOspJ7OScBnGdDN/yBr0sguwnwf0=
-github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q=
-golang.org/x/crypto v0.0.0-20181203042331-505ab145d0a9/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
-golang.org/x/sys v0.0.0-20181205085412-a5c9d58dba9a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
+github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
+github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
+github.com/spf13/cobra v1.8.1 h1:e5/vxKd/rZsfSJMUX1agtjeTDf+qv1/JdBF8gg5k9ZM=
+github.com/spf13/cobra v1.8.1/go.mod h1:wHxEcudfqmLYa8iTfL+OuZPbBZkmvliBWKIezN3kD9Y=
+github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
+github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
+github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
+github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
+golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
+golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
+golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
+golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
+golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
+golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
+golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
+golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200113162924-86b910548bc1 h1:gZpLHxUX5BdYLA08Lj4YCJNN/jk7KtquiArPoeX0WvA=
-golang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.22.0 h1:RI27ohtqKCnwULzJLqkv897zojh5/DwS/ENaMzUOaWI=
+golang.org/x/sys v0.22.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
 golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
-golang.org/x/tools v0.0.0-20180221164845-07fd8470d635/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
+golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
+golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
+golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
+golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY=
 gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
-gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=
-gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
-src.lwithers.me.uk/go/htpack v1.1.5 h1:2JzgqLZ1ROYc53+96NezfJ3S9fkwHZNd6QgJhMXnlSE=
-src.lwithers.me.uk/go/htpack v1.1.5/go.mod h1:JWofpm01RJbCTIyKfIPftUsxk6KlFkrYwyHgCVdKY+s=
+gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
+gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
+gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+src.lwithers.me.uk/go/htpack v1.3.3 h1:Xvl6qR9HfSblmCgPyu+ACQ9o3aLQSIy3l8CrMbzj/jc=
+src.lwithers.me.uk/go/htpack v1.3.3/go.mod h1:qKgCBgZ6iiiuYOxZkYOPVpXLBzp6gXEd4A0ksxgR6Nk=
 src.lwithers.me.uk/go/writefile v1.0.1 h1:bwBGtvyZfCxFIM14e1aYgJWlZuowKkwJx53OJlUPd0s=
 src.lwithers.me.uk/go/writefile v1.0.1/go.mod h1:NahlmRCtB7kg4ai+zHZgxXdUs+MR8VqWG8mql35TsxA=
--- a/cmd/htpacker/inspector.go
+++ b/cmd/htpacker/inspector.go
@ -32,7 +32,6 @@ var inspectCmd = &cobra.Command{

 // Inspect a packfile.
 //  TODO: verify etag; verify integrity of compressed data.
-//  TODO: skip Gzip/Brotli if not present; print ratio.
 func Inspect(filename string) error {
 	f, err := os.Open(filename)
 	if err != nil {
@ -65,10 +64,46 @@ func Inspect(filename string) error {
 					printSize(info.Brotli.Length), info.Brotli.Offset)
 			}
 		}
+		inspectSummary(dir)
 	}
 	return err
 }

+func inspectSummary(dir *packed.Directory) {
+	var (
+		n, ngzip, nbrotli int
+		s, sgzip, sbrotli uint64
+	)
+
+	for _, f := range dir.Files {
+		n++
+		s += f.Uncompressed.Length
+		if f.Gzip != nil {
+			ngzip++
+			sgzip += f.Gzip.Length
+		}
+		if f.Brotli != nil {
+			nbrotli++
+			sbrotli += f.Brotli.Length
+		}
+	}
+
+	fmt.Printf("Uncompressed:\n\tFiles: %d\n\tSize:  %s\n",
+		n, printSize(s))
+	if ngzip > 0 {
+		fmt.Printf("gzip compressed:\n\tFiles: %d (%.1f%% of total)\n"+
+			"\tSize:  %s\n\tRatio: %.1f%%\n",
+			ngzip, 100*float64(ngzip)/float64(n),
+			printSize(sgzip), 100*float64(sgzip)/float64(s))
+	}
+	if nbrotli > 0 {
+		fmt.Printf("brotli compressed:\n\tFiles: %d (%.1f%% of total)\n"+
+			"\tSize:  %s\n\tRatio: %.1f%%\n",
+			nbrotli, 100*float64(nbrotli)/float64(n),
+			printSize(sbrotli), 100*float64(sbrotli)/float64(s))
+	}
+}
+
 func printSize(size uint64) string {
 	switch {
 	case size < 1<<10:
--- a/cmd/htpacker/pack.go
+++ b/cmd/htpacker/pack.go
@ -10,11 +10,26 @@ import (
 	"github.com/spf13/cobra"
 	yaml "gopkg.in/yaml.v2"
 	"src.lwithers.me.uk/go/htpack/cmd/htpacker/packer"
+	"src.lwithers.me.uk/go/htpack/packed"
 )

 var packCmd = &cobra.Command{
 	Use:   "pack",
 	Short: "creates a packfile from a YAML spec or set of files/dirs",
+	Long: `When given a YAML spec file (a template for which can be generated
+with the "yaml" command), files will be packed exactly as per the spec. The
+--content-type flag cannot be used and no extra files can be specified.
+
+When given a list of files and directories to pack, the content type for each
+file will be automatically detected. It is possible to override the content
+type by specifying one or more --content-type flags. These take an argument in
+the form "pattern:content/type". The pattern is matched using common glob
+(* = wildcard), very similar to .gitignore. If the pattern contains any
+directory names, these must match the final components of the file to pack's
+path. If the pattern starts with a "/", then the full path must be matched
+exactly.
+`,
+
 	RunE: func(c *cobra.Command, args []string) error {
 		// convert "out" to an absolute path, so that it will still
 		// work after chdir
@ -50,6 +65,16 @@ var packCmd = &cobra.Command{
 			}
 		}

+		// parse content-type globs
+		ctGlobList, err := c.Flags().GetStringArray("content-type")
+		if err != nil {
+			return err
+		}
+		ctGlobs, err := parseGlobs(ctGlobList)
+		if err != nil {
+			return err
+		}
+
 		// if "spec" is not present, then we expect a list of input
 		// files, and we'll build a spec from them
 		if spec == "" {
@ -57,12 +82,16 @@ var packCmd = &cobra.Command{
 				return errors.New("need --yaml, " +
 					"or one or more filenames")
 			}
-			err = PackFiles(c, args, out)
+			err = PackFiles2(c, args, ctGlobs, out)
 		} else {
 			if len(args) != 0 {
 				return errors.New("cannot specify files " +
 					"when using --yaml")
 			}
+			if ctGlobs != nil {
+				return errors.New("cannot specify --content-type " +
+					"when using --yaml")
+			}
 			err = PackSpec(c, spec, out)
 		}
 		if err != nil {
@ -82,14 +111,22 @@ func init() {
 		"YAML specification file (if not present, just pack files)")
 	packCmd.Flags().StringP("chdir", "C", "",
 		"Change to directory before searching for input files")
+	packCmd.Flags().StringArrayP("content-type", "", nil,
+		"Override content type for pattern, e.g. \"*.foo=bar/baz\" (like .gitignore)")
 }

 func PackFiles(c *cobra.Command, args []string, out string) error {
+	return PackFiles2(c, args, nil, out)
+}
+
+func PackFiles2(c *cobra.Command, args []string, ctGlobs ctGlobList, out string) error {
 	ftp, err := filesFromList(args)
 	if err != nil {
 		return err
 	}
-	return packer.Pack(ftp, out)
+	ctGlobs.ApplyContentTypes(ftp)
+
+	return doPack(ftp, out)
 }

 func PackSpec(c *cobra.Command, spec, out string) error {
@ -103,5 +140,26 @@ func PackSpec(c *cobra.Command, spec, out string) error {
 		return fmt.Errorf("parsing YAML spec %s: %v", spec, err)
 	}

-	return packer.Pack(ftp, out)
+	return doPack(ftp, out)
+}
+
+func doPack(ftp packer.FilesToPack, out string) error {
+	prog := newUiProgress(ftp)
+	err := packer.Pack2(ftp, out, prog)
+	prog.Complete()
+
+	if err == nil {
+		fin, err := os.Open(out)
+		if err != nil {
+			return err
+		}
+		defer fin.Close()
+
+		_, dir, err := packed.Load(fin)
+		if err != nil {
+			return err
+		}
+		inspectSummary(dir)
+	}
+	return err
 }
--- a/cmd/htpacker/packer/packer.go
+++ b/cmd/htpacker/packer/packer.go
@ -45,7 +45,30 @@ type FileToPack struct {
 	DisableBrotli bool `yaml:"disable_brotli"`
 }

+// Progress is a callback object which reports packing progress.
+type Progress interface {
+	// Count reports the number of items that have begun processing.
+	Count(n int)
+
+	// Begin denotes the processing of an input file.
+	Begin(filename, compression string)
+
+	// End denotes the completion of input file processing.
+	End(filename, compression string)
+}
+
+type ignoreProgress int
+
+func (ignoreProgress) Count(_ int)       {}
+func (ignoreProgress) Begin(_, _ string) {}
+func (ignoreProgress) End(_, _ string)   {}
+
 const (
+	// minCompressionFileSize is the minimum filesize we need before
+	// considering compression. Note this must be at least 2, to avoid
+	// known bugs in go-zopfli.
+	minCompressionFileSize = 128
+
 	// minCompressionSaving means we'll only use the compressed version of
 	// the file if it's at least this many bytes smaller than the original.
 	// Chosen somewhat arbitrarily; we have to add an HTTP header, and the
@ -68,8 +91,18 @@ const (
 	sendfileLimit = 0x7FFFF000
 )

-// Pack a file.
+// Pack a file. Use Pack2 for progress reporting.
 func Pack(filesToPack FilesToPack, outputFilename string) error {
+	return Pack2(filesToPack, outputFilename, nil)
+}
+
+// Pack2 will pack a file, with progress reporting. The progress interface may
+// be nil.
+func Pack2(filesToPack FilesToPack, outputFilename string, progress Progress) error {
+	if progress == nil {
+		progress = ignoreProgress(0)
+	}
+
 	finalFname, w, err := writefile.New(outputFilename)
 	if err != nil {
 		return err
@ -79,7 +112,8 @@ func Pack(filesToPack FilesToPack, outputFilename string) error {
 	// we use this little structure to serialise file writes below, and
 	// it has a couple of convenience methods for common operations
 	packer := packer{
-		w: w,
+		w:        w,
+		progress: progress,
 	}

 	// write initial header (will rewrite offset/length when known)
@ -124,6 +158,7 @@ func Pack(filesToPack FilesToPack, outputFilename string) error {
 		Files: make(map[string]*packed.File),
 	}

+	var count int
 PackingLoop:
 	for path, fileToPack := range filesToPack {
 		select {
@ -132,6 +167,8 @@ PackingLoop:
 			break PackingLoop
 		default:
 			packer.packFile(path, fileToPack)
+			count++
+			progress.Count(count)
 		}
 	}

@ -199,12 +236,13 @@ func compressionWorthwhile(data []byte, compressed os.FileInfo) bool {
 // of compression. Unexported methods assume they are called in a context where
 // the lock is not needed or already taken; exported methods take the lock.
 type packer struct {
-	w       *os.File
-	lock    sync.Mutex
-	cpus    chan struct{}
-	errors  chan error
-	aborted chan struct{}
-	dir     *packed.Directory
+	w        *os.File
+	lock     sync.Mutex
+	cpus     chan struct{}
+	errors   chan error
+	aborted  chan struct{}
+	dir      *packed.Directory
+	progress Progress
 }

 // pad will move the file write pointer to the next padding boundary. It is not
@ -292,11 +330,14 @@ func (p *packer) packFile(path string, fileToPack FileToPack) {
 		return
 	}

-	data, err := unix.Mmap(int(f.Fd()), 0, int(fi.Size()),
-		unix.PROT_READ, unix.MAP_SHARED)
-	if err != nil {
-		p.Abort(fmt.Errorf("mmap %s: %v", fileToPack.Filename, err))
-		return
+	var data []byte
+	if fi.Size() > 0 {
+		data, err = unix.Mmap(int(f.Fd()), 0, int(fi.Size()),
+			unix.PROT_READ, unix.MAP_SHARED)
+		if err != nil {
+			p.Abort(fmt.Errorf("mmap %s: %v", fileToPack.Filename, err))
+			return
+		}
 	}

 	// prepare initial directory entry
@ -312,23 +353,29 @@ func (p *packer) packFile(path string, fileToPack FileToPack) {
 	// list of operations on this input file that we'll carry out asynchronously
 	ops := []func() error{
 		func() error {
+			p.progress.Begin(fileToPack.Filename, "uncompressed")
+			defer p.progress.End(fileToPack.Filename, "uncompressed")
 			return p.Uncompressed(fileToPack.Filename, info)
 		},
 	}
 	if !fileToPack.DisableCompression && !fileToPack.DisableGzip {
 		ops = append(ops, func() error {
+			p.progress.Begin(fileToPack.Filename, "gzip")
+			defer p.progress.End(fileToPack.Filename, "gzip")
 			if err := p.Gzip(data, info); err != nil {
 				return fmt.Errorf("gzip compression of %s "+
-					"failed: %v", fileToPack.Filename, err)
+					"failed: %w", fileToPack.Filename, err)
 			}
 			return nil
 		})
 	}
 	if !fileToPack.DisableCompression && !fileToPack.DisableBrotli {
 		ops = append(ops, func() error {
+			p.progress.Begin(fileToPack.Filename, "brotli")
+			defer p.progress.End(fileToPack.Filename, "brotli")
 			if err := p.Brotli(data, info); err != nil {
 				return fmt.Errorf("brotli compression of %s "+
-					"failed: %v", fileToPack.Filename, err)
+					"failed: %w", fileToPack.Filename, err)
 			}
 			return nil
 		})
@ -391,6 +438,10 @@ func (p *packer) Uncompressed(srcPath string, dir *packed.File) error {
 // Gzip will gzip input data to a temporary file, and then append that to the
 // output file.
 func (p *packer) Gzip(data []byte, dir *packed.File) error {
+	if len(data) < minCompressionFileSize {
+		return nil
+	}
+
 	// write via temporary file
 	tmpfile, err := ioutil.TempFile("", "")
 	if err != nil {
@ -432,6 +483,9 @@ func (p *packer) Gzip(data []byte, dir *packed.File) error {
 // Brotli will compress input data to a temporary file, and then append that to
 // the output file.
 func (p *packer) Brotli(data []byte, dir *packed.File) error {
+	if len(data) < minCompressionFileSize {
+		return nil
+	}
 	// write via temporary file
 	tmpfile, err := ioutil.TempFile("", "")
 	if err != nil {
--- a/cmd/htpacker/progress.go
+++ b/cmd/htpacker/progress.go
@ -0,0 +1,131 @@
+package main
+
+import (
+	"bytes"
+	"slices"
+	"sync"
+
+	"github.com/gosuri/uiprogress"
+	"src.lwithers.me.uk/go/htpack/cmd/htpacker/packer"
+)
+
+type uiProgress struct {
+	p                          *uiprogress.Progress
+	uncompressed, gzip, brotli *uiProgressBar
+}
+
+func newUiProgress(ftp packer.FilesToPack) *uiProgress {
+	up := &uiProgress{
+		p: uiprogress.New(),
+	}
+
+	up.uncompressed = newUiProgressBar(up.p, len(ftp), "uncompressed")
+	var nGzip, nBrotli int
+	for _, f := range ftp {
+		if !f.DisableCompression && !f.DisableGzip {
+			nGzip++
+		}
+		if !f.DisableCompression && !f.DisableBrotli {
+			nBrotli++
+		}
+	}
+	if nGzip > 0 {
+		up.gzip = newUiProgressBar(up.p, nGzip, "gzip")
+	}
+	if nBrotli > 0 {
+		up.brotli = newUiProgressBar(up.p, nGzip, "brotli")
+	}
+
+	up.p.Start()
+	return up
+}
+
+func (up *uiProgress) Count(_ int) {
+}
+
+func (up *uiProgress) Begin(filename, compression string) {
+	up.bar(compression).begin(filename)
+}
+
+func (up *uiProgress) End(filename, compression string) {
+	up.bar(compression).end(filename)
+}
+
+func (up *uiProgress) bar(compression string) *uiProgressBar {
+	switch compression {
+	case "uncompressed":
+		return up.uncompressed
+	case "gzip":
+		return up.gzip
+	case "brotli":
+		return up.brotli
+	}
+	return nil
+}
+
+func (up *uiProgress) Complete() {
+	up.p.Stop()
+}
+
+type uiProgressBar struct {
+	bar      *uiprogress.Bar
+	lock     sync.Mutex
+	inflight []string
+}
+
+func newUiProgressBar(p *uiprogress.Progress, total int, compression string) *uiProgressBar {
+	bar := &uiProgressBar{
+		bar: p.AddBar(total).AppendCompleted(),
+	}
+
+	var buf bytes.Buffer
+	bar.bar.PrependFunc(func(*uiprogress.Bar) string {
+		bar.lock.Lock()
+		defer bar.lock.Unlock()
+		buf.Reset()
+		buf.WriteString(compression)
+		if len(bar.inflight) > 0 {
+			buf.WriteString(" (")
+			for i, f := range bar.inflight {
+				if i > 0 {
+					buf.WriteString(", ")
+				}
+				buf.WriteString(f)
+			}
+			buf.WriteRune(')')
+		}
+		if buf.Len() < 40 {
+			buf.WriteString("                                        ")
+			buf.Truncate(40)
+		} else if buf.Len() > 40 {
+			buf.Truncate(39)
+			buf.WriteString("…")
+		}
+		return buf.String()
+	})
+
+	return bar
+}
+
+func (bar *uiProgressBar) begin(filename string) {
+	if bar == nil {
+		return
+	}
+	bar.lock.Lock()
+	defer bar.lock.Unlock()
+
+	bar.inflight = append(bar.inflight, filename)
+}
+
+func (bar *uiProgressBar) end(filename string) {
+	if bar == nil {
+		return
+	}
+	bar.lock.Lock()
+	defer bar.lock.Unlock()
+
+	bar.bar.Incr()
+	if idx := slices.Index(bar.inflight, filename); idx != -1 {
+		bar.inflight = slices.Delete(bar.inflight, idx, idx+1)
+	}
+}
--- a/cmd/htpacker/yaml.go
+++ b/cmd/htpacker/yaml.go
@ -3,6 +3,7 @@ package main
 import (
 	"errors"
 	"fmt"
+	"io"
 	"io/ioutil"
 	"net/http"
 	"os"
@ -125,13 +126,22 @@ func filesFromListR(prefix, arg string, ftp packer.FilesToPack) error {

 	case fi.Mode().IsRegular():
 		// sniff content type
+		var ctype string
 		buf := make([]byte, 512)
 		n, err := f.Read(buf)
-		if err != nil {
-			return err
+		switch err {
+		case nil:
+			buf = buf[:n]
+			ctype = http.DetectContentType(buf)
+
+		case io.EOF:
+			// Empty file; this is typically due to things like
+			// npm webpack producing empty .css files.
+			ctype = "text/plain; charset=UTF-8"
+
+		default:
+			return fmt.Errorf("failed to read %s: %v", arg, err)
 		}
-		buf = buf[:n]
-		ctype := http.DetectContentType(buf)

 		// augmented rules for JS / CSS / etc.
 		switch {
@ -140,7 +150,7 @@ func filesFromListR(prefix, arg string, ftp packer.FilesToPack) error {
 			case ".css":
 				ctype = "text/css"
 			case ".js":
-				ctype = "application/javascript"
+				ctype = "text/javascript"
 			case ".json":
 				ctype = "application/json"
 			case ".svg":
--- a/cmd/packserver/go.mod
+++ b/cmd/packserver/go.mod
@ -1,8 +1,15 @@
 module src.lwithers.me.uk/go/htpack/cmd/packserver

-go 1.13
+go 1.22

 require (
-	github.com/spf13/cobra v0.0.5
-	src.lwithers.me.uk/go/htpack v1.1.5
+	github.com/spf13/cobra v1.8.1
+	src.lwithers.me.uk/go/htpack v1.3.3
+)
+
+require (
+	github.com/gogo/protobuf v1.3.2 // indirect
+	github.com/inconshreveable/mousetrap v1.1.0 // indirect
+	github.com/spf13/pflag v1.0.5 // indirect
+	golang.org/x/sys v0.22.0 // indirect
 )
--- a/cmd/packserver/go.sum
+++ b/cmd/packserver/go.sum
@ -1,42 +1,45 @@
-github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
-github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6/go.mod h1:grANhF5doyWs3UAsr3K4I6qtAmlQcZDesFNEHPZAzj8=
-github.com/coreos/etcd v3.3.10+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=
-github.com/coreos/go-etcd v2.0.0+incompatible/go.mod h1:Jez6KQU2B/sWsbdaef3ED8NzMklzPG4d5KIOhIy30Tk=
-github.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
-github.com/cpuguy83/go-md2man v1.0.10/go.mod h1:SmD6nW6nTyfqj6ABTjUi3V3JVMnlJmwcJI5acqYI6dE=
-github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
-github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
-github.com/gogo/protobuf v1.2.1 h1:/s5zKNz0uPFCZ5hddgPdo2TK2TVrUNMn0OOX8/aZMTE=
-github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4=
-github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ=
-github.com/inconshreveable/mousetrap v1.0.0 h1:Z8tu5sraLXCXIcARxBp/8cbvlwVa7Z1NHg9XEKhtSvM=
-github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
-github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q=
+github.com/cpuguy83/go-md2man/v2 v2.0.4/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
+github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
+github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
+github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
+github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
+github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
 github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
-github.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ=
-github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
-github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
-github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic=
-github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
-github.com/russross/blackfriday v1.5.2/go.mod h1:JO/DiYxRf+HjHt06OyowR9PTA263kcR/rfWxYHBV53g=
-github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ=
-github.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=
-github.com/spf13/cobra v0.0.5 h1:f0B+LkLX6DtmRH1isoNA9VTtNUK9K8xYd28JNNfOv/s=
-github.com/spf13/cobra v0.0.5/go.mod h1:3K3wKZymM7VvHMDS9+Akkh4K60UwM26emMESw8tLCHU=
-github.com/spf13/jwalterweatherman v1.0.0/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo=
-github.com/spf13/pflag v1.0.3 h1:zPAT6CGy6wXeQ7NtTnaTerfKOsV6V6F8agHXFiazDkg=
-github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
-github.com/spf13/viper v1.3.2/go.mod h1:ZiWeW+zYFKm7srdB9IoDzzZXaJaI5eL9QjNiN/DMA2s=
-github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
-github.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8/go.mod h1:VFNgLljTbGfSG7qAOspJ7OScBnGdDN/yBr0sguwnwf0=
-github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q=
-golang.org/x/crypto v0.0.0-20181203042331-505ab145d0a9/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
-golang.org/x/sys v0.0.0-20181205085412-a5c9d58dba9a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
-golang.org/x/sys v0.0.0-20200113162924-86b910548bc1 h1:gZpLHxUX5BdYLA08Lj4YCJNN/jk7KtquiArPoeX0WvA=
-golang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
+github.com/spf13/cobra v1.8.1 h1:e5/vxKd/rZsfSJMUX1agtjeTDf+qv1/JdBF8gg5k9ZM=
+github.com/spf13/cobra v1.8.1/go.mod h1:wHxEcudfqmLYa8iTfL+OuZPbBZkmvliBWKIezN3kD9Y=
+github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
+github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
+github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
+github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
+golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
+golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
+golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
+golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
+golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
+golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
+golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
+golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.22.0 h1:RI27ohtqKCnwULzJLqkv897zojh5/DwS/ENaMzUOaWI=
+golang.org/x/sys v0.22.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
 golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
-golang.org/x/tools v0.0.0-20180221164845-07fd8470d635/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
+golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
+golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
+golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
+golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
-gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
-src.lwithers.me.uk/go/htpack v1.1.5 h1:2JzgqLZ1ROYc53+96NezfJ3S9fkwHZNd6QgJhMXnlSE=
-src.lwithers.me.uk/go/htpack v1.1.5/go.mod h1:JWofpm01RJbCTIyKfIPftUsxk6KlFkrYwyHgCVdKY+s=
+gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+src.lwithers.me.uk/go/htpack v1.3.3 h1:Xvl6qR9HfSblmCgPyu+ACQ9o3aLQSIy3l8CrMbzj/jc=
+src.lwithers.me.uk/go/htpack v1.3.3/go.mod h1:qKgCBgZ6iiiuYOxZkYOPVpXLBzp6gXEd4A0ksxgR6Nk=
--- a/cmd/packserver/main.go
+++ b/cmd/packserver/main.go
@ -5,12 +5,18 @@ package main

 import (
 	"bufio"
+	"context"
 	"errors"
 	"fmt"
 	"net/http"
 	"os"
+	"os/signal"
 	"path/filepath"
+	"sort"
 	"strings"
+	"sync/atomic"
+	"syscall"
+	"time"

 	"github.com/spf13/cobra"
 	"src.lwithers.me.uk/go/htpack"
@ -49,6 +55,12 @@ func main() {
 		"Name of index file (index.html or similar)")
 	rootCmd.Flags().Duration("expiry", 0,
 		"Tell client how long it can cache data for; 0 means no caching")
+	rootCmd.Flags().String("fallback-404", "",
+		"Name of file to return if response would be 404 (spa.html or similar)")
+	rootCmd.Flags().String("frames", "sameorigin",
+		"Override X-Frame-Options header (can be sameorigin, deny, allow)")
+	rootCmd.Flags().Duration("graceful-shutdown-delay", 3*time.Second,
+		"Number of seconds to wait after receiving SIGTERM before initiating graceful shutdown")

 	if err := rootCmd.Execute(); err != nil {
 		fmt.Fprintln(os.Stderr, err)
@ -80,6 +92,23 @@ func run(c *cobra.Command, args []string) error {
 		certFile = keyFile
 	}

+	// parse frames header
+	framesHeader := "SAMEORIGIN"
+	frames, err := c.Flags().GetString("frames")
+	if err != nil {
+		return err
+	}
+	switch frames {
+	case "sameorigin":
+		framesHeader = "SAMEORIGIN"
+	case "allow":
+		framesHeader = ""
+	case "deny":
+		framesHeader = "DENY"
+	default:
+		return errors.New("--frames must be one of sameorigin, deny, allow")
+	}
+
 	// parse extra headers
 	extraHeaders := make(http.Header)
 	hdrs, err := c.Flags().GetStringSlice("header")
@ -124,6 +153,21 @@ func run(c *cobra.Command, args []string) error {
 		return err
 	}

+	// optional 404 fallback file
+	fallback404File, err := c.Flags().GetString("fallback-404")
+	if err != nil {
+		return err
+	}
+
+	// graceful shutdown delay must be > 0
+	gracefulShutdownDelay, err := c.Flags().GetDuration("graceful-shutdown-delay")
+	if err != nil {
+		return err
+	}
+	if gracefulShutdownDelay <= 0 {
+		return errors.New("graceful shutdown delay must be > 0s")
+	}
+
 	// verify .htpack specifications
 	if len(args) == 0 {
 		return errors.New("must specify one or more .htpack files")
@ -149,6 +193,7 @@ func run(c *cobra.Command, args []string) error {
 	}

 	// load packfiles, registering handlers as we go
+	router := &routerHandler{}
 	for prefix, packfile := range packPaths {
 		packHandler, err := htpack.New(packfile)
 		if err != nil {
@ -157,27 +202,65 @@ func run(c *cobra.Command, args []string) error {
 		if indexFile != "" {
 			packHandler.SetIndex(indexFile)
 		}
+		if err = packHandler.SetNotFound(fallback404File); err != nil {
+			return fmt.Errorf("%s: fallback-404 resource %q "+
+				"not found in packfile", prefix, fallback404File)
+		}
+		packHandler.SetHeader("X-Frame-Options", framesHeader)

-		handler := &addHeaders{
+		var handler http.Handler = &addHeaders{
 			extraHeaders: extraHeaders,
 			handler:      packHandler,
 		}

 		if prefix != "/" {
-			http.Handle(prefix+"/",
-				http.StripPrefix(prefix, handler))
-		} else {
-			http.Handle("/", handler)
+			handler = http.StripPrefix(prefix, handler)
 		}
+		router.AddRoute(prefix, handler)
 	}

+	// HTTP server object setup
+	sv := &http.Server{
+		Addr:    bindAddr,
+		Handler: router,
+	}
+
+	// register SIGINT, SIGTERM handler
+	sigch := make(chan os.Signal, 1)
+	signal.Notify(sigch, syscall.SIGINT, syscall.SIGTERM)
+	var (
+		// if we are shut down by a signal, then http.ListenAndServe()
+		// returns straight away, but we actually need to wait for
+		// Shutdown() to complete prior to returning / exiting
+		isSignalled atomic.Bool
+		signalDone  = make(chan struct{})
+	)
+	go func() {
+		<-sigch
+		time.Sleep(gracefulShutdownDelay)
+		isSignalled.Store(true)
+		shutctx, shutcancel := context.WithTimeout(context.Background(), gracefulShutdownDelay)
+		sv.Shutdown(shutctx)
+		shutcancel()
+		close(signalDone)
+	}()
+
 	// main server loop
 	if keyFile == "" {
-		err = http.ListenAndServe(bindAddr, nil)
+		err = sv.ListenAndServe()
 	} else {
-		err = http.ListenAndServeTLS(bindAddr, certFile, keyFile, nil)
+		err = sv.ListenAndServeTLS(certFile, keyFile)
 	}
-	if err != nil {
+
+	// if we were shut down by a signal, wait for Shutdown() to return
+	if isSignalled.Load() {
+		<-signalDone
+	}
+
+	switch err {
+	case nil, http.ErrServerClosed:
+		// OK
+	default:
 		fmt.Fprintln(os.Stderr, err)
 		os.Exit(1)
 	}
@ -228,3 +311,53 @@ func (ah *addHeaders) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 	}
 	ah.handler.ServeHTTP(w, r)
 }
+
+// routeEntry is used within routerHandler to map a specific prefix to a
+// specific handler.
+type routeEntry struct {
+	// prefix is a path prefix with trailing "/" such as "/foo/".
+	prefix string
+
+	// handler for the request if prefix matches.
+	handler http.Handler
+}
+
+// routerHandler holds a list of routes sorted by longest-prefix-first.
+type routerHandler struct {
+	// entries are the list of prefixes, with longest prefix strings first.
+	// The sorting ensures we can iterate through from the start and match
+	// "/dir/subdir/" in preference to just "/dir/".
+	entries []routeEntry
+}
+
+// AddRoute adds a new entry into the handler. It is not concurrency safe; the
+// handler should not be in use.
+func (rh *routerHandler) AddRoute(prefix string, handler http.Handler) {
+	if !strings.HasSuffix(prefix, "/") {
+		prefix += "/"
+	}
+	rh.entries = append(rh.entries, routeEntry{
+		prefix:  prefix,
+		handler: handler,
+	})
+	sort.Slice(rh.entries, func(i, j int) bool {
+		l1, l2 := len(rh.entries[i].prefix), len(rh.entries[j].prefix)
+		if l1 > l2 {
+			return true
+		}
+		if l1 == l2 {
+			return rh.entries[i].prefix < rh.entries[j].prefix
+		}
+		return false
+	})
+}
+
+func (rh *routerHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
+	for _, entry := range rh.entries {
+		if strings.HasPrefix(r.URL.Path, entry.prefix) {
+			entry.handler.ServeHTTP(w, r)
+			return
+		}
+	}
+	http.NotFound(w, r)
+}
--- a/go.mod
+++ b/go.mod
@ -1,8 +1,8 @@
 module src.lwithers.me.uk/go/htpack

 require (
-	github.com/gogo/protobuf v1.2.1
-	golang.org/x/sys v0.0.0-20200113162924-86b910548bc1
+	github.com/gogo/protobuf v1.3.2
+	golang.org/x/sys v0.22.0
 )

-go 1.13
+go 1.22
--- a/go.sum
+++ b/go.sum
@ -1,7 +1,33 @@
-github.com/gogo/protobuf v1.2.1 h1:/s5zKNz0uPFCZ5hddgPdo2TK2TVrUNMn0OOX8/aZMTE=
-github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4=
-github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q=
+github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
+github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
+github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
 github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
-golang.org/x/sys v0.0.0-20200113162924-86b910548bc1 h1:gZpLHxUX5BdYLA08Lj4YCJNN/jk7KtquiArPoeX0WvA=
-golang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/tools v0.0.0-20180221164845-07fd8470d635/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
+github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
+golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
+golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
+golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
+golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
+golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
+golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
+golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
+golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.22.0 h1:RI27ohtqKCnwULzJLqkv897zojh5/DwS/ENaMzUOaWI=
+golang.org/x/sys v0.22.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
+golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
+golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
+golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
+golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
+golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
+golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
--- a/handler.go
+++ b/handler.go
@ -2,14 +2,12 @@ package htpack

 import (
 	"fmt"
-	"net"
 	"net/http"
 	"os"
 	"path"
 	"path/filepath"
 	"strconv"
 	"strings"
-	"syscall"
 	"time"

 	"golang.org/x/sys/unix"
@ -21,14 +19,13 @@ const (
 	encodingBrotli = "br"
 )

-// TODO: logging
-
 // New returns a new handler. Standard security headers are set.
 func New(packfile string) (*Handler, error) {
 	f, err := os.Open(packfile)
 	if err != nil {
 		return nil, err
 	}
+	defer f.Close()

 	fi, err := f.Stat()
 	if err != nil {
@ -37,19 +34,16 @@ func New(packfile string) (*Handler, error) {
 	mapped, err := unix.Mmap(int(f.Fd()), 0, int(fi.Size()),
 		unix.PROT_READ, unix.MAP_SHARED)
 	if err != nil {
-		f.Close()
 		return nil, err
 	}

 	_, dir, err := packed.Load(f)
 	if err != nil {
 		unix.Munmap(mapped)
-		f.Close()
 		return nil, err
 	}

 	h := &Handler{
-		f:         f,
 		mapped:    mapped,
 		dir:       dir.Files,
 		headers:   make(map[string]string),
@ -57,7 +51,7 @@ func New(packfile string) (*Handler, error) {
 	}

 	// https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Frame-Options
-	h.SetHeader("X-Frame-Options", "sameorigin")
+	h.SetHeader("X-Frame-Options", "SAMEORIGIN")

 	// https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Content-Type-Options
 	h.SetHeader("X-Content-Type-Options", "nosniff")
@ -67,11 +61,11 @@ func New(packfile string) (*Handler, error) {

 // Handler implements http.Handler and allows options to be set.
 type Handler struct {
-	f         *os.File
 	mapped    []byte
 	dir       map[string]*packed.File
 	headers   map[string]string
 	startTime time.Time
+	notFound  *packed.File
 }

 // SetHeader allows a custom header to be set on HTTP responses. These are
@ -110,6 +104,28 @@ func (h *Handler) SetIndex(filename string) {
 	}
 }

+// SetNotFound allows overriding the returned resource when a request is made
+// for a resource that does not exist. The default behaviour would be to return
+// a standard HTTP 404 Not Found response; calling this function with an empty
+// string will restore that behaviour.
+//
+// This function will return an error if the named resource is not present in
+// the packfile.
+func (h *Handler) SetNotFound(notFound string) error {
+	if notFound == "" {
+		h.notFound = nil
+		return nil
+	}
+
+	notFound = path.Clean(notFound)
+	dir := h.dir[path.Clean(notFound)]
+	if dir == nil {
+		return fmt.Errorf("no such resource %q", notFound)
+	}
+	h.notFound = dir
+	return nil
+}
+
 // ServeHTTP handles requests for files. It supports GET and HEAD methods, with
 // anything else returning a 405. Exact path matches are required, else a 404 is
 // returned.
@ -130,8 +146,11 @@ func (h *Handler) ServeHTTP(w http.ResponseWriter, req *http.Request) {

 	info := h.dir[path.Clean(req.URL.Path)]
 	if info == nil {
-		http.NotFound(w, req)
-		return
+		if h.notFound == nil {
+			http.NotFound(w, req)
+			return
+		}
+		info = h.notFound
 	}

 	// set standard headers
@ -177,87 +196,7 @@ func (h *Handler) ServeHTTP(w http.ResponseWriter, req *http.Request) {
 	if req.Method == "HEAD" {
 		return
 	}
-	h.sendfile(w, data, offset, length)
-}

-func (h *Handler) sendfile(w http.ResponseWriter, data *packed.FileData,
-	offset, length uint64,
-) {
-	hj, ok := w.(http.Hijacker)
-	if !ok {
-		// fallback
-		h.copyfile(w, data, offset, length)
-		return
-	}
-
-	conn, buf, err := hj.Hijack()
-	if err != nil {
-		// fallback
-		h.copyfile(w, data, offset, length)
-		return
-	}
-
-	tcp, ok := conn.(*net.TCPConn)
-	if !ok {
-		// fallback
-		h.copyfile(w, data, offset, length)
-		return
-	}
-	defer tcp.Close()
-
-	rawsock, err := tcp.SyscallConn()
-	if err == nil {
-		err = buf.Flush()
-	}
-	if err != nil {
-		// error only returned if the underlying connection is broken,
-		// so there's no point calling sendfile
-		return
-	}
-
-	var breakErr error
-	off := int64(data.Offset + offset)
-	remain := length
-
-	for breakErr == nil && remain > 0 {
-		// sendfile(2) can send a maximum of 1GiB
-		var amt int
-		if remain > (1 << 30) {
-			amt = (1 << 30)
-		} else {
-			amt = int(remain)
-		}
-
-		// behaviour of control function:
-		//  · some bytes written: sets written > 0, returns true (breaks
-		//                        out of loop on first write)
-		//  · EAGAIN: returns false (causes Write() to loop until
-		//            success or permanent failure)
-		//  · other error: sets breakErr
-		var written int
-		rawsock.Write(func(outfd uintptr) bool {
-			written, err = unix.Sendfile(int(outfd), int(h.f.Fd()), &off, amt)
-			switch err {
-			case nil:
-				return true
-			case syscall.EAGAIN:
-				return false
-			default:
-				breakErr = err
-				return true
-			}
-		})
-
-		// we may have had a partial write, or file may have been > 1GiB
-		remain -= uint64(written)
-	}
-}
-
-// copyfile is a fallback handler that uses write(2) on our memory-mapped data
-// to push out the response.
-func (h *Handler) copyfile(w http.ResponseWriter, data *packed.FileData,
-	offset, length uint64,
-) {
 	offset += data.Offset
 	w.Write(h.mapped[offset : offset+length])
 }
--- a/packed/packed.pb.go
+++ b/packed/packed.pb.go
@ -15,11 +15,15 @@
 */
 package packed

-import proto "github.com/gogo/protobuf/proto"
-import fmt "fmt"
-import math "math"
+import (
+	fmt "fmt"

-import io "io"
+	proto "github.com/gogo/protobuf/proto"
+
+	math "math"
+
+	io "io"
+)

 // Reference imports to suppress errors if they are not otherwise used.
 var _ = proto.Marshal
Author	SHA1	Message	Date
Laurence Withers	19b2560e2d	cmd/htpacker: swap out progress bar library Switch to a simpler progress bar library. I've been having trouble tracking down a panic that seemed to affect the prior code; the new code is much easier and uses far fewer async channels and things.	2024-08-24 09:54:25 +01:00
Laurence Withers	3974db129e	cmd/packserver: updated modules	2024-07-18 14:58:38 +01:00
Laurence Withers	7dffaaa5d7	cmd/htpacker: updated modules	2024-07-18 14:57:49 +01:00
Laurence Withers	2a1eafa306	Updated modules	2024-07-18 14:54:50 +01:00
Laurence Withers	3585b7943a	Update JavaScript MIME type It seems that application/javascript has been supplanted by text/javascript. See https://www.rfc-editor.org/rfc/rfc9239 and https://www.iana.org/assignments/media-types/application/javascript .	2024-07-18 14:52:12 +01:00
Laurence Withers	6cbbe7328a	Correct some display issues with mpb update	2024-05-03 17:14:12 +01:00
Laurence Withers	a83aedd502	cmd/htpacker: update modules	2024-05-03 16:49:33 +01:00
Laurence Withers	8474cfbc5d	Update mpb lib to v8	2023-04-28 16:12:19 +01:00
Laurence Withers	2f842a21f3	cmd/htpacker: updated modules	2023-04-28 15:57:34 +01:00
Laurence Withers	439bf2422b	cmd/htpacker: don't try to compress tiny files If we have some really tiny files, it's not worth compressing them. Among other things, this will work around a bug in go-zopfli for 0- or 1-byte files.	2023-04-28 15:57:28 +01:00
Laurence Withers	2b280de481	Correct handling of multiple packfiles During the 1.3.1 update, a change was made to stop using http.Handle (and ServeMux underneath), in order to have manual control over the http.Server object. Unfortunately, it was overlooked that nothing was doing routing / multiplexing, so when using multiple packfiles only one handler (picked arbitrarily due to map) would actually be active on any given invocation. Correct this by adding an explicit handler. We don't use ServeMux so as to avoid bringing in any of its more complex behaviours like path cleaning etc.	2022-12-12 10:26:55 +00:00
Laurence Withers	5398dddb02	Add --graceful-shutdown-delay This option allows a configurable delay after receiving SIGTERM (or SIGINT) but before the HTTP server stops accepting new connections. It is quite useful for distributed systems where callers are only notified asynchronously (e.g. via service discovery) that a service is being shut down; it prevents the shut down from occurring prior to callers processing the notification. This required some minor refactoring to allow the Shutdown() method on http.Server to be accessed.	2022-11-26 10:43:18 +00:00
Laurence Withers	e0ae6bb4b6	cmd/packserver: update to Go 1.19, update dependencies	2022-11-26 10:14:25 +00:00
Laurence Withers	6b836895a0	cmd/htpacker: update to Go 1.19, update dependencies	2022-11-26 10:12:42 +00:00
Laurence Withers	301dc0c7c8	Update to Go 1.19	2022-11-26 10:08:16 +00:00
Laurence Withers	a6c2991781	cmd/htpacker: updated dependencies	2022-07-06 13:31:22 +01:00
Laurence Withers	d827d8aace	cmd/htpacker: add --content-type flag This allows overriding the content-type of files being packed using the ad-hoc method (not YAML spec).	2022-07-06 13:29:55 +01:00
Laurence Withers	565a269cef	cmd/packserver: add --frames option Allows override of the X-Frame-Options header on the handler.	2022-07-06 10:33:54 +01:00
Laurence Withers	16d836da9a	cmd/packserver: dependency update	2022-07-06 10:17:32 +01:00
Laurence Withers	8cae4d0f8f	Correct X-Frame-Options value sameorigin→SAMEORIGIN	2022-07-06 10:15:28 +01:00
Laurence Withers	6ea49bb3b3	Updated dependencies	2022-07-06 10:00:24 +01:00
Laurence Withers	83a5226e1a	handler: drop sendfile(2) support After some experimentation, I found that the sendfile(2) support did not really save any time compared to just write(2) from an already memory-mapped file. After some reading, I think open/sendfile is supposed to be slightly more efficient than open/mmap/write — but if we already did the mmap step, then it doesn't save us much. Moreover, the code to support sendfile(2) is a bit icky, and also forces us to close the HTTP connection after serving a file.	2022-07-06 09:53:34 +01:00
Laurence Withers	1b84160dcf	cmd/htpacker: cope with zero-length input files Sometimes we might be asked to serve up a zero-length input file, typically from some machine-generated CSS etc. We make some very rudimentary guess about the content-type the caller wanted and skip the mmap(2) call.	2020-04-02 12:33:04 +01:00
Laurence Withers	52213cf67e	cmd/htpacker: report filename on I/O error	2020-04-02 12:25:23 +01:00
Laurence Withers	f70914aa38	cmd/packserver: add --fallback-404 arg for Angular-style SPA support	2020-02-15 12:35:05 +00:00
Laurence Withers	ca54fb8fbb	Add doc link to README.md	2020-02-15 12:27:45 +00:00
Laurence Withers	d5b4fcf0be	Implement library support for Angular-style single page applications	2020-02-15 12:26:01 +00:00
Laurence Withers	f08165b0f1	Update README to remove reference to external brotli command Since we're now using github.com/andybalholm/brotli to natively pack, there's no need to refer to an external brotli process any more.	2020-02-15 11:23:42 +00:00
Laurence Withers	6f532296ef	Add CLI progress bars to the packer tool Since packing can be quite slow, it is nice to display progress to the caller. We use the excellent github.com/vbauerster/mpb library to do so, and add a bit of colour with github.com/logrusru/aurora. Finally, augment the inspector and packer with a summary printer that displays the overall file size/count, and compression ratio for each compression type.	2020-02-15 11:22:24 +00:00