diff --git a/content/daily/2025-06-08.md b/content/daily/2025-06-08.md new file mode 100644 index 0000000..0fa2aa2 --- /dev/null +++ b/content/daily/2025-06-08.md @@ -0,0 +1,88 @@ ++++ +title = "SSL troubles and pushing the server to git" +date = 2025-06-08 ++++ + +Last night, this website saw hours of downtime because of a combination of silly mistakes. Let me explain. + +### TLS Trouble + +This site runs with the [Ferron](https://www.ferronweb.org/) webserver. When you have a website, you need to have TLS certificates so users can use HTTPS to access it. Ferron has these really great configuration options to do this automatically. + +> [Nix](https://git.mtgmonkey.net/server-configuration.git/tree/services/ferron.nix), used to define Ferron's config.yaml +> +> ```nix +> -- ferron-conf.nix +> { +> global = { +> enableAutomaticTLS = true; +> useAutomaticTLSHTTPChallenge = true; +> }; +> } +> ``` + +These automatic TLS certificates are issued by [Let's Encrypt](https://letsencrypt.org), a fantastic nonprofit you should definitely go check out. They do, however, have pretty strict [rate limits](https://letsencrypt.org/docs/rate-limits/). Luckily, they offer a [staging feature](https://letsencrypt.org/docs/staging-environment) with much higher rate limits. + +I made a number of mistakes. The first was testing in production rather than on a local server. + +The second was not taking advantage of Let's Encrypt's staging features. Ferron even has an option `automaticTLSLetsEncryptProduction` that, when `false`, lets one use Let's Encrypt's staging features. Even though there is a [whole page](https://www.ferronweb.org/docs/automatic-tls), and despite knowing not to experiment in a production environment, I thought I would be fine and forged ahead. + +It was the seventh `systemctl restart ferron` which broke my sites. Suddenly, all I got was `ERR_SSL_PROTOCOL_ERR` from visiting *any* of my sites. There it was: Lesson learned. I will do my experiments locally going forward. + +A problem still remained, however. This blog is updated daily, and is built with a [flake](https://git.mtgmonkey.net/blog.git/tree/flake.nix). This makes it fully declarative; a boon! Ferron, however, was configured to point to `${blog.packages.x86_64-linux.default}/wwwroot`, which would be outdated until I `systeml restart`ed the server. Ferron requests a new certificate from Let's Encrypt, though, meaning I would hit the rate limit if I restarted even a couple of times too many. To fix this, my Ferron https server points to a second http server, which hosts the blog. + +> My much-overcomplicated ferron setup +> +> ```nix +> # ferron-conf.nix +> { +> global = { +> # enable automatic tls for https +> secure = true; +> enableAutomaticTLS = true; +> useAutomaticTLSHTTPChallenge = true; +> # let an https connection travel to the blog-ferron http server without error +> disableProxyCertificateVerification = true; +> # enable proxying to local servers (reverse proxying) +> loadModules = ["rproxy"]; +> }; +> hosts = [ +> { +> # route requests for the blog to port 8181 +> domain = "blog.mtgmonkey.net"; +> proxyTo = "http://localhost:8181/"; +> } +> ]; +> } +> ``` +> +> ```nix +> # blog-ferron-conf.nix +> # take the blog flake as an input +> { blog, ... }: { +> global = { +> # expose server to port 8181, so the main ferron server will proxy to it +> port = 8181; +> # the default flake output includes the static site at wwwroot +> wwwroot = "${blog.packages.x86_64-linux.default}/wwwroot"; +> }; +> } +> ``` + +This means I only need to `systemctl restart blog-ferron`, rather than ...`ferron`, meaning I don't trigger a new Let's Encrypt cert request every time. I can update my blog *however frequently I want* and *still* ensure the reproducibility of Nix. + +All code above is, clearly, just simplified snippets; the actual file are linked below +- [ferron.nix](https://git.mtgmonkey.net/server-configuration.git/tree/services/ferron.nix), where ferron-conf.nix is in the let binding +- [blog.nix](https://git.mtgmonkey.net/server-configuration.git/tree/services/blog.nix), where blog-ferron-conf.nix is in the let binding +- [flake.nix](https://git.mtgmonkey.net/server-configuration.git/tree/flake.nix),the flake to which the above modules are imported + + +### Secrets Embarrassment + +I spent all morning trying to configure different secrets management programs, from [agenix](https://github.com/ryantm/agenix), to [spos-nix](https://github.com/Mic92/sops-nix), to even a simple `.gitignore`. After finally getting agenix configured properly, it occured to me that my only 'secrets' were my ssh *public keys*, which are harmless to share! I don't need *any* secrets management, much less something as complex as agenix. + +### By Tomorrow + +- [ ] Polish up [the blog](blog.mtgmonkey.net) a little bit +- [ ] Get a [translation api](https://github.com/LibreTranslate/LibreTranslate) set up +- [ ] Take an ACT practice test