Luke Hannan

On Transparent Metagenomics

2026-07-01T00:00:00+00:00

A Note on Terminology</h2>
The terminology used to describe microbial communities is diverse and often subject to debate. Common proposed definitions are:

Microbiota</dt>
The assemblage of living microorganisms (bacteria, archaea, fungi, and viruses) present in a defined environment.</dd>
Metagenome</dt>
Strictly, the collection of genes and genomes from the members of a microbiota.</dd>
Microbiome</dt>
The entire ecosystem, encompassing the microbiota, their genomes, and the surrounding environmental conditions.</dd> </dl>
While these biological distinctions are important, in the literature (and within this thesis) the terms microbiome data and metagenomic data are frequently used interchangeably to refer to the high-throughput sequencing read count tables derived from these communities, regardless of whether the underlying biological material was the organisms themselves or their extracted genetic material.
Introduction</h2>
In an ideal world, every research project would be perfectly reproducible. That is, anyone should be able to take the detailed description of your methods and reproduce your results, either with your data or with their own data. Much has been written about reproducible research, and in fact the field of metascience</a> itself is primarily concerned with reproducibility. The concepts that I will describe here are not new. However, while the general ideas behind reproducible research are broadly the same in every field, transparent and reproducible reporting in metagenomics requires substantially more effort.

Some recommended reading on reproducibility in general:

The Turing Way: Guide for Reproducible Research</a></li>
Reproducible Data Analysis Workflows</a></li>
Ten Simple Rules for Reproducible Computational Research</a></li> </ul>
The first resource is a particularly thorough introduction to reproducible data science. </aside> 
In metascience, a distinction between reproducibility and replicability is often made. Reproducibility is whether other scientists (including future you) are able to reproduce your analysis as-is, with your given data and code. Reproducibility, therefore, is primarily a matter of good house-keeping. Replicability on the other hand is more of a scientific concern. Are others able to find results that match yours in study populations other than the original study population? Both reproducibility and replicability are important, but we’ll start with reproducibility given that without reproducibility your results cannot be replicable. Reproducibility is necessary but not sufficient for replicability.
On Sampling Metagenomic Data</h2>
While the majority of this post will deal with analysing and reporting on metagenomic data, it is important to recognise that metagenomic data is extracted from real biological samples. The physical act of sampling is inherently messy, and often the stage which can most drastically alter study conclusions based on the decisions made. The journey to reproducible research begins the moment you collect that sample, long before the data ever gets to a computer.
Documenting this step is critical and the Genomic Standards Consortium</a> has provided some guidance on the appropriate information to document, see Minimum Information about any (X) Sequence (MIxS) standard.</a> </aside> Consider the simple impact of storage, on two samples from the same individual. If sample `x</code> remains at room temperature on a lab bench-top for two hours while sample y</code> is immediately stored in a -80°C freezer, you may generate drastically different microbial compositions. Differential degradation rates, sample lysis, as well as a host of other factors can impact extraction and replication efficiency. No downstream bioinformatics pipeline, regardless of how sophisticated, can magically recover information that is lost prior to sequencing.`
`Bit-rot and the End of Things</h2> Before I get to suggestions for a transparent metagenomic analysis workflow I need to point out that reproducibility is only as good as your available documentation. If you have great documentation on your workflow, but absolutely nobody can find it then unfortunately your study cannot be reproduced. I have lost track of the number of times I have searched for detailed methods for a metagenomics manuscript only to be met with this:`Error 404 Page Not Found The page you are looking for might have been removed, had its name changed, or is temporarily unavailable. </div> Often the links to the supplementary data maintained by the journals themselves point to moved or unavailable resources. As an author, while this is not your fault, it is ultimately your responsibility. Therefore, rule number one of transparent metagenomics is: Rule 1 </aside> You must have a permanent link to your detailed methods documentation that is attached directly to the manuscript and always points to the current documentation for said analysis. </blockquote> Ideally this would be located in the same place that your final analysis scripts are located, and the link to both of those things is somewhat permanent. A good option for this is zenodo</a>. See zenodo’s github documentation</a> for information on how to do this. Zenodo, will not only provide a Digital Object Identifier (DOI)</a> that links directly to your chosen repository, but will archive the version of your repository at the time you choose to publish on zenodo. The exact data and code you used to generate your findings, will be permanently linked to a specific identifier that is not tied to account names (which may change) or proprietary software enterprises. GitHub is usually pretty good at tracking account name changes, but sometimes it can still lead to dead links. Now that we are in agreement about having permanent documentation, you may be asking what it is you should document. Pipeline Pathways</h2> At a minimum, the exact code you used for your analysis should be documented. However, metagenomic analyses typically involve complex data cleaning and preparation before any actual analysis. The decisions made in the data processing pipeline can drastically alter your final results. Every single one of the steps in the pipeline below has the potential to alter the final results if different choices are made. Much like Gelman’s garden of forking paths</a>, although in this instance it is more like a forest of forking paths. To complicate matters further, metagenomic analyses often depend on externally maintained resources for many steps in the QC pipeline. For example, taxonomic classification often makes use of reference databases like silva</a> or greengenes</a> — for 16S rRNA sequencing. Thankfully, all of these resources are typically versioned, and therefore you merely need to document the specific version of the resource that you used. See Salter, et al., 2014. Reagent and laboratory contamination can critically impact sequence-based microbiome analyses. BMC Biol 12.</a> </aside> For quality control and contaminant checking, it is not enough to just record the names and versions of software that you use. Ideally you should report both the code that you used to identify contaminants, and the final list of contaminants so that readers of your work can determine for themselves if your approach and its results are reasonable. Versions and the Beginning of Things</h2> It goes without saying, but I’ll say it anyway — the choice of analysis software will also drastically alter your results. See Nearing, et al., 2022. Microbiome differential abundance methods produce different results across 38 datasets. Nat Commun 13, 342.</a> </aside> Metagenomic software moves fast. Not only are there new software packages for metagenomic analysis almost weekly, but existing software updates constantly. The results obtained by a script written today, might not be reproduced if someone — including yourself — re-runs the script in six months time. In fact, many R and python users can attest to the fact that code written today might not even work six months in the future, and a typical metagenomic analysis script will depend on a large number of external software packages. An example of a very simple analysis metagenomic analysis script is shown below. I’m omitting the full script for brevity, but assume that data cleaning and contaminant removal has already been done for the following analysis script. </aside> library("TreeSummarizedExperiment") library("mia") library("breakaway") library("vegan") library("maaslin3") library("ANCOMBC") library("radEmu") # I've excluded non-metagenomic software for brevity, but each of those packages have their own defaults. root = rprojroot::is_rstudio_project tse_analysis = readRDS(root$find_file("data", "processed", "tse_analysis.rds")) # This is a typical command call output_ancom = ancombc2( data = tse_analysis, assay_name = "counts", fix_formula = "hiv_status + bmi + age + months_post_partum", p_adj_method = "holm", pseudo_sens = TRUE, group = "hiv_status", struc_zero = TRUE, neg_lb = TRUE, alpha = 0.05 )</code></pre> All of the external libraries used above, as well as the function calls (like ancombc2()</code>) have two things that make reproducibility hard, implicit versions and implicit defaults. The options shown in the ancombc2</code> are not all of the available options in ancombc2</code> and as such, any option that is not explicitly set will be set to the default value the function expects. See this vignette</a> for all the options you can set in ancombc2</code>. If the default values have changed since you last ran your script then your results will unfortunately also change. Which leads me to rule number two for transparent metagenomics: Rule 2 </aside> Everything should have a version number attached. </blockquote> Ideally one would document all of the default options of all of the software used, but for modern metagenomics analysis contexts, this is almost always not a reasonable nor productive use of time. I would like to keep this post as software agnostic as possible, and therefore won’t go into detailed description of all of the available software for recording software versions there are a few standouts that I can recommended. If you are working in R only, then the venerable renv</a> — via the renv.lock</code> file — will record the versions of all of the R packages you use and all of their dependencies. In this way, you can hand off an R project to someone else and they should, in theory, be able to download and install the exact versions of all of the packages you used 1</a>. rix</a> is a modern contender, that aims to not only snapshot your R packages, but also all of the required system dependencies for your analysis. rix</code> uses the nix</a> package manager and sources under the hood, but attempts to simplify interacting with nix</code> through rix</code>. Finally, if you find yourself working in more than one language often, then you might find pixi</a> to provide a more frictionless versioning environment that the previous two packages. A minimal pixi.toml</code> to specify the required versions of the dependencies for your project. pixi</code> will use this file to generate a pixi.lock</code> file that has the exact versions and locations of all of your dependencies. With the pixi.toml</code> and pixi.lock</code> files, any collaborator can just type pixi install</code> to reproduce the entire environment. Pixi has some other very helpful features that I recommended investigating, such as task running and shell invocation. </aside> [workspace] authors = ["Luke Hannan <luke@lukehannan.com>"] channels = ["conda-forge", "bioconda"] name = "microbiome" platforms = ["linux-64"] version = "0.1.0" [dependencies] fastexcel = ">=0.19.0,<0.20" polars = ">=1.40.1,<2" r = ">=4.5,<4.6" r-qiime2r = ">=0.99.20,<0.100" bioconductor-treesummarizedexperiment = ">=2.18.0,<3" bioconductor-decontam = ">=1.30.0,<2"</code></pre> </div> </div> The tools described above are both useful and incredibly powerful, however all of the tools mentioned far are mainly designed to version one part of the full lifecycle of a metagenomic analysis. Often, bioinformatics processing happens separately to data exploration and analysis and as such are versioned separately. Domain specific languages aim to address that challenge by allowing you to specify the entire data preparation and analysis pipeline in a DSL. Snakemake</a> and Nextflow</a> are the two most popular workflow managers in bioinformatics. The example below shows how snakemake</code> allows you to reproduce the entire pipeline figure</a> in a single easily versioned text file. Importantly, snakemake</code> describes every step of your analysis and the causal pathways between inputs and outputs. We’ll come back to causal pathways and Directed Acyclic Graphs in the reproducibility section. It is able to figure out that in order to use data/dehost/{sample}_R1.fastq.gz</code> as an input target for the taxonomic_profiling</code> rule, the remove_host</code> rule (and all of its upstream rules) must first be executed before taxonomic_profiling</code> can be executed. SAMPLES = ["sample_A", "sample_B"] rule all: input: expand("results/taxonomy/{sample}_report.tsv", sample=SAMPLES), expand("results/functional/{sample}_genefamilies.tsv", sample=SAMPLES) rule quality_control: input: r1 = "data/raw/{sample}_R1.fastq.gz", r2 = "data/raw/{sample}_R2.fastq.gz" output: r1 = "data/qc/{sample}_R1.fastq.gz", r2 = "data/qc/{sample}_R2.fastq.gz", report = "results/qc/{sample}.json" shell: "fastp --in1 {input.r1} --in2 {input.r2} " "--out1 {output.r1} --out2 {output.r2} " "--json {output.report}" rule remove_host: input: r1 = "data/qc/{sample}_R1.fastq.gz", r2 = "data/qc/{sample}_R2.fastq.gz" output: r1 = "data/dehost/{sample}_R1.fastq.gz", r2 = "data/dehost/{sample}_R2.fastq.gz" params: index = "resources/GRCh38" shell: "bowtie2 -x {params.index} " "-1 {input.r1} -2 {input.r2} " "--un-conc-gz data/dehost/{wildcards.sample}_R%.fastq.gz " "> /dev/null 2>&1" rule taxonomic_profiling: input: r1 = "data/dehost/{sample}_R1.fastq.gz", r2 = "data/dehost/{sample}_R2.fastq.gz" output: report = "results/taxonomy/{sample}_report.tsv" params: db = "resources/kraken2_db" shell: "kraken2 --db {params.db} " "--paired {input.r1} {input.r2} " "--report {output.report}" rule functional_annotation: input: r1 = "data/dehost/{sample}_R1.fastq.gz", r2 = "data/dehost/{sample}_R2.fastq.gz" output: genefamilies = "results/functional/{sample}_genefamilies.tsv" shell: "humann --input {input.r1} " "--output results/functional/"</code></pre> Once you have reached the end of your analysis, and you are ready for publication, there are two more things that you need to version control. Your analysis code itself, and your data. Without version controlled analysis code, and data all of the fancy dependency tracking mentioned above will be useless because your project will not be reproducible. Versioning of metagenomic data is as challenging, if not more so, than versioning analysis code given the common scale of metagenomic data. While it is a crucial aspect of reproducibility, it is beyond the scope of the current article. For a thorough overview of the challenges and best-practices, see The metagenomic data life-cycle: standards and best practices</a> and the EMBL-EBI training on submitting metagenomic data to ENA</a>. Finally, you also need to upload your analysis code. Git and GitHub are the defacto standards for uploading and storing your code. And as mentioned earlier, integrates well with zenodo</a>. There are many, many</a> tutorials</a> on using Git and GitHub, but if the aim is to just archive the final analysis code, using GitHub is the simplest option. Just click the green upload button in the interface shown below, and give your file upload a meaningful comment and you are done. Once the analysis has been published, the GitHub code can be permanently archived as described above</a>. </div> </div> As a statistician or analyst, I believe advanced knowledge of git to be incredibly useful, both for version control and understanding the software you depend on, but for purely archival purposes directly interfacing with the GitHub web-ui will be enough for many researchers. Using the methods described above is sufficient to satisfy rule three of transparent metagenomics: Rule 3 </aside> If it is mentioned in your manuscript, it must be online and accessible. </blockquote> This excellent post by the Meren lab</a> describes how paying lip-service to data availability means nothing for reproducibility if interested parties cannot actually access the data. In particular, it shows that while the sequencing data itself is important, without the associated metadata it is functionally useless for answering epidemiological questions, and is at best fun training data for bioinformaticians. Causes and Effects</h2> Reproducibility is necessary but not sufficient for replicability. Generating an exact in silico reproduction of an analysis is a good first step, but it cannot determine if the association you determined both exists in vivo and is causal. Therefore, in order to make sure your science is replicable there are two important considerations. Firstly, are your results internally replicable. If you performed your analysis for a second time — from the beginning — using different but equivalent tools would you still come to the same conclusions. There are many alternatives for all the common tools used in metagenomic analyses. Often choosing between alternatives is a matter of preference not performance. Does choosing ANCOM-BC2</a> alter your final conclusions, compared to if you had used radEmu</a>. If you re-run your k-means clustering algorithm with a different random seed do your results change entirely? In metagenomic studies, the answer to both of the questions above is: likely yes. In fact, the analysis method you choose will drastically alter your final conclusions. Nearing et al.</a> tested 14 different differential abundance algorithms, across 38 datasets, and found: The percentage of significant ASVs identified by each DA method varied widely across datasets, with means ranging from 3.8–32.5% and 0.8–40.5%, respectively. Interestingly, we found that many tools behaved differently between datasets. Specifically, some tools identified the most features in one dataset while identifying only an intermediate number in other datasets. </blockquote> Furthermore, both within and across tool differences in performance were altered depending on whether or not the datasets were filtered prior to analysis or not 2</a>. Unfortunately, it is not possible to accurately determine the true associations from false positive associations when the method you use is so sensitive to many possible differences in study design. If your aim is differential abundance analyses, current recommended practice is then to run a variety of differential abundance methods and report concordance and discordance in the results generated by these tools. Reporting your results in this manner is much more work, however you gain much for your extra work. Reporting in this way allows you to explicitly report on the confidence of the associations you have found, and importantly when others are trying to reproduce your results they are able to discern which ASVs should reproduce and which might not due to the strength of the evidence. This also allows you, the researcher, to be appropriately cautious in interpreting your findings, significant ASVs that do not reproduce across methods within your own study should be interpreted with caution. Particularly when working with sample sizes of typical metagenomic studies. The need for caution in interpretation is much greater when the aim is to evaluate the relationship between microbial relative abundance and health biomarkers 3</a>. This would lead to rule four for transparent metagenomics: Rule 4 </aside> Make sure that your analyses reproduce within your study, across methods. </blockquote> If you’ll indulge me one concluding footnote, in metagenomics as in all science the most important concerns of reproducibility are not technical they are scientific. The recent article by Mitchell et al.</a> illustrates many of the aspects discussed in this post. The title is: “Conceptual and methodological flaws undermine claims of a link between the gut microbiome and autism”. The fundamental issue it raises is not about software versions or dead hyperlinks, it is about whether the associations being reported are meaningful in the first place. In metagenomics, as in all observational science, association does not imply causation. Microbial relative abundances are compositional, Meaning every taxon’s apparent abundance is defined in relation to all others, a feature that makes standard statistical tests misleading without appropriate correction. and the associations we observe between community composition and health outcomes may reflect confounding, reverse causation, or simply artefacts of the garden of forking paths described above. Making genuine causal claims from metagenomic data requires either experimental intervention or very careful application of causal inference methods. Without that, a perfectly reproducible and internally replicable association study may still leave the most important scientific questions with unsatisfactory answers. That question, the one that motivates almost all human microbiome research, is whether the microbiome can be modified to improve health. Probiotics, dietary interventions, and faecal microbiota transplantation are all downstream attempts to answer it in practice. But the intervention literature can only be as trustworthy as the observational literature that motivates it. If our reported associations are not reproducible, not replicable across methods, and do not reflect genuine causal relationships, then the clinical promise of the microbiome will remain perpetually just out of reach. Transparent, reproducible science is a prerequisite for doing research that is actually useful. There are some caveats to this, see renv caveats</a>. ↩</a> </li> It will not come as a surprise to anyone, that both filtering methods and their general utility are hotly debated topics in metagenomics. See Waste not, want not</a> for a rough idea. ↩</a> </li> Identifying reproducible biomarkers for microbiome association studies requires thousands of sample sizes</a>. ↩</a> </li> </ol> </section>

On Reservoir Sampling

2026-06-14T00:00:00+00:00

I personally find that one of the most fun ways to learn a new programming language (particularly a systems programming language</a>) is to take a relatively simple program and see how much you can optimise it. Just for the hell of it. I like to call this: Perhaps in a later blog post I’ll expand on this further. 

In pursuit of meaningless minimalism. </blockquote>
We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3% - Donald Knuth Recently, with the goal of learning Rust</a> I converted all of my old Fish shell scripts to Rust binaries. This will come as no surprise to anyone but Rust is far more capable than Fish, and so while translating my shell scripts to Rust I decided to see if I could make them faster. One utility that I thought would be ripe for premature optimisation is my wallpaper picker. The task is simple, select a single random wallpaper from a given directory containing all of my wallpapers.
My naïve Fish implementation looked something like this (I omit the full script for brevity):
# Wallpaper directory set -l WALLPAPER_DIR "$HOME/onedrive/wallpapers/" # Randomly select a wallpaper from the specified directory set -l WALL (find "." $WALLPAPER_DIR \ --type file -e jpg -e jpeg -e png -e webp \ | sort -R \ | head -n 1)</code></pre> As you can see, the Fish implementation loads the full list of files into memory, sorts them randomly and picks the first one. Between shell invocation overhead, firing up three separate utilities, and piping text streams between them, we incur significant overhead. All of that, just to discard the entire list of files bar one. To guard against the hypothetical edge case where I have more images than there are atoms in the universe. For my Rust utility, I wanted to make sure that I never have to read the entire list of files into memory.
Enter Reservoir Sampling</a>. Reservoir sampling is a family of algorithms for choosing a simple random sample of k elements, without replacement, from a population of size n — without needing to know the population size (n). I know, I know, em-dashes are signs of AI use. You can take my em-dashes from my cold, dead hands. The naïve implementation is known as Algorithm R, created by Alan Waterman1</a>, and is conceptually relatively straightforward.
First select the first k elements of n, where k is the sample size you would like. Next initialise a new index to track the remainder of our elements: i. Then for each element in the list of remainders (after the first k elements have been selected), we generate a random number between 1 and i. If that random number is between 1 and k we replace the element in k at that position with our new number, otherwise we discard the number. Iterate over every element of the remainder, and at the end you will have your random sample of k elements. When k = 1, it is even simpler as you can just check if the random number generated is equal to 1 (or 0 in zero-indexed arrays). In Rust it would look something like this:
fn algorithm_r<I>(inputs: I) -> Option<PathBuf> where I: Iterator<Item = PathBuf>, { let mut chosen = None; for (i, item) in inputs.enumerate() { if fastrand::usize(..=i) == 0 { chosen = Some(item); } } chosen }</code></pre> Items at the front of the remainder have a higher chance of initially entering the sample than items towards the end, but they must survive a brutal gauntlet to make it to the end. While conceptually simple, Algorithm R has two major drawbacks with respect to the pursuit of meaningless minimalism:

You must evaluate every element of the remainder, one at a time.</li>
You must generate a random number for every element of the remainder.</li> </ul>
Generating random numbers (particularly true RNG) is computationally expensive, doing so when you are going to discard 99% of the results is even more so. Thankfully, there exists a more efficient algorithm for reservoir sampling.
Enter Algorithm L. First select the first k elements of n, where k is the sample size you would like, and initialize a floating-point variable W to track the reservoir’s threshold property. Instead of generating a random number for every single remaining element, we use a geometric distribution formula based on W to calculate a random number of elements to skip entirely. After we calculate the skip, we generate a random number between 1 and k and replace the element in k at that position with our new random number. Finally we update our reservoir threshold with another random number. Algorithm L generates 3 random numbers for each evaluated element, but 0 random numbers for each skipped element. Given that the majority of elements are skipped, this works out to far fewer random number generations than Algorithm R.
A standard implementation of Algorithm L in Rust looks something like this:

For k = 1. </aside> 

A bit of Rust pedantry here: looking at inputs.nth(skip)</code>, it might seem like we are cleanly skipping over unneeded elements. However, because our iterator is an unknown-length filesystem stream, nth()</code> is forced to call next()</code> under the hood until it reaches the target. We are saving ourselves from generating random numbers, but we are still evaluating every single directory entry in between.
If we were sampling from a data structure that supported O(1) random access, Algorithm L would be significantly faster. But converting the directory stream into an array would require loading all the file paths into memory first, which obviates the entire raison d’être of reservoir sampling. </aside>
fn algorithm_l<I>(inputs: I) -> Option<PathBuf> where I: Iterator<Item = PathBuf>, { let mut inputs = inputs; let mut chosen = None; if let Some(item) = inputs.next() { chosen = Some(item); } else { return None; } let mut w: f64 = fastrand::f64(); loop { let skip = (fastrand::f64().ln() / (1.0 - w).ln()) as usize; if let Some(item) = inputs.nth(skip) { chosen = Some(item); w *= fastrand::f64(); } else { break; } } chosen }</code></pre>
We have achieved peak meaningless minimalism. End of story, or so I thought. However, every good meaningless minimalist knows that peak minimalism is only achieved once the benchmarks say it has been achieved. Andrew Gallant, aka BurntSushi</a>, has perhaps the best post on benchmarking</a> I have ever read, and it should be required reading for anyone comparing anything with benchmarks. Using the Rust implementations described above, we can benchmark just the random wallpaper selection section of code using the criterion crate</a>.

The benchmarks shown here were calculated on my Ryzen 5950x in a TTY.
Notably, I used the list of my wallpapers stored in RAM as the test case. I currently have 51 wallpapers in my directory. </aside>

Algorithm L</div>
Metric</th> Lower Bound</th> Estimate</th> Upper Bound</th></tr></thead>

Slope</td> 63.625 ns</td> 63.685 ns</td> 63.753 ns</td></tr>
R²</td> 0.9538650</td> 0.9544209</td> 0.9537203</td></tr>
Mean</td> 63.666 ns</td> 63.713 ns</td> 63.762 ns</td></tr>
Std. Dev.</td> 465.83 ps</td> 547.00 ps</td> 621.83 ps</td></tr>
Median</td> 63.512 ns</td> 63.534 ns</td> 63.553 ns</td></tr>
MAD</td> 187.12 ps</td> 210.03 ps</td> 240.41 ps</td></tr> </tbody></table> </div>

Algorithm R</div>
Lower Bound</th> Estimate</th> Upper Bound</th></tr></thead>

48.057 ns</td> 48.085 ns</td> 48.116 ns</td></tr>
0.9829652</td> 0.9831714</td> 0.9829132</td></tr>
48.072 ns</td> 48.094 ns</td> 48.117 ns</td></tr>
217.27 ps</td> 259.18 ps</td> 300.39 ps</td></tr>
47.980 ns</td> 47.985 ns</td> 47.992 ns</td></tr>
41.994 ps</td> 51.568 ps</td> 64.680 ps</td></tr> </tbody></table> </div> </div>
As we can see, Algorithm R is faster than Algorithm L … Wait, What? The keen-eyed might already suspect why this is the case. There are two related issues leading to the surprising performance difference, operation cost and mechanical sympathy. Namely, Big O says that the time complexity for algorithm R is O(n) and for algorithm L it is O(k(1 + log(n/k))). However, this is only true if all primitive operations take the same length of time or number of CPU cycles — which does not happen on modern CPUs.
On modern CPUs algorithm R is (roughly) an integer increment, a pseudo-random integer generation, and a simple integer comparison. All in all on most modern CPUs this will be between 3 to 7 CPU cycles per iteration. Algorithm L is (roughly) a pseudo-random floating-point generation, two natural logarithms, a floating-point division and then a cast to usize</code>. Logarithms are transcendental functions. They cannot be executed in a single CPU cycle. Instead, they require complex microcode approximations or iterative hardware loops. All in all algorithm L will take roughly 60 to 120 CPU cycles per iteration — before we even get to things like pipelining and branch prediction. Of course there are fewer iterations in algorithm L, but not enough to outweigh the raw speed of simple instructions for this use-case. If k and n were both much, much larger algorithm L would snatch the crown, as the cost of floating-point operations is dwarfed by the number of elements algorithm L would allow us to skip relative to algorithm R. We pursued meaningless minimalism, and discovered that the real meaningless minimalism was the friends we made along the way, and that mechanical sympathy beats theoretical complexity every single time.
To tie a bow in the wallpaper selector saga, there are a couple of things we can do outside of the reservoir sampling algorithm to make the final utility faster. Use the fastrand</a> crate to generate random numbers, they are not true random numbers but are very fast to generate. Avoid allocations by making use of Rust’s iterators which are lazily evaluated. Filter unsupported image types to reduce the number of elements evaluated for reservoir sampling. Use only DirEntry</code> not PathBuf</code>, so no allocations are needed. Putting that all together, the final version of my random wallpaper utility looks something like this:
The core implementation of algorithm R is in the commented reservoir sampling block. A plus side of using Algorithm R is that it is much easier to understand, even if you know nothing about reservoir sampling. </aside> fn pick_random_wallpaper(dir: &Path) -> Result<Option<walkdir::DirEntry>, Box<dyn Error>> { let candidates = WalkDir::new(dir) .into_iter() .filter_map(Result::ok) // Filter using borrowed DirEntry to avoid early allocations .filter(|entry| entry.file_type().is_file() && is_supported_image(entry.path())); let mut chosen: Option<walkdir::DirEntry> = None; let mut rng = fastrand::Rng::new(); // Reservoir sampling for a single item for (i, entry) in candidates.enumerate() { if rng.usize(..=i) == 0 { chosen = Some(entry); } } Ok(chosen) }</code></pre>Appendix</h2> Now that you know for a wallpaper directory of 51 images, algorithm R is faster. The natural next question is: at what directory size does algorithm L dethrone algorithm R? It turns out that my current directory size was right at the inflection point, if I had slightly more images this post might never have happened. The figure below shows implementations for algorithm R and algorithm L optimised in the same way as the snippet above: If you have more than approximately 70 wallpapers in your folder, you have my explicit permission to use Algorithm L, it will be faster. Who discovered Algorithm R?</a> ↩</a> </li> </ol> </section> On Blogging 2026-05-29T00:00:00+00:00 Like many academics before me, I have spent years upholding a storied tradition: attempting to maintain a public-facing website, failing, and trying again. My initial foray into the world of personal academic websites was with the Hugo</a> static site generator using the Academic theme. I went looking for the original Academic theme I used, and I can’t find it anywhere. There are still some screenshots of the old theme on Google Images. My problem with Hugo was two-fold. Firstly, I didn’t understand how to really use Hugo, and I didn’t put in any effort to rectify that shortcoming. Secondly, I did not keep up-to-date with any of the developments of Hugo or the Academic theme that I was using. This meant that updates to Hugo — or the academic theme — frequently broke my website. I would apply the updated theme, and then Hugo would fail to build as I hadn’t set one of the new configuration options or additional settings. Could I have remedied the situation by spending some time learning Hugo and how the themes worked? Absolutely. However, at the time I wanted something that just worked ™, without having to spend any time or effort. History has shown that this is not a winning strategy in any endeavour. I would routinely have to spend hours getting my site up-to-date with all of the new changes before I could even think of writing a blog post. Given how infrequently I actually worked on my website, the ratio of fixing to writing was sub-optimal to say the least. Eventually, I got sick of that vicious cycle and gave up on writing my own blog posts. Instead, I switched to a purely static personal website that was made up of just a single index.html</code> with a very brief biography and some social media links. Without the CSS, my entire website could fit into a single jumbo frame</a>. </aside> Although the above site did not look particularly aesthetically pleasing, it had the distinct advantage of minimalism. No updates would cause it to not work, and editing or adding content was extremely easy. There was no framework updating or tinkering with settings. Perhaps a version of myself born in the 80s would be appreciative of such a website. The downside was that adding additional non-homepage content — like say for example a blog — was extremely challenging, unless your idea of a good-looking blog was a loose collection of roughly related html files in the same directory. Like three honey badgers</a> in a trench coat. I have recently been spending most of my free time writing Rust</a> and connecting it to R, through the excellent extendr</a> project. Both the extendr website, and the blog</a> are excellent examples of relatively lightweight but good looking websites. Both are built using the Zola</a> static site generator. Zola promises that it is A fast static site generator in a single binary with everything built-in. Forget dependencies. Everything you need in one binary.</q> After dealing with Hugo dependency hell, no dependencies sounded like music to my ears. Particularly, Zola felt like a perfect middle ground between the extreme minimalism of hand-writing your own html</code> and using a fully defined Byzantine theme. A Zola site is just a collection of files and folders, with a specific structure. ~/posts/blog-setup </div> . ├── config.toml ├── content │ ├── about │ │ └── _index.md │ ├── posts │ │ ├── hello_world │ │ │ └── index.md │ │ └── _index.md │ └── _index.md ├── sass │ └── main.scss ├── static │ └─ font │ └── custom_font.woff2 └── templates ├── base.html ├── index.html └── post.html</code></pre> </div> </div> See getting started</a> for a more official introduction to Zola. </aside> What attracted me to this layout is how fiercely it aligns with my preference for ownership and predictability. Zola doesn’t abstract the machinery of a website away behind magic black boxes or deeply nested theme configurations. It offers a transparent, flat structure that says exactly what it does on the tin. If a page renders incorrectly, I don’t have to hunt through upstream GitHub theme repositories to find a breaking change; the issue can only be directly inside my own templates/</code> directory. It forces a certain level of intentionality that my previous Hugo setup lacked. When you run zola build</code>, Zola parses your Markdown files and automatically injects the text into the corresponding HTML templates. What this means is that you only have to write your core template logic once. From then on, creating a new post is straightforward, just add a new Markdown file into a folder and zola build</code>. The source for my website can be found here https://github.com/luhann/luhann.github.io</a> as a more detailed demonstration. What this gives me is a website that requires no external themes or dependencies — so cannot break like my previous Hugo site — but the flexibility of editing raw html</code> for each page of my site if I so desire. Of course, I am being entirely hypocritical here. My main problems from Hugo stemmed from using external themes, yet I tout the advantages of Zola by creating my own theme from scratch. Of course it will never break. If I had used an external Zola theme I might eventually run into the same problems as using the Hugo Academic theme. Conversely, if I had created my own Hugo theme from scratch it is unlikely I would have had any of the issues I did. What then does this mean for blogging and static site generation? Regardless of the tool you pick, spending some time upfront to learn the ins-and-outs of the tool is likely going to lead to a much less frustrating workflow in the long run. Or perhaps the true message is that one should do all they can to limit external dependencies</a> wherever possible. Hello World 2026-05-21T00:00:00+00:00 Some recent examples: In Defense of YAML</a></li> Writing Python like it’s Rust</a></li> Type safe(r) R code</a></li> Apache Arrow, Rust, and cross-langauge data science</a></li> </ul> and perhaps more controversially: GPL is holding R back</a></li> </ul> </aside> These days I spend much of my time reading excellent blog posts about making data science and R faster (typically using Rust). I have learnt a great deal from those posts, not just about computer science, but also about the value of writing. Both for your readers and for yourself as well. Writing forces can sometimes be laborious, but it forces you to convert your understanding of a thing into something that should (hopefully) be intelligible to your target audience. Given the above, I’ve decided to attempt to get more practice at both understanding and explaining the tools and methods I use regularly. I’ll mainly write for applied statisticians and data scientists working in R-heavy workflows. Expect notes on biostatistics, clinical-trial thinking, and computational methods for microbiome and related omics data. If any of the above sounds interesting to you, you’re welcome to follow along or to reach out to me: Elsewhere</h2>