Lately, I've been experimenting with using LLMs to find privacy-impacting bugs in differential privacy libraries. Some issues I've found in the process are particularly fun (for some definition of fun), so I thought I'd write up my favorite ones. Here's a first one.

Some DP tasks use thresholding: you only publish some statistics if sufficiently many people contributed to it (modulo some randomness). How many people are enough — the threshold value — depends on various parameters. One of these parameters is called \(\delta\), which is very roughly "how likely are you to publish data associated with a single person". We like \(\delta\) to be tiny, to avoid leaking information about single individuals. The smallest \(\delta\) is, the higher the threshold becomes.

Now, Google's DP library in Go computes the threshold using floating-point math. For some parameter values — in particular, when \(\delta\) is tiny — the threshold can get so large that it's represented as a positive infinity value. This should mean that nothing gets published. That would be sad, but safe.

The user count, and its noisy version, is an integer value. So to compare it to the floating-point threshold, the latter is first converted to an integer. This raises an interesting question: what happens when converting an infinity value to an integer value?

The first answer is bad. The Go spec answers simply says that the conversion must always succeed, but that the output of out-of-range values can be implementation-dependent. So the compiler is free to do whatever it wants, including different things on different CPU architectures.

The second answer is worse. On amd64, by far the most common architecture, the infinity value is converted to… -9223372036854775808, the smallest possible 64-bit integer. So the user count always ends up above this "threshold", everything gets published, and users get no privacy guarantee whatsoever. Oops.

This seems like absolutely bonkers behavior, and there is in fact a proposal to adopt a more reasonable approach in the Go specification. Hopefully it'll get adopted, and prevent future cursed bugs of the same kind!