Forums
Karen was maintaining some specification tests that were flaky. Not extremely flaky, but three or four times out of a thousand, the tests would just fail. The tests were complicated, and some of the operations were timing sensitive, so it wasn't precisely surprising- but the problem was that they were actually generous with their timing windows. The unit tests passed consistently, it was only these functional, specification-based tests that failed.
So, for example, there were sections in the tests where they wanted to wait at least 2ms. Since the code and tests were in TypeScript, they used the setTimeout function, which per standard JavaScript documentation warns that it may wait longer. But again, Karen was fine with longer.
Unfortunately for Karen, the documentation for NodeJS is less specific, as it makes no guarantees about when the timeout function gets invoked. This means that it can fire the timeout before the time has elapsed.
After many, many hours of debugging, that was exactly the situation that Karen found herself in. Which is why her very simple wait function went from:
export const wait = (ms:number) => new Promise((complete) => setTimeout(complete, ms));
To the much more awkward:
export const wait = (ms: number) {
const target = performance.now() + ms;
return new Promise((complete) => {
const checkReady = () => {
if (performance.now() > target) {
complete();
} else {
setTimeout(checkReady, 1);
}
}
setTimeout(checkReady, 1);
});
}
This version of the function checks the time every millisecond, and only completes the operation if we've waited at least as long as our target duration. This ensures that the timeout never fires too soon and it fixes the janky tests. But it's also terrible. Terrible that it exists. Terrible that this is the best solution. Terrible that our functional tests need to be so time sensitive. And terrible that the Node runtime actually breaks the one consistent scheduling guarantee that pretty much every other scheduler does: that it'll wait at least as long as you asked, but might wait much longer.
At best, we can say, "at least it's only testing code."