Oh Snap! Easy Snapshot Testing for Zig

It's hard to know if a program, or part of one, actually works. But it's easy to know if it doesn't: if there isn't a test for some part of the program, then that part doesn't work.

Snapshot testing is a great way to get fast coverage for data invariants in a program or library. The article I just linked to goes into great detail about the advantages of snapshot testing, and you should read it.

ohsnap is a Zig library for doing snapshot testing, one which is, in fact, based on the TigerBeetle library used in that post.

It includes some features not found in the original. TigerBeetle has a no-dependencies policy, and I'm confident that what they have serves their needs just fine. But a library like this is a dependency by definition, and I didn't mind adding a couple more.

Let me show you its features!

Installation

The best way to use ohsnap is to install it using the Zig Build System. From your project repo root, use zig fetch like this:

zig fetch --save "https://github.com/mnemnion/ohsnap/archive/refs/tags/v0.3.1.tar.gz"

Then add it to your test artifact like so:

    if (b.lazyDependency("ohsnap", .{
        .target = target,
        .optimize = optimize,
    })) |ohsnap_dep| {
        lib_unit_tests.root_module.addImport("ohsnap", ohsnap_dep.module("ohsnap"));
    }

That should be it! Now you're ready to write some snaps!

Using ohsnap

The interface will be familiar if you read the linked blog post, which, really, you should.

One difference between ohsnap and the original, is that ohsnap includes pretty, a clever pretty-printer for arbitrary data structures. So you don't need to write a custom .format method to use ohsnap, although if you have one, you can use that instead. Or both. Belt and suspenders kinda thing.

Writing a snap is simple, to get started, do something like this:

const OhSnap = @import("ohsnap");

test "snap something" {
    const oh = OhSnap{};
    // You can configure `pretty` by using `var oh` and changing settings
    // in `oh.pretty_options`.
    const snap_me = someFn();
    try oh.snap(@src(),
        \\
        ,
    ).expectEqual(snap_me);
}

Note that the call to @src() has to be directly above the string, and the string has to be multi-line style, with the double backslashes: \\. Both this:

try oh.snap(@src(),
    \\ etc
    ,).expectEqual(snap_me);

And this:

try oh.snap(
    @src(),
    \\ etc
  ,).expectEqual(snap_me);

Will work just fine.

This test will fail, because the snapshot generated by pretty won't be equal to the empty string. ohsnap will diff that empty string with what it gets out of snap_me, and print what it got in all-green, because that's what happens when you diff an empty string against a string which isn't empty.

If you like what you see, updating is simple. Change the file to the following:

const OhSnap = @import("ohsnap");

test "snap something" {
    const oh = OhSnap{};
    // You can configure `pretty` by using `var oh` and changing settings
    // in `oh.pretty_options`.
    const snap_me = someFn();
    try oh.snap(@src(),
    \\<!update>
    ,
    ).expectEqual(snap_me);
}

The snaptest will see the <!update>, which must be the beginning of the string, and replace it in your file with the output of the pretty printing. Easy!

If your data structure has a .format method, and you'd prefer to use that as a basis, simply use .expectEqualFmt instead of .expectEqual.

If, down the road, the snapshot doesn't compare to the expected string, ohsnap will use diffz¹, a Zig port of diff-match-patch, to produce a terminal-colored character-level diff of the expected string with the actual string, making it easy to see exactly what's changed. These changes are either a bug, or a new feature. If it's the former, fix it, if it's the latter, just add <!update> to the head of the string again, and ohsnap will oblige.

Pattern-Matching Snapshots

This is fine and dandy, if the data structure, exactly as it prints, will always be the same on every test run. But what if that's only true of some of the data?

Consider this example. We have a struct which looks like this:

const StampedStruct = struct {
    message: []const u8,
    tag: u64,
    timestamp: isize,
    pub fn init(msg: []const u8, tag: u64) StampedStruct {
        return StampedStruct{
            .message = msg,
            .tag = tag,
            .timestamp = std.time.timestamp(),
        };
    }
};

Which we want to snapshot test, like this:

test "snap with timestamp" {
    const oh = OhSnap{};
    const with_stamp = StampedStruct.init(
        "frobnicate the turbo-encabulator",
        31337,
    );
    try oh.snap(
        @src(),
        \\ohsnap.StampedStruct
        \\  .message: []const u8
        \\    "frobnicate the turbo-encabulator"
        \\  .tag: u64 = 31337
        \\  .timestamp: isize = 1721501316
        ,
    ).expectEqual(with_stamp);
}

But of course, the next time we run the test, the timestamp will be different, so the test will fail. We care about the message and the tag, we care that there is a timestamp, but we don't care what the timestamp is, because we know it will be changing.

For cases like this, ohsnap includes mvzr, the Minimum Viable Zig Regex library, which I wrote specifically for this purpose.

Simply replace the timestamp like so:

    try oh.snap(
        @src(),
        \\ohsnap.StampedStruct
        \\  .message: []const u8
        \\    "frobnicate the turbo-encabulator"
        \\  .tag: u64 = 31337
        \\  .timestamp: isize = <^\d+$>
        ,
    ).expectEqual(with_stamp);

Through the magic of diffing, ohsnap will identify the part of the new string which matches <^\d+$>, and try to match the regular expression against that part of the string. Since this matches, the test now passes.

Note that the regex must be in the form <^.+?$> (the exact regex we use is <\^[^\n]+?\$>, in fact), the ^ and $ are essential and are load-bearing parts of the expression. This prevents partial matches, as well as making the regex portions of a snapshot test easier for ohsnap to find. Note that because this is a multi-line string, you don't have to do double-backslashes: its <^\d+$>, not <^\\d+$>. To be very clear, the < and > demarcate the regex, they aren't part of it.

Let's say you make a change:

    const with_stamp = StampedStruct.init(
        "thoroughly frobnicate the encabulator",
        31337,
    );

The test will now fail: the word "thoroughly" will be highlighted in green, turbo- will be marked in red, and the timestamp will be cyan, indicating that the regex is still matching the pattern string. If a change in the test data means that the regex no longer matches, then the part of the test string which should match is highlighted in magenta.

Since this was an intentional change, we need to update the snap:

    try oh.snap(
        @src(),
        \\<!update>
        \\ohsnap.StampedStruct
        \\  .message: []const u8
        \\    "frobnicate the turbo-encabulator"
        \\  .tag: u64 = 31337
        \\  .timestamp: isize = <^\d+$>
        ,
    ).expectEqual(with_stamp);

Once again, through the magic of diffing, ohsnap will locate the regexen in the old string, and patch them over the new one.

    try oh.snap(
        @src(),
        \\ohsnap.StampedStruct
        \\  .message: []const u8
        \\    "thoroughly frobnicate the encabulator"
        \\  .tag: u64 = 31337
        \\  .timestamp: isize = <^\d+$>
        ,
    ).expectEqual(with_stamp);

Voila!

Usage note: in some cases, the changes to the new string will displace the regex, you can tell because some part of the regex itself will be exposed in red. When that happens, the update may not apply correctly either: the regex will always be moved to the new string intact, but it may or may not be in the correct place (usually, not). This can generally be fixed by making changes to the expected-value string until whatever part of the regex was sticking out of the diff is no longer exposed. Sometimes running <!update> twice will fix it as well.

Developing With Snapshots

When we're programming, there are always points in the process where a data structure is in flux, and ohsnap can help you out with that as well. Instead of .expectEqual(var), use .show(var), or .showFmt(var). This will print the snapshot, whether it diffs or not, and it doesn't count as a test. <!update> continues to work in the same way, but an updated .show snapshot counts as a failed test. The update logic is fairly simple, and updating often changes the line numbering of the file, so this helps update one at a time. Note that you can add the <!update> string to any number of snapshots, and just keep recompiling the test suite until they all pass. Also, if ohsnap can't find the snapshot because it moved, nothing untoward will happen, it will just report a failed test, and running it again will fix the problem if it was caused by a previous update.

This also works as a minimalist way to regress a snapshot test, when you aren't sure what the final value will be.

Whenever you're satisfied with the output, just change .show to its .expect cousin, and now you've got a test.

That's It!

One of the great advantages of snapshot testing is that it's easy, so ohsnap, like the library it's based upon, is intentionally quite simple. Simple, yet versatile, the latter to a large degree is owed to pretty, which can handle anything I've thrown at it, types, unions, you name it.

It's a new library, but I expect the core interface to remain stable. It's meant to do one thing, well, and otherwise stay out of the way. I'm willing to consider suggestions for ways to make ohsnap better at what it already does, however.

That said, the regex library mvzr is pretty new, and so is the added code in diffz, so version-bumps to fix any bugs in those can be expected over time. The build system doesn't currently do update checks, so you'll need to check for updates manually, for now.

I hope you enjoy it! Test early, test often, and do it the easy way.

Footnotes

1. The link is to a fork of the library which has the necessary changes for terminal printing. That branch is in code review, and these things take time. ohsnap will be updated to fetch from the main repo when that's possible. ↩