Testing

The JUnit suite and what it covers, the in-game smoke tests and how to read their reports, the dev tools inside the game, the checks over generated files, and what CI runs on a merge request.

This page is about knowing a change works before anyone plays it. Vantage has three layers of checks: a JUnit suite that runs in seconds without starting the game, in-game harnesses that boot a real client and write their verdict to a file, and Python tools that confirm the generated files still match the source. There is also a set of developer commands inside the game. It is written for anyone sending a change back or keeping a fork healthy.

The fast suite

./gradlew compileJava test                                  # the default Minecraft version
./gradlew compileJava test -Pminecraft_version=26.1.2       # the other one

Neither task produces a jar, so neither touches mod_version. A run ends with one summary line, printed whether it passed or not, so a suite that silently found nothing is visible too:

Vantage tests: <n> run, <n> failed, <n> skipped

A failing test prints its full stack trace and causes to the console. You do not need to open build/reports/tests to find out why.

What it covers

The suite lives in src/test and uses JUnit 5. It covers the parts of the mod written deliberately free of Minecraft types, which is why it can run without a client. As of 2.0.1:

The Stats tests also own the golden fixtures under src/test/resources/stats/golden/. The service's own test suite ingests the same bytes, so a change on either side that breaks the contract fails a test on both.

A test that never boots the game

Nothing in the suite starts the mod, and no mixin applies. Code that needs to know where the player is takes a SkyblockState, and tests hand it testing/FakeSkyblockState: a state whose answers are public fields. It starts on a SkyBlock island, outside any lobby, with a sidebar and a profile, and each test changes only what it is about:

private final FakeSkyblockState state = new FakeSkyblockState();

@Test
void afterADisconnectNothingIsRecordedUntilTheProfileIsReadBack() {
    state.profileName = null;
    assertFalse(ProfileGate.recording(state));
    state.profileName = "Pear";
    assertTrue(ProfileGate.recording(state));
}

That excerpt is from core/stats/record/ProfileGateTest.java, shortened.

When you need real items

A few tests need a real ItemStack, which cannot be built until the game's registries are loaded and each item's components are bound. testing/GameRegistries.boot() does exactly that with vanilla's own bootstrap, once per test JVM, and GameRegistries.skyblockItem(base, id, name, lore…) builds an item the way Hypixel sends one.

Properties for the Stats fixtures

Gradle runs tests in their own JVM and does not pass -D options through to it. The test task therefore forwards five properties by name, the ones that drive the Stats fixture emitter and its demo generator: vantage.writeFixtures, vantage.demoOut, vantage.demoSeed, vantage.demoCategories and vantage.demoMissingSections. python tools/stats_golden.py --write is the usual way to use them.

In-game harnesses

Some failures only exist in a running client: a screen that throws while it renders, a shader that does not compile, a chunk read on the wrong thread. Two Loom run configurations boot a real client, do one job, write a verdict file and quit. Both sign in as an offline player named Dev.

The UI smoke test

./gradlew runUiSmokeTest

It starts the client at 1920×1080 with -Dvantage.uiSmokeTest=true. Once the title screen is up and the loading overlay has faded, it walks a scripted list of Vantage screens: the menu in several states and two presets, the widget gallery, the HUD editor, feature pages, the Stats screens, the setup wizard, About, the bug report screen and more. Each one renders for 40 ticks and is saved as run/screenshots/vantage-<step>.png. Along the way it runs the in-game self-test and checks that the custom shaders compile. Then it writes its verdict and quits.

The verdict is run/screenshots/smoke-report.txt. Its first line is OK - 0 failure(s) or FAILED - <n> failure(s), followed by one line per failure. As of 2.0.1, a healthy run reports exactly one:

FAILED - 1 failure(s)
 - selftest: ravengard stack adapter: SKIPPED (item components unbound - runs on your first world join)

That line is expected. The Ravengard fixture check needs item components, which bind only when a world loads, and the smoke test never joins one. Any line beyond it is a real failure.

Look at the pictures as well. The menu-lit and gallery-lit frames are taken with the cursor parked over the content, so they show the pointer's light; menu-ember shows a second preset. If the sdf_shape shader fails to compile, the interface falls back to flat geometry and every screenshot still "renders"; the harness asks the pipeline directly and fails the run instead.

The pathfinding smoke test

./gradlew runPathSmokeTest

The pathfinder's rules are unit-tested against hand-built worlds, but those tests stop where Minecraft starts. This harness opens a fresh flat world at 1280×720, builds a row of walled lanes with one obstacle each (a carpeted low corridor, a door, a fence, stairs, a ladder, water, a pit), routes through every one, and writes the verdicts to run/screenshots/path-smoke-report.txt beside a screenshot per lane. Each lane is one block wide with walls four high, so a route cannot fake "passable" by walking round.

Playing from source

./gradlew runClient starts an ordinary development client as Dev. It has no real Minecraft session, so Mojang's joinServer answers it with a 401: anything that needs a proven account cannot work there. Vantage Stats has a development sign-in for exactly this case, which The service describes.

Dev tools in the game

Every build registers /vantage dev, also /vt dev. Its overlays are off until you turn them on.

The self-test checks the formatters, text parsing, SkyBlock time, the colour, maths, regex and JSON helpers, the tab widgets, the Ravengard fixtures, the Bazaar execution engine, the feature taxonomy, the Director and Vet. The taxonomy check is where a group with more than twelve features, or a feature with no tags, is caught. It runs here and inside the UI smoke test; it is not a Gradle step.

The player-facing commands are listed in Commands; the developer rows are left out there on purpose.

Checks the tools run

Several files in the repository are generated from the source, and a tool checks each one is current. All of them are plain Python 3 with no packages to install.

Each regenerates its file when run without --check. mixin_targets.py also needs unzip and javap on your PATH, and a build of each version first, so that Loom has downloaded the game jar it reads.

What CI runs

The pipeline runs for merge requests, for the default branch and for tags. A branch with no merge request open runs nothing. The Gradle jobs run in the eclipse-temurin:25-jdk-jammy image and the Python jobs in python:3.12-slim.

Before you ask anyone to merge

This is what "verified" means for a change to Vantage:

./gradlew compileJava test
./gradlew compileJava test -Pminecraft_version=26.1.2
./gradlew runUiSmokeTest                  # then read run/screenshots/smoke-report.txt
python tools/tracker.py --check
python tools/sitedocs.py --check

Add python tools/mixin_targets.py if you touched a mixin, and the two Stats tools if you touched Vantage Stats. Then write down what you ran and what it said; Contributing shows where that goes.

Related

• Building: the JDK and the Gradle tasks these commands assume.
• Multi-version: why every check runs once per Minecraft version.
• The UI toolkit: what the smoke test's screenshots should look like.
• Contributing: how a verified change is sent back.