Testing¶
make test # go test ./...
make test-race # go test -race -shuffle=on ./...
make lint # go vet + golangci-lint (v2, configuration in .golangci.yml)
make cover # coverage summary
make fuzz # every fuzz target, FUZZTIME each (default 30s)
make bench # benchmarks of Transform and Reverse on the example
CI runs lint, the race tests, a build, a conversion of the example in both
directions, a ten-second fuzz smoke of every target and govulncheck on
every push and pull request, and it builds the containerized environment and
runs the race tests and lint inside it.
In a container¶
environment/ provides the same targets inside a Docker image with the
toolchain CI uses (see the folder's README):
make -C environment build # one-time image build
make -C environment test ENV=dev # go test ./..., with Go's caches
make -C environment check ENV=test # race tests and lint, no test cache
Layers¶
| Tests | What they pin |
|---|---|
internal/jsondoc/jsondoc_test.go |
decoding with exact digits, encoding (and what cannot be encoded), deep copies, structural equality, Go and JSON numbers, non-finite text, number formatting at the edges, type names |
internal/jsonpath/path_test.go |
parsing of every path form and its error messages, the String() round trip, key template matching and rendering, matching with bound and unbound variables (leading zeros, out-of-range and non-integer bindings, errors inside iterations), concrete locations, reads and writes through holes and wrong containers, compaction |
internal/convert/convert_test.go |
every converter forward and back (each case is run both ways), argument and type validation, absent inputs, chains and their error propagation, zone rendering, non-finite text, structural lookups |
internal/mapping/compile_test.go |
a valid configuration with every construct, unusual but valid shapes, bind ordering and join detection, and one case per validation error with the message and the rule position it must produce |
internal/mapping/run_test.go |
table cases for copy rules, constants and templates, arrays and keyed objects, each in iterate and join mode, nested scopes, every condition operator, defaults, roots and nulls, and every runtime error path with its rule position and direction; each case checks forward and reverse |
pkg/t1k/t1k_test.go, pkg/t1k/config_test.go |
the public API, golden files for the default mapping, the round-trip property, the concurrency guarantee, error classification |
pkg/t1k/calliope_test.go |
the regression test for issue #1: every generated key and reference satisfies Calliope's identifier syntax, every reference resolves, the reverse restores EnerPlanET's ids, and an id the prefix cannot repair fails with the rule named |
pkg/t1k/mapping_test.go |
the default mapping's behaviour for every technology kind, payload edge cases (no topology, standalone buildings, zero demand and rating, resolutions, infinity spellings), values it must reject, the reverse of jobs T1K did not produce, the round trip over a payload with every kind, and degenerate inputs |
config/config_test.go |
the default mapping is embedded and is JSON |
cmd/t1k/main_test.go |
flags, files, standard streams, exit codes and error messages |
Fuzzing¶
Every parser and the transformation itself have a fuzz target next to their
tests: FuzzParse and FuzzParseTemplate in jsonpath (no expression
panics, accepted ones round-trip through String(), rendered keys match
their template), FuzzParse in convert (no spec panics, parsed converters
survive representative inputs in both directions) and FuzzAffix (the
affix round trip holds for arbitrary prefixes, suffixes and values), FuzzCompile and
FuzzRun in mapping (no configuration or document panics, every failure
wraps ErrConfig or ErrRule) and FuzzTransform in pkg/t1k (the
default mapping never panics on any input in either direction). go test
runs their seed corpora; make fuzz explores for FUZZTIME per target and
CI runs a short smoke. A crasher lands in testdata/fuzz/ of the package;
commit it as a regression case once fixed.
Golden files¶
pkg/t1k/testdata/meme-job.golden.json and
pkg/t1k/testdata/enerplanet-reverse.golden.json are the outputs of the default mapping for examples/enerplanet-calculation.json
in both directions. After a deliberate change to the mapping:
Review every changed line; the diff is the change's effect on the contract.
The round-trip property¶
TestDefaultMappingRoundTrip converts the example forward, back and forward
again and requires the second job to equal the first apart from the building
without a connection, which the reverse cannot place (see
the mapping page).
It then reverses the second job and requires that output to equal the first
reverse output, so the transformation is stable from the first round trip
on. A new lossy rule shows up here immediately.
Checking against the consumers¶
MEME's decoder rejects unknown fields and its validators check the model, so
the surest test of a mapping change is MEME itself. In a clone of the MEME
repository, a small program in cmd/ can decode a job with
json.Decoder.DisallowUnknownFields, call job.Model.Validate() and
job.Experiment.Validate(), and run each registered target's ValidateJob
to see the per-target gates and warnings; the compatibility table on the
mapping page was produced that way.
The reverse output can be checked against the simulation engine's
servicehub/schemas/calliope.json with any JSON Schema validator. It passes
once the placeholders the mapping documents (country, ...) are filled in.