Test-Mapper

Test::Mapper

Selective acceptance test runner — only run what a change can break.

Acceptance tests are expensive. Running the full suite on every commit burns CI budget fast. Test::Mapper solves this by persisting a many-to-many mapping between source files and the acceptance tests that exercise them, then using that map to narrow each run to only the tests that could be affected.

The problem

Unit and integration tests have a straightforward file-to-test mapping: lib/Foo/Bar.pm → t/lib-Foo-Bar.t. Acceptance tests don't. One acceptance test may exercise dozens of routes, loading dozens of source files. One source file may be exercised by dozens of acceptance tests. You can't derive this mapping mechanically — but the test author knows it, because they wrote the test to exercise specific routes/features.

Test::Mapper gives the test author a place to record that knowledge once, and then uses it automatically on every CI run.

How it works

1. Map phase (run once per test, update when routes change):

   use Test::Mapper;
   
   my $mapper = Test::Mapper->new(
      db          => '/var/lib/test-mapper/myapp.db',
      product     => 'MyApp',
      environment => $^O,
   );
   
   $mapper->map_test(
      test      => 't/acceptance/login.t',
      resources => [ 'lib/MyApp/Auth.pm', 'lib/MyApp/Session.pm' ],
   );
   

2. Select phase (run on each CI build, after git diff):

   my @changed = qw(lib/MyApp/Auth.pm);   # from git diff HEAD~1
   
   my @to_run = $mapper->tests_for_resources( resources => \@changed );
   # => ('t/acceptance/login.t')
   

3. Coordinate phase (distributed CI: multiple workers, one build):

   my $build = $git_sha;
   
   # Queue tests for this build
   my @queued = $mapper->start_build(
      build     => $build,
      resources => \@changed,
   );
   
   # Each worker claims and runs one test atomically
   for my $test ( $mapper->pending_tests( build => $build ) ) {
      next unless $mapper->claim_test(
          build  => $build,
          test   => $test,
          worker => "$$",
      );
      my $exit = system($^X, $test);
      $mapper->complete_test(
          build  => $build,
          test   => $test,
          result => $exit,
      );
   }
   
   # Inspect results
   my $results = $mapper->build_results( build => $build );
   for my $r (@$results) {
      printf "%s: %s\n", $r->{test_file}, $r->{result} == 0 ? 'PASS' : 'FAIL';
   }
   

Storage

SQLite database, one per product (or shared across products — each is namespaced internally). WAL mode enabled for concurrent readers. Schema is applied idempotently on first connection, so no migration scripts are needed.

Installation

perl Makefile.PL
make
make test
make install

Dependencies: DBI, DBD::SQLite, File::Touch (all on CPAN).

Concepts

Term	Meaning
product	The application being tested (namespace key)
environment	OS/platform string (e.g. linux, MSWin32) — resources may differ per platform
resource	A source file whose change may require tests to run
build	A build identifier (e.g. git SHA) used to coordinate a test run
test	An acceptance test filename

License

Artistic License 2.0. See LICENSE.