How to create tests

We take great care on crafting a stable product. One of the main keys is creating automated tests to check everytying works as expected.

As soon as a problem is found, the very first thing is to be able to reproduce it and then create a test case. Make this test fail, so when the code is fixed it should succeed.

Test Requirements

Tests run on an blank sqlite database that is an exact replica of the real mysql database used in production. The fields and data is the same but the data is empty. So you can run the tests in the same host when a real Ravada service is running.

To ease the process of creating this mock database it is required to install the module Test-SQL-Data .

Test Directory

Create a file in the directory t with the .t extension. There are subdirs there, try to put the file in one of them.

Test File Template

This is an empty tests that does nothing. It just loads a test environment with a blank sqlite database. Notice there is a cleaning of the test environment at the begin and end of the tests. This removes possible old leftovers from failed tests.

Let’s create a simple test that checks if a virtual machine is removed. So we edit a new file called 30_remove.t in the directory ravada/t.

#!perl

use strict;
use warnings;
use Test::More;
use Test::SQL::Data;

use lib 't/lib';
use Test::Ravada;

# create the mock database
my $test = Test::SQL::Data->new(config => 't/etc/sql.conf');

# init ravada for testing
init($test->connector);

##############################################################################

clean();

use_ok('Ravada');

clean();

done_testing();

Run

The prove command runs a test file, before you have to prepare the libraries and environment. Tests create remove and manage virtual machines. That requires root access to the system, so sudo should be used to run the tests.

The best thing is to run it this way:

$ cd ravada
$ perl Makefile.PL && make && sudo prove -b t/30_remove.t
t/user/30_grant_remove.t .. ok
All tests successful.
Files=1, Tests=1,  2 wallclock secs ( 0.02 usr  0.00 sys +  0.57 cusr  0.10 csys =  0.69 CPU)
Result: PASS

Trying the Virtual Managers

It is advisable to run the tests on all the virtual managers known to Ravada. To do so we add a loop that tries to load each of them.

Now the test file is like this:

#!perl

use strict;
use warnings;
use Test::More;
use Test::SQL::Data;

use lib 't/lib';
use Test::Ravada;

my $test = Test::SQL::Data->new(config => 't/etc/sql.conf');

init($test->connector);

##############################################################################

clean();

use_ok('Ravada');

for my $vm_name ( vm_names() ) {

    my $vm;
    eval { $vm = rvd_back->search_vm($vm_name) };

    SKIP: {
        my $msg = "SKIPPED test: No $vm_name VM found ";
        if ($vm && $vm_name =~ /kvm/i && $>) {
            $msg = "SKIPPED: Test must run as root";
            $vm = undef;
        }

        diag($msg)      if !$vm;
        skip $msg       if !$vm;

        diag("Testing remove on $vm_name");
    }
}

clean();

done_testing();

We also have a mock virtual manager that does nothing but it is used to test generic virtual machines. It is called the Void VM and it only should be used for testing. So the output of running the test should be like this:

$ perl Makefile.PL && make && sudo prove -b t/30_remove.t
t/user/30_grant_remove.t .. 1/?
# Testing remove on KVM
# Testing remove on Void
t/user/30_grant_remove.t .. ok
All tests successful.

Test Example: check machine removal

Now the test is there, let’s make it check something, like if a virtual machine has been removed.

#!perl

use strict;
use warnings;
use Test::More;
use Test::SQL::Data;

use lib 't/lib';
use Test::Ravada;

my $test = Test::SQL::Data->new(config => 't/etc/sql.conf');

init($test->connector);

##############################################################################

sub test_remove {
    my $vm = shift;

    my $domain = create_domain($vm->type);
#    $domain->remove( user_admin );

    my $domain2 = $vm->search_domain( $domain->name );
    ok(!$domain2,"[".$domain->type."] expecting domain already removed");

}
##############################################################################

clean();

use_ok('Ravada');

for my $vm_name ( vm_names() ) {

    my $vm;
    eval { $vm = rvd_back->search_vm($vm_name) };

    SKIP: {
        my $msg = "SKIPPED test: No $vm_name VM found ";
        if ($vm && $vm_name =~ /kvm/i && $>) {
            $msg = "SKIPPED: Test must run as root";
            $vm = undef;
        }

        diag($msg)      if !$vm;
        skip $msg       if !$vm;

        diag("Testing remove on $vm_name");

        test_remove($vm);
    }
}

clean();

done_testing();

Now let’s run the test:

$ perl Makefile.PL && make && sudo prove -b t/30_remove.t
t/user/30_grant_remove.t .. 1/?
# Texting remove on KVM
t/user/30_grant_remove.t .. 3/?
#   Failed test '[KVM] expecting domain already removed'
#   at t/user/30_grant_remove.t line 22.
# Texting remove on Void

#   Failed test '[Void] expecting domain already removed'
#   at t/user/30_grant_remove.t line 22.
# Looks like you failed 2 tests of 7.
t/user/30_grant_remove.t .. Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/7 subtests

Test Summary Report
-------------------
t/user/30_grant_remove.t (Wstat: 512 Tests: 7 Failed: 2)
  Failed tests:  4, 7

Whoah there ! It looks like the test failed, of course, someone commented the line 19 that actually removes the machine. Uncomment it and run the tests again. It should return OK.