Testing

On Test Automation:

• Not all automated tests are created equal. Some tests are better at testing certain things than others.
• Just because you can write a certain kind of automated test doesn’t necessarily mean you should.
• Speed and feedback matter. The longer it takes your test cases to run, the slower and less iterative your development cycle.

Cohen Test Pyramid

From: https://martinfowler.com/articles/practical-test-pyramid.html

Cohen Test Pyramid

Original test pyramid consists of three layers that your test suite should consist of (bottom to top):

• Unit Tests
• Service Tests
• User Interface Tests

Model is a tad simplistic in today’s world. Key take aways:

• Write tests with different granularity
• The more high-level you get the fewer tests you should have

Test Driven Development (Unit Tests)

Two types:

• Solitary unit tests
• Sociable unit tests

Solitary are tests that stub all collaborators

Sociable are tests that allow talking to real collaborators

What to Test

• A unit test class should at least test the public interface of the class
• Tests that are too close to the production code quickly become annoying
• Don't reflect your internal code structure within your unit tests.
  • Test for observable behaviour instead
• Think about

if I enter values x and y, will the result be z?

instead of

if I enter x and y, will the method call class A first, then call class B and then return the result of class A plus the result of class B?

Don’t test trivial code

Test Structure

A good structure for all your tests (this is not limited to unit tests) is this one:

• Set up the test data
• Call your method under test
• Assert that the expected results are returned

Or, Arrange, Act, Assert

Examples

Let's take a simplified version of the ExampleController class:

@RestController
public class ExampleController {

    private final PersonRepository personRepo;

    @Autowired
    public ExampleController(final PersonRepository personRepo) {
        this.personRepo = personRepo;
    }

    @GetMapping("/hello/{lastName}")
    public String hello(@PathVariable final String lastName) {
        Optional<Person> foundPerson = personRepo.findByLastName(lastName);

        return foundPerson
                .map(person -> String.format("Hello %s %s!",
                        person.getFirstName(),
                        person.getLastName()))
                .orElse(String.format("Who is this '%s' you're talking about?",
                        lastName));
    }
}

A unit test for the hello(lastname) method could look like this:

public class ExampleControllerTest {

    private ExampleController subject;

    @Mock
    private PersonRepository personRepo;

    @Before
    public void setUp() throws Exception {
        initMocks(this);
        subject = new ExampleController(personRepo);
    }

    @Test
    public void shouldReturnFullNameOfAPerson() throws Exception {
        Person peter = new Person("Peter", "Pan");
        given(personRepo.findByLastName("Pan"))
            .willReturn(Optional.of(peter));

        String greeting = subject.hello("Pan");

        assertThat(greeting, is("Hello Peter Pan!"));
    }

    @Test
    public void shouldTellIfPersonIsUnknown() throws Exception {
        given(personRepo.findByLastName(anyString()))
            .willReturn(Optional.empty());

        String greeting = subject.hello("Pan");

        assertThat(greeting, is("Who is this 'Pan' you're talking about?"));
    }
}

Additional Tests

• Integration Tests
  • Database
  • Other services
• Contract Tests
• UI Tests
• End-to-End Tests
• Functional/Acceptance Tests
• Exploratory Testing

Integration Tests

When writing unit tests these are usually the parts you leave out in order to come up with better isolation and faster tests. Still, your application will interact with other parts and this needs to be tested. Integration Tests are there to help. They test the integration of your application with all the parts that live outside of your application.

DB:

1. start a database
2. connect your application to the database
3. trigger a function within your code that writes data to the database
4. check that the expected data has been written to the database by reading the data from the database

It just extends the CrudRepository interface and provides a single method header. The rest is Spring magic.

public interface PersonRepository extends CrudRepository<Person, String> {
    Optional<Person> findByLastName(String lastName);
}

Application:

1. start your application
2. start an instance of the separate service (or a test double with the same interface)
3. trigger a function within your code that reads from the separate service's API
4. check that your application can parse the response correctly

• Note that you may want to avoid actual interactions with live (Production) systems when using automated tests. Tools such as Wiremock, or test databases should be utilized.

Contract Tests

Splitting your system into many small services often means that these services need to communicate with each other via certain (hopefully well-defined, sometimes accidentally grown) interfaces.

A naive implementation of CDC tests can be as simple as firing requests against an API and assert that the responses contain everything you need. You then package these tests as an executable (.gem, .jar, .sh) and upload it somewhere the other team can fetch it (e.g. an artifact repository like Artifactory).

• Pact is one of the recommend tools for this sort of testing

Pact Example

 @Pact(consumer="test_consumer")
    public RequestResponsePact createPact(PactDslWithProvider builder) throws IOException {
        return builder
                .given("weather forecast data")
                .uponReceiving("a request for a weather request for Hamburg")
                    .path("/some-test-api-key/53.5511,9.9937")
                    .method("GET")
                .willRespondWith()
                    .status(200)
                    .body(FileLoader.read("classpath:weatherApiResponse.json"),
                            ContentType.APPLICATION_JSON)
                .toPact();
    }

    @Test
    @PactVerification("weather_provider")
    public void shouldFetchWeatherInformation() throws Exception {
        Optional<WeatherResponse> weatherResponse = weatherClient.fetchWeather();
        assertThat(weatherResponse.isPresent(), is(true));
        assertThat(weatherResponse.get().getSummary(), is("Rain"));
    }
}

UI Tests

UI tests test that the user interface of your application works correctly. User input should trigger the right actions, data should be presented to the user, the UI state should change as expected.

• Selenium is the usual tool for UI tests, but there are others

End-to-End

Testing your deployed application via its user interface is the most end-to-end way you could test your application. The previously described, webdriver driven UI tests are a good example of end-to-end tests.

• Tools include Selenium and WebDriver, PhantomJS and Chromium for headless browser testing

Example

@RunWith(SpringRunner.class)
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
public class HelloE2ESeleniumTest {

    private WebDriver driver;

    @LocalServerPort
    private int port;

    @BeforeClass
    public static void setUpClass() throws Exception {
        ChromeDriverManager.getInstance().setup();
    }

    @Before
    public void setUp() throws Exception {
        driver = new ChromeDriver();
    }

    @After
    public void tearDown() {
        driver.close();
    }

    @Test
    public void helloPageHasTextHelloWorld() {
        driver.get(String.format("http://127.0.0.1:%s/hello", port));

        assertThat(driver.findElement(By.tagName("body")).getText(), containsString("Hello World!"));
    }
}

Acceptance Tests (BDD)

The higher you move up in your test pyramid the more likely you enter the realms of testing whether the features you're building work correctly from a user's perspective. You can treat your application as a black box and shift the focus in your tests from

when I enter the values x and y, the return value should be z

towards

given there's a logged in user

and there's an article "bicycle"

when the user navigates to the "bicycle" article's detail page

and clicks the "add to basket" button

then the article "bicycle" should be in their shopping basket

• Cucumber is the tool, Gherkin is the style.

Cucumber Example

# a sample acceptance test in Python

def test_add_to_basket():
    # given
    user = a_user_with_empty_basket()
    user.login()
    bicycle = article(name="bicycle", price=100)

    # when
    article_page.add_to_.basket(bicycle)

    # then
    assert user.basket.contains(bicycle)

Exploratory Testing

Even the most diligent test automation efforts are not perfect. Sometimes you miss certain edge cases in your automated tests. Sometimes it's nearly impossible to detect a particular bug by writing a unit test. Certain quality issues don't even become apparent within your automated tests (think about design or usability). Despite your best intentions with regards to test automation, manual testing of some sorts is still a good idea.

It is a manual testing approach that emphasises the tester's freedom and creativity to spot quality issues in a running system. Simply take some time on a regular schedule, roll up your sleeves and try to break your application. Use a destructive mindset and come up with ways to provoke issues and errors in your application. Document everything you find for later. Watch out for bugs, design issues, slow response times, missing or misleading error messages and everything else that would annoy you as a user of your software.

Writing Clean Test Code

As with writing code in general, coming up with good and clean test code takes great care. Here are some more hints for coming up with maintainable test code before you go ahead and hack away on your automated test suite:

1. Test code is as important as production code. Give it the same level of care and attention. "this is only test code" is not a valid excuse to justify sloppy code
2. Test one condition per test. This helps you to keep your tests short and easy to reason about
3. "arrange, act, assert" or "given, when, then" are good mnemonics to keep your tests well-structured
4. Readability matters. Don't try to be overly DRY . Duplication is okay, if it improves readability. Try to find a balance between DRY and DAMP code
5. When in doubt use the Rule of Three to decide when to refactor. Use before reuse.

DAMP (Descriptive And Meaningful Phrases) promotes the readability of the code.

To maintain code, you first need to understand the code. To understand it, you have to read it. Consider for a moment how much time you spend reading code. It's a lot. DAMP increases maintainability by reducing the time necessary to read and understand the code.

DRY (Don't repeat yourself) promotes the orthogonality of the code.

Removing duplication ensures that every concept in the system has a single authoritative representation in the code. A change to a single business concept results in a single change to the code. DRY increases maintainability by isolating change (risk) to only those parts of the system that must change.