Unit Testing and Test
Automation
CSE 4495 - Lecture 7 - 16/08/2022
Instructor : Md. Mohaiminul Islam
1
Today’s Goals
• Unit Testing
• Testing of individual classes
• Writing and executing test cases
• How to write unit tests in JUnit.
• Executing tests as part of a build script.
2018-08-27 Chalmers University of Technology 2
Testing Stages
API GUI CLI
• We interact with systems
API
through interfaces.
• APIs, GUIs, CLIs
• Systems built from subsystems.
• With their own interfaces. API
• Subsystems built from units.
• Communication via method calls.
• Set of methods is an interface.
Unit Testing
• Testing the smallest “unit” that can be tested.
• Often, a class and its methods.
• Tested in isolation from all other units.
• Mock the results from other classes.
• Test input = method calls.
• Test oracle = assertions on output/class variables.
Unit Testing
Account
• For a unit, tests should: - name
- personnummer
- balance
• Test all “jobs” associated with the unit.
• Individual methods belonging to a class. Account (name,
• Sequences of methods that can interact. personnummer, Balance)
• Set and check class variables. withdraw (double amount)
deposit (double amount)
• Examine how variables change after changeName(String name)
getName()
method calls. getPersonnummer()
getBalance()
• Put the variables into all possible states
(types of values).
Unit Testing - Account
Account Unit tests should cover:
- name ● Set and check class variables.
- personnummer
- balance ○ Can any methods change name,
personnummer, balance?
Account (name, ○ Does changing those create
personnummer, Balance)
problems?
withdraw (double amount)
deposit (double amount)
changeName(String name)
● Each “job” performed by the
getName()
getPersonnummer() class.
getBalance()
○ Single methods or method sequences.
■ Vary the order methods are called.
○ Each outcome of each “job” (error
Unit Testing - Account
Account
Some tests we might want to write:
- name • Execute constructor, verify fields.
- personnummer
- balance • Check the name, change the name, make
sure changed name is in place.
Account (name,
personnummer, Balance) • Check that personnummer is correct.
withdraw (double amount)
deposit (double amount) • Check the balance, withdraw money,
changeName(String name)
getName() verify that new balance is correct.
getPersonnummer()
getBalance() • Check the balance, deposit money,
verify that new balance is correct.
Unit Testing - Account
Account
Some potential error cases:
--name • Withdraw more than is in balance.
name
- personnummer
--balance
personnummer • Withdraw a negative amount.
- balance
• Deposit a negative amount.
Account
withdraw (name,
(double amount)
personnummer, Balance) • Withdraw/Deposit a small amount
deposit (double amount)
changeName(String
withdraw name)
(double amount) (potential rounding error)
deposit (double amount)
getName()
changeName(String
getPersonnummer() name) • Change name to a null reference.
getName()
getBalance()
getPersonnummer()
getBalance() • Can we set an “malformed” name?
• (i.e., are there any rules on a valid
name?)
Unit Testing and Test Automation
Writing a Unit Test
JUnit is a Java-based toolkit public class Calculator {
for writing executable tests. public int evaluate(String expression){
• Choose a target from the int sum = 0;
for (String summand
code base.
:expression.split("\\+"))
• Write a “testing class”
sum += Integer.valueOf(summand);
containing a series of unit return sum;
tests centered around }
testing that target. }
JUnit Test Skeleton
@Test annotation defines a single test:
Type of scenario, and expectation on outcome.
@Test I.e., testEvaluate_GoodInput() or testEvaluate_NullInput()
public void test<Feature or Method Name>_<Testing Context>() {
//Define Inputs
try{ //Try to get output.
}catch(Exception error){
fail("Why did it fail?");
}
//Compare expected and actual values through assertions or through
//if-statements/fail commands
}
Writing JUnit Tests Convention - name the test class
after the class it is testing.
Each test is denoted with keyword import static org.junit.Assert.assertEquals;
@test.
import org.junit.Test;
public class Calculator {
public class CalculatorTest {
public int evaluate (String
@Test
expression) {
void
int sum = 0; testEvaluate_Valid_ShouldPa
Initialization ss(){
Calculator calculator = new Calculator();
for (String summand: int sum = calculator.evaluate("1+2+3");
Test Steps Input
expression.split("\\+")) assertEquals(6, sum);
Oracle
sum += Integer.valueOf(summand); }
return sum; }
}
}
Test Fixtures - Shared Initialization
@BeforeEach annotation defines a common test
initialization method:
@BeforeEach
public void setUp() throws Exception
{
this.registration = new Registration();
this.registration.setUser(“MoI”);
}
Test Fixtures - Teardown Method
@AfterEach annotation defines a common test tear
down method:
@AfterEach
public void tearDown() throws Exception
{
this.registration.logout();
this.registration = null;
}
More Test Fixtures
@BeforeAll
• @BeforeAll defines public static void setUpClass() {
initialization to take myManagedResource = new
ManagedResource();
place before any tests }
are run.
• @AfterAll defines @AfterAll
public static void tearDownClass()
tear down after all throws IOException {
tests are done. myManagedResource.close(
); myManagedResource =
null;
}
Assertions
Assertions are a "language" of testing - constraints that you
place on the output.
• assertEquals, assertArrayEquals
• assertFalse, assertTrue
• assertNull, assertNotNull
• assertSame,assertNotSame
assertEquals
@Test
public void testAssertEquals()
● Compares two items for
{ assertEquals("failure - strings are equality.
not ● For user-defined classes,
equal", "text", "text");
} relies on .equals method.
○ Compare field-by-field
@Test
○ assertEquals(studentA.getName(),
studentB.getName())
public void testAssertArrayEquals() rather than
{ byte[] expected = "trial".getBytes(); assertEquals(studentA, studentB)
byte[] actual = "trial".getBytes(); ● assertArrayEquals
assertArrayEquals("failure - byte arrays
not same", expected, actual); compares arrays of items.
}
assertFalse, assertTrue
@Test
public void testAssertFalse()
{ assertFalse("failure - should be
● Take in a string and a
false", boolean expression.
(getGrade(studentA, “CSE4495”).equals(“A+”)); ● Evaluates the expression
}
and issues pass/fail based on
@Test outcome.
public void testAssertTrue()
● Used to check conformance
{ assertTrue("failure - should be
true", of solution to expected
(getCGPA(studentA) > 3.5)); properties.
}
assertSame, assertNotSame
@Test
public void testAssertNotSame()
{ assertNotSame("should not be same ● Checks whether two
Object",
studentA, new Object()); objects are clones.
} ● Are these variables aliases
@Test
for the same object?
public void testAssertSame()
○ assertEquals uses
{ Student studentB = .equals().
studentA;
○ assertSame uses ==
assertSame("should be same", studentA,
studentB);
}
assertNull, assertNotNull
@Test
public void testAssertNotNull()
{ assertNotNull("should not be
● Take in an object and
null", new Object());
} checks whether it is
null/not null.
@Test ● Can be used to help
public void testAssertNull()
diagnose and void null
{ assertNull("should be null",
null);
pointer exceptions.
}
Grouping Assertions
@Test
void groupedAssertions() {
● Grouped assertions are
Person person = Account.getHolder(); executed.
assertAll("person", ○ Failures are reported
() -> assertEquals("John", together.
person.getFirstName()), ○ Preferred way to
() -> assertEquals("Doe",
compare fields of two
person.getLastName()));
data structures.
}
assertThat
@Test both - two properties must be met.
public void testAssertThat{
assertThat("albumen", both(containsString("a")).and(containsString("b")));
assertThat(Arrays.asList("one", "two", "three"), hasItems("one", "three"));
assertThat(Arrays.asList(new String[] { "fun", "ban", "net" }),
everyItem(containsString("n")));
assertThat("good", allOf(equalTo("good"), startsWith("good")));
assertThat("good", not(allOf(equalTo("bad"),
equalTo("good")))); assertThat("good", anyOf(equalTo("bad"),
equalTo("good"))); assertThat(7,
not(CombinableMatcher.<Integer>
either(equalTo(3)).or(equalTo(4))));
}
assertThat
@Test everyItem - all items in list must match a
property.
public void testAssertThat{
assertThat("albumen", both(containsString("a")).and(containsString("b")));
assertThat(Arrays.asList("one", "two", "three"), hasItems("one", "three"));
assertThat(Arrays.asList(new String[] { "fun", "ban", "net" }),
everyItem(containsString("n")));
assertThat("godfather", allOf(equalTo("godfather"),
startsWith("go")));
assertThat("good", not(allOf(equalTo("bad"),
equalTo("good")))); assertThat("good", anyOf(equalTo("bad"),
equalTo("good"))); assertThat(7,
not(CombinableMatcher.<Integer>
either(equalTo(3)).or(equalTo(4))));
assertThat
@Test allOf - all listed properties must be true
public void testAssertThat{
assertThat("albumen", both(containsString("a")).and(containsString("b")));
assertThat(Arrays.asList("one", "two", "three"), hasItems("one", "three"));
assertThat(Arrays.asList(new String[] { "fun", "ban", "net" }),
everyItem(containsString("n")));
assertThat("good", allOf(equalTo("good"), startsWith("good")));
assertThat("good", not(allOf(equalTo("bad"),
equalTo("good")))); assertThat("good", anyOf(equalTo("bad"),
equalTo("good"))); assertThat(7,
not(CombinableMatcher.<Integer>
either(equalTo(3)).or(equalTo(4))));
}
assertThat
@Test anyOf - at least one of the listed
properties must be true
public void testAssertThat{
assertThat("albumen", both(containsString("a")).and(containsString("b")));
assertThat(Arrays.asList("one", "two", "three"), hasItems("one", "three"));
assertThat(Arrays.asList(new String[] { "fun", "ban", "net" }),
everyItem(containsString("n")));
assertThat("good", allOf(equalTo("good"), startsWith("good")));
assertThat("good", not(allOf(equalTo("bad"),
equalTo("good")))); assertThat("good", anyOf(equalTo("bad"),
equalTo("good"))); assertThat(7,
not(CombinableMatcher.<Integer>
either(equalTo(3)).or(equalTo(4))));
}
assertThat
@Test either - pass if one of these properties
public void testAssertThat{
assertThat("albumen", both(containsString("a")).and(containsString("b")));
assertThat(Arrays.asList("one", "two", "three"), hasItems("one", "three"));
assertThat(Arrays.asList(new String[] { "fun", "ban", "net" }),
everyItem(containsString("n")));
assertThat("good", allOf(equalTo("good"), startsWith("good")));
assertThat("good", not(allOf(equalTo("bad"),
equalTo("good")))); assertThat("good", anyOf(equalTo("bad"),
equalTo("good"))); assertThat(7,
not(CombinableMatcher.<Integer>
either(equalTo(3)).or(equalTo(4))));
}
Testing Exceptions
@Test ● When testing error
void exceptionTesting() {
Throwable exception =
handling, we expect
assertThrows( IndexOutOfBoundsExc exceptions to be thrown.
eption.class, ○ assertThrows checks
() -> { new ArrayList<Object>().get(0);} whether the code block
); throws the expected
assertEquals("Index:0, Size:0", exception.
exception.getMessage());
○ assertEquals can be
} used to check the
contents of the stack
trace.
Testing Performance
@Test
void timeoutExceeded()
{ assertTimeout( ofMillis(1
● assertTimeout can be
0), () ->
{ Order.process(); });
used to impose a time
} limit on an action.
@Test ○ Time limit stated using ofMilis(..),
void timeoutNotExceededWithMethod() { ofSeconds(..), ofMinutes(..)
○ Result of action can be captured as well,
String greeting = allowing checking of result correctness.
assertTimeout(ofMinutes(2),
AssertionsDemo::greeting);
assertEquals("Hello, World!",
greeting);
Unit Testing - Account
Account
• Withdraw money, verify balance.
@Test
- name
- personnummer public void testWithdraw_normal() {
- balance // Setup
Account account = new Account(“Test MrTest”, “19850101-1001”,
Account (name, 48.5);
personnummer, Balance) // Test Steps
withdraw (double amount) double toWithdraw = 16.0; //Input
deposit (double amount) account.withdraw(toWithdraw);
changeName(String name)
getName() double actual = account.getBalance();
getPersonnummer()
getBalance() double expectedBalance = 32.5; // Oracle
assertEquals(expected, actual); // Oracle
}
Unit Testing - Account
Account • Withdraw more than is in balance.
- name • (should throw an exception with appropriate
- personnummer
- balance error message)
@Test
public void testWithdraw_moreThanBalance() {
Account (name, // Setup
personnummer, Balance)
Account account = new Account(“Test MrTest”, “19850101-
withdraw (double amount) 1001”, 48.5);
deposit (double amount)
changeName(String name) // Test Steps
getName() double toWithdraw = 100.0; //Input
getPersonnummer()
getBalance() Throwable exception = assertThrows(
() -> { account.withdraw(toWithdraw); } );
assertEquals(“Amount 100.00 is greater than balance
48.50”,exception.getMessage()); // Oracle
Unit Testing - Account
Account
• Withdraw a negative amount.
- name • (should throw an exception with
- personnummer
- balance appropriate error message)
@Test
public void testWithdraw_negative() {
Account (name,
personnummer, Balance) // Setup
Account account = new Account(“Test MrTest”, “19850101-1001”, 48.5);
withdraw (double amount) // Test Steps
deposit (double amount)
changeName(String name) double toWithdraw = -2.5; //Input
getName() Throwable exception = assertThrows(
getPersonnummer()
getBalance() () -> { account.withdraw(toWithdraw); } );
assertEquals(“Cannot withdraw a negative amount: -2.50”,
exception.getMessage()); // Oracle
}
Let’s take a break.
Best Practices
• Use assertions instead of print statements
@Test
public void testStringUtil_Bad() {
String result = stringUtil.concat("Hello ", "World");
System.out.println("Result is "+result);
}
@Test
public void testStringUtil_Good() {
String result = stringUtil.concat("Hello ", "World");
assertEquals("Hello World", result);
}
• The first will always pass (no
assertions)
Best Practices
• If code is non-deterministic, tests should give deterministic results.
public long calculateTime(){
long time = 0;
long before =
System.currentTimeMillis(
);
veryComplexFunction();
long after = System.currentTimeMillis();
time = after - before;
return time;
}
• Tests for this method should not specify exact time, but properties of a
“good” execution.
• The time should be positive, not negative or 0.
• A range on the allowed times.
Best Practices
• Test negative scenarios and boundary cases, in
addition to positive scenarios.
• Can the system handle invalid data?
• Method expects a string of length 8, with A-Z,a-z,0-9.
• Try non-alphanumeric characters. Try a blank value. Try strings with
length < 8, > 8
• Boundary cases test extreme values.
• If method expects numeric value 1 to 100, try 1 and 100.
• Also, 0, negative, 100+ (negative scenarios).
Best Practices
• Test only one unit at a time.
• Each scenario in a separate test case.
• Helps in isolating and fixing faults.
• Don’t use unnecessary assertions.
• Specify how code should work, not a list of observations.
• Generally, each unit test performs one assertion
• Or all assertions are related.
Best Practices
• Make each test independent of all others.
• Use @BeforeEach and @AfterEach to set up state and clear state before
the next test case.
• Create unit tests to target exceptions.
• If an exception should be thrown based on certain input, make sure
the exception is thrown.
Thank You
38
