Overview

Content Server provides a unit testing framework for OScript developers. Unit tests are optional, but including them in your development cycle tends to improve code quality. Consider the following benefits of implementing unit tests as part of your development processes:

• Automating unit tests makes it easy to run tests frequently, which results in catching bugs earlier
• Keeping the scope of your unit tests small (in the ideal case) makes it easier to isolate issues when bugs are detected.

This framework aids with the following test-related activities:

• Creating test objects
• Executing tests
• Viewing test results
• Loading test OSpaces

In this framework, a unit test is the actual test to be run, while a test case is a collection of unit tests. A test case represents a specific method to be tested. A unit test represents the actual tests to validate that method.

Test OSpaces

By convention CSIDE places unit test code in another OSpace with the name ospacename_TEST (For example, if your code is in an OSpace called X then the unit test code should exist in X_TEST). The framework allows for an optional trailing digit in the OSpace name (For example, X_TEST2 is a valid unit test OSpace name for functions contained in the X OSpace). This digit allows you to compartmentalize your unit tests, if desired. The test OSpace should be in the same module as the OSpace with functions being tested.

By default, Content Server does not load test OSpaces on start-up. However Content Server will load test OSpaces on startup when the following conditions are true:

1. Content Server is started via CSIDE, and
2. The test OSpace file exists on the server, and
3. The source for the OSpace being tested exists in the Eclipse Workspace

Once Content Server has been started OSpaces can be loaded / unloaded in the same manner as 'regular' OSpaces.

Each test OSpace includes an object called UnitTest which inherits from UNITTEST::UnitTest. Each test case in the test OSpace inherits from this object.

Creating test objects

To create a test case for a method: right-click the function of interest in the OScript Explorer and select "Generate Unit Tests" from the menu. This results in the following actions:

• The appropriate test OSpace is loaded into the server (if it is not already). If the test OSpace doesn't yet exist: it is created.
• A OScript object representing the test case is added to the test OSpace. This object has the following characteristics:
  • Name #functionName. The resulting source file will be called _functionName.os, as # is not allowed in file names on some systems.
  • Inherit from the UnitTest object in the test OSpace (as mentioned in the previous section) via a series of objects representing the parent objects that the method being tested belongs to
• A pair of unit test objects are created that inherit from the test case object (called 'success' and 'failure')

For example, if you have something like the following in your workspace:

Then you generated a unit test for the 'sampleMethod' function you will see the following objects created:

Writing tests

The test case and test objects created by the framework contain skeleton code which allows you to add the logic for the test as desired. Upon object generation, the code provided by the framework will compile as-is, however, you need to do two things before the test executions provide value:

1. The tests need to be enabled. Set the fEnabled Boolean (which is inherited from UNITTEST::UnitTest) to TRUE)
2. Test logic needs to be introduced by the developer

Regarding testing logic: when writing your code the following functions should be overridden where necessary:

TestSubclassSetup: this function is expected to initialize input parameters to be used for the test case / unit test of interest. If the function does not accept parameters then this method does not need to be overridden. Input parameter values should be stored in the testInfo Assoc provided by the skeleton code. For example if the function mentioned has a String argument called argOne, and you want the value of this to be valOne when the test is run: add a line like the following to this function:

Another activity that can be performed during setup is the injection of mock functions (see the 'Mock Functions' section below for details on this option).

TestSubclassAssert: Overwrite this function to evaluate the data returned from running the method in the source OSpace. Depending on the unit test, the function could have one or more assert statements (see the next section for details on the built-in assertion functions provided).

TestSubclassTeardown: Override this function if changes to the system were made via TestSubclassSetup

Assertions

The framework provides a number of assertion functions. Although using these functions is optional, making use of these functions allows for standardized error messages to be provided upon failure and results in more uniform code.

For example, if you had two objects that you wanted to test for equality you can use AssertEquals(value1, value2). The full set of options can be found in UNITTEST::UnitTest. All assertion methods return an Assoc with the following entries:

• Ok (Boolean) – true if the assertion is valid
• ErrMsg (String) – if 'Ok' is false this should contain details about why the failure occurred.

Alternate assertion syntax

The unit test framework provides wrapper functions to the above methods which allow for chaining calls, which may reduce the volume of code needed and improve readability. For example, to check that something is an Assoc that contains a Boolean named 'ok' and that the value of 'ok' is TRUE you could write the following:

This code is easier to read than what you would need to put in place using the assertion syntax introduced earlier:

There are several test options available via this approach - have a look at the Should___ functions in UNITTEST::Frames::Verification object to see what is available.

As is seen in the example above: adding .which() to the function chain changes the context of the item being inspected to the feature being checked in the previous .shouldHaveFeature() call.

Mock functions

In some cases the function being tested calls other functions. Since unit tests are by definition designed to test a single unit of code (in our case an OScript function) it may be desirable to replace this function call with a dummy or mock function. It may also be desirable to replace a long-running function with a mock function.

Further, in some test cases replacing the function may be essential, as the behavior of the function may be impossible to predict for the test case. An example of this is when the behavior of the function is dependent on the state of data in the server.

In these cases you need to replace the function being called by adding a call to .Mock() from within TestSubclassSetup. The arguments for the call are a pointer to the object of interest, and the name of the function. For example if the method being tested makes a call to $Kernel.FileUtils.Copy() that you would like to mock you would replace this with the following:

The mock object can then be configured to return values via the WillReturn() function. Typically this is chained to the creation of the mock. In the case of $Kernel.FileUtils.Copy() the function is designed to return a Boolean, so if we wanted this to return true we would add the following:

If the mock is going to be called multiple times, and you would like a different value to be returned for the different calls you can specify multiple arguments:

If there are more calls made to the mock than there are arguments for WillReturn: the last argument is reused.

There are also functions available to validate the arguments and calls made to the mock function. To validate the number of times the mock is called during testing you would chain the following calls onto mock definition:

• .ExpectNever()
• .ExpectOnce()
• .ExpectAtLeastOnce()
• .ExpectAtLeast( min )
• .ExpectExactly( n )
• .ExpectBetween( min, max )
• .ExpectAtMost( max )

The arguments passed in can also be validated by chaining calls to .With(). If there are multiple calls to the mock, and each call should have different values, then you should chain multiple calls to .With():

The OUnit view in CSIDE

As of version 21.4 CSIDE ships with a new view to assist with unit test functionality:

This view provides improvements regarding:

1. Test execution
2. Test result viewing
3. Test OSpace management

Executing tests

There are two options for executing a test from CSIDE:

1. Right-click a test object or a test OSpace in either the OScript Explorer, Module Explorer, or OUnit view, then select "Run Tests" from the menu.
2. Select a test object in the OUnit view, then click the "Run Test" button ( )

Each option will run tests for the selected object and the children of that object. Options exist to allow you to filter out test cases by their prior run state, and to track code coverage during test execution (see The Unit Test Coverage Report below for details on what this provides).

By default, whenever CSIDE begins to execute a unit test the OUnit View is made visible. If you do not want the OUnit View to display when a test runs, you can disable the "Show OUnit when test is run" option from the above menu. This option can also be enabled / disabled via the OScript Unit Test preferences.

By default, CSIDE runs all selected tests in batches of 50. These test executions cannot be interrupted while they are executing (unless you wish to restart the server process). Further: feedback / test results are only visible once the batch has finished running. Should you wish to increase / decrease the number of tests executed at a time: the option to do so can be found in the OScript Unit Test preferences. Entering '0' (or leaving the field blank) results in all tests executing in a single batch. Note that decreasing the number of items in a batch does incur a minor performance penalty.

Running tests in multiple OSpaces

If you want to run unit tests in multiple OSpaces at the same time: select the "OScript -> Run Unit Tests for Multiple OSpaces ..." menu option. This brings up the following dialog:

The unit tests listed are available to the server (unit test code that exists only in the workspace is not included). To run the available tests for a given OSpace: move the OSpace from the 'Available' column to the 'Selected' column via the & buttons below the lists. Once the desired OSpaces are in the 'Selected' column: click the 'Run' button. Test results will be populated in the OUnit view, same as all other unit test executions.

Viewing test results

CSIDE scans the workspace and server for test objects and test OSpaces. Anything found gets loaded into the view's table. The table has the following columns:

Icons representing the state of the test objects are as follows:

Icon	Description
	Test has passed
	This object is not a test object, however all child test objects have passed
	Test has failed. The 'Message' column should show the reason why
	This object is not a test object, however at least one child test object failed
	Test has not yet been run
	This object is not a test object, however no child test objects have failed, and at least one child test object has not been run
	Test is currently executing
	This object is not a test object, however at least one child test object is executing
	Object is unavailable. The 'Message' column should show the reason why

Note that rows can be filtered by state via options in the view's menu:

Most test execution code will log progress and results to the console at the INFO log level.

Loading test OSpaces

In addition to the usual options to load and build OSpaces that CSIDE provides: the OUnit view provides the following options in the view's toolbar when the selected row is an OSpace:

• If a test OSpace is loaded click to unload it
• If a test OSpace is not loaded, but exists in the server, click to load it
• If a test OSpace exists in the workspace, but not the server, click to build it

If the button is greyed out ( ) the selected item in the view is not an OSpace, or the server is not running.

The Unit Test Coverage Report

In addition to assisting with unit test creation and execution: CSIDE provides some rudimentary assistance in seeing which functions were invoked while tests have been run.

To enable this option: select one of the "coverage" options when selecting tests to run:

This results in CSIDE recording which functions are invoked by the server while the selected tests are executed. The results of this are presented in the "Unit Test Coverage Report" view, which shows an expandable tree of the objects and functions encountered:

The first column shows the hierarchy of objects and functions. If the object icon is red this means that there is at least one function contained that does not have a unit test written for it. Green icons represent items where all functions have an associated test.

The second column in the tree shows the ratio of number of functions with tests to the number of functions overall for objects containing functions. The coloured bars help to illustrate this ratio. Since the functions themselves do not contain any other items the second column is blank for these rows

Clicking results in the coverage report view focusing on the selected row. Clicking this button a second time results in results for all objects / functions being displayed.

Clicking results in all data in the coverage report view being discarded.

Some notes about the implementation of this feature:

• Because of the significant increase in effort required to profile the running code: the time required to execute these unit tests increases significantly.
• Only functions that exist in the Eclipse Workspace / OScript Explorer are displayed in the 'Unit Test Coverage Report' view