Home | Up | Previous | Next


Test Case Implementation

This section describes how the test cases are implemented. Especially those test cases, which cannot be implemented using normal SSFNet classes and the tests requiring special or modified topologies are of interest here. Whenever possible the tests are implemented in a straightforward manner. That means, that a DML configuration is obtained from the test setup described in the IOL test suite. Detailed configuration of the particular routers is obtained from the test procedure description. For many test cases this is sufficient. The results can be obtained directly from the simulation logs. Because of the early development stage of OSPF and the limitations of SSFNet (missing broadcast and multicast, etc.) some tests cannot be performed yet or have to be modified to meet the current testing requirements. In particular, all broadcast networks are replaced by point-to-point networks. Features specific to broadcast networks (e.g. Designated Router election) cannot be tested. Another major difference between the current implementation and the OSPF standard concerns the use of mulicast. Multicast is used in OSPF to detect neighbors dynamically without having to configure each neighbor on every router. Additionally, multicast is used in broadcast networks to minimize the number of packets sent out of each interface. Since there are only point-to-point links in the SSFNet OSPF implementation at the moment, the only use of multicast would concern the automatic detection of neighbors. In the SSFNet environment, there is no need to configure neighbors manually, if multicast is not available. Each router has access to the complete network configuration in SSFNet. So, the neighbors can be obtained automatically from this configuration, without the use of multicast.

Since some features like virtual links or authentication are not needed in the OSPF implementation, these will not be implemented soon. Because of this it is not yet specified, how these features will be configured in DML. Therefore, the DML configurations for the according test cases can also not yet be prepared and are not considered in the following sections. However, no basic functionality of OSPF depends on these features.

Hello Protocol Tests

The hello protocol test suite consists of 12 test cases. They are used to verify, that the hello protocol works properly, neighbors are discovered and the DR and BDR are elected correctly. A complete overview of the test cases can be found in the IOL test suite description.

Most of the test cases for the Hello Protocol can be implemented with standard SSFNet components and the UnreliableIP class to take links up and down. The only exception to this is test 1.10, which verifies, that no adjacency can be formed, when the neighbors do not agree on certain parameters in their hello packets. One of these parameters is the network mask, which cannot be modified directly in SSFNet, since IP neworks (and the associated configuration) are assigned automatically. The IPwithErrorInjection class is used here to modify the Netmask field in all hello packets of one of the routers.

Test case 1.11 is a special case as well. When two routers do not agree about the network they are connected to (i.e. the network masks are the same, but the network addresses indicate another network), no adjacency may be formed. Because of SSFNet's automatic IP address assignment, this class of errors will never occur. All interfaces connected to the same link are always configured properly. Therefore this test will pass implicitly.

Flooding and Adjacency Tests

The flooding and adjacency test suite consists of 21 test cases which verify, that the process of building adjacencies and the flooding procedure work properly. This includes testing, that the link state databases are exchanged correctly, when an adjacency is formed. In addition, the process of flooding LSAs throughout the network must be tested, including ackowledgements and retransmissions. Again, many of the tests can be implemented using standard components of SSFNet and the UnreliableIP class to take links up and down. In addition, routers are restarted using the Reset class and receive configuration updates using the Configurator class. Other tests use the IPwithErrorInjection and PacketGenerator classes.

On multiaccess networks not every two routers form an adjacency. Test 2.1 is used to verify the correct building of adjacencies in these networks. Since multiaccess networks are not yet available, this test cannot be performed yet.

Test case 2.3 tests, how a router behaves, when the MTU field is not set correctly during the database description process. Since this value cannot be configured, it must be manipulated directly in the database description packets. For this, the IPwithErrorInjection class can be used.

When a router receives a self-originated LSA, that is newer than the instance of this LSA contained in its own link state database, it must originate a new instance of this LSA to keep the link state databases of all routers in the network consistent. This behavior is verified in test case 2.5. In order to do so, the router must be shut down, after it has synchronized its database with its neighbor. When the router is restarted it originates a new LSA, beginning with the InitialSequenceNumber (0x80000001). This is realized with the Reset class. To assure, that the LSA in the neighbor's database has a higher sequence number (indicating that it is newer), the link between both routers is taken down several times, everytime waiting for longer than RouterDeadInterval. This causes the adjacencies to be taken down and up again and assures, that new instances of the routers' LSAs are originated.

old_lsa_receipt

Test number 2.6 verifies, that a router properly handles the receipt of an LSA instance, that is older than an instance of this LSA that is already contained in its database. To test this, the tester must first flood an instance of the LSA having a new sequence number (e.g. 0x70000001). Then, after some time, the LSA is flooded again, having an older sequence number (e.g. 0x8FFFFFFE). This behavior is implemented in the PacketGenerator and can be activated using old_lsa_rcpt as the behavior name in the configuration of the PacketGenerator. To realize this test behavior, the PacketGenerator waits until the database exchange is over and the adjacency is established. Then it sends an update for its own router LSA, indicating a new instance of this LSA. After that, a timer is started, which fires after 10 seconds (enough time to have the neighbor install the new LSA in its database). When the timer fires, a new update is sent for the router LSA, which has an older sequence number. Since this behavior was specified differently in RFC 1583 and RFC 2328, the IOL test suite has different test steps which verify the behavior of a router against both specifications. However, the SSFNet implementation of OSPF only uses RFC 2328 as a reference. No downward compatibility is needed, so the tests for RFC 1583 can be omitted.

nbr_state_too_low

A router must discard link state requests and updates that are received before the master/slave negotiation for the database exchange process is over. This is tested in test case 2.7. To realize a tester, again an according bahavior of the PacketGenerator can be used (nbr_state_too_low). The PacketGenerator waits for the first database description packet from the neighbor and drops this packet. So the OSPF session does not know anything of the beginning database description process. Database description packets from the OSPF session are also dropped. When the first database description packet has been received by the PacketGenerator, a link state update packet containing the own router LSA is sent to the neighbor. After 10 seconds, a link state request packet is sent to the neighbor.

dd_retransmit

Test case 2.8 is used to verify, that database description packets are retransmitted when appropriate. Database description packets may only be retransmitted by the master. This test is divided into four independent parts, which test the correct behavior in different situations. The parts of this test case are implemented in a set of 4 test behaviors for the PacketGenerator.

The first part of the test verifies, that the slave does not retransmit database description packets. The tester must be the master for this test (i.e. it must have a higher Router ID). The PacketGenerator drops all database description packets, that are coming from the OSPF session, but the initial one. This behavior can be activated using the name dd_retransmit1.

The second part of this test case (dd_retransmit2) is used to assure, that the slave properly retransmits its previous database description packet, when it receives a retransmission from the master. For this, the link state database of the tester must be filled with enough LSAs to fill at least four database description packets. This is done by connecting a network to the tester, which contains enough routers (see Section \ref{sec:implementation:dictionary}). Again, the tester is the master in the database description process. The PacketGenerator now simply drops the third database description packet coming from the RUT. After RxmtInterval, the OSPF session on the tester will retransmit its last packet to the RUT. This test implicitly verifies, that the master correctly retransmits database description packets after RxmtInterval, when it does not receive a packet from the slave.

An explicit test for this can be done, using the dd_retransmit3 behavior. This time, the tester must be the slave. All database description packets from the master but the first two packets are dropped. So the OSPF session on the tester only receives these first two packets. Since the slave must not retransmit any database description packets unless it receives a retransmission from the master, it does not send any packets after that.

The fourth part of this test verifies, that the slave retains its last database description packet for RouterDeadInterval after it receives the final packet from the master. For this test, the tester must again be the master. The PacketGenera\-tor waits for the last database description packet from the slave. Once, this packet is received, a timer is started, which expires after RouterDeadInterval. When the timer fires, the last database description packet is retransmitted to the slave. The slave must now retransmit its last packet in response. This test behavior is activated using the name dd_retransmit4 in the DML configuration of the tester.

event_seq_number_mismatch

Test case 2.9 is used to verify, that a router reacts correctly on some errors during the database description process. The test is divided in eight independent test steps. Each test step can be implemented using a separate PacketGenerator behavior. Each of the test steps can be activated in the DML configuration using the name event_seq_number_mismatch, followed by the test step number (1-8).

When the options field in database description packets of a neighbor changes during the database description process, it must be aborted immediately and then be restarted. The PacketGenerator waits for the second database description packet from the neighbor. The options field of the database description packet, which is sent to the neighbor in response, is then manipulated.

The second test verifies, if a router correctly restarts the database exchange, when it receives an initial database description packet unexpectedly. For this, the PacketGenerator simply sets the Initial-bit in its third database description packet.

Whenever a database description packet is received during the database exchange process, which has a sequence number that is higher than expected, the database exchange must be restarted. To verify this, the PacketGenerator simply increases the sequence number of the third database description packet.

The fourth test step is used to verify, that a router correctly restarts he database exchange, when it receives a database description packet, which has the sequence number set too low. This is implemented in the PacketGenerator by decreasing the sequence number of the third database description packet.

The next test step verifies, that the database exchange is restarted, when the master does not claim to be the master anymore during the database exchange. For this, the tester must be the master during database exchange (i.e. it must have the higher router ID). When the neighbor sends its first non-initial database description packet, the PacketGenerator simply clears the Master-bit in its next database description packet.

When a router receives a database description packet more than RouterDeadInterval after the final database description packet has been sent, it must restart the database exchange process. So, after the final packet is received, the PacketGenerator starts a timer, which waits for 50 seconds. When this timer fires, an additional database description Packet is sent to the neighbor.

The seventh test step verifies, that the database exchange is restarted, when a router receives an LSA header with an unknown LS type in a database description packet. For this, the PacketGenerator adds a new LSA header with LS type 0x7F in its second Database Description packet.

The last test step verifies, that the database exchange is restarted, when a router receives a header of an AS-external LSA in a stub area. AS-external LSAs are not yet implemented in OSPF. Because of this, the according test behavior for the PacketGenerator is not yet implemented.

lsa_request_retransmit

Test case 2.17 is used to verify, that a router properly retransmits link ltate lequest packets, when it does not receive the requested LSAs within RxmtInterval. This test consists of two independent test steps. Testers for these test steps can be realized using the PacketGenerator. In the DML configuration the name lsa_request_retransmit followed by the test step number (1-2) can be used to activate the appropriate test behavior for the tests.

The first test step verifies, that a router properly retransmits requests for LSAs, that are not received within RxmtInterval. For this, the tester is connected to a network with 57 routers in order to have its link state database filled. After that it is connected to another router. During the database exchange it sends several database description packets full of LSA headers. These LSAs are then requested by the neighbor, but no updates for these LSAs are sent to the neighbor.

When a router receives updates for some of the requested LSAs, it must not retransmit requests for these LSAs anymore. This is verified in the second test step. The PacketGenerator is again connected to a network in order to fill its link state database. Again, the router is then connected to another router, which requests all LSAs. The PacketGennerator now drops all link state update packets but the first one, so some of the requested LSAs are sent to the neighbor.

event_bad_ls_req

During the database exchange, a router must request all LSAs from its neighbor, that are not contained in its own link state database or for which it has older copies. Whenever a router does not behave like this, the database must be restarted. Additionally, when a router receives an LSA that it has requested from its neighbor, but this LSA is older than an instance already contained in its link state database, the database exchange must be restarted. Test case 2.18 verifies this. To perform these tests, the PacketGenerator can be configured with the event_bad_ls_req behavior. Since this test consists of two independent test steps, the number of the according test step (1-2) must be appended to the behavior name in the DML configuration.

The first test step verifies, that the database exchange is restarted, when a neighbor requests an LSA that is not in the router's link state database. For this, the PacketGenerator adds a header for an imaginary LSA to the first link state request packet sent to its neighbor.

In the second test step it must be verified, that a router restarts the database exchange process, when it receives an LSA that it has requested from its neighbor, but which is older than an instance of the same LSA, that is already contained in the router's link state database. To test this, the PacketGenerator sends an update for its own Router LSA after the adjacency has been brought up. This LSA instance has sequence number 0x80000105. Then the OSPF session is shut down for longer than RouterDeadInterval to bring the adjacency down. When the OSPF session is restarted, the PacketGenerator indicates during the database description process, that its Router LSA has sequence number 0x80000205. This causes the neighbor to request this LSA. When the tester sends an update for the LSA, a sequence number smaller than 0x80000105 is used.

remove_lsa_from_retransmission_list

Test number 2.21 is used to verify, that a router properly removes LSAs from all its neighbors' retransmission lists, that are rmoved from its link state database. To realize this test, a router must originate two new instances of an LSA within RouterDeadInterval. On real routers, this ca be done by enabling and disabling an interface, which causes a new Router LSA to be originated immediately. In SSFNet, there is no mechanism to simulate this, since link failures are only detected after the RouterDeadInterval is over. So a PacketGenerator behavior must be used for this test (remove_lsa_from_retransmission_list). The test scenario is changed according to Figure 5.


 
Figure 5 - Modified test setup for test case 2.21

After all routes have synchronized their databases, the link between TR and RUT is taken down. The PacketGenerator on the tester now sends an update for its router LSA, including an imaginary stub network, so RUT begins retransmitting this update to TR. After 20 seconds, the tester again announces an update for its router LSA, this time without the imaginary stub network. When RUT receives this update it must delete the old instance from its link state database.

Link State Advertisement Tests

The link state advertisements test suite has 22 test cases, which are used to verify that all kinds of LSAs are originated and handled correctly.

Because of the early state of the OSPFv2 implementation, there are currently only router LSAs. Since most of the link state advertisement tests deal with the origination of different LSA types, most of these tests can not yet be performed. However, the DML configurations for these tests can be provided in a straightforward manner, using only standard SSFNet classes and the UnreliableIP class. Some of the test scenarios additionally require the use of the Configurator class to update link costs during the simulation. Since the test configurations are clear, the tests can be performed as soon as other LSA types become available. The tests dealing with router LSAs can also be performed only partially at the moment, since the only link types available at the moment are router links and stub network links. Again, the remaining tests can be performed as soon as the features are implemented in OSPF.

Tests number 3.13 and 3.14 are used to verify the correct use of area address ranges. Area address ranges can be used to help area border routers when originating summary LSAs. Multiple networks can so be combined and advertised together in a single summary LSA. Since SSFNet assigns IP addresses automatically, it is not easy to configure these address ranges from DML. Indeed, this problem will be solved, when summary LSAs are implemented. Until then, these tests can not yet be performed.

ls_sequence_number_wrap

Everytimes a router must increase the sequence number of a self originated LSA past MaxSequenceNumber (0x7FFFFFFF), it must delete this LSA from all routers' link state databases, before it can originate a new instance of this LSA with a sequence number set to InitialSequenceNumber (0x80000001). This is done by premature aging the old LSA. Since there are over 4 billion sequence numbers in the range from the initial to the maximum sequence numbers, it would take a long time until the router must increment the sequence number of one of its LSAs past the maximum value. To shorten the simulation, a behavior is implemented in the PacketGenerator, which simplifies this (ls_sequence_number_wrap). During the database exchange, the PacketGenerator receives a router LSA from its neighbor. After the adjacency is established, it simply modifies the sequence number of this LSA to MaxSequenceNumber and floods it back to its neighbor. The neighbor must then originate a new router LSA, since it has received a self-originated LSA which is newer than the copy contained in its own database. In this new instance, it must increase the sequence number past the maximum value.

Route Calculation Tests

The IOL test suite provides 9 test cases to verify the calculation of the routing tables.

The process of building the routing table is done in several stages (see Section \ref{sec:ospf:calculation}). The first stage calculates the shortest path tree for an area using only router- and network-LSAs. The following stages add stub-networks, inter-area routes and AS-external routes. A closer look at the routing tests in the IOL test suite shows, that the basic calculation of the shortest path tree and the calculation of next-hops is not tested explicitly. Tests for these calclations have to be added, to assure, that a router properly calculates the routing table and the next-hop interfaces. A description of these additional tests can be found later in this Section.

In the later stages of the routing table calculation, summary-LSAs and AS-external LSAs are used. In the current release of OSPF these are not yet implemented. As a consequence, these stages of the routing table calculation are not implemented as well. As a consequence, the routing tests from the IOL test suite cannot be performed yet. Nevertheless, the DML configurations for these tests can be obtained from the test descriptions using the UnreliableIP, Reset and Configurator classes.

Configuration and Formatting Tests

The fifth subsidiary testsuite provides 14 test cases to verify that certain parameters are configurable and that OSPF packets are formatted correctly.

Again, many of the tests verify things that are not yet implemented in the current version of OSPF. Other tests are not applicable in the SSFNet environment at all, because behavior that is tested in these test cases can never occur in the SSFNet implementation of OSPF. Tests 5.9 to 5.13 are used to verify different "length" fields, such as the packet length, the number of LSAs contained in a packet or the number of links in a router LSA. In OSPF implementations for real routers, packets are transmitted as arrays of bytes, which may be of variable length. The different length fields are needed to determine, where a packet ends. But in SSFNet, packets are not byte arrays. Packets are objects in SSFNet. So, there is no need to determine where the packet ends. Lists of LSAs or LSA headers are also not byte arrays in SSFNet. They are contained in the packtes as vectors of LSAs. The number of LSAs is not saved explicitly in the packets but can be obtained through the use of appropriate methods of the Vector class. For the test suite, this means, that the class of errors that is tested by these test cases will never occur in the SSFNet implementation. So these test cases can be omitted.

All OSPF packets should have the IP precedence field to Internetwork Control (0xC0). This is validated by test 5.4. SSFNet implements only a subset of the IP standard, which is sufficient to simulate most scenarios. The IP precedence field is not implemented in SSFNet, so this test cannot be performed. Most of the other tests are used to test features of OSPF which are not yet implemented in the current version. This includes authentication and virtual links. These test cases cannot be performed yet.

Test number 5.8 verifies, if a router properly evaluates the checksums of LSAs. Since the checksum field is not yet provided by the LSA datastructure, a tester for this test case can not be implemented. Once checksums are implemented for LSAs, the IPwithErrorInjection class can be modified to support the manipulation of the checksums in all LSA headers in order to model this test behavior.

Additional Tests

As seen before, tests for the first stage of the routing table calculation are missing from the IOL test suite. However, it must be shown, that the Shortest Path First algorithm works properly. The algorithm cannot be proven to be implemented correctly simply by testing one scenario. Nevetheless, when the algorithm works properly for an appropriate scenario, there is some confidence that it works correctly for all scenarios. To find such a scenario, the structure of Dijkstra's algorithm must be clear. The algorithm contains a main loop, which terminates when there are no nodes left to be added to the tree. Termination of the algorithm is guaranteed, since each iteration of the loop adds exactly one node to the tree. It can be assumed, that the OSPF implementation of Dijkstra's algorithm terminates for all scenarios, when it does for an arbitrary one. To test if the calculated tree is really the correct tree of shortest paths, a similar approach can be followed. This approach does not prove the correctness of the algorithm, but if the test scenario is chosen carefully the chances are good, that the algorithm works for all scenarios.

Test Case 6.1 -- spf_simple

This test case is used to verify the calculation of the shortest path tree and the set of next hops. The test setup together with the link costs is depicted in Figure 6. All link costs are chosen, so that no equal-cost routes occur. After all routers have synchronized their databases, all networks must be contained in the routing table of the RUT. The minimal cost and outgoing interfaces for each destination can now be calculated manually and compared to the entries in the routing table of the RUT.


 
Figure 6 - Test 6.1 -- Simple Shortest Path First calculation

Test Case 6.2 -- equal_cost_routes

OSPF explicitly supports multiple least-cost routes to the same destination. This is considered during the next-hop calculation. The link costs of the scenario shown in Figure 7 are chosen, so that there are multiple least-cost routes to several destinations. Again, after the link state databases of all routers have synchronized, all destinations must be contained in the RUT's routing table. Again, the next-hops and the cost for each destination must be calculated manually and then be compared to the entries in the RUT's routing table.


 
Figure 7 - Test 6.2 -- Calculation of equal-cost routes

Test Case 6.3 -- routes_use

This test case verifies, that the routes calculated by OSPF are correctly used, when a router forwards packets. For this, a simple TCP-client and an according server are connected to the network depicted in Figure 8. The client uses RUT as its default gateway, the server uses TR3 as its default gateway. Packets are captured on TR1. All packets from the TCP client and server must travel through TR1.


 
Figure 8 - Test 6.3 -- Use of routes calculated by SPF

Test Case 6.4 -- table_change

This test case verifies, that routing tables are changed correctly, when a link fails. For this, the topology from test case 6.2 (see Figure 7) can be used. After the link state databases of all routers have been synchronized, the link between RUT and TR1 is taken down using the UnreliableIP class. After RouterDeadInterval, the routers detect the failure and originate new LSAs. After the updates have been flooded throughout the network, the routing tables must be checked. For comparison purposes, the new routing table can again be calculated manually.



Home | Up | Previous | Next


Last update: 2002-02-05, Dirk Jacob (dirk@d-jacob.net)