Module 47 - T204 Part 1 of 2

T204 Part 1 of 2: How to Develop Test Procedures for an ITS Standards-Based Test Plan

HTML of the Student Supplement

(Note: This document has been converted from the Student Supplement to 508-compliant HTML. The formatting has been adjusted for 508 compliance, but all the original text content is included, plus additional text descriptions for the images, photos and/or diagrams have been provided below.)

T204 Part 1 of 2: How to Develop Test Procedures for an ITS Standards-Based Test Plan, Part 1 of 2

Table of Contents

Introduction/Purpose - 2

Samples/Examples - 2

Reference to Other Standards - 6

Glossary - 6

References - 7

Study Questions - 7

Module Description

This module is part of the non-SE path; thus, students must learn How to Write a Test Plan (T201); followed by an Overview of Test Design Specifications, Test Cases, and Test Procedures (T202); and then How to Develop Test Cases for ITS Standards-Based Test Plans (T203 Parts 1 and 2). This module extends this discussion by providing participants with detailed information on how to prepare their own test procedure specifications, including using the test cases, creating test logs, test summaries, and anomaly reports. This module ends with an overview of the Test Procedure Generator (TPG), which is an automated tool available at no cost from the U.S. Department of Transportation (USDOT) that students may download try before Part 2 of this module, which will provide examples of how to perform tests.

1. Introduction/Purpose

The purpose of this module is to teach the student how to develop a Test Procedure Specification (TPS) and show them how said document fits into the testing process and relates to the test plan, test design specification, and test case specification. In addition, the student will be taught how to develop the TPS to meet specific project requirements for the interface (as found in the Protocol Requirements List (PRL)) or Needs to Requirements Traceability Matrix (NRTM).

2. Samples/Examples

2.1. Understand the purpose and structure of a test procedure

2.1.1. Test Procedure Specification (TPS) details a specific example assigning values

2.1.2. Test Procedure Inputs and expected Outputs

2.1.3. Special needs outside of the standards

2.2. Understand the role of Test Procedure Specification within a test plan and the overall testing process

2.2.1. Test Design Specification details what a test is intended to demonstrate

2.2.2. Test Case Specification details a specific example assigning values

2.2.3. Test Procedure defines the steps to perform the test

(Extended Text Description: A graphic illustration created for this slide that depicts the testing tasks within the overall system workflow. The topmost gray box labeled "Test Plan" includes an arrow from the lower side of that box directed downward to a second gray box labeled "Test Design Specification" that includes three arrows as follows: Arrow 1 from the left side of "Test Design Specification" is directed to the left then down to a gray box labeled "Test Case 1". Arrow 2 from the lower side of "Test Design Specification" is directed downward to a gray box labeled "Test Case 2". Arrow 3 from the right side of "Test Design Specification" is directed to the right then downward to a gray box labeled "Test Case 3". The gray box labeled "Test Case 1" includes an arrow from the lower side of that box directed downward to a gray box labeled "Test Procedure 1". The gray box labeled "Test Case 2" includes an arrow from the left side of that box directed to the left then downward to the gray box labeled "Test Procedure 1". The gray box labeled "Test Case 3" includes two arrows, one arrow from that box directed to the left and then downward, and then to the left to the gray box labeled "Test Procedure 2". The second arrow is directed downward from the lower side of "Test Case 3" to a gray box labeled "Test Procedure 3".)

2.3. Synchronize the test procedure specification to the contract Terms and Conditions (T&Cs) for successful contract execution

2.3.1. The odds are against success. The typical highway project model does not work well for ITS projects

(Extended Text Description: Odds of Success: Pie chart graphic reproduced from FHWA Systems Engineering Handbook, 2007. The pie chart indicates 24% Succeeded, 15% Failed and 51% Challenged.)

2.3.2. Structure the contract Terms and Conditions from the viewpoint of the project's end, including test specification, test plan, test scripts and common equipment, if possible.

(Extended Text Description: Author's relevant notes: VEE model graphic reproduced from FHWA Systems Engineering Handbook, 2007, located to the right of the slide text. Above the V to the left, pointing to the left side of the V, is the text "Write Terms and Conditions here..." Above the V to the right, pointing to the right side of the V, is the text "as if you are here." The point of the graphic image is to convey the idea to synchronize the TPS to the contract trems and conditions.)

2.4.Write the reports produced at the end of testing and understand their relationship to successful Procurement Contracts

2.4.1. Logs, including the data, information, files and fulfilled requirements that are captured during the test

2.4.2. Anomaly Report, including a failure description and the investigation process to resolve

2.4.3. Level Test Report, providing a measure of success compared to the stated goals and scope of the Test Plan

2.4.4. Master Test Report covering Level Test Reports

(Extended Text Description: Master Test Report: Graphic identical to graphic shown in 2.2.3 above, with another row of four green boxes located below labeled "Output Records 1", "Output Records 2", "Output Records 3" and "Output Records 4" respectively from left to right. An arrow extends from the bottom of "Test Procedure 1" to the top of "Output Records 1" and "Output Records 2" boxes. A second arrow extends from the bottom of "Test Procedure 2" to the top of "Output Records 3" box. A third arrow extends from the bottom of "Test Procedure 3" to the top of "Output Records 4".)

2.5. Use tools to develop the Test Procedures for a sample TPS structure

2.5.1. Test Procedure Generator (TPG) is an automated tool

2.5.2. TPG guides the development of uniform Test Procedures

2.5.3. How to obtain the TPG from USDOT at no cost

2.5.4. Installation of TPG

2.5.5. Step by step TPG use and the results

2.5.6. Demonstrating a typical error and how it is handled

2.5.7. Use of uniform Key Words with a common understanding of the ITS standards

(Extended Text Description: Test Procedure 01.00: Screenshot taken from TPG. Author's key notes for illustration purposes: This image shows the screen when Requirements are then populated, in this case, 3.4.1.1 through 3.4.2.2, for example. The Current Test Procedure tab is selected, indicating the current document.)

2.5.7. XLM scripts can be used as inputs to automated test tools

(Extended Text Description: XML Report: Screenshot taken from TPG. Author's key notes for illustration purposes: This image shows the screen with the XML Representation of the example, in this case NTCIP C2F Device Interface Standard. The Reports tab is selected indicating the current XML representation.)

2.6. Preparing for T204 Part 2

Between T204 Part 1 and T204 Part 2, please become familiar with the installation and operation of the TPG. T204 Part 2 will use the TPG to develop Test Procedures for different equipment types using the NTCIP standards. Please resolve all TPG installation and operation issues before the start of Part 2, as we will begin immediately with the course work using the TPG as the tool.

• Request a copy of the Test Procedure Generator at the address:
  • blake.christie@noblis.org
• Register at the same link to receive updates
• Download and install the TPG on a computer
• Create a simple Test Procedure to become familiar with TPG before we begin T204 Part 2
• Become familiar with input data entry and the resulting XML outputs
• Have the TPG up and running when we begin Part 2
• No installation questions will be answered during the sessions

3. Reference to Other Standards

• IEEE 829-2008, IEEE Standard for Software and System Test Documentation, IEEE Computer Society, July 18, 2009
• Institute of Transportation Engineers, ATC5201 Advanced Transportation Controller (ATC) Standard Version 06. ATC Joint Committee, 30 July 2012.
• Institute of Transportation Engineers, ATC 5202 Model 2070 Controller Standard Version 03. ATC Joint Committee, 28 December 2012.
• Institute of Transportation Engineers, Intelligent Transportation System (ITS) Standard Specification for Roadside Cabinets v01.02.17b. ATC Joint Committee, 16 November 2006.
• National Electrical Manufacturers Association, NEMA Standards Publication TS 2-2003 v02.06 Traffic Controller Assemblies with NTCIP Requirements. NEMA, 2003

4. Glossary

Term	Definition
Agency Specification	A document that has been prepared by an agency to define requirements for a subject item or process when procured by the agency.
AKA	Also Known As
Compliance	A condition that exists when an item meets all of the requirements of an agency specification.
Concept of Operations	A document that describes the purpose for a system project, including a description of the current and proposed system, as well as key user needs that the new system is required to address.
Conformance	A condition that exists when an item meets all of the mandatory requirements as defined by a standard. It can be measured on the standard as a whole, which means that it meets all mandatory (and applicable conditional) requirements of the standard or on a feature level (i.e., it conforms to feature X as defined in section X.X.X), which means that it meets all mandatory (and applicable conditional) requirements of the feature.
IT	Information Technology
T&C	Terms and Conditions
TCS	Test Case Specification
TDS	Test Design Specification
TPG	Test Procedure Generator
TPS	Test Procedure Specification
XML	Extensible Markup Language

5. References

• IEEE 829: IEEE Standard for Software and System Test Documentation http://standards.ieee.org/findstds/standard/829-2008.html
• NTCIP 1204 v03 ESS, www.ntcip.org
• PCB Website: Module T201, T202 available stds training.aspx
• US Department of Transportation, Systems Engineering for Intelligent Transportation Systems, USDOT, January 2007
• Center to Field Test Procedure Generator User Manual, v1.7, Federal Highway Administration, October 18, 2012

6. Study Questions

1. Which of the following is a FALSE statement?

1. ITS standards define objects and dialogs to be tested
2. ITS standards provide format and steps for test procedures
3. Test procedures can merge with test cases
4. Test cases trace to requirements

2. In addition to inputs, outputs, and execution conditions, test case specification includes:

1. Test objective to provide guidance to the test operator
2. Test environment hardware and software
3. Special procedures, such as automated tools
4. All of the above

3. The TPS should be synchronized to the contract terms and conditions...

1. Project ending without unexpected issues
2. Minimizing the project planning costs and time
3. Replicating the wording of similar prior project
4. Enforcement after the project is late and over budget

4. Which of the following statements is FALSE?

1. Only one test case used as input to each test procedure
2. A test procedure can use multiple test cases as inputs

5. Which statement is TRUE? The TPG...

1. Takes test cases directly as inputs
2. Guides test procedures having MIBs, Dialogs, and RTM
3. Creates outputs in commonly understood XLS files
4. Executes test procedures automatically