Module 49 - T312

T312: Applying Your Test Plan to a Transportation Sensor System (TSS) Based on the NTCIP 1209 Standard v02

HTML of the PowerPoint Presentation

(Note: This document has been converted from a PowerPoint presentation to 508-compliant HTML. The formatting has been adjusted for 508 compliance, but all the original text content is included, plus additional text descriptions for the images, photos and/or diagrams have been provided below.)

Slide 1:

Welcome - Graphic image of introductory slide. Please see the Extended Text Description below.

(Extended Text Description: Welcome - Graphic image of introductory slide. A large dark blue rectangle with a wide, light grid pattern at the top half and bands of dark and lighter blue bands below. There is a white square ITS logo box with words "Standards ITS Training" in green and blue on the middle left side. The word "Welcome" in white is to the right of the logo. Under the logo box is the logo for the U.S. Department of Transpotation, Office of the Assistant Secretary for Research and Technology.)

Slide 2:

Welcome slide with Ken Leonard. Please see the Extended Text Description below.

(Extended Text Description: This slide, entitled "Welcome" has a photo of Ken Leonard, Director, ITS Joint Program Office, on the left hand side, with his email address, Ken.Leonard@dot.gov. A screen capture snapshot of the home webpage is found on the right hand side - for illustration only - from August 2014. Below this image is a link to the current website: www.its.dot.gov/pcb - this screen capture snapshot shows an example from the Office of the Assistant Secretary for Research and Development - Intelligent Transportation Systems Joint Program Office - ITS Professional Capacity Building Program/Advanced ITS Education. Below the main site banner, it shows the main navigation menu with the following items: About, ITS Training, Knowledge Exchange, Technology Transfer, ITS in Academics, and Media Library. Below the main navigation menu, the page shows various content of the website, including a graphic image of professionals seated in a room during a training program. A text overlay has the text Welcome to ITS Professional Capacity Building. Additional content on the page includes a box entitled What's New and a section labeled Free Training. Again, this image serves for illustration only. The current website link is: https://www.its.dot.gov/pcb.)

Slide 3:

Activity. A placeholder graphic with an image of hand over a computer keyboard to show that an activity is taking place.

Slide 4:

T312:

Applying Your Test Plan to a Transportation Sensor System (TSS)

Based on the NTCIP 1209 Standard v02

Slide 5:

Instructor

Headshot photo of Ralph W. Boaz

Ralph W. Boaz

President

Pillar Consulting, Inc.

San Diego, CA, USA

Slide 6:

Target Audience

Slide 7:

Recommended Prerequisites

Slide 8:

Recommended Prerequisites (cont.)

Slide 9:

Curriculum Path

A graphical illustration indicating the sequence of training modules that lead up to and follow each course. Please see the Extended Text Description below.

(Extended Text Description: This slide contains a graphical illustration indicating the sequence of training modules that lead up to this course. Each module is represented by a box with the name of the module in it and an arrow showing the logical flow of the modules and the preceding module and current module highlighted. There are 8 boxes in total; two rows of 3 boxes and a row of 2 boxes. The first box is labeled "I101 Using ITS Standards: An Overview." An arrow from this box connects it to a box labeled "A101 Introduction to Acquiring Standards-based ITS Systems." An arrow from this box connects it to a box labeled "A102 Introduction to User Needs Identification." An arrow from this box connects it to a box located at the start of the next row labeled "A201 Details on Acquiring Standards-based ITS Systems." An arrow from this box connects it to a box labeled "C101 Intro. To Comm. Protocols and Their Use in ITS Applications." An arrow from this box connects it to a box labeled "C201 Introduction to the SNMP and its Applications in the NTCIP Standards." An arrow from this box connects it to a box located at the start of the next row labeled "A312a Understanding User Needs for Transportation Sensor System Based on NTCIP 1209 Standard." An arrow from this box connects to a box labeled "A312b Specifying Requirements for Transportation Sensor Systems (TSS) Based on NTCIP 1209 Standard." An arrow from this box connects to a box labeled "T312 Applying Your Test Plan to Transportation Sensor Systems (TSS) Based on NTCIP 1209 Standard v02." It is highlighted indicating that it is the current course.)

Slide 10:

Learning Objectives

  1. Recognize the purpose, structure, and content of well-written test documentation based on Institute of Electrical and Electronics Engineers (IEEE) Std 829-2008
  2. Describe TSS testing and the role of test documentation within the context of the systems life cycle
  3. Identify a process to develop test documentation for a TSS specification based on National Transportation Communications for Intelligent Transportation Systems Protocol (NTCIP) 1209 Standard v02
  4. Describe the testing of a TSS using sample test documentation

Slide 11:

Learning Objective #1: Recognize the Purpose, Structure, and Content of Well-Written Test Documentation Based on IEEE Std 829-2008

IEEE Std 829-2008 - IEEE Standard for Software and System Test Documentation

Slide 12:

Learning Objective #1

Comparison of IEEE Testing Standards

IEEE Std 829-2008 IEEE Std 829-1998
Level Test Plan (LTP) Test Plan
Level Test Design (LTD) Test Design Specification
Level Test Case (LTC) Test Case Specification
Level Test Procedure (LTPr) Test Procedure Specification
Level Test Log (LTL) Test Log
Anomaly Report (AR) Test Incident Report
Level Interim Test Status Report (LITSR) No corresponding document
Level Test Report Test Summary Report

Using the term "test documentation" to refer to a collection of test documents.

Slide 13:

Learning Objective #1

Documents Used to Specify Testing

Test Document Purpose
Level Test Plan (LTP) Specifies scope, approach, resources, and schedule for a specified level of testing. "Level" can be replaced by name of the particular level of testing being covered by the plan.
Level Test Design (LTD) Specifies refinements of the test approach in the LTP and identifies the features to be tested by this design and associated tests.
Level Test Case (LTC) Defines the information needed as it pertains to inputs and outputs from the software or software-based system being tested. May represent a single test case or a group of test cases for a segment of an LTD.
Level Test Procedure (LTPr) Specifies the steps for executing a set of test cases or, more generally, the steps used to exercise a software product or software-based system item in order to evaluate a set of features.

Slide 14:

Learning Objective #1

Documents Used to Specify Testing (cont.)

Documents Used to Specify Testing. Please see the Extended Text Description below.

(Extended Text Description: This slide, entitled "Documents Used to Specify Testing," contains a graphic illustrating the relationships of the test documents that are used in specifying testing. Each document is represented by a graphic that has a rectangular-like shape except the bottom of the shape is curved. Each graphic has the name of the test document in the middle of it. When multiple documents of the same time are represented, the graphic is the same except that the right edges of two more documents are shown behind it. The graphic is arranged in four vertical areas or levels on the slide. At first level (top of slide) center of the slide is the document "Level Test Plan (LTP)." In the second level, on the left side of the slide is the document "Level Test Design (LTD)." On the right side of the slide is a rectangle with a dashed line labeled "Additional LTDs/LTCs/LTPrs if necessary." There is a thin solid arrow extending from the LTP document to the LTD document. There is thin dashed arrow extending from the LTP document to the dashed rectangle. In the third level, spaced evenly horizontally below the LTD document, are two multiple documents that are labeled "Level Test Case (LTC)" and "Level Test Procedure (LTPr)." There are thin solid arrows extending from the LTD document to the LTC and LTPr documents respectively. There is a thin dashed double arrow extending horizontally between the LTC document and LTPr document. In the fourth level, there is a rectangle about 10 times as wide as it is tall made with solid lines and extending most of the page. The rectangle is labeled "Test Execution." There are solid arrows extending from the LTC and LTPr documents to the Test Execution rectangle.)

Graphics: Ralph W. Boaz

Slide 15:

Learning Objective #1

Test Specification Documents Used in This Module

Test Specification Documents Used in This Module. Please see the Extended Text Description below.

(Extended Text Description: This slide, entitled "Test Specification Documents Used in This Module," contains a graphic similar to that of Slide #14 except the document names have been changed and an extra dashed rectangle is in the second level. The graphic is arranged in four vertical areas or levels on the slide. At first level (top of slide) center of the slide is the document "TSS Comm Test Plan." In the second level, on the left side of the slide is the document "TSS Comm Bench Test Design." On the right side of the slide there are two rectangles with dashed lines. One is labeled "TSS Comm Integrated Test Design, etc." and the other is labeled "TSS Comm Field Test Design, etc." There is a thin solid arrow extending from the TSS Comm Test Plan document to the TSS Comm Bench Test Design document. There are thin dashed arrows extending from the TSS Comm Test Plan document to the dashed rectangles. In the third level, spaced evenly horizontally below the TSS Comm Bench Test Design document, are two multiple documents that are labeled "TSS Comm Test Case" and "TSS Comm Test Procedure." There are thin solid arrows extending from the TSS Comm Bench Test Design document to the TSS Comm Test Case and TSS Comm Test Procedure documents respectively. There is a thin dashed double arrow extending horizontally between the TSS Comm Test Case document and TSS Comm Test Procedure document. In the fourth level, there is a rectangle about 10 times as wide as it is tall made with solid lines and extending most of the page. The rectangle is labeled "Test Execution." There are solid arrows extending from the TSS Comm Test Case and TSS Comm Test Procedure documents to the Test Execution rectangle.)

Graphics: Ralph W. Boaz

Slide 16:

Learning Objective #1

Documents Used in Test Reporting

Test Document Purpose
Level Test Log (LTL) Provides a chronological record of relevant details about the execution of tests.
Anomaly Report (AR) Documents any event that occurs during the testing process that requires investigation. This may be called a problem, test incident, defect, trouble, issue, anomaly, or error report.
Level Interim Test Status Report (LITSR) Summarizes the results of the designated testing activities and, optionally, provides evaluations and recommendations based on these results.
Level Test Report (LTR) Summarizes the results of the designated testing activities and provides evaluations and recommendations based on these results.

Slide 17:

Learning Objective #1

Documents Used in Test Reporting (cont.)

Documents Used in Test Reporting. Please see the Extended Text Description below.

(Extended Text Description: This slide, entitled "Documents Used in Test Reporting," contains a graphic illustrating the relationships of the test documents that are used in test reporting. The document graphics are the same as that in Slide #14 except that the document names are different. The graphic is arranged in three vertical areas or levels on the slide. At first level (top of slide), there is a rectangle about 10 times as wide as it is tall made with solid lines and extending about 2/3 of the slide. The rectangle is labeled "Test Execution." To the right of the Test Execution rectangle is a multiple document labeled "Level Interim Test Status Report (LITSR)." There is a thin solid arrow extending from the Test Execution rectangle to the LITSR document. In the second level, spaced evenly horizontally below the Test Execution rectangle, are two multiple documents that are labeled "Level Test Log (LTL)" and "Anomaly Report (AR)." There are thin solid arrows extending from the Test Execution rectangle to the LTL and the AR documents respectively. In the third level, centered below the LTL and AR documents is the document "Level Test Report (LTR)." There are thin solid arrows extending from LTL and the AR documents to the LTR.)

Graphics: Ralph W. Boaz

Slide 18:

Learning Objective #1

Test Reporting Documents Used in This Module

Test Reporting Documents Used in This Module. Please see the Extended Text Description below.

(Extended Text Description: This slide, entitled "Test Reporting Documents Used in This Module," contains the same graphic as used in Slide #17 except the document names have been changed. The graphic is arranged in three vertical areas or levels on the slide. At first level (top of slide), there is a rectangle about 10 times as wide as it is tall made with solid lines and extending about 2/3 of the slide. The rectangle is labeled "Test Execution." To the right of the Test Execution rectangle is a multiple document labeled "TSS Comm Interim Test Status Report." There is a thin solid arrow extending from the Test Execution rectangle to the TSS Comm Interim Test Status Report document. In the second level, spaced evenly horizontally below the Test Execution rectangle, are two multiple documents that are labeled "TSS Comm Test Log" and "TSS Comm Test Incident Report *." There are thin solid arrows extending from the Test Execution rectangle to the TSS Comm Test Log and the TSS Comm Test Incident Report documents respectively. In the third level, centered below the TSS Comm Test Log and TSS Comm Test Incident Report documents is the document "TSS Comm Test Report." There are thin solid arrows extending from TSS Comm Test Log and the TSS Comm Test Incident Report documents to the TSS Comm Test Report. There is separate text towards the bottom right of the slide as follows: "* IEEE 829-2008 allows "anomaly report" to be replaced by problem, test incident, defect, trouble, issue, or error report." )

Graphics: Ralph W. Boaz

Slide 19:

Activity. A placeholder graphic with an image of hand over a computer keyboard to show that an activity is taking place.

Slide 20:

Learning Objective #1

Which of the following is a TRUE statement?

Answer Choices

  1. There is usually one Level Test Case per Level Test Design
  2. Always use the word "Level" in test document names
  3. Anomaly Reports provide a chronological record of tests
  4. A Level Test Report summarizes the results of testing

Slide 21:

Learning Objective #1

Review of Answers

A small graphical red and yellow X representing incorrect.a) There is usually one Level Test Case per Level Test Design
Incorrect. There are typically many Level Test Cases per Level Test Design.

A small graphical red and yellow X representing incorrect.b) Always use the word "Level" in test document names
Incorrect. "Level" refers to the level or type of testing that is to be performed. Replace it with something more descriptive.

A small graphical red and yellow X representing incorrect.c) Anomaly Reports provide a chronological record of tests
Incorrect. Anomaly Reports document any event that occurs during the testing process that requires investigation.

A small graphical green and yellow check mark representing correct.d) A Level Test Report summarizes the results of testing
Correct! A Level Test Report summarizes the results of the designated testing activities.

Slide 22:

Summary of Learning Objective #1

Recognize the Purpose, Structure, and Content of Well-Written Test Documentation Based on IEEE Std 829-2008

Slide 23:

Learning Objective #2: Describe TSS Testing and the Role of Test Documentation Within the Context of the Systems Life Cycle

Slide 24:

Learning Objective #2

Definition of a TSS Within the NTCIP 1209 Standard

Transportation Sensor System (TSS) is defined as any system or device capable of sensing and communicating near real-time traffic parameters using NTCIP.

Slide 25:

Learning Objective #2

Clarification of Terminology

Slide 26:

Learning Objective #2

NTCIP 1209 Detection Architecture

NTCIP 1209 Detection Architecture. Please see the Extended Text Description below.

(Extended Text Description: This slide, entitled "NTCIP 1209 Detection Architecture," contains a graphic illustrating the detection architecture defined by the NTCIP 1209 Standard. The slide is composed of four graphic images. There is a computer terminal located in the middle left area of the slide. The computer terminal has a label above the graphic with the words "Management Station." To the right of the computer terminal are three graphic images arranged vertically down the center of the slide. The top most graphic is one of a traffic controller connected to detectors as described in Slide 44. There is a label above this graphic with the words "Traffic Controllers." The graphic beneath the traffic controller is a video detection camera which is shown as a cylindrical device with a camera lens at one end. The camera has a label above the graphic with the word "Video." The graphic beneath the video detection camera is a rectangle containing the words "Radar, Magnetometer, Acoustic, Etc." The rectangle has a label above the graphic with the words "Other Technologies." There are solid lines connecting the computer terminal to the other three graphics. There is a legend at the bottom left of the slide indicating that the solid lines identify "NTCIP TSS Communications." The slide is animated. As the speaker discusses the different sensing technologies ovals appear around items as follows: 1) Traffic Controllers and Loop Detectors; 2) Video; 3) Radar; and 4) Magnetometer.)

Graphics: Ralph W. Boaz

Slide 27:

Learning Objective #2

Types of Testing Mentioned in IEEE Std 8292008

Types of Testing Mentioned in IEEE Std 8292008. Please see the Extended Text Description below.

(Extended Text Description:

Note: A red oval appears over "Acceptance/Qualification Testing.")

Slide 28:

Learning Objective #2

Acceptance/Qualification Testing of a TSS

Acceptance/Qualification Testing of a TSS. Please see the Extended Text Description below.

(Extended Text Description:

Note: a red oval appears over "Communications.")

Slide 29:

Learning Objective #2

Stages for NTCIP Communication Testing of a TSS (suggested)

  1. Bench Communications Testing
  2. Integrated Communications Testing
  3. Field Communications Testing

Slide 30:

Learning Objective #2

Bench Communications Testing for a TSS

Slide 31:

Learning Objective #2

Integrated Communications Testing for a TSS

Slide 32:

Learning Objective #2

Field Communications Testing for a TSS

Slide 33:

Learning Objective #2

Testing TSS Communications in the Context of the Systems Life Cycle

Testing TSS Communications in the Context of the Systems Life Cycle. Please see the Extended Text Description below.

(Extended Text Description: Author's relevant description: This slide, entitled "Testing TSS Communications in the Context of the Systems Life Cycle," contains a graphic that illustrates the Systems Life Cycle in the form of the standard Vee diagram. The life-cycle steps include the following: Regional Architectures, Feasibility Study / Concept Exploration, Concept of Operations, System Requirements, High-level Design, Detailed Design, Software / Hardware Development Field Installation, Unit Device Testing, Subsystem Verification, System Verification & Deployment, System Validation, Operations and Maintenance, Changes and Upgrades, Retirement / Replacement. An arrow runs down the bottom left of the Vee and is labeled "Decomposition and Definition". The bottom of the Vee has a label that says "Implementation" and a timeline arrow running from left to right is labeled "Development Process". Another arrow runs up the right underside of the Vee and is labeled "Integration and Recomposition".)

Slide 34:

Learning Objective #2

Testing TSS Communications in the Context of the Systems Life Cycle (cont.)

Slide 35:

Learning Objective #2

Testing a TSS in the Context of the Systems Life Cycle

Testing a TSS in the Context of the Systems Life Cycle. Please see the Extended Text Description below.

(Extended Text Description: This slide, entitled "Testing a TSS in the Context of the Systems Life Cycle," contains a graphic showing the bottom portion of the systems life cycle "Vee" diagram from slide 33 which in effect makes a smaller "vee". This bottom portion includes the five processes "High-Level Design," "Detailed Design," "Software/Hardware Development Field Installation," "Unit/Device Testing" and "Subsystem Verification." It is located in the left half of the slide with the title above it "Subsystem Development." On the right side of the slide at the level of "Unit/Device Testing" is the text "Bench Comm Testing." On the right side of the slide at the level of "Subsystem Verification" are two sets of text "Field Comm Testing" and "Integrated Comm Testing." Above the text items is the title, "Communications Testing.")

Slide 36:

Activity. A placeholder graphic with an image of hand over a computer keyboard to show that an activity is taking place.

Slide 37:

Learning Objective #2

When bench testing the communications for a TSS, the primary objective is to:

Answer Choices

  1. Test the TSS communications with other components
  2. Exercise as much of the NTCIP 1209 protocol as possible
  3. Test the TSS communications under real-world conditions
  4. Test the central system communications to the TSS

Slide 38:

Learning Objective #2

Review of Answers

A small graphical red and yellow X representing incorrect.a) Test the TSS communications with other components
Incorrect. Testing other components is a part of integrated testing.

A small graphical green and yellow check mark representing correct.b) Exercise as much of the NTCIP 1209 protocol as possible
Correct! This is usually performed using a software tool.

A small graphical red and yellow X representing incorrect.c) Test the TSS communications under real-world conditions
Incorrect. Testing under real-world conditions is a part of field communications testing.

A small graphical red and yellow X representing incorrect.d) Test the central system communications to the TSS
Incorrect. Testing using the central system communications with the TSS is part of integrated testing and field testing.

Slide 39:

Summary of Learning Objective #2

Describe TSS Testing and the Role of Test Documentation Within the Context of the Systems Life Cycle

Slide 40:

Learning Objective #3: Identify a Process to Develop Test Documentation for a TSS Specification Based on NTCIP 1209 Standard v02

Slide 41:

Learning Objective #3

Key Elements of NTCIP 1209 Standard v02 and the Agency Specification

Slide 42:

Learning Objective #3

Developing Test Documentation Based on an NTCIP 1209 v02 Specification

Developing Test Documentation Based on an NTCIP 1209 v02 Specification. Please see the Extended Text Description below.

(Extended Text Description: This slide, entitled "Developing Test Documentation Based on an NTCIP 1209 v02 Specification," contains a flowchart where rectangles represent processes and diamonds (rhombuses) represent decision points. The right point of the diamond is labeled with an "N" for "no" and the bottom point of the diamond is labeled "Y" for "yes." There are three vertical areas or levels on the slide. The first (top) level has four processes evenly spaced across the slide. They are labeled "Develop TSS Comm Test Plan," "Develop TSS Comm B/I/F* Test Design," "Develop TSS Comm Test Case," and "Develop TSS Comm Test Procedure" respectively. The second level has three diamonds, one diamond centered below each of the first three processes. They are labeled "Do Test Designs cover Test Plan?," "Are Test Cases sufficient for purpose?," and "Are Test Procedures sufficient for purpose?" respectively. Third level has a single process centered below the first diamond labeled "Test Execution." The flow is shown using elbow-type connector arrows between the items as follows:

)

Graphics: Ralph W. Boaz

Slide 43:

Learning Objective #3

Develop a TSS Communications Test Plan

Develop a TSS Communications Test Plan. Please see the Extended Text Description below.

(Extended Text Description: This slide, entitled "Develop a TSS Communications Test Plan," has a graphic in the bottom portion of the slide. On the left, there is a graphic that looks like a sheet of paper labeled "Agency TSS Specification." In the middle, there is a thick arrow pointing from the left to the right labeled "PRL: Features to be / not to be tested." On the right, there is a graphic that looks like a sheet of paper labeled "TSS Comm Test Plan.")

Slide 44:

Learning Objective #3

Use the PRL to Identify the Features to Be / Not to Be Tested

Use the PRL to Identify the Features to Be / Not to Be Tested. Please see the Extended Text Description below.

(Extended Text Description: Author's relevant description, for illustration only: This slide, entitled "Use the PRL to Identify the Features to Be / Not to Be Tested," has a graphic of a Protocol Requirements List (PRL) table in the top half of the slide and two sets of text large text in the bottom half of the slide. The PRL has the column headers "User Need Section Number," "User Need," "FR Section Number," "Functional Requirement," "Conformance," "Support / Project Requirement," and "Additional Specifications."  The PRL is only partially filled to be represented of a completed PRL as follows:

The large text on the bottom left of the slide has a title "Features to be Tested" and below the title is the text "2.5.4.1 Retrieve In-Progress Sample Data."  The large text on the bottom right of the slide has a title "Features not to be Tested" and below the title is the text "2.5.5 Multi-Version Interoperability (Backward Compatibility."  There is an arrow from the circled "Yes" for User Need 2.5.4.1 on the table to the text on the left. There is an arrow from the circled "No" for User Need 2.5.5 on the table to the text on the right.)

Slide 45:

Learning Objective #3

Develop TSS Communications Test Designs

Develop TSS Communications Test Designs. Please see the Extended Text Description below.

(Extended Text Description: This slide, entitled "Develop TSS Communications Test Designs," has a graphic in the bottom portion of the slide. On the left, there is a graphic that looks like a sheet of paper labeled "Agency TSS Specification." In the middle, there is a thick arrow pointing from the left to the right labeled "PRL: Features and requirements to be tested for the test design."  On the right, there is a graphic that looks like a sheet of paper labeled "TSS Comm Bench Test Design.")

Slide 46:

Learning Objective #3

Use the PRL to Identify the Features and Requirements

Use the PRL to Identify the Features and Requirements. Please see the Extended Text Description below.

(Extended Text Description: Author's relevant description, for illustration only: This slide, entitled "Use the PRL to Identify the Features and Requirements," has a graphic of a Protocol Requirements List (PRL) table in the top half of the slide and two sets of text large text in the bottom half of the slide. The PRL has the column headers "User Need Section Number," "User Need," "FR Section Number," "Functional Requirement," "Conformance," "Support / Project Requirement," and "Additional Specifications."  The PRL is only partially filled to be represented of a completed PRL as follows:

The large text on the bottom left of the slide has a title "Features to be Tested" and below the title is the text "2.5.4.1 Retrieve In-Progress Sample Data."  The large text on the bottom right of the slide has a title "Requirements to be Tested" and below the title is the text "3.4.3.1.1 Get Historical Sample End Time."  There is an arrow from the circled "Yes" for User Need 2.5.4.1 on the table to the text on the left. There is an arrow from the circled "Yes" for Requirement 3.4.3.1.1 on the table to the text on the right.)

Slide 47:

Learning Objective #3

Develop TSS Communications Test Cases

Develop TSS Communications Test Cases. Please see the Extended Text Description below.

(Extended Text Description: This slide, entitled "Develop TSS Communications Test Cases," has two rows of graphics in the bottom middle and bottom portions of the slide. Describing the first row: On the left, there is a graphic that looks like a sheet of paper labeled "Agency TSS Specification." In the middle, there is a thick arrow pointing from the left to the right labeled "PRL: Requirement to be tested, addt'l specs."  On the right, there is a graphic that looks like a sheet of paper labeled "TSS Comm Test Cases." Describing the second row: On the left, there is a graphic that looks like a sheet of paper labeled "NTCIP 1209 Standard v02." In the middle, there is a thick arrow pointing from the left to the TSS Comm Test Cases graphic labeled "RTM: Data objects to be tested for test case - MIB: Standard values for data objects.")

Slide 48:

Learning Objective #3

Use the PRL, RTM, and MIB to Develop Test Cases

Use the PRL, RTM and MIB to Develop Test Cases. Please see the Extended Text Description below.

(Extended Text Description: Author's relevant description, for illustration only: This slide, entitled "Use the PRL, RTM and MIB to Develop Test Cases," has a graphic of a Protocol Requirements List (PRL) table in the top half of the slide, a Requirements Traceability Matrix (RTM) table in the middle left of the slide, large text large text in the middle right of the slide, and text at the bottom center of the slide. The PRL has the column headers "User Need Section Number," "User Need," "FR Section Number," "Functional Requirement," "Conformance," "Support / Project Requirement," and "Additional Specifications."  The PRL is only partially filled to be represented of a completed PRL as follows:

The RTM has the column headers "Requirement ID," "Requirement," "Dialog ID," "Dialog," "Object ID," and "Object."  The RTM is only partially filled to be represented of a completed RTM as follows:

The large text on the middle right of the slide has a title "Test Case" and below the title is the text "Verify Zone Class Labels."  The text at the bottom of the slide is "zoneClassLabel  OBJECT-TYPE SYNTAX  OCTET STRING (SIZE (1..255))."

There is an arrow from the requirement 3.4.3.1.7 row in the PRL to the requirement 3.4.3.1.7 row in the RTM. There is an arrow from the dialog 4.3.3.5 row in the RTM to the "Verify Zone Class Labels" text to the right. There is an arrow from the "ZoneClassLabel OBJECT-TYPE …" text at the bottom of the slide to the "Verify Zone Class Labels" text to the right.)

Slide 49:

Learning Objective #3

Develop TSS Communications Test Procedures

Develop TSS Communications Test Procedures. Please see the Extended Text Description below.

(Extended Text Description: This slide, entitled "Develop TSS Communications Test Procedures," has two rows of graphics in the bottom middle and bottom portions of the slide. Describing the first row:
On the left, there is a graphic that looks like a sheet of paper labeled "Agency TSS Specification." In the middle, there is a thick arrow pointing from the left to the right labeled "PRL: Requirement to be tested, addt'l specs."  On the right, there is a graphic that looks like a sheet of paper labeled "TSS Comm Test Procedures." Describing the second row: On the left, there is a graphic that looks like a sheet of paper labeled "NTCIP 1209 Standard v02." In the middle, there is a thick arrow pointing from the left to the TSS Comm Test Procedures graphic labeled "RTM: Data objects & dialogs to be tested - MIB: Standard values for data objects.")

Slide 50:

Learning Objective #3

Use the PRL, RTM, and MIB to Develop Test

Procedures

Use the PRL, RTM, and MIB to Develop Test Procedures. Please see the Extended Text Description below.

(Extended Text Description: Author's relevant description, for illustration only: This slide, entitled "Use the PRL, RTM, and MIB to Develop Test Procedures," has a graphic of a Requirements Traceability Matrix (RTM) table in the upper left of the slide, text in the lower left of the slide, text to the right of the RTM, and large text in the bottom right of the slide. The RTM is as described in Slide #48 above. The text in the lower left of the slide describes the ordered steps for dialog 4.3.3.5 Retrieve Sensor Zone Class Labels. The text to the right of the RTM is "zoneClassLabel  OBJECT-TYPE SYNTAX  OCTET STRING (SIZE (1..255))." The large text on the bottom right of the slide has a title "Test Procedure" and below the title is the text "Verify Zone Class Labels Procedure." There is an arrow from the dialog 4.3.3.5 row in the RTM to text of the dialog in the lower left of the slide.  There is an arrow from the dialog 4.3.3.5 row in the RTM to the "Verify Zone Class Labels" text to the right. There is an arrow from the "5.4.5.1 zoneClassLabel" row in the RTM to the object definition text to the right of the RTM. There is an arrow from the object definition to the text in the lower right of the slide. There is an arrow from the bottom of the text of the dialog in the lower left of the slide to the "Verify Zone Class Labels Procedure" text in the lower right of the slide. The text in the lower left side reads, Where: x = sensorZoneNumber; y = sampleZoneClass - a)(Precondition) The management station shall determine that the sensorZoneNumberis less than or equal to the maxSensorZones. The TSS supports sampling features - b)The management station shall GET zoneSequenceTableEntry:numSensorZoneClass.x - c)sampleZoneClass= zoneSequenceTableEntry:numSensorZoneClass.xfrom Step b - d)The management station shall GET zoneClassLabel.y.x - e)If zoneClassEntryis greater than 0, then zoneClassEntry= zoneClassEntry-1 and go to Step d - f)Retrieval of class labels for this sensor zone is complete.)

Slide 51:

Learning Objective #3

Create a Test Traceability Matrix (TTM)

Req ID Req Test Case ID Test Case Test Proc ID Test Procedure
3.4.3.1.7 Get Zone Class Label
    TC3.4.3. 1.7-1 Verify Zone Class Labels
        TP3.4.3. 1.7-1 Verify Zone Class Labels Procedure
3.4.3.1.8 Get Number of Sample Data Entries
    TC3.4.3. 1.8-1 Get Number of Sample Data Entries Nominal
        TP3.4.3. 1.8-1 Get Number of Sample Data Entries Nominal Procedure 1
        TP3.4.3. 1.8-2 Get Number of Sample Data Entries Nominal Procedure 2

Slide 52:

Learning Objective #3

Create a Test Traceability Matrix (TTM) (cont.)

Create a Test Traceability Matrix (TTM) (cont.). Please see the Extended Text Description below.

(Extended Text Description: This slide, entitled "Create a Test Traceability Matrix (TTM)," contains a Test Traceability Matrix (TTM) from slide 51. The "Test Case ID" column is has a rounded rectangle highlighting the column.)

Slide 53:

Learning Objective #3

Create a Test Traceability Matrix (TTM) (cont.)

Create a Test Traceability Matrix (TTM) (cont.). Please see the Extended Text Description below.

(Extended Text Description: This slide, entitled "Create a Test Traceability Matrix (TTM)," contains a Test Traceability Matrix (TTM) from slide 51. The "Test Case" header and the test case names "Verify Zone Class Labels" and "Get Number of Sample Data Entries Nominal" have rounded rectangles highlighting them.)

Slide 54:

Learning Objective #3

Create a Test Traceability Matrix (TTM) (cont.)

Create a Test Traceability Matrix (TTM) (cont.). Please see the Extended Text Description below.

(Extended Text Description: This slide, entitled "Create a Test Traceability Matrix (TTM)," contains a Test Traceability Matrix (TTM) from slide 51. The "Test Proc ID" column is has a rounded rectangle highlighting the column.)

Slide 55:

Learning Objective #3

Create a Test Traceability Matrix (TTM) (cont.)

Create a Test Traceability Matrix (TTM) (cont.). Please see the Extended Text Description below.

(Extended Text Description: This slide, entitled "Create a Test Traceability Matrix (TTM)," contains a Test Traceability Matrix (TTM from slide 51). The "Test Procedure" header and the test procedure names "Verify Zone Class Labels Procedure," "Get Number of Sample Data Entries Nominal Procedure 1" and "Get Number of Sample Data Entries Nominal Procedure 2" have rounded rectangles highlighting them.)

Slide 56:

Learning Objective #3

Describe Test Tools Available for NTCIP Communications

Slide 57:

Learning Objective #3

Describe Test Tools Available for NTCIP Communications (cont.)

Slide 58:

Activity. A placeholder graphic with an image of hand over a computer keyboard to show that an activity is taking place.

Slide 59:

Learning Objective #3

True or False: The best way to start developing your test documentation is with a Test Design.

Answer Choices

  1. True
  2. False

Slide 60:

Learning Objective #3

Review of Answers

A small graphical red and yellow X representing incorrect.a) True
Incorrect. The test design is developed after the test plan and it may only cover a portion of the testing to be performed.

A small graphical green and yellow check mark representing correct.b) False
Correct! The test plan (not the test design) is the best starting point for developing test documentation. It covers the scope, approach, resources, and schedule for testing.

Slide 61:

Activity. A placeholder graphic with an image of hand over a computer keyboard to show that an activity is taking place.

Slide 62:

Learning Objective #3

What is the most appropriate test document in which to include a Test Traceability Matrix

(TTM)?

Answer Choices

  1. TSS Communications Test Cases
  2. TSS Communications Test Procedures
  3. TSS Communications Test Design
  4. TSS Communications Test Report

Slide 63:

Learning Objective #3

Review of Answers

A small graphical red and yellow X representing incorrect.a) TSS Communications Test Cases
Incorrect. A test case defines the input and output specifications for a test identified in a test design.

A small graphical red and yellow X representing incorrect.b) TSS Communications Test Procedures
Incorrect. A test procedure specifies the steps for executing one or more test cases.

A small graphical green and yellow check mark representing correct.c) TSS Communications Test Design
Correct! A test design identifies the features to be tested and the test cases associated with the design.

A small graphical red and yellow X representing incorrect.d) TSS Communications Test Report
Incorrect. A test report summarizes the results of the designated testing activities.

Slide 64:

Summary of Learning Objective #3

Identify a Process to Develop Test Documentation for a TSS Specification Based on the NTCIP 1209 Standard v02

Slide 65:

Learning Objective #4: Describe the Testing of a TSS Using Sample Test Documentation

See the Participant Supplement for the outlines of the Test Reporting documents.

Slide 66:

Learning Objective #4

Variations

Slide 67:

Learning Objective #4

Example of TSS Deployment

The city of Buena Vida, CA, has a central system that uses NTCIP communications to communicate with their signal systems and dynamic message signs. The city wants to be able to manage its video detection systems from the same central system. The city developed a TSS Communications Specification based on NTCIP 1209 Standard v02. One of the current equipment providers, ICU Detection Systems, implemented NTCIP 1209 Standard v02 for its video detection system products. The city needs to test the communications to verify compliance to the specification.

Slide 68:

Learning Objective #4

Test Specification Documents

Test Specification Documents. Please see the Extended Text Description below.

(Extended Text Description: This slide, entitled "Test Specification Documents," has the same graphic as Slide 15.)

Graphics: Ralph W. Boaz

Slide 69:

Learning Objective #4

TSS Communications Test Plan

1 Introduction

This document is a TSS Communications Test Plan as part of the Buena Vida Central System Project. This Introduction includes the following subsections: Document Identifier, Scope, References, etc.

1. 1 Document Identifier

TSSCommTP v01.04

TSS Communications Test Plan v01.04

30 March 2015

1.2 Scope

This test plan covers the communications for video detection devices as part of an overall acceptance process for traffic control field equipment under the Buena Vida Central System Project.

Slide 70:

Learning Objective #4

TSS Communications Test Plan (cont.)

1.3 References

Buena Vida TSS Communications Specification Buena Vida Central System Project Management Plan Buena Vida Video Detection Acceptance Test Plan NTCP 1209 Standard v02 IEEE Std 829-2008

1.4 Level in the Overall Sequence

TSS Communications Test Plan. Please see the Extended Text Description below.

(Extended Text Description: This slide, entitled "TSS Communications Test Plan," has a graphic in paragraph "1.4 Level in the Overall Sequence." There is a rectangle labeled "Video Detection Acceptance Test Plan." There is a second rectangle labeled "TSS Communications Test Plan." There is a solid slanted line between the two boxes.)

1.5 Test Classes and Overall Test Conditions

This test plan covers the NTCIP communications for a TSS. Testing will be performed in both a laboratory environment with test software and in integrated testing with the city's central system both in the lab and in the field.

Slide 71:

Learning Objective #4

TSS Communications Test Plan (cont.)

2 Details for this level of test plan

2.1 Test items and their identifiers

VIDS Model 2020 from ICU Detection Systems

VIDS Model 2020 User's Manual

VIDS Model 2020 Installation Guide

2.2 Test Traceability Matrix

[Including this in the TSS Communications Test Designs]

2.3 Features to be tested

2.5.1 Configure the TSS

2.5.1.1 Determine the TSS Identity

2.5.1.2 TSS Capabilities

2.5.1.2.1 Determine TSS Support for Sampling

Slide 72:

Learning Objective #4

TSS Communications Test Plan (cont.)

2.4 Features not to be tested

2.5.5 Multi-Version Interoperability (MVI-Backward Compatibility)

2.5.5.1 NTCIP 1209:2005 (v01) Conformant

2.5.5.1.1 Retrieve NTCIP 1209:2005 (v01) Conformant Most Recent Sample Data

2.5 Approach

This testing will include: a) bench testing the TSS in the city's lab using a test tool, b) integrating the TSS with the city's system within the lab and exercising the communications using the central system software, and c) field testing the TSS at the city's test intersection using the central software.

2.6 Item pass/fail criteria

The test item will be considered to have passed if it successfully completes all test cases and their associated test procedures.

Slide 73:

Learning Objective #4

TSS Communications Test Plan (cont.)

2.7 Suspension criteria and resumption requirements

There are four major feature areas of the TSS communications: a) Configure the TSS, b) Control the TSS, c) Monitor the TSS and d) Collect data from the TSS. If there are more than five failed test cases in any one major area, testing for the area will be suspended until the issues are resolved.

2.8 Test deliverables

TSS CommTest Plan
TSS Comm Test Designs
TSS Comm Test Cases
TSS Comm Test Procedures
TSS Comm Test Logs
TSS Comm Test Incident Reports
TSS Comm Interim Test Status Reports
TSS Comm Test Reports (one for each test design)

Slide 74:

Learning Objective #4

TSS Communications Test Plan (cont.)

3 Test management

3.1 Planned activities and tasks; test progression

The TSS Comm testing will not be performed until the item under test passes the Video Detection Acceptance Test Plan.

3.2 Envi ronment/i nfrastructure

For bench TSS communications testing, the city's lab will be equipped with a laptop running NTCIP Tester Software, loaded with the NTCIP 1209 MIB, test cases, and test procedures. For integration communications testing, ...

3.3 Responsibilities and authority

The overall TSS communications testing is under the management of Jane Doe, the city's Deputy Traffic Engineer. Testing Consultants, Ltd, has been hired to produce the test documents and carry out the testing process.

Slide 75:

Learning Objective #4

TSS Communications Test Plan (cont.)

3.4 Interfaces among the parties involved

No additional interfaces required.

3.5 Resources and their allocation

No additional resources required.

3.6 Training

No additional training required.

3.7 Schedules, estimates, and costs

The time frame for TSS Communications Testing is determined by the project schedule in the Buena Vida Central System Project Management Plan.

3.8 Risk(s) and contingency(s)

Risks are being tracked as part of the project risk management plan.

Slide 76:

Learning Objective #4

TSS Communications Test Plan (cont.)

4 General

4.1 Quality assurance procedures

The testing quality will fall under the Testing QA Procedures established by the city and Testing Consultants, Ltd.

4.2 Metrics

The percentage of test cases passed per test design will be recorded.

4.3 Test coverage

All data elements specified by the PRL and RTM shall be included in at least one test using nominal values.

4.4 Glossary

4.5 Document change procedures and history

Slide 77:

Learning Objective #4

TSS Communications Bench Test Design

1 Introduction

1.1 Document identifier

TSSCommBenchTD v01.02

TSS Communications Bench Test Design v01.02

30 March 2015

1.2 Scope

This test design describes the TSS bench communications testing for the TSS Communications Test Plan.

1.3 References

Slide 78:

Learning Objective #4

TSS Communications Bench Test Design (cont.)

2 Details of the Level Test Design

2.1 Features to be tested

2.5.1 Configure the TSS

2.5.1.1 Determine the TSS Identity

2.5.1.2 TSS Capabilities

2.5.1.2.1 Determine TSS Support for Sampling

2.2 Approach refinements

The bench testing will be conducted using laptop running NTCIP Tester Software, loaded with the NTCIP 1209 MIB, test cases, and test procedures. The TestCo NTCIP Tester Software will be used to drive the testing and capture results.

2.3 Test identification

[Test Traceability Matrix]

Slide 79:

Learning Objective #4

TSS Communications Bench Test Design (cont.)

2.4 Feature pass/fail criteria

The test item will be considered to have passed the testing under this test design if it successfully completes all test cases and their associated test procedures.

2.5 Test deliverables

TSS Comm Bench Test Design

Associated test documents resulting from this test design:

Slide 80:

Learning Objective #4

TSS Communications Bench Test Design (cont.)

3 General

3.1 Glossary

3.2 Document change procedures and history

Slide 81:

Learning Objective #4

TSS Communications Test Cases

1 Introduction (once per document)

1.1 Document identifier

TSSCommBenchTC v01.02

TSS Communications Test Cases v01.02

30 March 2015

1.2 Scope

This document contains all of the test cases used for the TSS Communications Test Design.

1.3 References

1.4 Context

No additional context necessary.

Slide 82:

Learning Objective #4

TSS Communications Test Cases (cont.)

1.5 Notation for description

Test cases have identifiers that are associated with a feature being tested having the form TCf-n.vv, where: TC indicates the identifier is a test case, f indicates the feature paragraph number within the NTCIP 1209 Standard, -n is a sequential number uniquely identifying the test case for the feature, and .vv is the two digit version number of the test case.

Slide 83:

Learning Objective #4

TSS Communications Test Cases (cont.)

2 Details (once per test case)

2.1 Test case identifier

TC3.4.3.1.7-1.03 Verify Zone Class Labels

TSS Comm Test Case 1 Version 03 for Feature 3.4.3.1.7

30 March 2015

2.2 Objective

This test case validates that the TSS has zone classification labels according to the city's specification.

2.3 Inputs

File containing the vehicle classification labels from the city specification: "Classes 11-13," "Classes 8-10," "Classes 47," "Classes 2-3," and "Class 1." Each entry is on a single line in file ClassLabels.txt.

Slide 84:

Learning Objective #4

TSS Communications Test Cases (cont.)

2.4 Outcome(s)

Self-validating test case

2.5 Environmental needs

Windows 7 PC with i5 Intel CPU @ 2 GHz processor, 8 GB RAM, and 200 GB available storage

TestCo NTCIP Tester Software v02.05

2.6 Special procedural requirements

None

2.7 Intercase dependencies

None

Slide 85:

Learning Objective #4

TSS Communications Test Cases (cont.)

3 Global

3.1 Glossary

3.2 Document change procedures and history

Slide 86:

Learning Objective #4

TSS Communications Test Procedures

1 Introduction

1.1 Document identifier

TP3.4.3.1.7-1.08 Verify Zone Class Labels Procedure

TSS Comm Test Procedure 1 Version 08 for Feature 3.4.3.1.7

17 May 2015

1.2 Scope

This test procedure is used to validate that the TSS has zone classification labels according to the city's specification.

1.3 References

TC3.4.3.1.7-1.03

1.4 Relationship to other procedures

None

Slide 87:

Learning Objective #4

TSS Communications Test Procedures (cont.)

2 Details

2.1 Inputs, outputs, and special requirements

Inputs: See TC3.4.3.1.7-1.02

Outputs: Logs, Pass/Fail

2.2 Ordered description of the steps to be taken

Step 1: Open FILE = ClassLabels.txt;

Step 2: SZCLASSENTRY = Get zoneSequenceTableEntry:numSensorZoneClass.0;

Step 3: Read FILEENTRY from FILE;

Step 4: SZCLASSLABEL = Get zoneClassLabel.SZCLASSENTRY.0;

Step 5: If (FILEENTRY = SZCLASSLABEL) Then log ("Step:6 Pass Label", SZCLASSLABEL; Pass) Else log ("Step:6 Pass Label", SZCLASSLABEL; Fail);

Step 6: If (SZCLASSLABEL > 0) Then SZCLASSLABEL = SZCLASSLABEL - 1

Goto Step 3

Slide 88:

Learning Objective #4

TSS Communications Test Procedures (cont.)

3 General

3.1 Glossary

3.2 Document change procedures and history

Slide 89:

Activity. A placeholder graphic with an image of hand over a computer keyboard to show that an activity is taking place.

Slide 90:

Learning Objective #4

Which of the following is a TRUE statement?

Answer Choices

  1. Only manufacturers need to be concerned with testing
  2. Well-written agency TSS specifications facilitate testing
  3. Good testing is easy
  4. The only thing that matters is the level test report

Slide 91:

Learning Objective #4

Review of Answers

A small graphical red and yellow X representing incorrect.a) Only manufacturers need to be concerned with testing
Incorrect. Agencies and their consultants cannot rely on the manufacturer to be sure that a product meets the specification.

A small graphical green and yellow check mark representing correct.b) Well-written agency TSS specifications facilitate testing
Correct! This has been demonstrated through this module and the entire Professional Capacity Building (PCB) program.

A small graphical red and yellow X representing incorrect.c) Good testing is easy
Incorrect. Good testing can be difficult and tedious but it is necessary. In the long run, it saves the agency money, reputation, and shows public accountability.

A small graphical red and yellow X representing incorrect.d) The only thing that matters is the level test report
Incorrect. While some people want to hold to the bottom line, understanding and performing good testing practices are essential.

Slide 92:

Summary of Learning Objective #4

Describe the Testing of a TSS Using Sample Test Documentation

Slide 93:

What We Have Learned

  1. IEEE 829-2008 provides for test documents that are used in test specification and test reporting.
  2. There are many types of testing can be performed on a TSS. When testing a TSS for compliance to an NTCIP 1209-based specification, we are concerned with communications .
  3. Both NTCIP 1209 Standard and the agency's specification are critical in developing good test documentation.
  4. A Test Traceability Matrix is a key element of a TSS Communications Test Design.
  5. In practice, the IEEE Std 829-2008 recommended document outlines may be tailored to improve their effectiveness on a given project.

Slide 94:

Resources

Slide 95:

Questions? A placeholder graphic image with word Questions? at the top, and an image of a lit light bulb on the lower right side.