You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 45 Next »

Welcome to the TM Forum Open API Conformance Testing and Validation Instructions Page!

The following guidelines will take you through the various steps needed to run self-testing for your API to achieve conformance certification.

Self-Certification is based on the TM Forum Open API conformance profiles, it is the process by which an implementation is confirmed to support the standard definition of a TM Forum Open API.  The TM Forum provides a set of scripts useful to self-certify your implementation of TM Forum Open API specifications. You will need to install and run the applicable API CTK. Currently there is a number of CTKs available for conformance testing and the list is steadily increasing. Currently available conformance profiles and CTKs can be found through the following links: 

The following instructions describe how to install an Open API CTK 

Tools needed for Open API CTK testing and validation:

Newman (needed for CTK test execution), this is the command line version of 'Postman'; installation instructions available at: https://www.getpostman.com/docs/newman_intro

Newman is built on Node.js. To run Newman, make sure you have Node.js installed. Node.js can be downloaded and installed from here on Linux, Windows, and Mac OSX.

Node.js offers a server-side environment, which allows the use of JavaScript language to generate web pages. Basically, it replaces server languages such as PHP, Java EE, etc. Once Node.js is installed, Newman is just a command away:

npm install -g newman

This installs Newman from npm globally on your system allowing you to run it from anywhere.

npm makes it easy for JavaScript developers to share and reuse code, and it makes it easy to update the code that you're sharing. (You can download npm from here).

Once Node.js and Newman are installed follow the next steps: 

1  Import CTK from TM Forum

Download and unzip postman collection file for selected API from the corresponding page (see CTK links provided above)

Store downloaded file into the folder in your system where Newman runs

2  Configure environment

Download environment file (TMFENV) from the following link: TMFENV File

Modify environment file to refer to the endpoint where the implementation to be certified is running (see step 3 below)

Store modified file into the folder in your system where Newman runs

3  You need to create your testing environment. The environment is a text file without extension written in JSON. It has the following format:


You can use this as a template. Create a file with this format, replace the values for the name, the host and your API addresses and put it in the same folder as the postman collections.

4  Now create the right script or batch file for your platform (either Windows or Linux/Mac), put it in the same folder and run script execution command. For example, you can use a batch file (runtest.bat) which would look like the following example:
 

set http_proxy=http://proxyinternet.tesa:8080
newman -c DSTestedAPITestCollection.postman_collection -e TMFENV -H DSTestedAPICTKResult.html -o DSTestedAPICTKResult.json
pause >nul

5  A folder name results will be created and filled with the output of the tests. You should see something like the following: 


If they are no failures, then you have passed the CTK and your API is conformant with all the Mandatory features.

The results of the CTK are in the DSTestedAPICTKResult.html and DSTestedAPICTKResult.json files and should look like the following (example with Customer Management API):


 

All the information related to the execution of the CTK will be contained in the DSTestedAPICTKResult.json file.

 The following is a complete example for the Trouble Ticket API:

 

Newman test report

Collection: TMForum - TroubleTicket API CTK

Iterations: 1

Description:

Requests:

 

troubleTicket/{{id}} 404

Description:

Mean time per request: 6123 ms

Total tests that passed: 2

Total tests that failed: 0

Individual tests:

Test name

Pass count

Fail count

Content-Type is present application/json

1

0

Status code is 404

1

0

{apiRoot}/productOffering

Description: retrieve a product offering

Mean time per request: 3675 ms

Total tests that passed: 2

Total tests that failed: 0

Individual tests:

Test name

Pass count

Fail count

Content-Type is present application/json

1

0

Status code is an error

1

0

{apiRoot}/productOffering

Description: retrieve a product offering

Mean time per request: 4320 ms

Total tests that passed: 2

Total tests that failed: 0

Individual tests:

Test name

Pass count

Fail count

Content-Type is present text/plain

1

0

Status code is an error

1

0

/troubleTicket

Description: create a trouble ticket

Mean time per request: 3863 ms

Total tests that passed: 5

Total tests that failed: 0

Individual tests:

Test name

Pass count

Fail count

Status code is 201

1

0

Content-Type is present application/json

1

0

Response contains ID

1

0

Response contains all required fields

1

0

POST Body Response equals Request Body

1

0

/troubleTicket

Description: retrieve /troubleTickets

Mean time per request: 7652 ms

Total tests that passed: 5

Total tests that failed: 0

Individual tests:

Test name

Pass count

Fail count

Content-Type is present application/json

1

0

Status code is 200

1

0

Response contains all required fields

1

0

Response contains created resource

1

0

Element contained equals previous creation

1

0

/troubleTicket/{{troubleTicketId}}

Description: retrieve a troubleTicket

Mean time per request: 1300 ms

Total tests that passed: 6

Total tests that failed: 0

Individual tests:

Test name

Pass count

Fail count

Content-Type is present application/json

1

0

Status code is 200

1

0

Response contains ID

1

0

Response contains all required fields

1

0

ID is correct

1

0

Body Response equals previous creation

1

0

/troubleTicket

Description: create a trouble ticket

Mean time per request: 1315 ms

Total tests that passed: 5

Total tests that failed: 0

Individual tests:

Test name

Pass count

Fail count

Status code is 201

1

0

Content-Type is present application/json

1

0

Response contains ID

1

0

Response contains all required fields

1

0

POST Body Response equals Request Body

1

0

/troubleTicket

Description: retrieve /troubleTickets

Mean time per request: 5085 ms

Total tests that passed: 5

Total tests that failed: 0

Individual tests:

Test name

Pass count

Fail count

Content-Type is present application/json

1

0

Status code is 200

1

0

Response contains all required fields

1

0

Response contains created resource

1

0

Element contained equals previous creation

1

0

/troubleTicket/{{troubleTicketId}}

Description: retrieve a troubleTicket

Mean time per request: 4147 ms

Total tests that passed: 6

Total tests that failed: 0

Individual tests:

Test name

Pass count

Fail count

Content-Type is present application/json

1

0

Status code is 200

1

0

Response contains ID

1

0

Response contains all required fields

1

0

ID is correct

1

0

Body Response equals previous creation

1

0

/troubleTicket

Description: retrieve troubleTickets

Mean time per request: 4481 ms

Total tests that passed: 7

Total tests that failed: 0

Individual tests:

Test name

Pass count

Fail count

Content-Type is present application/json

1

0

Status code is 200

1

0

Response contains all required fields

1

0

Response contains created resource 1

1

0

Element contained equals previous creation 1

1

0

Response contains created resource 2

1

0

Element contained equals previous creation 2

1

0

/troubleTicket?severity=High

Description: retrieve tickets with high severity

Mean time per request: 2135 ms

Total tests that passed: 6

Total tests that failed: 0

Individual tests:

Test name

Pass count

Fail count

Content-Type is present application/json

1

0

Status code is 200

1

0

Response contains all required fields

1

0

Response does not contain created resource 2

1

0

Response contains created resource 1

1

0

Element contained equals previous creation 1

1

0

/troubleTicket?type=connectivity

Description: retrieve tickets with connectivity problems

Mean time per request: 1560 ms

Total tests that passed: 6

Total tests that failed: 0

Individual tests:

Test name

Pass count

Fail count

Content-Type is present application/json

1

0

Status code is 200

1

0

Response contains all required fields

1

0

Response does not contain created resource 1

1

0

Response contains created resource 2

1

0

Element contained equals previous creation 2

1

0

troubleTicket/{{id}}?fields=description

Description: retrieve the product offering that are active

Mean time per request: 887 ms

Total tests that passed: 3

Total tests that failed: 0

Individual tests:

Test name

Pass count

Fail count

Content-Type is present application/json

1

0

Status code is 200

1

0

Element contained equals previous creation

1

0

troubleTicket/{{id}}?fields=severity,status

Description: retrieve the product offering that are active

Mean time per request: 620 ms

Total tests that passed: 3

Total tests that failed: 0

Individual tests:

Test name

Pass count

Fail count

Content-Type is present application/json

1

0

Status code is 200

1

0

Element contained equals previous creation

1

0

troubleTicket?severity=High&fields=description

Description: retrieve the product offering that are active

Mean time per request: 598 ms

Total tests that passed: 5

Total tests that failed: 0

Individual tests:

Test name

Pass count

Fail count

Content-Type is present application/json

1

0

Status code is 200

1

0

Response contains created resource 1

1

0

Element contained equals previous creation

1

0

Response does not contain created resource 2

1

0

When you have completed these tests successfully, please send your test results (html and json results files) to [email protected], as they will be posted to our TM Forum Self Certification Web Page. 

The following link points to the TM Forum Open API Self Certification Web Page in which conformance results are published: TM Forum Open API Self Certification Web Page

You can also submit a link to your webpage along with your test results.


  • No labels