...
...
...
...
...
...
...
...
Trouble Ticket API Conformance Profile R16.5.0 (API REST Specification TMF661)
...
in our TM Forum Open API portal.
Appears in the following playbooks
...
The following instructions describe how to install and run an Open API CTK:
Tools needed for Open API CTK testing and validation:
Newman (needed for CTK test execution), this is the command line version of 'Postman'; installation instructions available at: https://www.getpostman.com/docs/newman_intro
...
(This example is covering Product Offering Qualification for the 18.0 Release, but the steps to run a CTK are similar, unless stated on the readme.md that comes with the CTK).
1.
Installing and Running Conformance Test Kit
The CTK is dependent on the installation of node.js and npm to work.
Node.js and NPM
...
can be downloaded and installed from
...
...
Node.js offers a server-side environment, which allows the use of JavaScript language to generate web pages. Basically, it replaces server languages such as PHP, Java EE, etc. Once Node.js is installed, Newman is just a command away:
npm install -g newman
This installs Newman from npm globally on your system allowing you to run it from anywhere.
...
Once Node.js and
...
npm are installed
...
1 Import CTK from TM Forum
Download and unzip postman collection file for selected API from the corresponding page (see CTK links provided above)
Store downloaded file into the folder in your system where Newman runs
2 Configure environment
Download environment file (TMFENV) from the following link: TMFENV File
Modify environment file to refer to the endpoint where the implementation to be certified is running (see step 3 below)
Store modified file into the folder in your system where Newman runs
3 You need to create your testing environment. The environment is a text file without extension written in JSON. It has the following format:
You can use this as a template. Create a file with this format, replace the values for the name, the host and your API addresses and put it in the same folder as the postman collections.
4 Now create the right script or batch file for your platform (either Windows or Linux/Mac), put it in the same folder and run script execution command. For example, you can use a batch file (runtest.bat) which would look like the following example:
set http_proxy=http://proxyinternet.tesa:8080
newman -c DSTestedAPITestCollection.postman_collection -e TMFENV -H DSTestedAPICTKResult.html -o DSTestedAPICTKResult.json
pause >nul
5 A folder name results will be created and filled with the output of the tests. You should see something like the following:
If they are no failures, then you have passed the CTK and your API is conformant with all the Mandatory features.
The results of the CTK are in the DSTestedAPICTKResult.html and DSTestedAPICTKResult.json files and should look like the following (example with Customer Management API):
All the information related to the execution of the CTK will be contained in the DSTestedAPICTKResult.json file.
The following is a complete example for the Trouble Ticket API:
Newman test report
Collection: TMForum - TroubleTicket API CTK
Iterations: 1
Description:
Requests:
Description:
Mean time per request: 6123 ms
Total tests that passed: 2
Total tests that failed: 0
Individual tests:
Test name
Pass count
Fail count
Content-Type is present application/json
1
0
Status code is 404
1
0
Description: retrieve a product offering
Mean time per request: 3675 ms
Total tests that passed: 2
Total tests that failed: 0
Individual tests:
Test name
Pass count
Fail count
Content-Type is present application/json
1
0
Status code is an error
1
0
Description: retrieve a product offering
Mean time per request: 4320 ms
Total tests that passed: 2
Total tests that failed: 0
Individual tests:
Test name
Pass count
Fail count
Content-Type is present text/plain
1
0
Status code is an error
1
0
Description: create a trouble ticket
Mean time per request: 3863 ms
Total tests that passed: 5
Total tests that failed: 0
Individual tests:
Test name
Pass count
Fail count
Status code is 201
1
0
Content-Type is present application/json
1
0
Response contains ID
1
0
Response contains all required fields
1
0
POST Body Response equals Request Body
1
0
Description: retrieve /troubleTickets
Mean time per request: 7652 ms
Total tests that passed: 5
Total tests that failed: 0
Individual tests:
Test name
Pass count
Fail count
Content-Type is present application/json
1
0
Status code is 200
1
0
Response contains all required fields
1
0
Response contains created resource
1
0
Element contained equals previous creation
1
0
/troubleTicket/{{troubleTicketId}}
Description: retrieve a troubleTicket
Mean time per request: 1300 ms
Total tests that passed: 6
Total tests that failed: 0
Individual tests:
Test name
Pass count
Fail count
Content-Type is present application/json
1
0
Status code is 200
1
0
Response contains ID
1
0
Response contains all required fields
1
0
ID is correct
1
0
Body Response equals previous creation
1
0
Description: create a trouble ticket
Mean time per request: 1315 ms
Total tests that passed: 5
Total tests that failed: 0
Individual tests:
Test name
Pass count
Fail count
Status code is 201
1
0
Content-Type is present application/json
1
0
Response contains ID
1
0
Response contains all required fields
1
0
POST Body Response equals Request Body
1
0
Description: retrieve /troubleTickets
Mean time per request: 5085 ms
Total tests that passed: 5
Total tests that failed: 0
Individual tests:
Test name
Pass count
Fail count
Content-Type is present application/json
1
0
Status code is 200
1
0
Response contains all required fields
1
0
Response contains created resource
1
0
Element contained equals previous creation
1
0
/troubleTicket/{{troubleTicketId}}
Description: retrieve a troubleTicket
Mean time per request: 4147 ms
Total tests that passed: 6
Total tests that failed: 0
Individual tests:
Test name
Pass count
Fail count
Content-Type is present application/json
1
0
Status code is 200
1
0
Response contains ID
1
0
Response contains all required fields
1
0
ID is correct
1
0
Body Response equals previous creation
1
0
Description: retrieve troubleTickets
Mean time per request: 4481 ms
Total tests that passed: 7
Total tests that failed: 0
Individual tests:
Test name
Pass count
Fail count
Content-Type is present application/json
1
0
Status code is 200
1
0
Response contains all required fields
1
0
Response contains created resource 1
1
0
Element contained equals previous creation 1
1
0
Response contains created resource 2
1
0
Element contained equals previous creation 2
1
0
Description: retrieve tickets with high severity
Mean time per request: 2135 ms
Total tests that passed: 6
Total tests that failed: 0
Individual tests:
Test name
Pass count
Fail count
Content-Type is present application/json
1
0
Status code is 200
1
0
Response contains all required fields
1
0
Response does not contain created resource 2
1
0
Response contains created resource 1
1
0
Element contained equals previous creation 1
1
0
/troubleTicket?type=connectivity
Description: retrieve tickets with connectivity problems
Mean time per request: 1560 ms
Total tests that passed: 6
Total tests that failed: 0
Individual tests:
Test name
Pass count
Fail count
Content-Type is present application/json
1
0
Status code is 200
1
0
Response contains all required fields
1
0
Response does not contain created resource 1
1
0
Response contains created resource 2
1
0
Element contained equals previous creation 2
1
0
troubleTicket/{{id}}?fields=description
Description: retrieve the product offering that are active
Mean time per request: 887 ms
Total tests that passed: 3
Total tests that failed: 0
Individual tests:
Test name
Pass count
Fail count
Content-Type is present application/json
1
0
Status code is 200
1
0
Element contained equals previous creation
1
0
troubleTicket/{{id}}?fields=severity,status
Description: retrieve the product offering that are active
Mean time per request: 620 ms
Total tests that passed: 3
Total tests that failed: 0
Individual tests:
Test name
Pass count
Fail count
Content-Type is present application/json
1
0
Status code is 200
1
0
Element contained equals previous creation
1
0
troubleTicket?severity=High&fields=description
Description: retrieve the product offering that are active
Mean time per request: 598 ms
Total tests that passed: 5
Total tests that failed: 0
Individual tests:
Test name
Pass count
Fail count
Content-Type is present application/json
1
0
Status code is 200
1
0
Response contains created resource 1
1
0
Element contained equals previous creation
1
0
Response does not contain created resource 2
1
0
...
download and unzip the TMF679-ProductOfferingQualification ZIP file within your test directory.
You should see the following files between many others :
Windows-RUNCTK.ps1
Linux-RUNCTK.sh
2.
For Windows:
Right click Windows-RUNCTK.ps1 and select run with PowerShell, press Y and Enter, wait for the dependencies to be installed go to Step 2.
For Linux and Mac:
You need to give executable permission for the Linux-RUNCTK.sh file, you can do that by opening a terminal and typing:
bash Linux-RUNCTK.sh
Wait for NPM to install the dependencies and go to step 2.
3.
Enter the URL for the base of your API, for example, if you can get a resource on:
You should input:
https://tm-forum-open-api-reference-implementation.mybluemix.net/tmf-api/productOfferingQualificationManagement/v1
The script will now run for a few minutes and when it ends, you will have a resultsHTML.html file inside the folder, this is the file you need to forward to TMForum to get your certification if your API passed every test, the file should look like this:
If there are no failures then you have passed the CTK and your API is conformant with all
the Mandatory features.
The following link points to the TM Forum Open API Self Certification Web Page in which conformance results are published: TM Forum Open API Self Certification Web Page
You can also submit a link to your webpage along with your test results .