...
All the information related to the execution of the CTK will be contained in the DSTestedAPICTKResult.json file.
The following is a complete example See example output for the Trouble Ticket API:
Newman test report
Collection: TMForum - TroubleTicket API CTK
Iterations: 1
Description:
Requests:
Description:
Mean time per request: 6123 ms
Total tests that passed: 2
Total tests that failed: 0
Individual tests:
Test name
Pass count
Fail count
Content-Type is present application/json
1
0
Status code is 404
1
0
Description: retrieve a product offering
Mean time per request: 3675 ms
Total tests that passed: 2
Total tests that failed: 0
Individual tests:
Test name
Pass count
Fail count
Content-Type is present application/json
1
0
Status code is an error
1
0
Description: retrieve a product offering
Mean time per request: 4320 ms
Total tests that passed: 2
Total tests that failed: 0
Individual tests:
Test name
Pass count
Fail count
Content-Type is present text/plain
1
0
Status code is an error
1
0
Description: create a trouble ticket
Mean time per request: 3863 ms
Total tests that passed: 5
Total tests that failed: 0
Individual tests:
Test name
Pass count
Fail count
Status code is 201
1
0
Content-Type is present application/json
1
0
Response contains ID
1
0
Response contains all required fields
1
0
POST Body Response equals Request Body
1
0
Description: retrieve /troubleTickets
Mean time per request: 7652 ms
Total tests that passed: 5
Total tests that failed: 0
Individual tests:
Test name
Pass count
Fail count
Content-Type is present application/json
1
0
Status code is 200
1
0
Response contains all required fields
1
0
Response contains created resource
1
0
Element contained equals previous creation
1
0
/troubleTicket/{{troubleTicketId}}
Description: retrieve a troubleTicket
Mean time per request: 1300 ms
Total tests that passed: 6
Total tests that failed: 0
Individual tests:
Test name
Pass count
Fail count
Content-Type is present application/json
1
0
Status code is 200
1
0
Response contains ID
1
0
Response contains all required fields
1
0
ID is correct
1
0
Body Response equals previous creation
1
0
Description: create a trouble ticket
Mean time per request: 1315 ms
Total tests that passed: 5
Total tests that failed: 0
Individual tests:
Test name
Pass count
Fail count
Status code is 201
1
0
Content-Type is present application/json
1
0
Response contains ID
1
0
Response contains all required fields
1
0
POST Body Response equals Request Body
1
0
Description: retrieve /troubleTickets
Mean time per request: 5085 ms
Total tests that passed: 5
Total tests that failed: 0
Individual tests:
Test name
Pass count
Fail count
Content-Type is present application/json
1
0
Status code is 200
1
0
Response contains all required fields
1
0
Response contains created resource
1
0
Element contained equals previous creation
1
0
/troubleTicket/{{troubleTicketId}}
Description: retrieve a troubleTicket
Mean time per request: 4147 ms
Total tests that passed: 6
Total tests that failed: 0
Individual tests:
Test name
Pass count
Fail count
Content-Type is present application/json
1
0
Status code is 200
1
0
Response contains ID
1
0
Response contains all required fields
1
0
ID is correct
1
0
Body Response equals previous creation
1
0
Description: retrieve troubleTickets
Mean time per request: 4481 ms
Total tests that passed: 7
Total tests that failed: 0
Individual tests:
Test name
Pass count
Fail count
Content-Type is present application/json
1
0
Status code is 200
1
0
Response contains all required fields
1
0
Response contains created resource 1
1
0
Element contained equals previous creation 1
1
0
Response contains created resource 2
1
0
Element contained equals previous creation 2
1
0
Description: retrieve tickets with high severity
Mean time per request: 2135 ms
Total tests that passed: 6
Total tests that failed: 0
Individual tests:
Test name
Pass count
Fail count
Content-Type is present application/json
1
0
Status code is 200
1
0
Response contains all required fields
1
0
Response does not contain created resource 2
1
0
Response contains created resource 1
1
0
Element contained equals previous creation 1
1
0
/troubleTicket?type=connectivity
Description: retrieve tickets with connectivity problems
Mean time per request: 1560 ms
Total tests that passed: 6
Total tests that failed: 0
Individual tests:
Test name
Pass count
Fail count
Content-Type is present application/json
1
0
Status code is 200
1
0
Response contains all required fields
1
0
Response does not contain created resource 1
1
0
Response contains created resource 2
1
0
Element contained equals previous creation 2
1
0
troubleTicket/{{id}}?fields=description
Description: retrieve the product offering that are active
Mean time per request: 887 ms
Total tests that passed: 3
Total tests that failed: 0
Individual tests:
Test name
Pass count
Fail count
Content-Type is present application/json
1
0
Status code is 200
1
0
Element contained equals previous creation
1
0
troubleTicket/{{id}}?fields=severity,status
Description: retrieve the product offering that are active
Mean time per request: 620 ms
Total tests that passed: 3
Total tests that failed: 0
Individual tests:
Test name
Pass count
Fail count
Content-Type is present application/json
1
0
Status code is 200
1
0
Element contained equals previous creation
1
0
troubleTicket?severity=High&fields=description
Description: retrieve the product offering that are active
Mean time per request: 598 ms
Total tests that passed: 5
Total tests that failed: 0
Individual tests:
Test name
Pass count
Fail count
Content-Type is present application/json
1
0
Status code is 200
1
0
Response contains created resource 1
1
0
Element contained equals previous creation
1
0
Response does not contain created resource 2
1
...
You will see 2 files:
TroubleTicketCTK.html which will show the full detailed report, plus the json output file.
When you have completed these tests successfully, please send your test results (html and json results files) to [email protected], as they will be posted to our TM Forum Self Certification Web Page.
...