Rest APIs for CI/CD

Integrating REST APIs into CI / CD pipelines

This article describes how to use the apptest.ai with CI(Continuous Integration) tools. You can use apptest.ai with the test creation, status view, and result view APIs.

1. [POST] Run New Testset

This Run New Testset allows you to create a TestSet and run. Testset is a group of unit Tests.

Some options for the new test run follow the configuration information stored in the specific project.

[Request]

POST /openapi/v2/testset

Host : https://api.apptest.ai

Authorization : Basic {user_id}:{access_key}

Request Body Multipart Form Data #1

KeyTypeDescriptionRequired
app_fileFileTarget APP File to Test.
Automatic test only.
Yes

Request Body Multipart Form Data #2

KeyTypeDescriptionRequired
pidPositive Number-Unique number of Project
ex) 509
Yes
testset_nameStringMax 100 CharactersName of Testset
ex) “Testset name Example#1”
Yes
os_typeStringANDROID | iOSOS type of test target
ex) “ANDROID”
Yes
scenario_idNumber0 | PositiveIf Automated test, 0 (default).
Otherwise, a unique number of Scenario.
ex) 509
No
source_typeStringfile | app_idCurrently scenario test only
Type of Test Source (App file | App ID)
Type of test source set in App Repository of the project
Yes
time_limitPositive NumberMin : 5
Max : 30
Test time limit (Minutes)
If the value is empty, it follows the time-limit saved in the project settings.
ex) 5
No
use_voBooleanDefault: falseAutomated test only
Whether AT&T Video Optimizer(ARO) is used (true | false )
ex) true
No
callbackStringMax 250 charactersCallback URL to be called after test completion.
ex) ‘https://127.0.0.1/callback/url/example’
No
credentialslogin_idStringMax 150 CharactersAccount information(ID) of the test target app to be used to test the app
(Test credentials info - Login ID)
ex) ’test@gmail.com’, ‘01012345678’, ’testaccount’
No
login_pwStringMax 150 CharactersAccount information(Password) of the test target app to be used to test the app
(Test credentials info - Login PW)
ex) ‘password1234!’
No

[ Request Example #1 - Automated Test]

curl --request POST \
    --user {user_id}:{access_key} \
    -F 'app_file=@/path/of/your/app_file' \
    -F 'data={\
        "pid": 509, \
        "testset_name": "Testset Name Example #1", \
        "os_type": "ANDROID", \
        "time_limit": 15, \
        "use_vo": false, \
        "credentials": { \
            "login_id": "test@gmail.com", \
            "login_pw": "password1234!" \
        } \
    }' https://api.apptest.ai/openapi/v2/testset

[ Request Example #2 - Scenario Test]

curl --request POST \
    --user {user_id}:{access_key} \
    -F 'data={\
        "pid": 509, \
        "testset_name": "Testset Name Example #1", \
        "os_type": "ANDROID", \
        "scenario_id": 1204, \
        "source_type": "file", \
        "time_limit": 15, \
    }' https://api.apptest.ai/openapi/v2/testset

[ Response ]

KeyTypeDescription
test_countPositive NumberNumber of Unit tests run on the Testset
ex) 3
testset_idPositive NumberUnique number of the Testset
ex) 251929

[ Response Example ]

{
    "data": {
        "test_count": 1,
        "testset_id": 251929
    },
    "result_code": 0,
    "result_msg": "",
    "reason": ""
}

[ Error Code ]

result_coderesult_msgreason
4000Missing Request ParameterThe required parameter was not found in the request parameters : {{ PARAMETER KEY }}
4030App File Analysis FailediOS plistlib Parsing Error : {{ PARSING ERROR MSG }}
4031Test creation failed : There was a problem pre-processing your app file.
The IPA file must be a development version that is signed by your development certificate.
4032Test creation failed. Invalid app file.
There was a problem pre-processing your app file.
4040Android Manifest Parsing Error : {{ PARSING ERROR MSG }}
4041No launchable activity found. (android.intent.category.LAUNCHER)
4050Invalid Request ParameterFile extension not supported. ( Support : ipa, zip, app, apk, xapk, apks )
4051The required parameter was not found in the request parameters : (app_file)
4052Invalid request parameter : (project_id: {{ PROJECT_ID }})
4053Invalid request parameter : (testset_id: {{ TESTSET_ID }})
4055Device is not compatible with this app : ( APP os type : {{ APP_OS_TYPE }} )
4080Invalid Request ParameterDevice info does not exist in preset project data.
4081Available devices don’t exist in preset project data.
5001Run New Testset ErrorGlobal Exception Error - {{ ERROR_MSG }}
6001App File Upload ErrorApp file upload failed.
8000System MaintenanceThe service is currently under system maintenance.

2. [ GET ] Testset Status

Check the progress of the testset

[ Request ]

GET /openapi/v2/testset/{testset_id}

Host : https://api.apptest.ai

Authorization : Basic {user_id}:{access_key}

Request URL Parameter

KeyTypeDescriptionRequired
testset_idPositive numberUnique number of TestsetYes

[ Request Example ]

curl --request GET \
    --user {user_id}:{access_key} \
    https://api.apptest.ai/openapi/v2/testset/55716

[ Response ]

Response Body Data Type : JSON

KeyTypeDescription
testset_statusStringTestset Status ( Complete, Running )
testset_status_detailJSONCount of the result for each test.
total_test_cntPositive numberTotal test counts
error_cntPositive numberError detected test counts
fail_cntPositive numberFailed test counts
initializing_cntPositive numberInitializing test counts
pass_cntPositive numberPassed test counts
running_cntPositive numberRunning test counts
stop_cntPositive numberStopped test counts
response_timeDatetimeResponse time (timezone : UTC)

[Response Example ]

{
    "data": {
        'testset_status': 'Complete',
        'testset_status_detail': {
           'total_test_cnt': 3,
           'error_cnt': 1,
           'fail_cnt': 1,
           'initializing_cnt': 0,
           'running_cnt': 1,
           'stop_cnt': 0,
           'pass_cnt': 0
        },
        'response_time': 'Mon, 25 May 2020 06:40:24 GMT'
    },
    "result_code": 0,
    "result_msg": "",
    "reason": ""
}

[ Error Code ]

result_coderesult_msgreason
5002Get Testset Status ErrorGlobal Exception Error - {ERROR_MSG}
8000System MaintenanceThe Service is currently under system maintenance.

3. [ GET ] Testset Result

This API can be called only when the test is completed. Returns the result data of the testset.

[ Request ]

GET /openapi/v2/testset/{testset_id}/result

Host : https://api.apptest.ai

Authorization : Basic {user_id}:{access_key}

Request URL Parameter

KeyTypeDescriptionRequired
testset_idPositive numberUnique number of TestsetYes

Request Query Parameter

KeyTypeDescriptionRequired
data_typeStringTest result data type (all | xml | bare_xml | json | html)
Default : all
ex) https://api.apptest.ai/openapi/v2/testset/{testset_id}/result?data_type=all
No

[ Request Example ]

curl --request GET \
    --user {user_id}:{access_key} \
    https://api.apptest.ai/openapi/v2/testset/55716/result
curl --request GET \
    --user {user_id}:{access_key} \
    https://api.apptest.ai/openapi/v2/testset/55716/result?data_type=xml

[ Response ]

Response Body Data Type : JSON

KeyTypeDescription
completeBooleanWhether the test is running or completed ( true | false)
result_xmlBooleanResult data in XML format in JUnit format
result_bare_xmlBooleanResult data in XML format in JUnit format
Exclude “?xml” and “testsuites” tag
Use to run multiple tests at once
result_htmlBooleanResult data in HTML format
result_jsonBooleanResult data in JSON format in JUnit format

[ JUnit XML Format ]

KeyTypeDescription
testsuitenameapptest.ai’s project name
testcasenameName of the device under test (Unit Test)
timeTime the test was performed
system-outTest result link of apptest.ai
errormessageTest results link where errors were found

[ JUnit XML Format Example ]

<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="TestBot Test">
    <testsuite name="{PROJECT NAME}.TestBot">
        <testcase name="{DEVICE NAME}" time="{TEST TIME SECONDS}">
            <system-out>{RESULT PAGE LINK URL}</system-out>
        </testcase>
        <testcase name="{DEVICE NAME}" time="{TEST TIME SECONDS}">
            <error message="{RESULT PAGE LINK URL}" />
        </testcase>
    </testsuite>
</testsuites>

[ Error Code ]

result_coderesult_msgreason
5003Get Testset Result Data ErrorGlobal Exception Error - {ERROR_MSG}
6002Get Testset Result Data FailedTest is running yet. Please request when the test is complete.
6003Data Does Not ExistTest data does not exist.
8000System MaintenanceThe Service is currently under system maintenance.

4. [ GET ] List of Scenario IDs

This is an API to get the scenario information to be used when running the scenario test.

[ Request ]

GET /openapi/v2/project/{project_id}/scenarios

Host : https://api.apptest.ai

Authorization : Basic {user_id}:{access_key}

Request URL Parameter

KeyTypeDescriptionRequired
project_idPositive numberUnique number of projectYes

Request Query Parameter

KeyTypeDescriptionRequired
os_typeStringOS type selected when registering the scenario (ANDROID | iOS)
ex) https://api.apptest.ai/openapi/v2/project/{project_id}/scenarios?os_type=ANDROID
No
qStringA query string to search in the scenario name
ex) https://api.apptest.ai/openapi/v2/project/{project_id}/scenarios?q=login
No

[ Request Example ]

curl --request GET \
    --user {user_id}:{access_key} \
    https://api.apptest.ai/openapi/v2/project/2019/scenarios?os_type=ANDROID&q=login_fail

[ Response ]

Response Body Data Type : JSON

KeyTypeDescription
scenariosArraylist of Scenario IDs
response_timeDatetimeResponse time (timezone: UTC)

[ Response Example ]

{
    "data": {
        "scenarios": [1204,1206],
        "response_time": "Mon, 25 May 2020 06:40:24 GMT",
    },
    "result_code": 0,
    "result_msg": 0,
    "reason": ""
}

[ Error Code ]

result_coderesult_msgreason
2001Get Project Data FailProject data does not exist
4051Invalid request parameterInvalid request parameter : (project id)
5005Get Scenario List ErrorGlobal Exception Error - {ERROR_MSG}
8000System MaintenanceThe Service is currently under system maintenance.

Appendix. API Result Codes

result_coderesult_msgreason
4000Missing Request ParameterThe required parameter was not found in the request parameters : ({PARAMETER KEY})
4030App File Analysis FailediOS plistlib Parsing Error : {PARSING ERROR MSG}
4031App File Analysis FailedTest creation failed : There was a problem pre-processing your app file.
The IPA file must be a development version that is signed by your development certificate.
4032App File Analysis FailedTest creation failed : Invalid app file.
There was a problem pre-processing your app file.
4040App File Analysis FailedAndroid Manifest Parsing Error : {PARSING ERROR MSG}
4041App File Analysis FailedNo launchable activity found. (android.intent.category.LAUNCHER)
4050Invalid Request ParameterFile extension not supported. (support : ipa, zip, app, apk, xapk, apks)
4051Invalid Request ParameterThe required parameter was not found in the request parameters: (app_file)
4052Invalid Request ParameterInvalid request parameter : (project_id : {PROJECT_ID})
4053Invalid Request ParameterInvalid request parameter : (testset_id : {TESTSET_ID})
4055Invalid Request ParameterDevice is not compatible with this app : (App os type : {APP_OS_TYPE})
4080Invalid Request ParameterDevice info does not exist in preset project data.
4081Invalid Request ParameterAvailable devices don’t exist in preset project data.
5001Run New Testset ErrorGlobal Exception Error - {ERROR_MSG}
5002Get Testset Status ErrorGlobal Exception Error - {ERROR_MSG}
5003Get Testset Result Data ErrorGlobal Exception Error - {ERROR_MSG}
6001App File Upload ErrorApp file upload failed.
6002Get Testset Result Data FailedTest is running yet. Please request when the test is complete.
6003Data Does Not ExistTest data does not exist.
8000System MaintenanceThe Service is currently under system maintenance.

Last modified October 5, 2022