3. Circle CI

apptest.ai Integration for Circle CI

Starting with CircleCI version 2.1, the build script sharing method called orb allows you to have Testbot test your app and report the results in every build with just a few lines of scripts.

To enable the apptest.ai service during a build in CircleCI: 1. Register the apptest.ai access key in the build environment variable.

Adding apptest.ai access key onto the build environment

  1. Copy the access key string from the apptest.ai service profile page

  2. Connect to CircleCI and click the Setting icon to the right of the build project name to go to the build configuration page.

  3. From the Setup page, select Environment variables menu and click Add Variables button to create the variable APPTEST_AI_ACCESS_KEY.

    Enter the value of the variable as “apptest.ai ID: apptest.ai Access Key”. For example, if the ID is jean@apptest.ai and the access key is “ab8f3e321d631c84c9b1113”. Register one as follows.


    The following is the normal registration and access_key is masked and printed.

Adding apptest.ai test step to the build

  1. Include apptest.ai Orb

    If the app you want to test is in Android, include android-app-test orb as shown below.

       apptest-ai: apptestai/android-app-test@volatile

    To test your iOS App, include ios-app-test orb as shown below.

       apptest-ai: apptestai/ios-app-test@volatile
  2. Adding apptest.ai test build step

    You can add apptest-ai test steps anywhere in your build workflow.

    - apptest-ai/run-test:
       binary_path: app/build/outputs/apk/debug/app-debug.apk
       project_id: '818'

    At least two arguments, binary_path and project_id, are required to invoke the service. binary_path is the path to the binary you want to test: * .apk files for android and * .ipa files for iOS. project_id is the ID for apptest.ai that contains the settings such as test time and device list for testing. The Project ID of apptest.ai can be found on the Project Information page as shown below.

  3. Execute the build

    The apptest-ai/run-test step sends a request to test with apptest.ai and after the test is complete, prints the test result to the console as shown below.

    If you want to move on to the next build step without waiting for the apptest.ai test result, you can set FALSE to the waiting_for_test_results argument as shown below.

    - apptest-ai/run-test:
       binary_path: app/build/outputs/apk/debug/app-debug.apk
       project_id: '818'
       waiting_for_test_results: 'false'
  4. (Optional) Storing test result

    apptest.ai test result printed in the console is saved under “test-results” directory by default. To include the test results in the Test Summary provided by CircleCI, add the store-test-results step provided by CircleCI and specify the test result path of apptest.ai as shown below.

    - store_test_results:
       path: test-results

    If you add the store-test result step, CircleCI analyzes the test result of apptest.ai along with other test results in the Test Summary tab as below.

  5. (Optional) Storing test result

    Because Test Summary only provides very limited pass/fail information, apptest.ai provides a more detailed test result report in a separate HTML format. To save this report in the Artifact section of CircleCI, add the store_artifacts step provided by CircleCI as below and specify the path where test results of apptest.ai are saved.

    - store_artifacts:
       path: test-results

    Saved test report is inquired as below in Artifacts tab in CircleCI.

    If you click this html file, you can see the detailed report as below.

    If you want to change the directory that stores the test results, you can pass the path to the test_result_path argument when you run the apptest.ai test step as below.

    - apptest-ai/run-test:
       binary_path: app/build/outputs/apk/debug/app-debug.apk
       project_id: '818'
       test_result_path: app/build/outputs/apk/debug

    Finally, here’s the sample the final .circleci / config.yml file. Below is for Android.

    And iOS:

Last modified October 4, 2022