[ Azure DevOps ] apptest.ai Integration for Azure DevOps
Author : Apptest.ai
With just a few lines of script, you can enable apptest.ai along with Azure DevOps Pipeline for our testbot to explore your apps and report results.
To use the apptest.ai service within Azure DevOps Pipeline, you can take the following two step:
1. register the apptest.ai Access Key, and 2. add the apptest.ai to your build script (azure-pipelines.yml).
1. Adding apptest.ai Access Key onto the build environment
- Copy the access key from the Profile Page in apptest.ai
- Register APPTEST_AI_ACCESS parameters at the Variable tab within Azure Pipeline settings.
Once registered, you can click the Lock Icon and modify the setting to the secret variable. This provents other users from looking up the variables.
The format for APPTEST_AI_ACCESS_KEY is set as “apptest.ai ID: apptest.ai Access Key“.
For example, if the ID is jean@apptest.ai and the access key is “ab8f3e321d631c84c9b1113” as follows.
jean@apptest.ai:ab8f3e321d631c84c9b1113
2. Calling the script from azure-pipelines.yml
- Now, add a script that executes the CI Tool Integration Script provided by apptest.ai as follows.
In this case, you need to pass the environment variable created above to the Env Section. Azure DevOps Pipeline requires an explicit mapping to variable regarding secret variables.
- bash: |
git clone https://github.com/apptestai/ci_addon
export binary_path=HackerNews.ipa
export project_id='831'
bash ./ci_addon/step.sh
displayName: 'Tested by TestBot'
env:
APPTEST_AI_ACCESS_KEY: $(APPTEST_AI_ACCESS_KEY)
At least two arguments, binary_path and project_id, are required to invoke the service.
binary_path is the path to the binary you want to test, i.e. * .apk files for Android and * .ipa files for iOS.
project_id is the project ID of apptest.ai that contains the settings such as test time and device list for testing.
The project ID of apptest.ai can be found on the Project Information page as shown below.
If you don’t want to download apptest.ai’s CI Integration script every time during the build, include apptest.ai’s ci_addon repository in the project repository in advance.
If project ID is also registered in Pipeline variable like the access key, you can compose the build YML file as below.
- bash: bash ./ci_addon/step.sh
displayName: 'Tested by TestBot'
env:
APPTEST_AI_ACCESS_KEY: $(APPTEST_AI_ACCESS_KEY)
- Once you commit the build script, Azure DevOps will start the build process.
When the script reaches apptest.ai, the test is executed as shown below and the result is printed on the console.
Currently, the Azure Devops Pipeline has some errors while handling the output to the console.
Logs that use ASCII color code cannot be displayed during the build, but only after the build is finished.
Also, there is an error which ignores the align setting through the printf command in bash.
For this reason, the alignment of the result column is broken as shown above.
However, given the current evolution of Azure Devops services, we hope it will be resolved soon.
- The test results are stored in test-results apptest.ai result.html with a detailed HTML report and an XML document in JUnit Test result format in test-results /apptestai/result.xml.
You can save this file using the various deploy features provided by Azure DevOps. Below is an example of publishing as an artifact and test result in Azure.
If you want to change the location of the folder where the test results are stored, declare and pass test_result_path environment variable.
If you want to execute the next Build Script before finishing the test, declare waiting_for_test_results environment variable and pass “false” value.
Finally, let me show you the complete azure-pipelines.yml example. First case is Android.
Next, iOS case:
As you can see, the part that is different from Android is the actual building and packaging of the binary but the process for calling the apptest.ai testbot and saving the result is the same.
