XCUITest — CircleCI configuration for different TestPlan in iOS app Testing (using Fastlane)

When automating UI tests on native apps, one may have to deal with testing not only the native app features, but also some of in-app external functionalities such as integration of web-views, advertisements etc., within the app.

There may be a need to segregate running of iOS tests in CI for different type of PR commits and merges.

When the ACI pipeline is integrated to run tests on every Git commit/push, it might make sense to run different type of test suites for different Source Code Branches.

One such scenario arose in one of our iOS app projects where, there was a need for:

• running smoke tests in most dev-committed initial PR branches, while
• run end-to-end tests for release/develop/master branches.

Also, at times it makes sense to save on time and expense, and run only mocked UI tests for all Git branches, over hitting the network more frequently with the other tests which are otherwise dependent on external dependencies and may turn your tests be frequently flaky.

Apple introduced concept of TestPlans in the year 2019 with XCTest and from then, test runs have become much more flexible.

While it is easy and handy to configure TestPlans locally. It may turn a bit tricky to configure the same in the CI.

Below configuration is a working copy where different test plans are running with UI tests in CircleCI for different Github branches.

Fastlane file — Fastfile

platform: ios do
  before_all do
    setup_circle_ci
  end

Setting lane for smoke test runs, where value of TestPlan passed below is same as the file name of TestPlan set in Xcode test project, say example: ‘SmokeTest.xctestplan’

 desc "Execute - Only Native App UI Tests"
 lane: UITestSmokeTests do
   TestParameterised(
     scheme: "XyzUITests",      
     build_configuration: "Debug", 
     testplan: "SmokeTest")
 end

Similarly, Set Another lane for end-to-end test runs where configured test plan in Xcode test project is ‘EndToEndTest.xctestplan’

 desc "Execute - All UI Tests"
 lane: UITestAllTests do
   TestParameterised(
     scheme: "XyzUITests", 
     build_configuration: "Debug", 
     testplan: "EndToEndTest")
 end

where TestParameterised lane has the following parameters:

private_lane :TestParameterised do |options|
    scan(
      workspace: "Xyz.xcworkspace",
      device: "iPhone 11",
      reset_simulator: true,
      scheme: options[:scheme],
      configuration: options[:build_configuration],
      output_directory: "output/scan",
      include_simulator_logs: false,
      testplan: options[:testplan],      
      clean: true
    )
  end
end

CircleCI file — config.yml

version: 2.1
executors:
  macos:
    macos:
      xcode: "11.6.0"
commands:
  execute-test-command:
    description: "Building and running tests"
    parameters:
      to:
        type: string
    steps:
      - run:
          name: "Set Bundle Path"
          command: bundle config set path 'vendor/bundle'
      - run:
          name: "Execute Tests"
          command: bundle exec fastlane << parameters.to >>
default: &DEFAULT
  working_directory: ~/project
  executor: macos
  environment:
    FL_OUTPUT_DIR: output
test_all: &TEST
  <<: *DEFAULT
  parameters:
    to:
      type: string
  steps:
    - attach_workspace:
        at: ./
    - execute-test-command:
        to: << parameters.to >>
    - store_artifacts:
        path: output
    - store_test_results:
        path: output/scan
jobs:                              # define jobs for test runs 
  setup_macos:
    <<: *DEFAULT
    steps:
      - ....
  
  test_ui_smoketests:
    <<: *TEST
  test_ui_alltests:
    <<: *TEST
workflows:                        # define workflows for tests
  version: 2
build_and_test:
  
    jobs:
      - setup_macos
            
      - test_ui_smoketests:
          to: "UITestSmokeTests"
          requires:
            - setup_macos
          filters:                # run smoke tests on dev branches
            branches:
              ignore:
                - develop 
                - master 
            tags:
              ignore:
                - /^release.*/
      - test_ui_alltests:
          to: "UITestAllTests"
          requires:
            - setup_macos
          filters:               # run all tests on merged branches
            branches:
              only:
                - develop 
                - master 
            tags:
              only:
                - /^release.*/

Thus, different tests can be segregated and run separately in any desired manner as per business need in CI as well, irrespective of branch variations.

Thanks for reading this article! Leave a comment below if you have any questions. Be sure to click on the 👏 icon to let me know that you encourage my writing.

Connect with me:

• https://www.linkedin.com/in/suparnakhamaru/
• https://github.com/suparna-khamaru