Android UI testing

Theory and Practice

Agenda for today

  • 1 Deep dive into UI testing
  • 2 Native Android UI Testing
  • 3 Mocks
  • 4 Conclusion

1

How many UI tests do you have in your project?

1

Dive into UI testing

1 Deep dive into UI testing

Software testing pyramid

center

1 Why do we need tests?

  • Confidence (Don't repeat your mistakes)
  • Continuous deployment
  • Screenshots generation
  • Application assessment speed up

1 Feature development process

center

  • Discussion of functionality
  • Discussion of assessment criteria
  • Feature implementation
  • Unit and UI tests implementation
  • Validation by QA

1 Basic feature state machine

center

1 Assessment criteria

  • Loading
  • Empty
  • Reload from Empty
  • Content
  • Reload from Content
  • Error
  • Reload from Error
  • Cross feature navigation

1 Abstract test case scenario

.../featureA/assesement.yaml

- case: 
  - id: featureAContentIsDisplayed
  - scenario:
  Open Feature A screen
  Wait for Content
  Verify that Loading is not displayed  
  Verify that Error is not displayed
  Verify that 3 items displayed in list
  Verify 1 item title is equals "Test title 1"  
  Verify 2 item title is equals "Test title 2"  
  Verify 3 item title is equals "Test title 3"

Agenda for today

  • 1 Dive into UI testing
  • 2 Native Android UI Testing
  • 3 Mocks
  • 4 Conclusion

2

Native Android UI Testing

1

What framework are you using for UI testing?

2 Native Android UI Testing

  • How to write?
    • Native / Cross platform
    • Content / Screenshot
  • How to run?
    • Locally / Remotely
    • Environment
  • How to scale?
    • Run in parallel

2 Frameworks

center

2 Native Android Frameworks

Espresso Kakao Kaspresso
Espresso Kakao Kaspresso

2 Pros and Cons (Espresso Espresso)

Pros Cons
✅ Supported by Google ❌ Lots of boilerplate
✅ Native Android test library ❌ Depends on API version
✅ Most popular UI testing framework ❌ Resources assertion and matching not supported out of the box
✅ Fast

2 Pros and Cons (Kaspresso Kaspresso)

Pros Cons
✅ Testing for ADB calls such as SMS, Calls, Geolocation etc ❌ Requires ADB server
✅ Built-in flakiness retries ❌ Tricky for scaling
✅ Allure report ❌ Parallel runs on CI
✅ Steps support
✅ Based on Kakao

2 Why Kakao?

  • Espresso
onView(allOf(withId(R.id.price_item),hasDescendant(withText("Standard Rate"))))
        .check(matches(withEffectiveVisibility(Visibility.VISIBLE)));
  • Kakao
onScreen<FormScreen> {
    price {
        isVisible()
    }
}

2 Typed page objects

class FormScreen : Screen<FormScreen>() {
    val phone = KEditText { withId(R.id.phone) }
    val email = KEditText { withId(R.id.email) }
    val submit = KButton { withId(R.id.submit) }
}

onScreen<FormScreen> {
    phone {
        hasText("971201771")
    }
    submit {
        click()
    }
}

2 Recycler support

class Item(parent: Matcher<View>) : KRecyclerItem<Item>(parent) {
    val title: KTextView = KTextView(parent) { withId(R.id.title) }
}

val recycler: KRecyclerView = KRecyclerView(
  { withId(R.id.recycler_view) },
  itemTypeBuilder = { itemType(::Item) })

onScreen<RecyclerScreen> {
    recycler {
        firstChild<Item> {
            title { hasText("Title 1") }
       ...

2 Compose support

class MainActivityScreen(semanticsProvider: SemanticsNodeInteractionsProvider) :
    ComposeScreen<MainActivityScreen>(
        semanticsProvider = semanticsProvider,
        viewBuilderAction = { hasTestTag("MainScreen") }
    ) {

    val myButton: KNode = child {
        hasTestTag("myTestButton")
    }

    val myButton2: KNode = myButton.child {
        hasTestTag("myTestButton2")
    }
}

2 Compose support

class ExampleInstrumentedTest {
    @Rule
    @JvmField
    val composeTestRule = createAndroidComposeRule<MainActivity>()

    @Test
    fun simpleTest() {
        onComposeScreen<MainActivityScreen>(composeTestRule) {
            myButton {
                assertIsDisplayed()
                assertTextContains("Button 1")
            }

            onNode { hasTestTag("doesNotExist") }.invoke {
                assertDoesNotExist()
            }
        }
    }
}

2 Quick refresh ...

.../featureA/assessment.yaml

- case: 
  - id: featureAContentIsDisplayed
  - scenario:
  Open Feature A screen
  Wait for Content
  Verify that Loading is not displayed  
  Verify that Error is not displayed
  Verify that 3 items displayed in list
  Verify 1 item title is equals "Test title 1"  
  Verify 2 item title is equals "Test title 2"  
  Verify 3 item title is equals "Test title 3"

2 Scenario Implementation

@Test
fun featureAContentIsDisplayed()
onScreen<FeatureAScreen> {
    waitFor(R.id.title)
    loading { isNotDisplayed() }
    error { isNotDisplayed() }
    list { 
      isDisplayed() 
      childAt(0)<Item> { title { hasText("Test title 1") } }
      childAt(1)<Item> { title { hasText("Test title 2") } }
      childAt(2)<Item> { title { hasText("Test title 3") } }
    }
}

2 Kakao + Marathon = 💞

Kakao Marathon
✅ Nice and simple DSL ✅ Flakiness control strategies
✅ Typed page objects ✅ Easy to scale
✅ Recycler support ✅ Can be integrated with CI
✅ Compat API support ✅ Supports custom configurations for your needs
✅ Lots of Matcher and Assertions ✅ Allure reporting

Agenda for today

  • 1 Dive into UI testing
  • 2 Native Android UI Testing
  • 3 Mocks
  • 4 Conclusion

2

Mocks

1

Do you have mocks for UI testing?

How flaky are your tests?

3 Mocks

  • What are we testing?
    • Feature
    • Integration
  • Why do we need mocks?
    • Performance improvement
    • Tests results stability
    • Control for 3rd party code

3 Mocks

  • Deployment
    • Local
    • Remote
  • Type
    • Stateless
    • Stateful

3 Testing with remote Prod server

Pros Cons
✅ Testing whole HTTP stack ❌ Extra time for request-response processing
✅ Good for Integration testing ❌ Bad for Feature testing
❌ Stateful
❌ Unstable results and branching in tests logic

3 Testing with remote server (Mocks)

center

3 Testing with remote server (Mocks)

Pros Cons
✅ Testing whole HTTP stack ❌ Extra time for request-response processing
✅ Single remote instance ❌ Required redeploy in case of mock updates
✅ Stable results for UI testing ❌ Not supporting custom error codes
✅ Stateless

3 Local Mocks (Entry level)

center

3 Local Mocks (Entry level)

Pros Cons
✅ Easy to implement ❌ HTTP Stack not tested
✅ Fast responses ❌ Not shared between platforms
✅ Static mock going as part of application
✅ Can be executed anywhere

3 Local Mocks (Intermediate)

  • Custom TestRule's for custom behaviours
    • Response codes
    • Throttling
    • Responses aligned with Test Suits requirements

3 Local Mocks (Advanced)

center

3 Local Mocks (Advanced)

Pros Cons
✅ HTTP Stack tested ❌ Requires extra effort to implement
✅ Mocks responses in-app ❌ Unsynchronized with "Production server"
✅ Good for Features test ❌ Bad for Integration test
✅ Can be run as "Remote server"
✅ Customizable behaviours

Agenda for today

  • 1 Dive into UI testing
  • 2 Native Android UI Testing
  • 3 Mocks
  • 4 Conclusion

4

Conclusion

4 Conclusion

  • Always understand what you want to test
  • Choose the right tooling set
  • Ask yourself "How will this code change in 3 years"
  • Think about scaling
  • Questions?