Testing Strategy: Unit vs Instrumentation vs UI Tests

Real-world analogy

What is it?

Testing strategy defines which types of tests you write, how many of each, and in what order — before writing any test code. The Android testing pyramid guides you to invest heavily in fast, cheap unit tests, moderately in integration tests, and minimally in slow UI tests. Understanding the cost-benefit of each level and the difference between test doubles (mock, fake, stub) lets you build a test suite that actually catches bugs without becoming a maintenance burden.

Real-world relevance

When building a school management app from scratch, the team had zero tests. Rather than starting with UI tests (the most obvious choice for 'does the app look right'), the lead engineer prioritized unit tests for the grade calculation engine, attendance rule logic, and report card generation — the core business rules. These caught 12 bugs in the first week without running the app once. UI tests were added only for the parent-facing grade report flow and the admin login — the two journeys where bugs would cause the most damage.

Key points

• The Android testing pyramid — The recommended distribution: ~70% unit tests (fast, no device needed), ~20% integration/instrumentation tests (moderate speed, may need device or Robolectric), ~10% end-to-end UI tests (slow, require emulator or physical device). Invert this pyramid and your test suite becomes slow, brittle, and expensive to maintain.
• Unit tests — what they are — Unit tests run on the JVM (no Android framework). They test a single class or function in isolation. Dependencies are replaced with test doubles. Run in milliseconds. The default location is src/test/java. Use JUnit, MockK, Truth. No Activity, no Context, no Room database — pure Kotlin/Java logic only.
• Instrumentation tests — what they are — Instrumentation tests run on a real Android device or emulator. They have access to the full Android framework — Context, databases, file system, system services. Located in src/androidTest/java. Run with AndroidJUnitRunner. Much slower (30s–5min per test run). Use sparingly for things that genuinely require the device.
• UI tests with Compose Testing or Espresso — UI tests drive the app's interface — find elements, click them, type text, assert visible state. Espresso for View-based UI, Compose Testing API for Jetpack Compose. These are instrumentation tests (need a device). Very brittle — UI changes break tests. Reserve for critical user journeys only.
• Robolectric — the middle ground — Robolectric is a framework that simulates the Android runtime on the JVM without a device. Allows testing code that uses Android APIs (Context, SharedPreferences, etc.) faster than instrumentation tests. Not perfect — some APIs are stubbed or behave differently. Good for ViewModel, Repository tests that need lightweight Android context.
• Cost vs benefit of each level — Unit tests: lowest cost, highest ROI — run in <1s, catch 80% of bugs, easy to debug. Integration tests: medium cost — run in seconds to minutes, catch wiring bugs. UI tests: highest cost — run in minutes, most brittle, catch only the most obvious UI regressions. Always maximize unit test coverage first before investing in higher-level tests.
• What to test first in a new project — Priority order: (1) Business logic in use cases / domain layer — these encode the core rules of your product. (2) ViewModel state transitions. (3) Repository data fetching and caching logic. (4) Edge cases: null inputs, empty states, error states. UI tests last and only for mission-critical flows like login, checkout, or payment.
• Test doubles: mock vs fake vs stub — Stub: returns hardcoded values, no verification. Mock: you verify exactly which methods were called (MockK, Mockito). Fake: a working lightweight implementation (e.g., an in-memory List-based repository instead of a real Room database). Fakes are more maintainable and realistic; mocks are faster to write but can lead to over-specification of implementation details.
• Fragile test smell — over-mocking — Tests that mock every dependency and verify every method call become coupled to implementation rather than behavior. When you refactor (e.g., rename a method), 50 tests break even though the behavior is correct. Prefer testing observable output (returned values, emitted Flow items, UI state) over verifying internal method calls.
• Testing philosophy — test behavior, not implementation — A good test answers: 'Does this class do what it is supposed to do?' not 'Does this class call method X in exactly this order?' Write tests from the user's perspective of the class. This makes tests resilient to refactoring and actually useful as regression protection.
• Code testability and design — Hard-to-test code is a design smell. If you cannot unit test a class without spinning up a database, the class has too many responsibilities. Dependency injection (Hilt/Koin) makes classes testable by allowing dependencies to be swapped. Pure functions with no side effects are trivially testable. Design for testability from day one.
• Testing in CI/CD pipelines — Unit tests run on every commit in CI (fast, <2 min for large projects). Instrumentation and UI tests run on PRs or nightly (slow, require emulator). Firebase Test Lab and AWS Device Farm provide real device testing at scale. A failing unit test blocks a merge; a failing UI test triggers investigation.

Code example

// Project structure — where tests live
//
// src/
//   main/java/com/schoolapp/       ← Production code
//   test/java/com/schoolapp/       ← Unit tests (JVM only, fast)
//   androidTest/java/com/schoolapp/ ← Instrumentation tests (device required)

// ---- UNIT TEST example (src/test/) ----
// Tests pure Kotlin business logic — no Android dependencies
class GradeCalculatorTest {

    private val calculator = GradeCalculator()  // Real instance, no mocks needed

    @Test
    fun `passing score returns PASS grade`() {
        val result = calculator.evaluate(score = 75, maxScore = 100, passMark = 60)
        assertThat(result.grade).isEqualTo(Grade.PASS)
        assertThat(result.percentage).isEqualTo(75.0)
    }

    @Test
    fun `score below pass mark returns FAIL grade`() {
        val result = calculator.evaluate(score = 45, maxScore = 100, passMark = 60)
        assertThat(result.grade).isEqualTo(Grade.FAIL)
    }

    @Test
    fun `zero max score throws IllegalArgumentException`() {
        assertThrows<IllegalArgumentException> {
            calculator.evaluate(score = 0, maxScore = 0, passMark = 60)
        }
    }
}

// ---- Test doubles: Stub, Mock, Fake ----

// STUB — returns hardcoded value, no verification
class StubStudentRepository : StudentRepository {
    override suspend fun getStudent(id: String) = Student("1", "Alice", grade = 10)
    override suspend fun saveStudent(student: Student) { /* no-op */ }
}

// FAKE — real working implementation backed by in-memory storage
class FakeStudentRepository : StudentRepository {
    private val storage = mutableMapOf<String, Student>()

    override suspend fun getStudent(id: String) = storage[id]
        ?: throw NotFoundException("Student $id not found")

    override suspend fun saveStudent(student: Student) {
        storage[student.id] = student
    }
}

// MOCK — created by MockK, allows call verification
// val mockRepo = mockk<StudentRepository>()
// every { mockRepo.getStudent("1") } returns Student("1", "Alice", 10)
// coVerify { mockRepo.getStudent("1") }   ← verifies the call happened

Line-by-line walkthrough

1. 1. src/test/java is for unit tests — Gradle runs these on the JVM with no Android device, so they run in milliseconds.
2. 2. src/androidTest/java is for instrumentation tests — these require a connected device or emulator and take much longer to run.
3. 3. GradeCalculatorTest instantiates the real GradeCalculator — no mocks needed for pure logic classes with no external dependencies.
4. 4. Each @Test method tests exactly one behavior with a descriptive backtick string name — 'passing score returns PASS grade' reads like a specification.
5. 5. assertThrows verifies that the class correctly validates its inputs and fails fast with a meaningful error.
6. 6. StubStudentRepository always returns the same hardcoded Student — useful when you only care about the system under test and not the repository behavior.
7. 7. FakeStudentRepository uses a mutableMapOf as in-memory storage — it behaves like a real repository (throws on missing keys) without needing Room or SQLite.
8. 8. The MOCK approach uses MockK's every { } to define return values and coVerify { } to assert that specific calls happened — useful when testing that side effects occur.

Spot the bug

// A developer wrote these tests for a payment processing use case.
// Identify what is wrong with the testing approach.

class ProcessPaymentUseCaseTest {

    @Test
    fun testProcessPayment() {
        // Bug 1 — test setup
        val mockRepo = mockk<PaymentRepository>()
        val mockNotifier = mockk<NotificationService>()
        val mockLogger = mockk<AuditLogger>()
        val mockValidator = mockk<PaymentValidator>()
        val mockFormatter = mockk<CurrencyFormatter>()
        val useCase = ProcessPaymentUseCase(
            mockRepo, mockNotifier, mockLogger, mockValidator, mockFormatter
        )

        every { mockValidator.validate(any()) } returns true
        every { mockFormatter.format(any()) } returns "$100.00"
        every { mockRepo.save(any()) } returns Unit
        every { mockNotifier.send(any()) } returns Unit
        every { mockLogger.log(any()) } returns Unit

        // Bug 2 — test body
        useCase.process(Payment(amount = 100.0, currency = "USD"))

        // Bug 3 — assertions
        coVerify { mockValidator.validate(any()) }
        coVerify { mockFormatter.format(100.0) }
        coVerify { mockRepo.save(any()) }
        coVerify { mockNotifier.send(any()) }
        coVerify { mockLogger.log(any()) }
    }
}

Explain like I'm 5

Fun fact

Hands-on challenge

More resources

• Android Testing Fundamentals (Android Developers)
• Test Doubles in Android (Android Developers)
• Robolectric Documentation (Robolectric)
• Testing on Android (Guide) (Android Developers)