marqably
Testing in Flutter
Description
Simon Auer von marqably nimmt in seinem devjobs.at TechTalk das Thema von automatischen Softwaretests in Flutter aus verschiedenen Blickpunkten unter die Lupe.
By playing the video, you agree to data transfer to YouTube and acknowledge the privacy policy.
Video Summary
In "Testing in Flutter," Simon Auer makes the case for automated tests to improve quality, reduce manual QA, enable safer refactoring, and serve as living documentation—accepting a 30–40% upfront time cost that pays off over the long run. He walks through the testing pyramid (unit, widget, integration), promotes naming conventions and the AAA pattern, and live‑demos tests for a simple search app using finders, pumpWidget, pumpAndSettle, and Flutter’s integration test package. Viewers come away knowing how to structure and prioritize Flutter tests and automate end‑to‑end flows—practices that map well to other ecosystems.
Testing in Flutter the right way: Unit, Widget, and Integration tests you’ll actually keep – Lessons from “Testing in Flutter” by Simon Auer (marqably)
Why this talk matters
In “Testing in Flutter,” Simon Auer (CEO of marqably) makes a straightforward case for automated testing as a non‑negotiable part of building quality apps. Whether you ship for startups or multinational brands, his baseline is the same: quality and maintainability come first. Automated tests aren’t about clicking through screens after every code change—they’re code that checks your code.
From our DevJobs.at editorial seat, what stood out is how approachable Auer keeps the topic. Flutter ships with testing built in, the syntax is familiar if you’ve used modern frameworks, and the mental model scales across ecosystems. This session covers why and when to test, the three core test types in Flutter, and a live demo based on a tiny fruit-search app that ties it all together.
The case for testing
Auer starts with the why—and it’s refreshingly practical:
- Quality and continuous verification:
- Tests validate all the cases you’ve implemented, every time they run. If a new change breaks something that used to work, you’ll know immediately and precisely where to look.
- Less manual testing:
- Run automated tests on every deployment, then do manual verification only after a green pass. You’ll cut the back-and-forth cycles significantly.
- Forced thinking about edge cases:
- Writing tests makes you reason about what can go wrong. You’ll discover edge cases you hadn’t considered.
- Refactoring becomes a joy:
- “Inputs and outputs stay the same; only the implementation changes.” With good tests, you can reshape the internals with confidence. If you break behavior, tests will tell you instantly.
- Shield against dependency regressions in teams:
- When someone changes a method another part of the code depends on, tests surface the break early so it can be fixed before merging or shipping.
- Even if you have a QA team, automate:
- Automation saves your testers’ time and can surface internal edge cases that manual testers might miss.
- Confidence at deployment time:
- Ship without second-guessing. If the pipeline goes green, you proceed; if not, you get immediate, actionable feedback.
- Tests as documentation:
- Auer highlights this often-overlooked benefit: tests are living examples sitting next to your code. They show how to call a method or compose a widget, what to expect as a result, and which variants are supported.
When to test
The short answer: almost always. The only exception Auer concedes is code you know will never be touched again. Otherwise, tests preserve knowledge over time. He’s candid about costs: expect roughly 30–40% extra time when developing with tests. Over a project’s lifetime, though, you save hours (more likely many hours) you’d otherwise spend rediscovering behavior, manually regression-testing, or diagnosing bugs late.
The Flutter testing pyramid
Auer frames Flutter testing in three tiers, moving from cheap/fast to more expensive/complex:
- Unit tests
- Fast to write and maintain; test classes, services, and functions with clear inputs and outputs.
- Widget tests
- Validate UI components (widgets) in isolation; moderate maintenance, higher leverage than unit tests for UI-related logic.
- Integration tests
- Exercise full workflows—tapping, swiping, navigating, filling out forms—like a human tester would. Powerful, but costlier and more brittle when UIs change.
There are other test types (accessibility, acceptance, performance), but the talk intentionally focuses on these three.
Three guiding practices
Auer offers three habits that pay dividends:
- Naming conventions:
- Test files mirror the lib/ structure, live under test/, and end with “_test.dart”. It keeps navigation and organization frictionless.
- The AAA pattern (Arrange – Act – Assert):
- Arrange: set up data and environment.
- Act: execute the behavior under test (call a method, render a widget, tap a button).
- Assert: verify outcomes.
- Test-driven development (TDD), wherever you can:
- Writing the expected results before implementation helps you avoid false positives and design clearer APIs.
Test types in detail
Unit tests: Treat code like a black box
- Purpose: Validate classes, services, and functions with deterministic input/output.
- Properties: fast, readable, independent.
- Anti-pattern: Don’t write implementation logic to test implementation logic. That spirals into testing your test instead of the target behavior.
- Example from the session: A search method that filters a hard-coded list of fruits. Given “berry”, it should return “strawberries” and “raspberries”.
- Documentation payoff: A tight unit test spells out the contract—what goes in and what comes out—so newcomers understand usage at a glance.
Widget tests: Verify UI behavior in isolation
- Purpose: Render a single widget in a test harness and assert on observable cues.
- Keep it independent: Don’t create extra widgets solely for testing. Test the component, not a synthetic environment.
- Golden tests exist in Flutter: pixel-perfect snapshots to detect visual diffs. Auer mentions them but focuses here on behavior rather than colors or pixels.
- Typical flow: pump the widget into an offscreen canvas, use finders (by text, by type), then assert on presence, count, or absence.
- Demo use cases:
- With an empty query, the screen shows “please enter a search term”.
- With a non-empty query, at least one ListTile appears in the results.
Integration tests: Drive real user flows
- Purpose: Automate sequences like opening the app, navigating, entering text, scrolling, and selecting items.
- Similar syntax to widget tests, but they run against a live app (emulator or headless). You’ll use Flutter’s integration test package.
- Overhead: Higher dev and maintenance cost. A small UI change can break a flow; use integration tests strategically for critical paths (e.g., login).
- Mechanics highlighted in the talk:
- Ensure the app is initialized, pump the app widget, enter text, call “pump and settle” to re-render, assert visibility, and tap through.
- CI/CD:
- Auer notes these tests are resource intensive; keep them in a separate folder and run them in the pipeline against emulators or headless environments.
The demo app: a simple fruit search
To connect concepts to code, Auer showcases a minimal Flutter app:
- Structure:
- A single screen with a search field at the top and results below.
- The data source is a hard-coded list of fruits.
- Behavior:
- Empty input: show “please enter a search term”.
- Query “berry”: return matches like “strawberries” and “raspberries”.
- Key widgets:
- A top-level “Super Search” widget.
- A “Super Search Results” widget that takes the query, calls
getSearchResultswhen the query is non-empty, and renders a list accordingly.
It’s a small example by design. Each test type gets a crisp target: pure logic for the unit test, UI state for the widget test, and a mini end-to-end flow for the integration test.
Unit test walkthrough: Verifying search logic
- Target:
getSearchResultstakes a string query and filters the fruit list. - Expectation: For “berry”, the result includes “strawberries” and “raspberries”.
- Structure: Use the AAA pattern with a clear test description so failures are self-explanatory.
- Good hygiene: Intentionally break the test once to ensure it fails when it should. Don’t trust a test that never turns red.
Widget test walkthrough: Validating UI for empty vs. non-empty input
- Case 1: Empty query
- Expect exactly one instance of the text “please enter a search term”.
- Method: Pump the widget, use a text finder, assert the count.
- Case 2: Non-empty query
- Expect at least one ListTile (the list of matches is visible).
- Method: Pump the widget with a non-empty query, use a type finder, assert presence.
Widget tests in this pattern remain independent and easy to read, allowing fast feedback on UI behavior without spinning up the whole app.
Integration test walkthrough: Typing, rendering, tapping
- Goal: Simulate a user flow—launch the app, confirm the initial state, type into the search field, let the UI re-render, assert the presence of results, ensure visibility, and tap an item.
- Key steps highlighted:
- Initialize the app, pump the main app widget.
- Enter text into the field, then call “pump and settle” to refresh the UI.
- Use finders to assert the presence of ListTiles.
- Ensure elements are visible before tapping them.
- Guidance: Integration tests are fantastic for critical flows but come with maintenance overhead. Keep them for the features you most care about.
Tooling and workflow
- VS Code quality-of-life:
- The dev tools include a “go to test/implementation” feature that jumps between counterpart files and can scaffold missing test files in the correct location.
- Running tests and interpreting output:
- Flutter’s test runner reports failures with expected vs. actual output, making root-cause analysis quick.
- Folder organization:
- Production code under lib/, tests under test/, and integration tests often separated due to their runtime cost in CI.
Maintenance, stability, and focus
- Keep tests independent:
- Unit and widget tests should avoid unnecessary dependencies. Test the unit, not the framework.
- Don’t create test logic that itself needs testing:
- Complex logic inside tests is a smell. Keep tests declarative and intention-revealing.
- Be selective with integration tests:
- They’re invaluable yet brittle against UI changes. Reserve them for high-impact workflows (e.g., login).
- Combine automation with manual QA:
- When the pipeline guards regressions, manual testing confirms behavior rather than rediscovering bugs.
TDD in practice: When it helps most
- Use it when expectations are clear:
- Write the failing test first, then implement to make it pass. This reduces the chance of false positives and clarifies design.
- Be pragmatic:
- Auer acknowledges you can’t always do TDD. Where it fits, it sharpens APIs and stabilizes behavior.
- Don’t forget negative paths:
- Tests should also prove what must not happen. Explicitly cover empty inputs, invalid states, and edge conditions.
A practical checklist for Flutter testing
- Unit tests
- For each function with clear input/output, add at least one positive and one negative test.
- Avoid logic in tests; stick to Arrange, Act, Assert.
- Widget tests
- Render widgets in isolation and assert on visible cues (text presence, widget types, counts).
- Consider golden tests for stable visual components.
- Integration tests
- Pick critical user journeys: login, search, checkout.
- Use “pump and settle”, ensure visibility before tapping, verify navigation and state changes.
- Structure & tools
- Mirror lib/ under test/; name files with “_test.dart”.
- Use VS Code shortcuts to move between test and implementation.
- Keep integration tests in a separate folder for targeted pipeline runs.
- Culture
- Treat tests as documentation: expressive names, clear Arrange/Act/Assert sections.
- Occasionally force a test to fail to validate its usefulness.
Closing thoughts: Tests as quality and knowledge infrastructure
“Testing in Flutter” by Simon Auer (marqably) underscores a simple truth: automated testing is the backbone of reliable Flutter apps. Unit, widget, and integration tests complement each other—from fast logic checks, to on-demand UI verification, to realistic user journeys. The benefits are tangible: less manual effort, broader coverage, living documentation, smoother refactors, and genuine confidence at deploy time.
The best line to carry forward might be the feeling Auer describes when all tests go green. That’s not just aesthetic satisfaction—it means your system’s contracts are holding, your assumptions are verified, and your team can move fast without fear. Combine a pragmatic pyramid approach with tight structure, AAA discipline, and selective TDD, and you’ll end up with a test suite you can trust—and maintain.
The fruit-search demo keeps it deliberately simple to highlight the essence: tests that are easy to write and read are the ones you’ll keep running. Start with a few unit tests, layer on widget tests for UI rules, and protect key flows with integration tests. That’s the path to fast, fearless Flutter development.