Testing on all Levels: A pragmatic blueprint for embedded credential quality – insights from “Testing on all Levels” by Stephan Puri-Jobi (HID Global GmbH)

Why testing “on all levels” matters for credentials

In “Testing on all Levels,” Stephan Puri-Jobi (Senior Lead Test Engineer at the ID Tech Lab, HID Global GmbH) walks through how his team tests embedded secure devices across the full stack. Based in Graz with 19 years of software and embedded testing experience, he leads a 15-person, multi-location team covering test specification, implementation, automation, and CI/CD.

HID Global operates globally (HQ in Austin, Texas; 4,000+ employees; offices in 100 countries) and is part of Assa Abloy (51,000 employees). Within this landscape, the ID Tech Lab focuses on future credential technologies. The problem space is security-critical and diverse: a credential might be an RFID card or live on a phone, yet both must prove identity securely and behave consistently.

Puri-Jobi frames the mission succinctly: provide identification and authentication in a secure way. That mission sets the bar for testing: correctness, robustness against misuse, and feasibility across heterogeneous platforms.

Credentials, APDUs, and the challenge of heterogeneity

Credentials store keys and encrypted data. They speak via APDUs as defined by ISO 7816, with a common command set that includes Select, Authentication, Get Data, and Put Data.

Here’s the tension for engineering teams: cards and phones are fundamentally different platforms. On a card, code is typically C on a constrained CPU with minimal RAM, non-volatile memory, and crypto coprocessors, drawing power from the field (RFID). On a phone, you’ll see Java, Swift, or Kotlin, depending on OS. These worlds are not drop-in compatible – a fact that drives the test strategy.

Start at the spec: static testing of the PRS

Before any code is written, Puri-Jobi’s team starts with the Product Requirement Specification (PRS). The PRS defines device behavior and maps statements to requirements. The testing team performs static analysis through reviews:

• Understand requirements and their interactions
• Derive test cases directly from requirements
• Detect under-specification and gaps early

A key reason to test the PRS itself: if development starts on an ambiguous requirement, teams may implement what “seems right.” Tests, by contrast, demand an unambiguous initial state and a clear assertion. If a postcondition can’t be stated, the issue must go back to specification. This early loop prevents implicit assumptions from becoming code.

Calibrating the test levels: unit, component integration, and system

The conventional testing pyramid emphasizes unit tests as the foundation. Puri-Jobi recognizes the principle but calibrates it for embedded credentials: time-to-market pressure and the inability to reuse low-level tests across languages and hardware platforms force trade-offs.

Unit and component integration: important, but not the center of gravity

• Unit tests isolate functions with dedicated drivers and mocks.
• Component integration reduces mocking and wires functionality together; for instance, actually invoking a crypto algorithm and verifying the return value.
• These tests are language- and hardware-specific. What’s written in C for a card rarely helps for a Swift/Kotlin/Java phone implementation.

Given the limited reuse, the team keeps the volume of unit/component tests intentionally lean. It’s not about skipping quality – it’s about investing where a test will exercise the same behavior across all implementations.

System testing: leverage the common APDU interface

The unifier is the APDU interface. Regardless of whether the credential runs on a card, phone, or watch, the system-level interface is the same. The ID Tech Lab connects its test bench at this layer:

• The bench implements PRS requirements directly, driving requirements-based testing with the aim of full coverage.
• It exercises the standardized RFID/APDU command set across implementations.

This is where testing scales. One system-level test suite can validate multiple form factors and languages – a huge efficiency gain compared to duplicating low-level tests per platform.

Requirements-based testing in practice: positive and negative

Puri-Jobi distinguishes two classes of tests and emphasizes the need for both.

Positive testing: expected paths, but with intent

• Typical flows: Select the credential, issue Get Data for known objects, validate against what was previously put onto the credential.
• Techniques: boundary value analysis and equivalence class partitioning to choose values that are likely to uncover issues, not just “happy path” defaults.

An observation: developers naturally verify with safe, mid-range values during implementation. That’s not enough to trigger edge issues. The test team deliberately targets boundaries (e.g., 1 and 10 rather than 5–7), even for “positive” cases, to extract more signal from each run.

Negative testing: assume hostile or invalid use

• Everything outside the spec: unintended command sequences, too-large/too-small inputs, malformed structures, out-of-range values.
• Objective: in every situation, the credential must return the correct error and never leak secrets.
• Real-world ratio from their work: roughly 1 positive test to 4 negative tests per requirement to fully cover behavior and error handling.

Negative tests matter because a card in the field isn’t protected from arbitrary probing. Testing must simulate this reality.

Simulator vs. real hardware: speed, visibility, and truth

System tests can run on simulators and on physical devices, each with trade-offs.

• Simulators are faster and provide transparency (gray-box) with code coverage, showing which parts of the code were executed and which were not. Uncovered code can then be targeted with static analysis or review.
• The experience varies by platform. On Android, simulation is fast and accurate with mature tooling. For smart cards in C, the cycle is slower: compile, load onto the card or emulator, then test.

To reduce developer friction, the team consolidates as much as possible at system level – where shared interfaces enable reuse – and uses simulators for coverage and iteration speed, while reserving key measurements and validations for real hardware.

Non-functional priorities: analog performance, timing, memory

Beyond functional correctness, Puri-Jobi’s team targets non-functional dimensions that are crucial in the field.

Analog performance (cards)

• Cards are passive; they modulate the field. The question: how close must the card be to the reader for a transaction to succeed consistently?
• This includes measuring field coupling and ensuring robust analog behavior.

Timing

• How long must the card be in the field before the door opens, a police officer confirms a license, or a payment completes?
• These end-to-end timings matter for user experience and operational reliability.

Memory consumption and hygiene

• Phone memory is comparatively abundant; card memory is not. Teams must track RAM in the kilobyte range and non-volatile storage.
• Tests verify that object creation and deletion are clean and do not waste precious memory.

Security features and attack countermeasures are part of the test scope as well, culminating in a card-specific concern: anti-tiering.

Anti-tiering: integrity across power loss

With a passive card, computation and writes occur only while in the field. Remove the card mid-write – for example, during a key update – and power drops instantly. What state does the card end up in?

Puri-Jobi’s team runs explicit tests and methods to ensure the implementation always guarantees one of two valid outcomes:

• The new data is fully written and consistent, or
• The old data remains fully and safely intact.

There is no room for an in-between, partially written state. Anti-tiering (as referred to in the session) is essential to preserve integrity under abrupt power loss.

System integration: personalization, readers, interoperability, endurance

Once the team’s system tests pass, the product moves into higher-level integration with other parts of the company – because in the real world, separate parties handle the credential, personalization, and readers.

• In the lab, the team personalizes the credential (write keys and data) and runs tests knowing exactly what’s on the device.
• In the field, personalization is performed by someone else (e.g., a hotel), while readers might come from yet another unit.

Integration testing validates the whole chain:

• Interoperability across various cards, credentials, phones, and reader devices
• Endurance: how many times can the card be presented before it wears out?
• User experience: how long to wait before the door unlocks; any awkward angles or positions required for successful reads?

This stage approximates real-world deployment and ensures that components built by different teams truly work together.

What stood out to us at DevJobs.at

Puri-Jobi’s approach is methodical and grounded in practical constraints. A few actionable insights we took away:

• Start testing at the PRS. Static reviews reduce ambiguity and prevent the “implement what you think it should be” trap.
• Place your testing center of gravity where reuse is highest: the common APDU system interface.
• Positive tests prove correctness; negative tests enforce safety. The 1:4 ratio illustrates the necessary emphasis on error handling.
• Boundary value analysis and equivalence partitioning are table stakes if you want tests to actually fail when they should.
• Use simulators for speed and coverage visibility (gray-box), but confirm critical behaviors on hardware – especially analog performance and timing.
• Memory matters on cards. Track both RAM and non-volatile usage, and verify clean object lifecycle.
• Anti-tiering is essential for passive media – explicitly test interrupted writes and validate resulting states.
• End-to-end integration is a first-class test objective: personalization flows, readers, interoperability, endurance, and UX under real conditions.

A practical sequence for engineering teams

Anchored in what we heard, here’s a sequence teams can adopt in similar embedded scenarios:

1. Operationalize requirements

• Review the PRS early; identify interactions and edge cases.
• Attach explicit preconditions and postconditions to each requirement’s test.

1. Design for the system interface

• Build your test bench around the shared, standardized interface (APDUs per ISO 7816 in this context).
• Implement requirements-based tests to pursue full coverage.

1. Balance positive and negative

• For each positive path, add multiple negative cases (invalid lengths, ranges, sequences).
• Choose boundary values intentionally; don’t rely on mid-range defaults.

1. Combine simulator and hardware

• Use simulators to accelerate iteration and expose coverage gaps.
• Validate key performance and analog behaviors on physical devices.

1. Measure resources and robustness

• Monitor RAM and NVM usage; verify clean create/delete cycles.
• Define anti-tiering tests to simulate interrupted writes and assert integrity guarantees.

1. Plan for system integration

• Include personalization processes and diverse readers in your test matrix.
• Test interoperability, endurance, and user experience under realistic conditions.

Memorable lines and cues from the session

A few lines serve as helpful cues when shaping a test strategy for credentials:

• “Spoiler alert. We do some testing … on all different levels.” Testing must be multi-layered.
• “We really do a requirements-based testing here.” Let the PRS be your test plan.
• “We cannot reuse it …” Low-level tests have limited reuse across platforms; system-level tests scale.
• “This is really annoying … very frustrating.” Embedded workflows must acknowledge developer experience and iteration speed.

Closing thoughts: a scalable, system-test-first approach for heterogeneous credentials

“Testing on all Levels” presents a pragmatic way to adapt the testing pyramid for a heterogeneous, security-critical domain. By anchoring on the shared APDU interface at system level, keeping unit and component tests focused, expanding negative coverage, combining simulator insight with hardware truth, and validating real-world integration (personalization, readers, UX, endurance), the approach delivers reusable, meaningful assurance across card and phone alike.

For teams building embedded credentials, these practices translate into a resilient path: start with the spec, test where behavior converges, prioritize negative coverage, guarantee anti-tiering, and prove interoperability in realistic scenarios. It’s a strategy designed to hold up not only in the lab, but—crucially—out in the field.

• 

  

  Importance of secure coding

  Bassem Taamallah von HID Global spricht in seinem devjobs.at TechTalk darüber, wie sich die Praktiken von Secure Coding auf die Qualität der Produkte auswirkt.

  Watch now

• 

  

  Stephan Puri-Jobi, Software Test Lead von HID Global

  Software Test Lead bei HID Global Stephan Puri-Jobi gibt im Interview Einblicke in den Aufbau des Unternehmens und des agilen Testing Teams, mit welchen Technologien gearbeitet wird und wie das Recruiting im Team gestaltet ist.

  Watch now

• 

  

  Anna Dziarkowska, Software Engineer bei HID Global

  Anna Dziarkowska von HID Global erzählt im Interview wie sie zum Software Engineering gekommen ist, mit welchen Themen sie sich aktuell als Software Engineer beschäftigt und was sie Anfängern empfehlen würde.

  Watch now
• 

  

  Stephan Puri-Jobi, Software Test Lead bei HID Global

  Stephan Puri-Jobi von HID Global redet im Interview über seine Anfänge im Programmieren bis hin zum Software Testing, was seine Arbeit beinhaltet und gibt Tipps für Neueinsteiger.

  Watch now