The simulator is not a Pencil
Endeo's core loop is handwriting on an iPad with Apple Pencil: strokes stay authoritative, Vision turns them into searchable text, and a live session ties those notes to a transcript on one clock. That loop can look fine in the simulator and still fail the moment a real Pencil hits glass. Device testing for us starts with that distinction — what the simulator is allowed to prove, and what only hardware is allowed to.
What the simulator is allowed to prove
We run most of the day in simulators, and we should. Layout across size classes, navigation state, SwiftData migrations, the offline-default path with no backend configured, permission prompt copy — none of that needs a GPU or a stylus. Pull requests stay cheap when those checks do not wait on a cable or a device farm queue.
The trap is treating a green simulator suite as a hardware claim. The simulator runs on the Mac's stack. It does not reproduce Pencil latency, palm rejection, thermal throttling after twenty minutes of recognition, or what Vision does with a shaky stroke on a warm iPad resting on a table instead of floating in a perfect input stream.
So the rule we use is narrow: the simulator proves software structure. It does not prove input fidelity. Anything whose correctness depends on a sensor, a radio, a stylus, or real memory pressure has to be named as a device gate, not assumed covered because XCTest went green.
PencilKit fails in ways buttons do not
Endeo keeps PencilKit strokes as the source of truth and treats OCR text as a rebuildable index. That architecture only matters if the capture path itself is trustworthy. On a real iPad, a note written at minute fourteen has to land with the same clock as the speech recognizer; the stroke data has to survive backgrounding; recognition has to run without locking the UI when the user keeps writing.
Those failures do not show up as assertion errors in a button-tap UI test. They show up as a laggy tip, a palm mark that becomes a stray glyph, a recognition pass that stalls the canvas, or a search index that drifts from what the person actually wrote. The simulator's mouse-drawn strokes are a different input channel with different timing, and they flatter the happy path.
When we test Endeo before a release candidate, the checklist is physical: write for several minutes with a Pencil, rotate, split view, interrupt with a listening session, lock the device mid-stroke, come back, and confirm the strokes are still the authority and the derived text still rebuilds. If that session only happens on a desk Mac, we have not tested the product.
- Layout and navigation
- SwiftData and sync state
- Permission copy and empty states
- Logic that does not need a stylus
- Stroke latency and palm rejection
- Vision under sustained writing
- Session clock across pen and mic
- Thermal and memory pressure
Green tests on a Mac are not a Pencil claim. Name the gate that is.
Build the matrix from the dependency
A device catalog is not a test plan. Buying every recent iPhone and running the same smoke on each one mostly proves the smoke is portable. The useful matrix starts from the feature's hardware dependency and the form factor people actually hold.
For Endeo that means iPad models people write on, not every phone size class. At least one current iPad with Pencil, one older supported iPad that still shows up in analytics, and the orientations and multitasking modes readers use with a Bible open beside notes. For RuleCaddie and BuilderHelp's field app the dependency is different: small phones outdoors, one-handed reach, sunlight, and radios that lie — which is why those apps already have their own outdoor and Network Link Conditioner practice.
The matrix stays small on purpose. Three or four intentional devices beat twelve that nobody has time to exercise before a TestFlight cut. If a device is on the list, it has a reason tied to a failure mode. If it does not, it is inventory.
Tier the gates so the farm does not own every commit
The industry pattern for 2026 mobile CI is the same one we already practice informally: simulators on the pull request for speed, a reduced real-device pass on merge or nightly, and the full intentional matrix on the release candidate. That is not bureaucracy. It is how you keep hardware truth without making every commit wait on a Pencil.
Automated UI on a device farm still earns its keep for launch, navigation, and critical paths that do not need a stylus. It does not replace the handwritten Endeo session. Farms are weak at sustained Pencil input and at the messy interruptions that happen when someone is studying with a mic open. Those stay on a physical iPad with a script of actions, not a script of taps alone.
TestFlight sits after the device gate, not instead of it. External testers are good at finding the journey we forgot to write down. They are a poor substitute for confirming that recognition still keeps up after a long writing stretch on the hardware we claim to support.
What we write down before we call it tested
Every release candidate gets a short device note: which machines ran, which OS versions, which hardware-dependent flows were exercised by hand, and what was left to the simulator on purpose. The point is not paperwork. The point is that "tested" stops being a vague claim about CI.
When something only failed on device, the fix ships with that path named — the same habit as keeping the purchase-order fixture that exposed a parser bug. A Pencil stall that only appeared after fifteen minutes of writing becomes a regression check on that iPad, not a hope that the next simulator run will invent the same timing.
Simulators keep us fast. Devices keep us honest. For an app whose product is a pen on glass, confusing the two is how you ship a study tool that only works in Xcode.