A developer released Phone Harness, an open source tool that lets AI agents such as Claude or Codex control a real phone instead of a simulator, according to the project's GitHub repository.
On iPhone, the tool captures a Mac's screen-mirroring window, reads on-screen text and coordinates with Apple's Vision framework for optical character recognition, and simulates taps through low-level input events, all without jailbreaking the device, installing Xcode or putting an app on the phone itself, the repository says.
On Android, it captures the screen with the built-in screencap tool and reads the accessibility tree to identify tappable elements precisely, sending commands over adb by USB or wireless, according to the project. Neither platform runs a persistent background process; each operation is stateless.
The tool supports taps, long presses, drags, scrolls and typing, but the author lists real limits: no multi-touch gestures like pinch-to-zoom, no camera flows, no support for DRM-protected video, and ASCII-only text input on Android, according to the repository.
Agents that only click through a simulator miss whatever breaks on a real device: carrier quirks, background app interference, actual notification permissions. A harness that drives the physical phone closes that gap, at the cost of the device being unusable for anything else while an agent is running it.