Eye Tracking: Calibrate Once, Use for A Lifetime?

Eye-tracking technology has long been a core interaction method for XR headsets, yet performance often degrades if the headset shifts even slightly. Is there a way to calibrate it just once and have that calibration last a lifetime?

Skarredghost—a renowned overseas XR reviewer and the founder of the XR website The Ghost Howls—encountered the startup Immersix at a VR/AR industry expo in Shanghai. Immersix is taking a radically different approach to eye tracking: instead of tracking the pupil, it tracks the retina.

The following text has been slightly edited for readability.

Breaking the mold: While others target the pupil, it targets the retina

While almost all eye-tracking solutions currently on the market track the pupil, Immersix has chosen a completely different path: tracking the user's retina.

According to the company, retinal tracking offers several major advantages over traditional methods:

1. The retina is unique to each individual, making it suitable for identity authentication and extremely difficult to forge;

2. The retina remains virtually unchanged over time, so eye-tracking calibration needs to be performed only once and remains valid thereafter;

3. The retina represents the "ground truth" of the eye; regardless of how the headset shifts on the user's head, the initial calibration remains effective—meaning no recalibration is required even after removing and putting the device back on;

4. Tracking precision can be extremely high;

5. The modules can be made very small (requiring only two tiny units near the bridge of the nose), operate with ultra-low power consumption (less than 10mW), and achieve very high speeds (up to 120Hz).

This effectively resolves virtually all the pain points associated with eye tracking in current MR headsets—such as the drop in performance after putting the headset on and taking it off a few times, which necessitates repeating the tedious calibration process. With Immersix’s solution, such hassles are theoretically eliminated.

How does it work

The approach is highly ingenious.

Each eye-tracking module consists of just a single camera and one infrared LED used to illuminate the eye. Notably, a single LED suffices, allowing the module to be much more compact than traditional designs that require a ring of LEDs.

During the calibration phase, the user performs specific eye movements. Each camera module observes the eye, but instead of focusing outward, it attempts to "look inside." This enables the camera to capture data on how the retina changes over time.

The design concept behind the calibration process is to have the eye move in various directions, thereby exposing the entire retina to the camera step by step. The software then reconstructs a complete retinal image and extracts its unique "fingerprint." In effect, this calibration process generates a comprehensive feature map of the user's retina. Since the retina remains stable over time (at least until advanced age), this map can be used for many years.

How does the system actively track the eye after calibration? At this stage, each camera continues to monitor its respective eye—specifically, the area of the retina visible based on the eye's current orientation. By using a specialized algorithm to match this small, visible retinal area against the complete retinal map created during calibration, the system can precisely determine the eye's orientation.

According to Immersix, this tracking is highly precise (claiming sub-degree accuracy). Furthermore, because the camera tracks the eyeball's absolute rotation relative to the retina—rather than the pupil's position relative to the glasses—the tracking is extremely robust against camera movement. Consequently, you can put the glasses on or take them off at will, and eye tracking continues to function without recalibration, even if the position and orientation vary slightly each time.

Real-world experience: Does it really work

By now, you’re probably wondering if it really lives up to the hype.

A company representative took me backstage to try out the system. I wasn't allowed to record anything, so I’m using official videos and images to help convey my experience.

I was handed a few pairs of frames equipped with Immersix technology. These weren't smart glasses or AR glasses—just frames designed to test the tracking technology. I could clearly see the circuitry, so I was testing a prototype rather than a finished product. It was clearly a technology demonstration using a development kit.

After putting on the special glasses, they were connected to a laptop, triggering an eye-tracking calibration routine. I could see large markers (likely ArUco markers) on the laptop screen, with a small red rectangle in the center. These markers were clearly intended to help the prototype glasses track the screen's position; since the calibration couldn't be performed on the device itself—given that it lacked AR or VR capabilities and consisted merely of frames—it had to be done via the laptop, requiring the glasses to locate the screen. I imagine these markers wouldn't be necessary if the software were embedded directly into the glasses.

In short, a small cursor was linked to my eyes, allowing me to move it by shifting my gaze. My task was to sweep this cursor across the screen to completely erase the small rectangle. It was akin to using a small-radius eraser tool in MS Paint to wipe away a rectangle. I had to move my eyes back and forth repeatedly to fully erase it. I noticed that as I progressed, the eye-controlled cursor seemed to "lag," forcing me to move my eyes more vigorously from side to side to clear the rectangle. This might have been intentional, designed to encourage me to shift my gaze toward the periphery so the cameras could capture the full range of my retinal view.

The calibration took a few minutes. It was easy to follow, though slightly longer than I had expected, and watching the pointer move during the final few seconds caused a bit of eye strain.

After that, the eye-tracking session could begin. To demonstrate that the eye-tracking was working, a staff member launched another application; I saw a different set of ArUco markers and a grid of red circles on the laptop screen. I could look at a specific circle to turn it blue, which confirmed the system was functioning correctly. I have to say, it worked quite well. The accuracy was impressive, and the color of the elements I looked at changed in real-time. It was pretty cool.

Of course, I cannot verify claims regarding sub-degree accuracy or a 120Hz frame rate. However, I can confirm that the system works exactly as advertised—at least in a context similar to the Apple Vision Pro. That is already a significant achievement, given that some current eye-tracking solutions struggle to handle even that level of performance.

I was also able to shift the headset slightly to see if the system would continue to function correctly without recalibration; it performed well in my quick tests.

Final impression: Promising, but privacy is a concern

This solution really surprised me. It represents a novel approach to eye tracking that eliminates calibration issues and enables the creation of compact, low-power, and precise eye-tracking modules.

My hands-on experience confirmed that it works effectively. I only had a few minutes to try it out—limited to a technical demonstration rather than extensive testing on an actual augmented reality device—so I’m reserving final judgment. However, the tests I did conduct yielded positive results, and the performance fully lived up to the claims.

My only concern is privacy. The system requires a full scan of my retina, implying that complete retinal data must be stored somewhere. That said, this issue could be resolved by keeping the data local so that it never leaves the user's device.

I believe this is a highly promising product.

Source: VRAR Planet