The new Logitech MX Ink
We’re very excited that Logitech has announced its mixed reality stylus, MX Ink (https://www.logitech.com/en-us/products/vr/mx-ink.html), for use with Meta Quest headsets. This device marks the first third party tracked peripheral for Meta devices and will bring valuable precision to mixed reality apps, including those for clinical use.
Since before Realize Medical was even officially a company, we have believed in the crucial importance of precise and familiar tools for the clinical use of mixed reality. The concept of a mixed reality stylus is important for three reasons:
- The familiar form factor that allows for easy and fluid adoption of new modalities
- The precision that a stylus form factor affords, and
- The for-work-use sentiment that styluses embody.
It’s for these reasons that a stylus was a core component of Elucis, even when it was an initial prototype without a name.
Early Elucis prototypes and stylus integration
See our co-founder, Dan La Russa, using a Vive controller as an oversized crayon with an early proof-of-concept at The Ottawa Hospital.
While developing early versions of Elucis at the academic predecessor of Realize Medical, we tested the (physical and virtual) use of home-made styluses using Vive trackers: ranging from duct-taped monstrosities to designed and 3D printed prototypes.
Our early experiments with our physician colleagues illustrated the value in having a familiar tool that would bridge that gap between every-day use tools, like tablets with styluses, and the extremely novel platform of virtual reality.
We also designed the early prototype of Elucis to act like a 3D powered tablet for that reason.
The virtual stylus
Elucis makes use of the interaction with a hard writing surface like a desk for actions that are performed on image cross section planes like the one seen above. With no widely available mixed reality styluses at hand in the last several years, we’ve still made use of the concept of a stylus in Elucis, except this time only the virtual version. We accomplished this by overlaying virtual styluses on mixed reality controllers in such a way that they could be comfortably used by resting the controller on a flat writing surface.
In this way, users can still make use of a precision tool in the virtual space, even if they don’t have the familiar stylus form factor in their hands.
When switching to hand-tracking, we display a virtual stylus held in the user’s hand, when appropriate, which gives a convenient and understandable tip the point of content generation (like markups or measurements).
Unfortunately, both of these form factors aren’t ideal: virtual styluses overlayed on controllers still required physician end users to learn the use of controller buttons with something that feels made for video games, and the hand tracking stylus lacks any tactile or haptic feedback and noticeably lacks the true precision of a controller. Both also lack the press-tip-against surface engagement mode of a stylus.
The precise actions afforded by a mixed reality stylus
With a comfortable, accurately tracked, and well integrated stylus, there are many 3D anatomical modeling and surgical planning activities that can be done with high precision.
For example, the stylus can be used to write on a desk surface (turned image viewer in VR) to quickly segment 3D anatomy from a CT scan. The benefit of using a mixed reality platform for this activity is the ability to see the result in true 3D immediately and to seamlessly switch from working on a surface in 2D to working in air in 3D. With the Logitech MX Ink you can press the tip on the surface and then press the action button in air.
In a very similar way, measurements on images or of 3D structures can be made precisely. Again, switching seamlessly between 2D and 3D.
Novel and useful interactions can also be achieved quite intuitively, like this anchor tool which uses the stylus to hold a 3D point of the image in place on the surface so that the other hand can grab and rotate to quickly find a desired multi-planar reformation of the image.
3D markups can be drawn to illustrate various concepts in the 3D anatomical structures.
Or specific clinical decisions can be easily and quickly tested like in this example where a baffle is rapidly created to consider for the closure of a congenital heart defect.
Conclusion
At Realize Medical, we are always keen to use what mixed reality technology has to offer to its fullest potential and we look forward to providing the best clinical tools possible.
If you’re interested in a free full version trial of Elucis then please get in touch (https://www.realizemed.com/try-elucis/). And read more about Logitech’s new MX Ink here.