On Monday Patently Apple posted a report titled "Samsung Invents an In-Air Gesturing System Designed for a Next-Gen VR Headset." We noted in the report that in-air gesturing was likely to be one of the technology features coming to various devices in 2019 from smartphones to Mixed Reality headsets. While Samsung invented a Headset that could use in-air gesturing to control various features, we learned yesterday that Google isn't only patenting an in-air system but has been given approval by the FCC to use next-generation short-range Soli sensors that support "Wide-Field Radar Based In-Air Gesturing."
Today the US Patent & Trademark Office published a patent from Apple that illustrates that they're also working towards a future headset that may use slip-on finger sensor devices that will provide accurate hand feedback allowing users to work with content seen on the display of a headset using three-dimensional air gestures and more.
To explain Apple's invention, they first provide some background to help us understand why they've invented an alternative to using sensor gloves with a mixed reality headset. Apple notes that electronic equipment such as computers and head-mounted display systems are sometimes controlled using input-output devices such as gloves. A glove may have sensors that detect user hand motions. The user hand motions can be used in controlling electronic equipment.
The use of wearable devices to gather input for controlling electronic equipment can pose challenges. If care is not taken, a device such as a glove may affect the ability of a user to feel objects in the user's surroundings, may be uncomfortable to use, or may not gather suitable input from the user.
Apple invention covers a finger-mounted device that may include finger-mounted units coupled to control circuitry. The control circuitry may wirelessly transmit information gathered with the finger mounted units to an external device to control the external device.
The control circuitry may also use the finger-mounted units to provide a user's fingers with feedback such as haptic feedback.
For example, the control circuitry may supply haptic output to a user's fingers based on wirelessly received information from the external device. The haptic output may correspond to virtual reality or augmented reality haptic output.
The finger-mounted units may each have a body. The body serves as a support structure for components such as force sensors, accelerometers, and other sensors and for haptic output devices. During operation, a user may wear the finger mounted units on the tips of the user's fingers while interacting with external objects.
The body of each finger-mounted unit may have sidewall portions coupled by portion that rests adjacent to a user's fingernail. A user's fingertip may be received between the sidewall portions.
The body may be formed from deformable material such as metal or may be formed from adjustable structures such as sliding body portions that are coupled to each other using magnetic attraction, springs, or other structures. This allows the body of the finger-mounted unit to be adjusted to accommodate different finger sizes.
The body of each finger-mounted unit may have a U-shaped cross-sectional profile that leaves the finger pad of each finger exposed when the body is coupled to a fingertip of a user's finger. The control circuitry may gather finger press input, lateral finger movement input, and finger tap input using the sensors and may provide haptic output using the haptic output device.
Apple's patent FIG. 1 below is a schematic diagram of an illustrative device such as a finger-mounted device; FIG. 2 is top view of a user's hand and illustrative finger-mounted device components on finger tips of the user's hand; FIG. 4 is a perspective view of an illustrative finger-mounted device; FIGS. 14 and 15 are views showing illustrative mounting arrangements for finger-mounted devices.
Further to FIG. 1, Apple notes that the "device #20" may be coupled to one or more additional devices in the system. For example, a head-mounted device with a display may be used for displaying visual content (virtual reality content and/or augmented reality content) to a user. This head-mounted device may be coupled to an electronic device such as a cellular telephone, tablet computer, laptop computer, or other equipment using wired and/or wireless communications links.
These devices (#20) may communicate with the finger devices to gather input (e.g., user finger position information) and to provide output (e.g., using haptic output components in device).
Apple's patent FIG. 26 below is a side view of an illustrative finger-mounted device being worn on a finger at a location other than the tip of the finger; FIG. 27 is a side view of an illustrative finger-mounted device with optical sensors for gathering touch input from the upper surface of a user's finger; and FIG. 28 is a diagram showing how markers may be used in calibrating a system in which a finger-mounted device is used.
Further to FIG. 28, Apple notes that the user may interact with the displayed visual content in the head mounted display by supplying force input, motion input (e.g., air gestures, three-dimensional air gestures), taps, shearing force input, and other input.
Apple's patent FIG. 29 below is a diagram showing how visual elements can be manipulated by a user who is wearing a finger-mounted device; FIG. 30 is a diagram in which a user is selecting an item in a list using a finger-mounted device; and FIG. 31 is a perspective view of an illustrative finger-mounted device with visual markers.
Apple's patent FIG. 3 is an alternative ring style device design. Apple notes that FIG. 3 may, as an example, be formed from a soft elastomeric material, fabric, or other flexible material that allows the user to feel surfaces through the ring (unit #22). If desired, sensors, haptic devices, and or other components may be mounted under the pad of finger #32 in locations such as location #36.
Apple's patent application 20190004604 was filed back in Q2 2018 and published today by the U.S. Patent Office. Considering that this is a patent application, the timing of such a product to market is unknown at this time.