The study of natural surface interaction is a cutting edge area of research that aims to answer, “How to turn any surface into a computer”. Read how the work of Professors Sean and Natasha Banerjee, using Sierra-Olympia’s Viento G Thermal Camera, may one day allow a human to interact with any surface as though it is a computer.
Introduction
This study describes a March 2019 survey of Sierra-Olympia Technologies (SOTI) Customers, Professors Sean and Natasha Banerjee of Clarkson University.
Professor(s) Banerjee co-direct the Terascale All-sensing Research Studio (TARS) at Clarkson University, where they perform research in human-driven artificial intelligence using capture and analysis of dense multi-person interactions in online and real-world environments.
They are currently building a massively dense multi-modal sensor laboratory as part of the studio, employing multiple Sierra-Olympia Viento G thermal imaging cameras.
The Sierra-Olympia Viento G has proven to be perfect for their application because of the excellent thermal imaging capability combined with robustness and ease of setup and use.
Application
Professors Sean and Natasha Banerjee lead research on Natural Surface Interaction at Clarkson University Terascale All-sensing Research Studio (TARS). The study of natural surface interaction is to answer the problem of ‘how to turn any surface into a computer’.
Professors Sean and Natasha Banerjee use infrared cameras to study how humans interact with surfaces. SOTI infrared cameras are used to study user gestures on a surface and allow visualization of the ‘change in thermal signatures of the surface by heat transfer from the fingers and hands of a user’. The professors have demonstrated use of infrared cameras to detect differences in human gestures on a surface including ‘pressure based, swipe based and drag and drop’.
The applications of this cutting edge research are tremendous – the work they are doing at the TARS lab to characterize the modality of surface interaction with thermal cameras may one day allow a human to interact with any surface as though it is a computer. Professors Banerjee imagine that “the entire world is an interactive touch surface where we are no longer limited to our computer screens, our desktops, our phones and our tablets”. Imagine the day that “you’re cooking in your kitchen and you can project your content onto your countertop… you tap on your countertop [you have a thermal camera and projector on the ceiling] and you can interact with that content”.
Solution
To meet Clarkson University’s needs, SOTI application sales engineer Craig Beal worked extensively with the customer to find the right camera configuration.
Professor Sean Banerjee indicated “One of the great things about the Viento G is that straight out the box it was easy to use …in about 5 minutes it was working. The camera is simple, robust and reliable and worked significantly better than the previous camera we were using”.
A Viento G camera and 9 mm lens were selected for the TARS lab application. The Viento G was selected based on:
- high spatial resolution [640 x 480 and 320 x 240],
- high frame rate [30hZ, 9, 30, 60Hz models available]
- configurable bit depth [8 and 16] and high thermal sensitivity [50mK at f/1.0]
- triggerable (can be frame synchronized with other cameras / sensors)
- Gig E interface and POE
- Ease of SW integration including SOTI SDK
The 9 mm lens was selected for the TARS application – the Viento G supports a wide range of fixed focal length lenses that provide FOVs from 6.2 deg to 90 deg [7.5 – 100 mm].
SOTI cameras ability and benefits:
- Operate in any lighting condition even in dark environment
- Increase the interaction space of projected surfaces
- Detect hand and body part better than compared to RGB and depth cameras
- Provide reliable heat trace detection that can be used to differentiation between hovering and touching hand gestures
- Detect finger and hand movement despite the projector’s light since the light it emits is not in the Infrared spectrum.
Learn More about the Viento family of cameras and lenses on the Viento Product Page as well as all of Sierra-Olympia’s camera offerings for Lab and Research applications.