What are Natural User Interfaces?

A growing list of devices built with natural user interfaces (NUIs) accept input in the form of taps, swipes, and other ways of touching; hand and arm motions; body movement; and increasingly, natural language. Tablets and smartphones are the first in a growing array of devices that allow computers to recognize and interpret natural physical gestures as a means of control. These natural user interfaces allow users to engage in virtual activities with movements similar to what they would use in the real world, manipulating content intuitively. The idea of being able to have a completely natural interaction with your device is not new, but neither has its full potential been realized. What makes natural user interfaces especially interesting this year is the increasing high fidelity of systems that understand gestures, facial expressions, and their nuances, as well as the convergence of gesture-sensing technology with voice recognition. Users interact with their devices in an almost natural fashion, with gesture, expression, and voice communicating their intentions. The next wave of NUIs will likely be electrovibration, while involves the use of an electrostatic force to produce detailed tactile sensations that users can feel. Seen as the next evolution of touchscreen technology, it will allow users to not only provide touch-based input, but also tactile output via a wide variety of textures, topography, and other features as they interact with the screen.

INSTRUCTIONS: Enter your responses to the questions below. This is most easily done by moving your cursor to the end of the last item and pressing RETURN to create a new bullet point. Please include URLs whenever you can (full URLs will automatically be turned into hyperlinks; please type them out rather than using the linking tools in the toolbar).

Please "sign" your contributions by marking with the code of 4 tildes (~) in a row so that we can follow up with you if we need additional information or leads to examples- this produces a signature when the page is updated, like this: - Larry Larry Feb 8, 2012

(1) How might this technology be relevant to the educational sector you know best?

  • The ability for students to control devices and communicate with motion and voice allows them to express themselves more easily; however, the assistive technology use by special needs students is probably more relevant.- kathyschrock kathyschrock Jan 30, 2015
  • Haptics:We're already getting used to using haptics on tablets, but they have many more uses in interactive learning experiences, according to a recentMIT Technology Review article- marcia.mardis marcia.mardis Feb 22, 2015
    [Editor's Note: Added her from RQ2.]

(2) What themes are missing from the above description that you think are important?

  • With the description above including "tactile output via a wide variety of textures, topography, and other features as they interact with the screen", it seems as if you have included electrovibration in this area, too. Perhaps merging these two together?- kathyschrock kathyschrock Jan 30, 2015
  • Motion Capture Systems ...and virtual production for creative story telling in classrooms. These systems are beginning to become affordable for K-12 schools that wish to specialise in this form of "animation" production. In the short term, this is most likely to occur in upper secondary schools, perhaps with specialist media production courses. Motion capture fundamentally changes the way animations / 3D models can be created. See http://www.optitrack.com/ for an instance of a company that supplies systems for the education market. See the following as an example of a University making motion capture technology available to secondary school students: https://blogs.deakin.edu.au/motionlab/education/school-workshops/ - dezuanni dezuanni Feb 20, 2015 [Editor's Note: Added here from RQ2.]

(3) What do you see as the potential impact of this technology on teaching, learning, or creative inquiry?

  • I see students being able to have access to virtual manipulatives (in conjunction with the electrovibration component) and be able to do things in a virtual world that are more difficult in the real world (like taking apart a skull as illustrated in the video in question 4.- kathyschrock kathyschrock Jan 30, 2015
  • add your response here

(4) Do you have or know of a project working in this area?

Please share information about related projects in our Horizon Project Sharing Form.