Automatic adaptation of the contents and interaction with the virtual universes from cerebral activity of the user
OpenVIBE2 : context
In the last project OpenViBE1 (ANR-RNTL 2005-2008 - and noted “project HEADLIGHT ANR”), was developed a software platform open-source allowing to conceive, test and use ICO (Interaction Brain Computer). Several research orientations in the field of the ICO and the treatment of signals EEG (electroencephalography) and in the use of the virtual reality techniques were approached to improve the training of a ICO.
Research in this field is currently in full explosion and impressive prototypes are already available. However, the level of control offered by a ICO does not equalize yet that offered by a traditional interface such as a joystick or a mouse. That's why, the use of the ICO is currently reserved to the handicapped people who cannot have access to these other interfaces.
OpenVIBE2 : project
In OpenViBE2, the project proposes to radically change the prospect for use of the ICO. It initially proposes to be able to jointly use the ICO with the traditional methods of interaction like the joystick or the mouse. Thus, the ICO is not regarded any more as a replacement solution but as a complement of the traditional man-machine interface. Secondly, the project wishes to use the cerebral activity with an aim of adapting in real-time and in an automatic way the protocol of interaction and the contents of the virtual environment.
This project is focused on the video games, which is a relatively new scope of application for the ICO with a consequent market potential. The aim of the OpenViBE2 is “to identify and use the mental state and the cerebral answers of the user collected by EEG, to adapt at the same time the manner by which the user can interact with the play and the contents of the video game itself”.
Various mental states can be identified starting from signals EEG and used: the attention (visual or auditive), the intention (of movement for example), the state of relieving, the cognitive load, etc According to these states, several modifications can be planned to improve the “gameplay”, while adapting for example:
protocol of interaction (to anticipate an action, to assist or guide an action, to carry out an action)
contents of the virtual environments (addition of new characters or change of the behavior of the one of them for example).
The OpenViBE2 project raises several scientific challenges and techniques which can be gathered according to three research orientations:
at the neurophysiological level, for the experimentation and the validation of the use of markers EEG in the context of the video games
at the technological level, concerning the design and the development of new methods of treatment of the signal specific to the EEG and the context video game
at the technological level, concerning the man-machine interaction and the development of new methods for the interaction in virtual environments based on the cerebral activity.
The repercussions we can await from this research are first new knowledges in very varied fields (neurosciences, treatment of the signal, IHM); and of new methods, tools and techniques for the adaptation of the contents and the interaction with the video games and the virtual environments, using the cerebral activity and the mental states of the user. Moreover, one software platform open-source integrating the whole advances in the field of the treatment of signal EEG in real time will be proposed. Lastly, the project aims three demonstrators (plays) based on technologies of virtual reality and video games:
a technological illustrator of type “casual range” (play of sport for example)
a “brain to trail” (cerebral drive),
a “serious game" for the rehabilitation of the children suffering from disorders attentionnels.
OpenVIBE2
