Abstract A Pivotable Head Mounted Camera System that is Aligned by Three-Dimensional Eye Mo

A Pivotable Head Mounted Camera System

that is Aligned by Three-Dimensional Eye Movements Philipp Wagner?1Klaus Bartl2Wolfgang G¨u nthner1Erich Schneider2Thomas Brandt2Heinz Ulbrich1

1Institute of Applied Mechanics,Technical University of Munich

2Department of Neurology,Ludwig-Maximilians University of Munich

Figure1:The eye movement driven head camera in action.

Abstract

The?rst proof of concept of an eye movement driven head cam-

era system was recently presented.This innovative device utilized

voluntary and re?exive eye movements,which were registered by

video-oculography and computed online,as signals to drive servo

motors which then aligned the camera along the user’s gaze direc-

tion.However,with just two degrees of freedom,this camera mo-

tion device could not compensate for roll motions around the op-

tical axis of the system.Therefore a new three-degree-of-freedom

camera motion device that is able to reproduce the whole range of

possible eye movements has now been implemented.In doing so,

it allows a freely mobile user to aim the optical axis of the head

mounted camera system at the target(s)in the visual?eld at which

he/she is looking,while the ocular re?exes minimize image shak-

ing by naturally counter-rolling the“gaze in space”of the camera

during head and visual scene movements as well as during locomo-

tion.A camera guided in this way mimics the natural exploration

of a visual scene and acquires video sequences from the perspective

of a mobile user,while the oculomotor re?exes naturally stabilize

the camera on target during head and target movements.Various

documentation and teaching applications in health care,industry,

and research are conceivable.This work presents the implementa-

tion of the new camera motion device and its integration into a head

camera setup including the eye tracking device.

no effort is taken to stabilize the camera system.A stabilized cam-era system might,for example,consist of an inertial measurement unit and a motion device to rotate the camera into certain gaze di-rections.But even a stabilized camera system has the disadvantage that the recorded video stream will not contain the view of the user, since the eyes move relative to the head.Therefore two major ben-e?ts can be gained when human eye movements are used as input signals for the camera motion device:on the one hand,the bio-logical gaze stabilization can be used to keep the gaze direction of the video camera stable,and on the other,the user’s gaze direction will control the video camera so that the camera always sees what the user sees.A system guided in this way is the ideal tool for ad-dressing the increased awareness of the importance of using natural stimuli to study and model the visual system[Betsch et al.2004; Olshausen and Field2005].Such a camera overcomes the weak-nesses of other state-of-the-art techniques that provide only rough approximations of the retinal content.

A?rst proof of concept that orients a camera’s optical axis parallel to the user’s gaze direction was presented in[Schneider et al.2005]. This system used a video-oculography device(VOG)to detect eye motions of the user.From these eye motions the gaze direction of the user was calculated and used as an input signal for the camera motion device.Thus,the optical axis of the camera was kept par-allel to the human’s gaze direction.Assuming a suf?ciently small latency,the human gaze stabilization based on the vestibulo-ocular, optokinetic,and smooth pursuit re?exes can be used to stabilize the camera image.In this camera setup the biological equilibrium or-gan–the inner ear labyrinth–controls a technical system,hence rendering a technical inertial measurement unit obsolete.

Major drawbacks of the above-described system were a high la-tency in the range of100ms due to the use of eye tracking with standard video sampling rates(50Hz)as well as slow servo ac-tuators and electronics.Also the two-degree-of-freedom camera motion device was not able to follow all eye movements,since the human eye can rotate along three perpendicular axes.Therefore, a three-degree-of-freedom(3DoF)camera motion device has been designed to facilitate full camera control by means of eye move-ments.

This paper presents the design and implementation of a wearable head camera system that meets the requirements claimed above. Details of the eye tracking device that served as a basis for both mechanical support and motion control of the head camera are pre-sented in a companion paper[Boening et al.2006].A user wearing the complete system consisting of the eye tracker and the head cam-era is shown in Figure1.The focus of this work is on a?rst3DoF prototype that serves as an experimental platform as well as a pos-sible solution for a head camera system for consumer products.

2System Overview

The eye movement driven head-mounted camera consists of?ve main modules:

1.lightweight head mount

2.eye tracker

3.wearable computer

4.servo driver

5.gaze camera mount with Cardan joint

The lightweight head mount mainly consists of swimming gog-gles(Aqua Sphere Crystal,Technisub,Genova)to which a?exi-ble,curved?at spring is attached to?t the head in the mid-sagittal plane(see Figure2).Electronic modules like a servo controller are also attached directly to the head mount.Hot mirrors for infrared (IR)light which are transparent for visible light re?ect the IR rays from the eyes to laterally attached video cameras.The invisible IR illumination is provided by a set of5LEDs mounted on the lower rim of the goggles.Similar camera,mirror,and illumination con-?gurations are implemented in many commercially available eye trackers(e.g.,SMI,Teltow,and Chronos Vision,Berlin).Video-oculography is used to track the eyes of the user.The video images of the laterally attached cameras are transferred to a wearable com-puter,whose image processing algorithms detect the position of the pupil.The eye tracker results are then used as signals to drive the actuators of a parallel kinematics setup on which the gaze camera is mounted.The eye tracker details are described in a companion paper[Boening et al.

2006].

Figure2:Illustration of the complete head camera system.

A picture of the complete head camera setup is shown in Figure2 with descriptions of the main parts.The whole setup is powered and controlled by a single off-the-shelf laptop(Acer TravelMate 382TMi).The eye tracker cameras are connected to the laptop with an IEEE1394interface and the servo controller is connected via a USB-RS232-adapter to the laptop’s US

B port.

2.1Calibration

Once eye position is detected in the image coordinate system,it is then transformed into a coordinate system suitable for aligning the gaze camera with the user’s gaze direction.The mapping or transformation between the two different coordinate systems of the eye tracker cameras and the actuators is accomplished with a set of calibration variables that are determined in a calibration proce-dure.This procedure was especially designed to minimize user in-teraction and was described previously together with the involved algorithms[Schneider et al.2005].In short,a small laser pointer that is rigidly attached to the camera module projects a bright dot that appears centered in the head camera image.The user is asked to?xate it as it jumps to positions prede?ned by motor commands that are known in the actuator coordinate system.The eye position values are then linearly?tted by two3rd order2D-polynomials to the corresponding motor commands(see Figure3).

3Design of the Camera Motion Device The major purpose of the newly designed camera motion device is to rotate the camera’s viewing direction around three perpendicular

(x,y)

(x,y)

Figure3:Schematic setup of the calibration technique.The user ?xates a laser dot that is moved together with the pivotable camera. The mapping between measured eye positions in image coordinates and corresponding motor commands is achieved by3rd order poly-nomial?ts.

axes with kinematic properties that are similar to those of the hu-man oculomotor system.Panning camera motions are achieved by rotations around a vertical(pan)axis.For elevation movements the camera is rotated around the horizontal(tilt)axis,and?nally,the camera can also move around its optical(roll)axis.To describe the considerations that led to the presented design,some general solu-tions for camera motion devices will be discussed in this section. The construction of a small three-degree-of-freedom camera mo-tion device based on a serial design is described in[Mayol et al. 2000].The use of extremely small cameras allows the choice of servo actuators designed for miniature aircraft models.An impor-tant advantage of the presented serial construction is the linear be-havior and the simple inverse kinematics of the system.The nu-merous moving components,however,constitute the major disad-vantage of such a serial system.As only the?rst actuator of the system is stationary,it not only moves the camera,but also all other actuators,leading to a loss of acceleration and velocity capabilities of the system.To compensate for this loss of dynamic abilities, stronger and usually heavier actuators have to be chosen.Depend-ing on the structure connecting the different components,additional stiffness problems may easily occur.

Therefore,the desired system would have stationary actuators that do not perform motions as they do in a serial construction,in which the actuators themselves add to the inertia of the moving compo-nents.This can be achieved by using parallel structures at the cost of more complex kinematics[Riebe and Ulbrich2003].Parallel motion devices especially designed for use with cameras without this drawback can be found in[Truong et al.2000]as well as in[Gosselin and Hamel1994].Note that the system presented in[Truong et al.2000]is used as a stereo robot head,which is a common application of moving cameras(see also[Scassellati 1998]).These systems reach extremely high velocities and accel-erations.However,compared to the cameras used,the complete systems are fairly big.Furthermore,a complex kinematic structure that uses many mechanical linkages to move the camera becomes necessary.The kinematic components add weight to the construc-tion and may have or develop backlash.

Hence the design goal for the head camera system was to create a mechanism that on the one hand has the undoubted advantages of a parallel mechanism(stationary actuators)and on the other, avoids the described drawbacks(backlash and high number of link-ages).Furthermore,the construction has to be extremely compact and lightweight in order to affect the user as little as possible.

3.1Mechanical Setup

An early attempt to construct a two-degree-of-freedom head mounted camera system can be found in[Schneider et al.2005]. Here the camera was mounted on a Cardan joint,allowing the cam-era to perform motions around two perpendicular axes,namely the pan and tilt axes.Motions around the camera’s optical axis,the roll axis,were not possible.This concept proved to be precise and stiff.

A similar mechanism had already been implemented in a humanoid robot’s ankle joint([Gienger et al.2001]and[Gienger2005]).Dis-advantages were the non-linearities in the inverse kinematics and a smaller pivoting range than in serial motion devices.

After considering the issues discussed above,we decided to up-grade the design given in[Schneider et al.2005]with a third axis to a full three-degree-of-freedom motion device.As in the previous system,the pan and tilt servo actuators are kept stationary.A major requirement was to?nd a solution that kept the third actuator sta-tionary in the same location next to the other actuators.

The earlier system did not reach the velocities of the human eye (500deg/s,see[Leigh and Zee1999]),thus the next main require-ment was to implement a system that could reach dynamic proper-ties comparable to that of the human oculomotoric system.Further requirements were a lightweight and compact construction as well as backlash-free joints for increased system precision. Photographs of the newly developed camera motion device can be seen in Figure4and Figure5from different viewing perspectives. The main parts are denoted.The camera motion device has been designed as a modular setup,consisting of the Cardan module, the servo module,and the camera module,thus allowing the user to change these components easily to achieve another setup,e.g., a servo module with different actuators or another camera mod-ule.All servo actuators are mounted together in the servo module, which is connected to the base plate of the Cardan module.The third module is mainly the camera carrier with the camera and an optional laser mount for calibration.The most important part of the system is the Cardan module,which hosts a concentric Cardan joint.The outer Cardan joint consists of a ring connecting the steer-ing plate with the base which can be?xed to the ergonomic head mount.The ring can rotate around a vertical axis in the base(pan axis).The steering plate is connected to the ring with a pivot joint. This joint allows rotation along a horizontal axis through the ring (tilt axis)perpendicular to the pan axis.With this design three de-grees of freedom are realized for the steering plate,enabling the

Figure4:Prototype of the three-degree-of-freedom camera motion

device.

Figure5:Prototype of the camera motion device.Note the inner Cardan joint in comparison to a coin.

camera to move in different gaze directions.To perform these mo-tions,two servo actuators move the steering plate by push rods and universal joints.

To control motions around the camera’s optical axis,the inner Car-dan joint is induced.This joint connects the camera carrier with a shaft that is coupled to the servo actuator’s output shaft by a push rod.The roll actuator has been placed above the other two actuators to keep the space below the other actuators free.Thus,the camera motion device can be placed closer to the human eyes to reduce the parallax error.It is a drawback that a third push rod becomes nec-essary.With this setup the camera carrier,which is mounted on the steering plate with ball bearings,can be rotated around the roll axis independently of the steering plate’s current orientation.For any given gaze direction,the camera can be rolled around its optical axis.

To keep the Cardan joints as small as possible,the inner Cardan joint has a maximum outer diameter of just6mm,and the steel bolts have a diameter of0.8mm(see Figure5).The device is there-fore based on two parallel mechanisms,in which the outer joint in?uences the inner joint,while the outer joint is completely

inde-Figure6:Kinematics of the three-degree-of-freedom camera mo-tion device.The upper part of the?gure shows a CAD model of the device with the description of the axes and the Cardan angles. In the lower part of the?gure the pan tilt mechanism(right)and the roll mechanism(left)are shown.Other components have been hidden for easier understanding.

pendent of the inner joint.

Except for the servo module made of POM(polyoxymethylene), the complete setup is machined from aluminum.Lightweight design has been implemented wherever possible.All joints are equipped with miniature precision ball bearings.The overall mass of the setup is as small as160g,with the concentric Cardan module having a mass of only15g.If smaller and lighter servo actuators are used,the mass of the complete system could be reduced to less than100g.The camera module itself(Lechner CCTV MK-0326) weighs15g.

3.2Inverse Kinematics and Workspace Optimiza-

tion

The challenge of dealing with parallel systems is usually the com-plex inverse kinematics.The solution to this problem is therefore a major task in this area.In the presented camera motion device, two different mechanisms are combined so that they partly in?u-ence each other.The?rst mechanism is the already known parallel mechanism that combines the motions of the two actuators,which perform the pan and tilt motions of the steering plate,thus steering the camera into the desired gaze directions.This mechanism will be referred to as the pan tilt mechanism.For the camera motion device,the Cardan angle de?nition has been used.Here the angular motion along the pan axis is described with the Cardan angleα.For rotations around the horizontal tilt axis the Cardan angleβis used. The task of the inverse kinematics is to calculate the correct actua-tor anglesφ1andφ2for the left and the right servo actuators from the known Cardan anglesαandβ.A CAD model of the kinematics of the camera motion device is shown in Figure6.The described angles are denoted.Pan,tilt,and roll axes have been included to illustrate how the system functions.For the given kinematic struc-ture,an analytical solution has been derived.

The second mechanism basically consists of the inner Cardan joint that connects the camera carrier with the output shaft of the roll ac-tuator;it will be called the roll mechanism in the following.This mechanism transfers the rotational motionφ3of the roll actuator to the camera,thus leading to an angular camera motion with the Car-

dan angleγ.While the roll mechanism does not in?uence the kine-

matics of the pan tilt mechanism,the pan tilt mechanism strongly affects the roll mechanism.For any combination of Cardan angles αandβ,a new gaze direction is set,leading to new constraints for the roll mechanism.Thus,the actuator angleφ3not only depends

on the desired roll angleγ,but also on the current pan and tilt angles αandβ.

We will not focus here on the solution of the inverse kinematics,be-cause the calibration described in section2.1renders exact knowl-edge of the mechanical behavior unnecessary.With the given cali-bration a direct coordinate mapping from eye tracker camera space to the actuator anglesφ1andφ2is possible without the need to know the anglesαandβ.Nevertheless,if the head mounted camera sys-tem is used in the various?elds of oculomotor research which re-quire exact knowledge of the eye movement angles and velocities, the analytical solution of the inverse kinematics would help to mea-sure these relevant oculomotor variables.In this application,how-ever,the inverse kinematics of the pan tilt mechanism have only been used to optimize the workspace,which will be described in the following.

The desired workspace of the camera motion device has been cho-sen to be in the range of±45deg for all three Cardan anglesα,β,andγ.For the roll angle,this is more than the physiologically required range of±15deg[Tweed et al.1998];however,the range of±45deg leaves enough room for arti?cial augmentation of the physiologically restricted eye movements in roll by,for example,a gain enhancement.The roll angleγis not involved in the pan tilt mechanism and since it is therefore not limited by the constraints of the parallel kinematics,it has not been considered in the optimiza-tion.The angleγcan easily cover the given range of±45deg. Due to the kinematic constraints of the parallel pan tilt mechanism, the desired range cannot be reached in the?rst attempt.Hence,to enlarge the workspace of the pan tilt mechanism,an optimization algorithm has been used in combination with certain geometrical deliberations.The optimization aim was to maximize the number of possible combinations of the anglesαandβwhich the mechanism can adjust.To achieve this,certain geometrical parameters of the kinematics were subjected to the variations of the algorithm.The result of the workspace optimization is shown in Figure7.Every point in the diagram represents a certain gaze direction of the cam-era motion device.This gaze direction is described by the corre-sponding combination of certain anglesαandβ.The area inside the inner shape symbolizes the workspace of the?rst camera mo-tion device without geometrical optimizations(starting con?gura-tion).That is,the mechanism can adjust all combinations of angles αandβwithin the shape.Note that for greater absolute values of α,only small variations ofβare possible,and vice versa.The area inside the outer shape symbolizes the possible combinations ofαandβafter an optimization using the Implicit Filtering algorithm (optimized con?guration).A description of this algorithm can be found in[Choi et al.2001]and[Kelley1998].Note that almost any point in the complete area between?45and45deg can be achieved for both angles.The radius of the actuator levers in the?rst setup was chosen to be R=10.75mm.It is a trivial observation that for longer actuator levers,the workspace will become broader.To achieve a compromise between long actuator levers and a compact system,the new radius of the actuator levers in the optimization has been?xed to R=20mm.Additionally,the universal joints at the steering plates have been set into the plane that is described by the pan and tilt axes forαandβbeing equal to zero.In neutral position the push rods and the line between the universal joints at the steer-ing plate are nearly perpendicular.The other geometrical values for joint positions and servo positions have been optimized using the Implicit Filtering algorithm to?nd the ideal geometry in terms of maximization of the desired workspace.The proposed changes in the geometrical values are relatively small.In general,the algo-rithm suggests that the distance between the universal joints at the steering plate should be shortened and moved closer to the tilt axis. This result is not surprising.The most important advantage of the optimization,however,is that under the given constraints,optimum values can be derived for the geometry.

Figure7:Results of the optimization of the parallel camera motion device’s workspace.αdescribes the camera’s orientation around the vertical(pan)andβthe orientation around the horizontal(tilt) axis.The curves surround the area of possible combinations ofαandβof the parallel mechanism.The optimization was achieved using the Implicit Filtering algorithm.Note the difference between the area inside the starting con?guration line and the area inside the line of the optimized con?guration.

3.3Drive Speci?cation

The actuators must be able to reach high velocities and accelera-tions to move the selected camera so that it performs like the human oculomotor system.Other important requirements are low weight, small dimensions,and high reliability.After an online search on different actuator systems,we have chosen electric DC motors with reduction gears.The most appropriate actuators turned out to be servo actuators for model airplanes.A major advantage of these ac-tuators is the fact that DC motor,gear,and a position control are all integrated in one package,making the complete actuator extremely compact.Furthermore,these actuators deliver both high velocities and high torque.They are also almost backlash-free and contain a motion control system with position feedback through a poten-tiometer.Therefore,no additional sensors have to be used as limit or reference switches.In the?rst camera motion device[Schnei-der et al.2005]Graupner DS281Actuators,which belong to the smallest servo motors available,were used.The price paid for the small size and weight(9g)was a low maximum speed(250deg/s). Now as the system had to be capable of performing eye-like mo-tions,much faster actuators were needed.Especially during fast eye movements,the angular velocity of the camera was not high enough to deliver good enough image quality.So for the new prototype,the chosen actuators(Graupner DS3781,Kirchheim,Germany)are among the fastest servo actuators that still have a reasonable size and weight(1000deg/s at a mass of just28g).With these compo-nents a camera motion device can be demonstrated with dynamic properties that come close to or even excel those of the human eye.

4Electronic Setup

A major drawback of the?rst system described in[Schneider et al. 2005]was a high over-all latency due to high camera latencies and slow servo actuators and electronic devices.The over-all latency of that system was on the order of100ms and therefore more than an order of magnitude above the10ms latency of the vestibulo-ocular re?ex[Leigh and Zee1999].With this high latency,suf?cient im-age stabilization could not be achieved.An important milestone of development was therefore to minimize the over-all latency.

In a?rst step,we replaced the comparably slow(50Hz)analog video cameras of the underlying eye tracker system.The new sys-tem is equipped with a IEEE-1394digital camera(Flea,Pointgrey, Vancouver).With pixel binning it can operate at an image resolu-tion of320x240pixels and a frame rate of100Hz.

The off-the-shelf servo-controllers used in[Schneider et al.2005] were limited in their sampling rate to the standard remote-control pulse width modulation(PWM)rate of50Hz.The actuators used here(see section3.3),however,are capable of treating PWM sig-nals up to approximately400Hz.Besides,the baud rate supported by the previously used controller was limited to19200kBaud. Therefore,the latency for transmitting control commands via the RS232-interface to the controller was inacceptably high.

To overcome these limitations,a new servo controller based on a PIC18F4431microcontroller has been designed.It can control up to four servo actuators at arbitrary pulse rates.The maximum rate, however,is limited by the required PWM signal,which has a max-imum pulse width of2ms.Therefore,the sampling rate must be lower than500Hz to be able to generate separate pulses.In the given setup,a frame rate of300Hz has been chosen to ensure cor-rect functioning of the actuators.The servo controller is connected to a RS232interface working at115200kBaud.

Figure8:Schematic setup for latency measurements.A servo mo-tor rotates an arti?cial eye whose pupil is detected by an eye tracker system.Control commands for the head mounted camera are cal-culated from the pupil movements.Gyroscopes measuring the an-gular velocities are attached on both the arti?cial eye and the head mounted camera.

The controller board additionally serves as a power supply system for the actuators,the calibration laser,and the head mounted cam-era.A switching voltage regulator and a DC/DC-converter generate the required voltages from the voltage of the IEEE-1394connector. The calibration laser of the camera motion device is also switched on and off by the controller board.A Firewire-hub(Macally FH-220)is used to connect eye tracking cameras for both eyes.It is assembled together with the servo controller in a small plastic case and attached to the back of the head mount.

For the?rst tests on mobility and functionality,the system was con-nected to an Acer TravelMate382TMi Laptop(Intel Pentium M725 processor,1.6Ghz,400Mhz FSB,2MB L2cache,512MB DDR RAM).Since the laptop adheres to the“legacy-free”design con-siderations,a USB-to-RS232converter(Manhattan products Part 205146)has been used.It introduces an additional latency of ap-proximately1ms compared to a direct coupling with a RS232in-terface.A color analog video camera module with standard PAL output(Mintron MTV-03X10H)has been used as the gaze camera.

Figure9:Delay of the complete system:Measured angular veloc-ities(see Figure8)at3.33Hz.The solid black curve indicates the measured(raw)velocity of the head mounted camera.The dashed black curve shows the sine?t to this velocity from the regression calculation.The solid gray line indicates the measured velocity of the arti?cial eye,and the dashed gray line shows the corresponding sine?t.

5Ergonomic Head Mount

The lightweight head mount is attached to the eye tracker goggles. It consists of a curved aluminum plate on the front and a curved elastic plastic plate which continues onto the back of the head.The goggles and the head mount are connected by a frame joint,which can be adjusted and then rigidly?xed at an angle matching the wearer’s head geometry.The curved plastic plate,which is rigidly attached to the aluminum plate,extends beyond it to the back of the head.With this con?guration,the head mount also serves to stabi-lize the eye tracker goggles.An initially loose,pliable,broad strap is placed between the aluminum plate and the user’s head in order to increase the contact surface,thereby improving the comfort and wearability by decreasing the surface load on the head.Adjustable straps like those on a bicycle helmet can be tightened around the head to prevent slippage.The base plate of the camera motion de-vice described in section3.1is mounted above the user’s forehead on the curved metal plate of the head mount.For further adaptabil-ity to different head geometries,two slots down the aluminum plate allow the gaze camera position to be adjusted until appropriate;it is then rigidly?xed.Supply cables are attached to the aluminum and plastic plates,which also serve as carriers for electronic modules controlling the actuators,the eye tracker,the actuated camera,and the calibration laser.A small plastic case is attached to the outer side of the plastic plate in back.It contains the necessary electronic

components and acts as a cord grip for the supply line as well as a counterweight to the camera motion device in front.

5°

Figure10:Head camera view of the used calibration pattern.Dark dots are?xation targets,the bright spot is the laser projection. Crosshaired markers show the positions from automated target de-tection.Differences between?xation targets(dark)and actual cam-era position(bright)are a measurement for the precision of the com-plete system.

6Results

An overview of our?rst experiences and measurements with the head mounted camera system is given in this section.

6.1Latencies

To measure the latency of the complete system,we attached an ar-ti?cial eye to an additional actuator,which is controlled indepen-dently by a pro?le generator.We used sinusoidal pro?les at5fre-quencies ranging from1Hz to10Hz.The eye tracking system detects the arti?cial eye’s pupil as it would with a real eye.The detected eye position serves as input to control the head mounted camera.Both the angular velocities of the arti?cial eye and of the head mounted camera are synchronously measured by the attached, integrated gyroscopes(see Figure8).

We calculated the phase shift of both angular velocities by?tting sine curves to the measured(noisy)signals with means of regres-sion https://www.360docs.net/doc/585260466.html,ing the known frequency,we can compute the latency from the phase shift.Figure9shows the measurement val-ues at3.33Hz.

Our measurements at different frequencies yielded the following mean latencies:As a?rst major result the overall latency has been signi?cantly reduced compared to that of the?rst eye movement driven head camera.The latency of the new setup is relatively close to the la-tency of the human vestibulo-ocular re?ex,which is on the order of 10ms[Leigh and Zee1999].

6.2Precision

To measure the accuracy of the camera positioning,we calibrated the system as described in the above section2.1.A subject wearing the head mount was asked to?xate25targets in front of him/her (viewing distance1.18m).The targets were spread out regularly at distances of8.5degrees of viewing angle in each horizontal and vertical direction.At each?xation point,two images of the head mounted camera were recorded in fast sequence.The?rst image required no additional prerequisites,but for the second image,the calibration laser was?ashed for approximately100ms.This proce-dure allowed the automatic detection of both the?xation target and the alignment of the head mounted camera(indicated by the laser spot)by means of image processing.

Figure10shows a representative camera image after the analysis of one?xation of the precision measurement;the different locations of the target dot(black)and the bright laser clearly indicate the camera’s view direction.The deviations between target locations and the corresponding camera positions for all25?xation tasks are summarized in Figure6.2.This precision measurement yielded a mean value of0.5deg for all distances between the?xation tar-get and the camera alignment(accuracy)and a standard deviation (resolution)of0.25deg.

Figure11:Target points and camera directions for the precision measurement.Circles indicate the target points,which have to be ?xated by the subject.Crosses show the actual gaze camera align-ment.The mean distance between targets and corresponding cam-era positions is a measure of the spatial accuracy of the complete system.

7Conclusion

Video recordings from the subjective perspective of a mobile user are possible with this more advanced three-degree-of-freedom pro-

totype of a gaze-driven camera.The camera has proven to be usable with lower-frequency head movements and with ocular following as well as during locomotion.The calibration procedure required only minimal user interaction and therefore it did not distract the user from his main tasks.The accuracy of the whole system–includ-ing the user–is in the range of0.5deg,which is well within the 2deg of foveal vision.Such a camera system can already be used for documentation and teaching purposes in,e.g.,health care,qual-ity assurance,movie making,and even sports.A consumer product based on this prototype is also conceivable.Future developments will focus on improving the latency even further with faster eye tracker cameras,on implementing image processing mechanisms that detect eye tracker slippage during fast head movements,and on optimizing the ergonomic constraints. Acknowledgments

This study was supported by the Bayerische Forschungsstiftung (FORBIAS)and the Deutsche Forschungsgemeinschaft(GL342/1-1).We would like to thank one reviewer for valuable comments and Dr.Hamish MacDougall for fruitful discussions on the design of the eye tracker goggles.

References

B ETSCH,B.Y.,E INH¨AUSER,W.,K¨ORDING,K.P.,AND K¨ONIG,

P.2004.The world from a cat’s perspective–statistics of natural videos.Biol Cybern90,41–50.

B OENING,G.,B ARTL,K.,D ERA,T.,B ARDINS,S.,S CHNEI-

DER,E.,AND B RANDT,T.,2006.Mobile eye tracking as a basis for real-time control of a gaze driven head-mounted video camera.Unpublished.

C HOI,T.D.,E SLINGER,O.J.,G ILMORE,P.A.,K ELLEY,C.T.,

AND P ATRICK,H.A.,https://www.360docs.net/doc/585260466.html,er’s Guide to IFFCO.Center for Research in Scienti?c Computation,North Carolina State Uni-versity.

G IENGER,M.,L¨OFFLER,K.,AND P FEIFFER,F.2001.Towards

the design of a biped jogging robot.In International Conference on Robotics and Automation(ICRA).

G IENGER,M.,2005.Entwurf und Realisierung einer zweibeinigen

Laufmaschine.VDI-Berichte,Reihe1,Nr.378,VDI-Verlag.

G OSSELIN,C.M.,AND H AMEL,J.-F.1994.The agile eye:a

high-performance three-degree-of-freedom camera-orienting de-vice.In Proceeding of the IEEE International Conference on Robotics and Automation.

K ELLEY,C.T.,1998.Unconstrained implicit?ltering code:Im-plicit?ltering with sr1and bfgs quasi-newton methods.Source code online,https://www.360docs.net/doc/585260466.html,/ctk/darts/im?l.m.

L EIGH,R.J.,AND Z EE,D.S.,1999.The neurology of eye move-ments.Oxford University Press,New York,Oxford.

M AYOL,W.W.,T ORDOFF,B.,AND M URRAY,D.W.2000.

Wearable visual robots.In Proceedings of the International Sym-posium on Wearable Computing.

O LSHAUSEN,B.A.,AND F IELD,D.J.2005.How close are we to understanding v1?Neural Comput17,1665–1699.P ATLA,A.E.,AND V ICKERS,J.N.2003.How far ahead do we look when required to step on speci?c locations in the travel path during locomotion?Exp Brain Res148,133–138.

P ELZ,J.B.,AND C ANOSA,R.2001.Oculomotor behavior and perceptual strategies in complex tasks.Vision Res41,3587–3596.

R IEBE,S.,AND U LBRICH,H.,2003.Modelling and online-computation of the dynamics of a parallel kinematic with six-degrees-of-freedom.Archive of Applied Mechanics,V ol.72, No.11-12,Springer-Verlag,Berlin,Heidelberg,New York,July.

S CASSELLATI,B.,1998.A binocular,foveated active vision sys-tem.MIT AI Memo1628,March.

S CHNEIDER,E.,B ARTL,K.,B ARDINS,S.,D ERA,T.,B OEN-ING,G.,AND B RANDT,T.2005.Eye movement driven head-mounted camera:It looks where the eyes look.In Proceed-ings of the IEEE Conference on Systems,Man and Cybernetics (SMC2005),2437–2442.

T RUONG,H.,A BDALLAH,S.,R OUGEAUX,S.,AND Z ELINSKY,

A.2000.A novel mechanism for stereo active vision.In Pro-

ceedings of the Australian Conference on Robotics and Automa-tion(ACRA2000).

T WEED,D.,H ASLWANTER,T.,AND F ETTER,M.1998.Optimiz-ing gaze control in three dimensions.Science281,1363–1366.