Downloaded 15 times
![Model-based Gaze Estimation
• Detailed geometric modeling between light sources, corneal, and
camera [Guestrin and Eizenman, 2006]
• Pros
• Accurate (reported performance < 1o)
• 3D gaze direction
• Head pose invariant
• Cons
• Need careful hardware calibration
Figure from [Guestrin and Eizenman, 2006]](https://image.slidesharecdn.com/towardaccurateandrobustcross-ratiobasedgazetrackers-etra2014-140421174541-phpapp02/75/Toward-Accurate-and-Robust-Cross-Ratio-based-Gaze-Trackers-Through-Learning-From-Simulation-ETRA-2014-8-2048.jpg){kind=tracking}
![Interpolation-based Gaze Estimation
• Learn polynomial regression from subject-dependent calibration
• Directly map from normalized to Point of Regard (2D PoR)
[Cerrolaza et al., 2008]
• Pros
• Simple to implement
• No need for hardware calibration
• Cons
• Head pose sensitive](https://image.slidesharecdn.com/towardaccurateandrobustcross-ratiobasedgazetrackers-etra2014-140421174541-phpapp02/75/Toward-Accurate-and-Robust-Cross-Ratio-based-Gaze-Trackers-Through-Learning-From-Simulation-ETRA-2014-9-2048.jpg){kind=tracking}
![Cross-Ratio based Gaze Estimation
• Gaze estimation by exploiting invariance of a plane
projectivity [Yoo et al. 2002]
• Pros
• Simple to implement
• No need for hardware calibration
• Head pose invariant
• Cons
• Large subject dependent bias occur
because simplifying assumptions Figure from [Coutinho and Morimoto 2012]](https://image.slidesharecdn.com/towardaccurateandrobustcross-ratiobasedgazetrackers-etra2014-140421174541-phpapp02/75/Toward-Accurate-and-Robust-Cross-Ratio-based-Gaze-Trackers-Through-Learning-From-Simulation-ETRA-2014-10-2048.jpg){kind=tracking}
![Two Sources of Errors [Kang et al. 2008]
• Angular deviation of visual axis and optical axis
• Virtual image of pupil center is not coplanar with corneal
reflections](https://image.slidesharecdn.com/towardaccurateandrobustcross-ratiobasedgazetrackers-etra2014-140421174541-phpapp02/75/Toward-Accurate-and-Robust-Cross-Ratio-based-Gaze-Trackers-Through-Learning-From-Simulation-ETRA-2014-12-2048.jpg){kind=tracking}
![Improve Accuracy for Stationary Head
CR [Yoo-2002]
CR-Multi [Yoo-2005]
CR-HOM [Kang-2007]
CR-HOMN [Hansen-2010]
CR-DV [Coutinho-2006]
No correction
Scale correction
Scale and translation correction
Homography correction
Homography correction
+ Residual interpolation](https://image.slidesharecdn.com/towardaccurateandrobustcross-ratiobasedgazetrackers-etra2014-140421174541-phpapp02/75/Toward-Accurate-and-Robust-Cross-Ratio-based-Gaze-Trackers-Through-Learning-From-Simulation-ETRA-2014-13-2048.jpg){kind=tracking}
![Improve Robustness for Head
Movements
No adaptation Adapt to eye
depth variations
Adapt to eye movements
Assumptions
1) weak perspective
2) fixed eye parameters.
CR [Yoo-2002] CR-DD [Coutinho and
Morimoto 2010]
PL-CR [Coutinho and
Morimoto 2012]](https://image.slidesharecdn.com/towardaccurateandrobustcross-ratiobasedgazetrackers-etra2014-140421174541-phpapp02/75/Toward-Accurate-and-Robust-Cross-Ratio-based-Gaze-Trackers-Through-Learning-From-Simulation-ETRA-2014-14-2048.jpg){kind=tracking}
![Accuracy of Gaze Prediction for
Stationary Head
Robustness to Head
Movement
No adaptation
CR [Yoo-2002]
CR-Multi [Yoo-2005]
CR-DV [Coutinho-2006]
CR-HOM [Kang-2007]
CR-HOMN [Hansen-2010]
No correction
Scale correction
Scale and translation
correction
Homography correction
Homography correction +
Residual interpolation
CR-DD [Coutinho-2010]
Adapt to eye depth
variations only
PL-CR [Coutinho-2012]
Adapt to eye movements
Assumptions
1) weak perspective
2) fixed eye parameters.
Adapt to eye movements
No assumptions on
1) weak perspective
2) fixed eye parameters
This paper](https://image.slidesharecdn.com/towardaccurateandrobustcross-ratiobasedgazetrackers-etra2014-140421174541-phpapp02/75/Toward-Accurate-and-Robust-Cross-Ratio-based-Gaze-Trackers-Through-Learning-From-Simulation-ETRA-2014-15-2048.jpg){kind=tracking}
![How? The Main Idea
• Build upon the homography normalization method [Hansen et al
2010]
• Improving accuracy and robustness simultaneously by introducing the
Adaptive Homography Mapping](https://image.slidesharecdn.com/towardaccurateandrobustcross-ratiobasedgazetrackers-etra2014-140421174541-phpapp02/75/Toward-Accurate-and-Robust-Cross-Ratio-based-Gaze-Trackers-Through-Learning-From-Simulation-ETRA-2014-16-2048.jpg){kind=tracking}
![Experimental Results – Synthetic data
• Setup
• Screen size 400mm x 300mm
• Four IR lights
• Camera 13mm focal length, placed slighted below the screen border
(FoV~31 degree)
• Calibration position and eye parameters
• Eye parameters from [Guestrin and Eizenman, 2006]](https://image.slidesharecdn.com/towardaccurateandrobustcross-ratiobasedgazetrackers-etra2014-140421174541-phpapp02/75/Toward-Accurate-and-Robust-Cross-Ratio-based-Gaze-Trackers-Through-Learning-From-Simulation-ETRA-2014-26-2048.jpg){kind=tracking}
![Conclusions
• A learning-based approach for simultaneously compensating (1)
spatially varying errors and (2) errors induced from head movements
• Generalize previous work on compensating head movements using
glint geometric transformation [Cerroaza et al. 2012] [Coutinho and
Morimoto 2012]
• Leveraging simulated data avoid the tedious data collection](https://image.slidesharecdn.com/towardaccurateandrobustcross-ratiobasedgazetrackers-etra2014-140421174541-phpapp02/75/Toward-Accurate-and-Robust-Cross-Ratio-based-Gaze-Trackers-Through-Learning-From-Simulation-ETRA-2014-41-2048.jpg){kind=tracking}
Uploaded byJia-Bin Huang
Toward Accurate and Robust Cross-Ratio based Gaze Trackers Through Learning From Simulation (ETRA 2014)
This document proposes a learning-based approach to improve the accuracy and robustness of cross-ratio based gaze estimation. It introduces an adaptive homography mapping method that uses both head pose variables and pupil center position as predictor variables in a quadratic regression model. This approach is trained on large amounts of simulated eye tracking data to minimize errors across different head poses and eye parameters. Experimental results show the method achieves state-of-the-art accuracy for both stationary gaze and head movements, and is robust to variations in eye features, sensor resolution, and noise.
More Related Content
Similar to Toward Accurate and Robust Cross-Ratio based Gaze Trackers Through Learning From Simulation (ETRA 2014)
Toward Accurate and Robust Cross-Ratio based Gaze Trackers Through Learning From Simulation (ETRA 2014)
- 1. Toward Accurate andRobust Cross- Ratio based Gaze Trackers Through Learning from Simulation Jia-Bin Huang1, Qin Cai2, Zicheng Liu2, Narendra Ahuja1, and Zhengyou Zhang2 21
- 2. Why? • Multimodal naturalinteraction • Gaze + touch, gesture, speech If I were an iron man…
- 3. Why? • Understanding userattention and intention
- 4. Why? • Understanding interactionamong people Before sunrise 1995
- 5. Sclera Limbus Pupil Iris Glint Cornea (like aspherical mirror) Mike @ Monster University
- 6. Geometric Model ofan Eye
- 7. Gaze Estimation using PupilCenter and Corneal Reflections Interpolation- based Cross-Ratio based Model-based
- 8. Model-based Gaze Estimation •Detailed geometric modeling between light sources, corneal, and camera [Guestrin and Eizenman, 2006] • Pros • Accurate (reported performance < 1o) • 3D gaze direction • Head pose invariant • Cons • Need careful hardware calibration Figure from [Guestrin and Eizenman, 2006]
- 9. Interpolation-based Gaze Estimation •Learn polynomial regression from subject-dependent calibration • Directly map from normalized to Point of Regard (2D PoR) [Cerrolaza et al., 2008] • Pros • Simple to implement • No need for hardware calibration • Cons • Head pose sensitive
- 10. Cross-Ratio based GazeEstimation • Gaze estimation by exploiting invariance of a plane projectivity [Yoo et al. 2002] • Pros • Simple to implement • No need for hardware calibration • Head pose invariant • Cons • Large subject dependent bias occur because simplifying assumptions Figure from [Coutinho and Morimoto 2012]
- 11. The Basic Formof Cross-Ratio Method Image Corneal Display
- 12. Two Sources ofErrors [Kang et al. 2008] • Angular deviation of visual axis and optical axis • Virtual image of pupil center is not coplanar with corneal reflections
- 13. Improve Accuracy forStationary Head CR [Yoo-2002] CR-Multi [Yoo-2005] CR-HOM [Kang-2007] CR-HOMN [Hansen-2010] CR-DV [Coutinho-2006] No correction Scale correction Scale and translation correction Homography correction Homography correction + Residual interpolation
- 14. Improve Robustness forHead Movements No adaptation Adapt to eye depth variations Adapt to eye movements Assumptions 1) weak perspective 2) fixed eye parameters. CR [Yoo-2002] CR-DD [Coutinho and Morimoto 2010] PL-CR [Coutinho and Morimoto 2012]
- 15. Accuracy of GazePrediction for Stationary Head Robustness to Head Movement No adaptation CR [Yoo-2002] CR-Multi [Yoo-2005] CR-DV [Coutinho-2006] CR-HOM [Kang-2007] CR-HOMN [Hansen-2010] No correction Scale correction Scale and translation correction Homography correction Homography correction + Residual interpolation CR-DD [Coutinho-2010] Adapt to eye depth variations only PL-CR [Coutinho-2012] Adapt to eye movements Assumptions 1) weak perspective 2) fixed eye parameters. Adapt to eye movements No assumptions on 1) weak perspective 2) fixed eye parameters This paper
- 16. How? The MainIdea • Build upon the homography normalization method [Hansen et al 2010] • Improving accuracy and robustness simultaneously by introducing the Adaptive Homography Mapping
- 17. Adaptive Homograph Mapping •Two types of predictor variables • : capture the head movements relative to the calibration position • Affine transformation between the glints quadrilateral • : capture gaze direction for spatially-varying mapping • Pupil center position in the normalized space • : polynomial regression of degree two with parameter
- 18. Training Adaptive HomographyMapping • Exploit large amount of simulated data • the set of sampled head position in 3D • the set of calibration target index in the screen space • Objective function
- 19. Minimizing the ObjectiveFunction • Minimize an algebraic error at each sampled head position • Use the solution from algebraic error minimization as initialization Minimize the re-projection errors using the Levenberg-Marquardt algorithm
- 20. Visualize the TrainingProcess • Eye gaze prediction results using the bias-correcting homography computed at the calibration position
- 21. RMSE Error ComparisonsUsing Different Training Models • Differences are small in linear regression • Linear model is not sufficiently complex • Compensation using both predictor variables achieve the lowest errors
- 22.
- 23. Linear Regression Adding thenormalized pupil center corrected spatially-varying errors
- 24.
- 25.
- 26. Experimental Results –Synthetic data • Setup • Screen size 400mm x 300mm • Four IR lights • Camera 13mm focal length, placed slighted below the screen border (FoV~31 degree) • Calibration position and eye parameters • Eye parameters from [Guestrin and Eizenman, 2006]
- 27.
- 28.
- 29. Stationary Head Varying (horizontal)angle between optical/visual axis
- 30. Stationary Head Varying (vertical)angle between optical/visual axis
- 31. Head Movements Parallelto the Screen
- 32. Head Movement alongDepth Variation
- 33. Tested at AnotherHead Position
- 34.
- 35. Effect of SensorResolution (at calibration) Focal Length = 13 mm Focal Length = 35 mm
- 36. Effect of SensorResolution (at new position) Focal Length = 13 mm Focal Length = 35 mm
- 37. Real Data Evaluation– Programmable Hardware Setup Off-axis IR light sources Stereo camera (We use one only in this work) On-axis ring light
- 38. Real Data Evaluation– Feature Detection • Detecting glints and pupil center
- 39. Averaged Gaze EstimationError at calibration position
- 40. Averaged Gaze EstimationError Calibrated at 600mm from screenCalibrated at 500mm from screen
- 41. Conclusions • A learning-basedapproach for simultaneously compensating (1) spatially varying errors and (2) errors induced from head movements • Generalize previous work on compensating head movements using glint geometric transformation [Cerroaza et al. 2012] [Coutinho and Morimoto 2012] • Leveraging simulated data avoid the tedious data collection
- 42. Future Work • Considersubject-dependent parameters in the learning and inference the adaptive homography adaptation • Integrate binocular information, please see poster Zhengyou Zhang, Qin Cai, Improving Cross-Ratio-Based Eye Tracking Techniques by Leveraging the Binocular Fixation Constraint • Extensive user study using a physical setup
- 43. Comments or questions? Jia-BinHuang [email protected] Narendra Ahuja [email protected] Zhengyou Zhang [email protected] Qin Cai [email protected] Zicheng Liu [email protected]