Head tracking & smooth camera movement

Head tracking devices are not perfect. They are not accurate, have input lag. Result of that imperfection is jittering of image, distortions of 3D image. To make head tracking results acceptable to player, data from head tracking device must be filtered.

Right now BAM try to automatically select best filtering method. If you did not find results good enough, you may try to manually adjust filtering params or select different filtering method.

First, lets define unwanted distortions/problems:

  1. jittering - when you don't move head tracking device in every frame finds you in little different place. Result: virtual camera randomly moves in all directions when it should stay in one place.
  2. input lag - time is very important. Head tracking device need time to caputer your position. It need time to pass data to computer. OS need time to pass that data to program. Program need time to interpret data and change it to player position. Result: computer don't know your current position but your position some times ago. So you will see, that image on screen is adjusted with some delay.
  3. camera movement is not smooth - player position is not updated by tracking device for every rendered frame. Some times it is every second frame, sometimes less frequently. Result: your move is smooth, but image on screen change in "jumps".

Above problems are minimized with data filtering. I will explain only newest filter: "Predict V3".

  • Where player tracking menu
    Where is menu
  • Player tracking filter menu
    Player tracking menu

Options in Player tracking menu:

  • Filtering - you can allow BAM to select filtering method and params (Auto) or chose to set it manualy (Manual)
  • Mode - select filtering algorithm. I strongly recomend "Predict V3".

Why is "Predict V3" better than other filters? Because timing is super important. There is one more big problem with input lag. It is not constant. BAM recives data from head tracking device in variable intefvals. For example webcam can capture 30 FPS. So interval between frames should be 33.3 ms, but sometimes it is 50 ms and next time it is 20 ms. Well, it is never 33.3 ms. Source of problem is in operating system (Windows). This new filter in first step try to determine real time when tracking data are caputerd. All what it needs to find that missing variable lag is framerate of head tracking device. 

  • Predict player position section: Set framerate & Framerate - for precise math, filter need to know framerate of tracking device. By default it try to detect that framerate. But detection is not perfect, so in case if it faill you may set framerate manualy. Set framerate switch with options auto and manual controls if framerate should be auto-detected or set by user.
  • Jittering filter section: Range & Exponent - that params are used to remove jittering (problem no. 1). If Exponent is set to 1.0 its don't do anything. This setting is for devices like both Kinects with own jittering filters. If Exponent is grater than 1.0 it small movment will be suppressed. Range is just used to limit range of that move suppression. For WebCam jittering is very stong. I set Range and Exponent to 400.0 and 2.5.
  • Lag filter section: avg speed steps & in future - based on detected position, time of detected position & move speed, player position in future may be estimated. It is basic phisics. It is not perfect solution for removing input lag (problem no. 2) but it is all i have. Here you can set how far in future we are guessing and how many frames of head tracking data are used for speed calculation. Why we don't use "acceleration"? It creates to big errors. So that two options alow to minimize "input lag". Please note, that if you set in future to 0.0 filter will already eliminate that "variable" part of lag. Thats why i allow to set negative value for in future.
  • Smooth cam move section: Frames in advance - this alow to solve problem no. 3. Head tracking data don't always come in time. When data cames, we calc predicted position for expected time when new data should come. Very often that data comes later than we expected. This delay create aditional jittering. This params controls what to do with that delay and how many frames delay filter may expect.

Good new is: all should work without any user actions with Filtering set to auto.



Q: Webcam can change framerate based on how bright is image. For example in bright room webcam can deliver 30 FPS and in dark room only 15 FPS. What should i set as framerate?

A: Bigger value. If you set 30 FPS and webcam deliver only 15 FPS when every second frame will be missing. Filter should detect it and work fine. I tested it. Problems may occur when change in frame rate is not "two times". If in bright room webcam give 30 FPS and in dark 20 FPS it may create troubles. I don't know if any webcam can do it.


Q: Webcam can capture 30 FPS but my CPU is slow and it can detect face only on 20 FPS. I see only variable number of FPS in plugin. What to do?

A: For Webcam plugins decrease processing res, increase step scale. It is important to have constant numer of frames. It is more important than bigger accuracy. Also constant smaller framerate from HT device is better than big and variable.