Plugins

BAM has supported plugins since version v0.6.

 

Plugin installation.

Copy plugin dll (and other required files) to BAM/plugins dir.

Plugins sometimes require installation of drivers and third-part software. Remember to do this  before you try to use the plugin. If some hardware or software components required by plugin to run are missing,  then plugin will not appear in BAM menu. 

Plugin usage

Select the desired plugin in the mode menu as you would for any other mode. 

  • Free Track

  • Simple Cam menu

    Simple Cam

    You can use any webcam to track player's head position in real time.  Players head is traced only in 2D. You can move "virtual cam" in X & Y axis. Cam Z value is fixed. On default it is above cab/display border closest to player.

     

    Face detection is cpu power hungry. If You have 2 (or more) cores in Your machine it will not do big harm to FP frame rate.

    Installation

    This plugin is added to BAM as default.  Merely,  attach your webcam to your computer.  You can place camera on top of , or in front of Back box.

     

    Setup

    You have to make sure that Face Detection works. The green square should be displayed consistently on your face . If you have a problem with face detection first try to tweak cam params.
    For starters you can try:
    - Run the  program for your CAM from producer and try to:

    • Disable automatic gain & exposure regulation
    • Set it manually to make face clearly visible 

    Automatic regulation almost always has "gain"set  too high and face on cam is like a big bright spot.

    - Run BAM and test face recognition. If you don't see green box on "face" with default setting try again to set CAM params from previous point.

    Set values in plugin menu:

    - Detection pattern - LBP is fastest, but make more "jittering", I use "Haar 1".

    - Min Face Size - green square displayed on "cam view" have width/height. You can set minimal accepted size. Try to increase value. If you increase this value too much, then "green" box will disappear. Then reduce value little bit. By doing this you will prevent detection of objects "smaller" than your face that may be interpreted as a "false face". It will increase performance too.
    - Max Face Size -  is maximal accepted size. 

    - Step Scale - lower value will incease detection accuracy but reduce performance.

    Other Menu options:

    - Cam Z - You can move virtual camera position in Z axis.

    - Max threads - face detection can start multiple threads. More thread will increase CPU usage and number of processed frames from camera, but reduce whole game performance. Default value (2) is probably best.

    - Face Detection - switch to enable / disable FD algorythm.

     

    3 points

    Once you have accomplished "good" tracking, go to "3 points" menu. BAM needs to know distance between left and right edge of your table, and how high they are in cam. You do this by showing BAM the  first 2 points in "3 points" setup.

     

    You can "put" face on points, but it is inconvenient. (But it can be done). I use an object (plastic tube) with known height and set these points at known distances (only verticaly) from screen. This distance is measured to "eye" level. Put one end of "plastic tube" on "red diamond" displayed on screen. Put the other end at your face/"eye". Then press "set point" (face detection is required). Do likewise for setting the second point. 


    BAM will ask for third point. This point is used only to trace in 3D. With "Simple Cam" it is unused. You shoud set "any" point. This point is not used in calculations.

     

    About "distance" betwen screen and eye.

    This is distance between "red diamond" displayed on screen and "eye" in the moment when You press "Set point".

     

    When you finish "3 point" setup BAM recognizes the points as "level 0" (or CAM Y=0). Now you need to tell BAM, that "level 0" is lower. To do this You should  set this distance with "Shift Y"under "Reality" Submenu
    (Please remember to set size of your TV screen, because this value is used to translate values in CM or INCHes to Future Pinball Unit).

     

    Example

    This mode is simplest way to test head tracking. You do not need any equipment except webcam. You can see how this works in the following movies:


    by The Loafer


    by open6l

     

  • Simple PS3Eye

     

    This plugin is very similar to "Simple Cam". Only difference is the ability to change PS3Eye: Gain, Exposure and number of captured frames per sec (FPS).

     

    Installation:

    Plugin is added to default BAM installation. You only have to install special drivers:
    1. Uninstall EyeToy drivers.
    2. Download & install (with "developer" libs):
      CL-Eye-Platform-SDK.
    3. Install EyeToy drivers (same ones that you removed in step 1):
      CL-Eye-Driver
  • HT with PS3Eyes

    • Menu
      Menu

    This is similar plugin to WiiMote Cam.

    Installation:

    Plugin is added to default BAM installation. You have to only install special drivers (same like for Simpe PS3Eye):
    1. Uninstall EyeToy drivers.
    2. Download & install (with "developer" libs):
      CL-Eye-Platform-SDK.
    3. Install EyeToy drivers (same as removed in step 1):
      CL-Eye-Driver.

     

    Setup: dots detection:

    This is "a dot" or "how PS3Eye see IR-LED":
    dot

     

     Now look at one line of pixels:

    In the menu, you can set:

    • Min. brightnes - all points in picture from cam with brightnes below this level will be will be considered black (brightnes = 0). This value is used mostly to eliminate "noise" and separate dots from background. If you set Min. brightnes value to 0 and set gain & exposure like "Simple PS3Eye", you will see in "cam view" normal, picture from cams (well... almost).
    • Req. brightnes - every dot has one pixel with maximum brightnes, this value is used to eliminate "dots" to dark. I usly don't use this value.
    • Min. dot weight - for every "dot" is calculated "weight". It is the sum of brightnes of all pixels in dot. This value is used to eliminate "noise" from cams. 

    How to set it up:

    • gain - this is the typical param of camera. If you increase this value, it will increase brightnes of all pixels... but at the same time it will increase amount of "noise". It is like the digital zoom in a photo camera. It will allow you see better but will not increase quality of information for BAM. You should set this to zero for dot detection.
    • exposure - this is the time it takes for the camera sensor to gather light. This is the best way to increase brightness of "dots" because if you increase it, you will not increase "noise". I set it to: 200-300.
    • Min. brightnes - level of "noise" in camera picture, when i set gain=0 is ~30. You should initially set this value to ~20 (small amount of "noise" is not a problem for BAM). If you see "room" in cam view, you can increase this value. It will be no problem to set Min. brightnes=100. When you increase Min. brightnes,  you will probably have to increase exposure.
    • Req. brightnes - now it is not very useful. I usually set it low (=100). If You have object visible in cam view but this object "maximum brightnes" is lower than brightest of IR-LED You can increase this value to eliminate detection of this "spot" as IR-LED. Be carefull with this. If you move IR-LED to this bright spot, then around the IR-LED will not be a black border, BAM can detect pixels of LED and this spot as one big "dot".
    • Min. dot weight - i set this to 200-300. It can eliminate realy small false "dots" from background. 

    To get best "subpixel" accuracy BAM should get "dots" with high brightness. In the menu, there are few more options:

    • FPS - how many frames per second will be taken from cams. I set this to 60, but you can test some other setting.
    • Resolution - with 640x480 dot detection is more accurate, but FPS is limited to 75. Only in very rare situations you will see difference betwen 640x480 and 320x240. You can change this in any time. When you change it no calibration is required. Higher resolution requires more CPU power. Notice: When BAM draws picture from cams in menu, a lot of CPU time is used to manipulate pictures. This "CPU power" is used only when BAM menu is visible. "normal" game pixel detection doesn't consume a lot of CPU time.

    In menu there is option OSD. You can select what additional information will be displayed at "cam view". If you select "histogram" on top of cam view, a red line will be drawn. It represents number of pixels with specific brightes on picture. On right are pixels with lower values (starts from 1). It can help to set right "dots detection".

    • Calibration submenu
      Calibration submenu

    Calibration:

     This is submenu is to calibrate cams for head tracking. Most of the options are similar to "WiiMote cams":

    • Swap Cams
    • Calibration Board
    • Start calibration
    • "3 Points"

    I will not describe here how to use this (same procedure as "WiiMote cams"), but here are a few tips:

    1. During calibration don't try to "cover" whole visible space for cams.
    2. Start at "center" for both cams.
    3. Try to not rotate board.
    4. Keep calibration board in radius ~20 cm from start position.
    5. First 5-10 frames are most important. Try at begin have one fram at center, and 4 frames at square corners at same distance from cams.

    Problem with camera lenses: greater FOV gives greater distortions of picture.

     

    To make cams usable for "head tracking" BAM has to correct pictures taken from PS3Eye camera and make the straight lines, straight again. To accomplish this,  use the following  menu commands:

    • Fix Cam - LEFT
    • Fix Cam - RIGHT

    When You select this command, BAM will take 40 frames (with 4 dots of calibration board), and use this data to correct distortions. Process is very similar to "calibration of 2 cams", but You should "cover" whole visible space (in same distance from camera). You do it for one cam at time.  Then BAM collects 40 frames and will calculate "fix" for cams. Calculation will take 1-2 minutes (2x-4x longer than calibration for 2 cams).

    You can set OSD to grid. BAM will display grid on image. It will give You better view how  corrected picture from cam looks for BAM. It will probably look like this:

    Sometimes calibration goes wrong, and You will see really weird things in cam view.

     

    You should repeat "Fix cam lense" calibration (and make sure, that you set correct values in calibration board).

     

    • Fix Cam - more options
      Fix Cam - more options

    Fix Cam

    If you select more options in calibration menu, then you have access to Fix Cam params. In this menu:

    • osd - select additional info displayed over Cam View
    • cam - selet on which cam you work (Left / Right)
    • Load params - every time you change this option, plugin will load "fix cam" param to menu:
      - current - setting of Fix Cam you use now
      - Last modified - If you manually modify values and change again "Load" for example to "current", before loading of params, change values are stored in "Last modified"
      - Second Cam - If you want copy values from one cam to another, you can use this option
      - Last Saved - Every time you "save" setting (last option in this menu) values are: 1. set as fix cam for selected Cam. 2. Save to disk.
      - ravarcade 3.6mm - my settings. Plugin have built-in default params used to fix distortions.
    • k1, k2, k3 - are values from "polynomial" and describe lens curve. 
    • size - this value limits range of used "polynomial"
    • Save - set values for lens and save it to file: HT-PS3Eyes-FixCamExport.dat

    If you try to manipulete these values, you will see "real time" effects on cam view. When you change values remember to "save" it. If you leave menu without saving, there will be no changes in "fix cam". After you save "fix cam" params,  you have to do normal "stereo calibration". (If you don't, these new values will not be used. You will only see how they look in cam view).

    If you use modified manualy "fix cam" params plugin will set "Fix Cam Quality" to 1.0. Don't worry about this value. It is only valid if You use own "Fix Cam".

    You can share "HT-PS3Eyes-FixCamExport.data" file. From this file "FixCam" params can be loaded by others. (or entered manually).

    Why I have created these options?

    1. It is really hard to do good "Fix Cam".
    2. Fix Cam created for one cam, usually works very well with the other. For example my values created with replaced lenses work quite well with original PS3Eye lenses. It is strange, it should not work, but it does.
    3. This is fortunate , and if you do only normal stereo calibration (without making own "Fix Cam") head tracking will work very well.
    4. Try to do test like me on video: https://www.youtube.com/watch?v=C4tknTMgLXo 

    NOTICE:

    • Data calculated during "Fix cam" is valid as long as you don't change lenses (or change focus).
    • If you move cams, only normal calibration for 2 cams is required.
    • Under cam view BAM displays Camera Fix Quality. It should be lower than 0.5. I sometimes get it lower than 0.1. Very small values don't guarantee better results, but big values mean: calibration fails.
    • After "Fix cam," you can increase gain and exposure and set Min. brightnes to 0. This way you will see in cam view normal image of your room. You can check if the string lines are realy straight. (Use some long object with straight edges).
    • You will get better results of "Fix cam" and "Calibration (for 2 cams)" if dot detection has better accuracy. You can set resolution to "640x480" only for calibration.
  • Menu

    Kinect

    Kinect is GREAT. It can trace You in 3D without any other tools! So here is Kinect Head Tracking for BAM!!!

    ***if you purchased kinect with xbox360 console you may need to purchase the usb/power adapter that will connect it to your PC.  It can be ordered online through Amazon/Ebay.  I believe this adapter comes automatically when kinect is purchased separately from console.

     

    Plugin created by Andreas Benamou
    email: benamou.andreas@gmail.com

     

    Installation

    1. Download and install Kinect SDK
      http://download.microsoft.com/download/3/1/3/313117D2-315D-4450-86F7-0A761DA19F57/KinectSDK-v1.7-Setup.exe 
    2. Connect Kinect to PC. (allow system to install drivers. You can test/experiment with Kinect for pc with Kinect explorer demo program that come with the KinectSDK)
    3. Run BAM and select plugin: Kinect  

     

    Menu

    (**Note the following description applies to older versions of BAM which have since been updated to include automatic point detection as well as "skeleton" mode.  Skeletal mode performs skeletal tracking as opposed to head tracking. Skeletal tracking is very useful when player glasses and especially 3d glasses (when used in combination with active 3d mode) obstruct the Kinect IR signal.  Discussed under Bam Corner at Gopinball.com)

    Only two options:

    • Angle - You can use Kinect motor to tilt camera
    • Eye -  Kinect face detection can find eyes on face. You can chose left or right eye to be tracked.

    "3 Points"

    Befor BAM will track your head You have do "3 Points" setup. You have 3 ways to do it:

    • Easy "3 points" low - on screen is drawn in checkerboard. Kinect camera is rotated down. If checkerboard drawn on screen is recognized, then "3 points" setup is done.
    • Easy "3 points" high - same as "low", but checkerboard is drawn by a little different way.
    • "3 points" - old version. Use it if none of "Easy" option works.

    If You put Kinect "low", almost at screen, You should use "low" version. (recomended).
    If You put Kinect "high" for example on top of backbox - use "high" version.

    How to do old "3 Points" setup:

    1. Select "3 Points".
    2. Grab some object with know size (You can use book, anything).
    3. Put "object" on red diamond (left/bottom of screen).
    4. Put your head over it. (your chin should touch the "object").
    5. Make sure, that Kinect recognizes your face. (You will see yellow "mask").
    6. Press enter (set point).
    7. Repeat 3-6 with second point (right/bottom).
    8. Repeat 3-6 with third point. But... you don't have to put "object" exactly on red diamond. Just as far as possible from bottom screen line, when Your face is detected.
    9. Tracking will now work, but BAM will get Your position "too low".
    10. Enter "object" height in "shift-y" in "reality" menu. Add few more inches/cm (2-5in./5-12cm).
      Position is tracked to "center" of Your head. When you press Enter in "3 Points", tracked point You set is: ["object" height] + [distance from chick to center of head] above "red diamond".
    11. Test. Look at red diamond when BAM menu is visible. If your head is above screen, red diamond shoud be directly under head.
    12. Smoothening. Cam position is not updated very often. Go to: "Reality" menu and set "Latency" to ~60-100ms.
      (If You want to make nice move, increase it to 200ms and move camera realy slowly).

    After "3 Points" setup, you can change "tilt" angle. If you move Kinect, You have to repeat "3 Points".

     

    • KINECT placment
      KINECT placment

    Where to put Kinect

    • best place is low, before the Backbox. This way you will be able to look down at table almost vertically. But there is one problem. Kinect for XBOX 360 can trace objects 0.8m away from camera. It will be hard to set third point in "3 Points". Kinect for Windows can trace 0.4m away from camera. 
    • top of backbox. It will be "easier" to set third point, but Kinect will probably lose tracking, when you look down.
    • if you use Easy "3 Points", Kinect depth range limit is not a problem (only color camera is used), so try to put kinect "low".
    • Kinect camera must see "checkerboard" on screen. If it fails in Easy "3 points" low, try to put Kinect 1-2 inches (2-5cm) higher on book or small box.

     

    Tip: DO NOT RUN BAM-TRACKER.EXE WHEN YOU WANT TO PLAY FUTURE PINBALL.
    (or you will get message "Kinect not found")

  • Kinect placment

    Kinect 2

    To use Kinect 2 with BAM you will need.

    Hardware:

    • Kinect 2 (Kinect for XBOX One)
    • Kinect  Adapter for Windows: Microsoft store
    • Windows 8 or newer
    • USB 3.0 port. (Warning: some USB 3.0 controlers are not supported by Kinect).

    Software:

    • KinectRuntime: https://download.microsoft.com/download/A/7/4/A74239EB-22C2-45A1-996C-2F8E564B28ED/KinectRuntime-v2.0_1409-Setup.exe

     

    Where Kinect should stand?

    You will see what is visible to Kinect in BAM. Kinect must see 2 things: player and part of payfield screen. If Kinect is placed "high" (at top of backbox) it must see ~ 40% of screen. It is same space where BAM menu is displayed. If Kinect is placed "low", it must see ~80% of screen.

     

    Calibration

    Before Kinect 2 will be able to trace you, it must find where is playfield display. To do this, you need to run calibration command from Kinect 2 menu. BAM will display checkerboard patern on screen. If Kinect successfully detect that pattern, you will see message on screen and all will work. If you move Kinect to other place or rotate it, you will have to repeat calibration.

    Kinect v2 uses skeleton tracking. It detect head position, not eyes. So you have 3 more params to adjust where is "eye". Default values works fine for me. You may change it for example when you try to make video with phone camera.

     

    Tip: DO NOT RUN BAM-TRACKER.EXE WHEN YOU WANT TO PLAY FUTURE PINBALL.
    (or you will get message "Kinect not found")

  • TrackIR

    • FP-Settings
      FP Settings

    FP Settings.

    • Enable TrackIR (red box on image)
    • Arcade Mode (green box)

     

    Why?

    Future Pinball can communicate with TrackIR only with this settings. BAM don't send or receive anything from TrackIR. Only overhears what TrackIR send to Future Pinball.

     

    • TrackIR-Settings
      TrackIR Settings

    TrackIR Settings.

    1. Set profile to "One:One"
    2. Set checkbox "Exclusice" 
    3. Set Action Key "Center" to key You like to use
    4. Disable Smooth

     

    Why? 

    Default settings aren't "linear". It means, that if you move head 1 inch, sometimes TrackIR report 1/4" move and another time 3/2". If you set smooth to 0, you will reduce input lag. TrackIR have very good resolution.

     

    • BAM-Settings
      BAM Settings

    BAM Settings

    Select TrackIR plugin in BAM and enter "base head position" (as CamX, CamY, CamZ). You will have to put head at this position every time, you will press "Center" key (see TrackIR Settings, point 3). When you press this "center" key, TrackIR will interprete this position as "base position" and report player position as diference, between this "base position" and current. Ok ... it sounds complicated ... don't panic ... now easy version:

    As default BAM set this position to 6 inches above edge of display close to player. Only "TrackIR Reflector Clip" must be at this point. So do it this way:

    • Reflection Clip
      Base position
    1. Get small box and put it on edge of display. (This may also be the head of the dummy).
    2. Put hat with reflector clip over this box. (see image on right).
    3. Press "Center" key.
    4. Now play (with reflector clip on your head).

    This last procedure You will have to repeat every time, you start Future Pinball with new table.

    You also may change scale or rotation of axis, but default values (Scale = 1.0, all rotations = 0.0 deg) should work fine.

    You may also enter different "base position". For example on your head (like this: CamY = 32 inch, CamZ = 3 inch). After "centering" of base position You may pass hat with reflector clip to other player (with different height). You don't have to repeat "centering".

    Limitations

    • TrackIR with BAM works only with Future Pinball. It will not work with BAM-Tracker.
    • Every time, after "centering", TrackIR will set axis little different. If you stay in exacly same place 10 times, for BAM it will be 10 different places. Error is small and don't break "feel of depth"... but if you look carefully you will see. For me it is main pitfall of TrackIR compared to IR-Led tracking (HT w/PS3 Eyes) or Kinect.
    • TrackIR require BAM v.1.2-84 or newer.
    • It work only with BAM - Forced Arcade Mode. Switch is BAM [Addons] menu.

    TrackIR has very small CPU usage, realy great accuracy, minimal lag (like my IR-LED tracking) and is easy to use. I put it on second place in list of best tracking solution for BAM.