Stressor: A video pipeline with target detection

Stressor is a video pipeline application that uses XNOR for target detection.  This program captures images from a sensor, encodes them and stores them as a elemental stream in a file.  Along the way it uses XNOR to find targets in the frames and mark them with boxes.

The program is called 'stressor' because you can 'spin up' as many XNOR detection engines as you like from the command line.  If you run 'top' at the same time you can see the effect on CPU load as the program runs.  The program also reports important encoding and evaluation times.

Click here for build and usage instructions.

Theoretically this application should run on any computer that supports video4linux2, XNOR and OMX hardware encoding.  However, I have only tested it on a Raspberry pi 3 b+.

6replies Oldest first
  • Oldest first
  • Newest first
  • Active threads
  • Popular
  • Wow, this is great! I'm fascinated, and a bit surprised, by the results you got. The 0.3 version of the SDK only allows evaluating models using as many threads as there are cores on the machine. I guess if there's enough head room, there's no reason you couldn't run multiple at once!

    In any case, the next version of the Xnor developer SDK, coming soon, will include an API for running models in multi-threaded or single-threaded mode, which may help with these kind of throughput-oriented applications. Keep an eye out for that, and thanks for sharing your project!

    Reply Like 1
    • Andrew   I suspected as much (about threads) but I wanted to see just how much I could get out of a little rpi3.  I think I will buy an rpi0 and give it a try.  The code should move over pretty easily.  

      My day job is implementing video pipelines on obscure video processors.  I would like to get the XNOR SDK to run on an Allwinner V5 but I can't seem to make it work.  I can't get any of the models to link with the Armv7 arch. 

      That is a good idea to allow the models to run in multi thread mode.  However, let me specify the number of threads so I can hold back some of my CPU for other purposes.

      Reply Like
      • Andrew
      • Developer
      • andrew_xnor
      • 6 mths ago
      • Reported - view

      Tyler Brooks Thanks for the feedback! We are rolling out on a different Armv7 device in the next release I mentioned earlier, which may let you test out our models on the Allwinner. That release unfortunately won't have a way to specify a particular number of threads, as we're still figuring out the best way to expose that control surface in a way that's effective and meaningful. We're aware it is a useful knob to have, so it will likely be coming to another release soon after.

      Reply Like
  • I have added USB camera support so you don't have to buy a camera module.  Just any webcam will do.

    Reply Like
  • I have added Raspberry Pi Zero support.  Xnor takes about 3.2sec to process a 640x480 image so the rpi0 is not a viable realtime video platform.  See results at the bottom of this page.

    Reply Like
  • I have updated Stressor to use SDK 0.9 and noted some test results on the bottom of this page.  See the 'update' note.  In single-threaded mode, a single xnor inference takes about 295ms on the rpi3 (ouch).  In multi-threaded mode, a single xnor inference takes about 60ms on the rpi3.  Multi-threaded mode is consistent with the results I got for SDK 0.3 so I can only assume the standard mode for SDK 0.3 was multi-threaded.  As before, if you have extra CPU headroom it still is an advantage to run more than one xnor engine as once.  On the rpi3 the sweet spot is three engines consuming about 87% of the cpu.

    Reply Like
Like1 Follow
  • 1 Likes
  • 6 mths agoLast active
  • 6Replies
  • 124Views
  • 3 Following