May 26 2020
“I’m Givin’ Her All She’s Got, Captain”
Those immortal words come from Montgomery “Scotty” Scott, Chief Engineer on the starship Enterprise. Who hasn’t heard them uttered at least once in their lifetime in reference to a situation where we just “need more power”?
I’ve heard this phrase repeated often in meetings, in social gatherings, and even once after a trip around the race track in an antique car. However, I never expected this 1960’s catch phrase to become synonymous with the value proposition of an NVMe device…….and yet this perfectly describes the real contributions made to Neural Network environments that deploy Computational Storage Drives (also known as CSD’s).
For those of you not familiar with Computational Storage I’ll digress for a moment and share that Computational Storage is the idea of adding “extra processors” into an NVMe SSD. These processors are then used to deliver microservices on the data which sits on that SSD. Research shows that in very large datasets, processing the data locally, where it resides (on the CSD) is significantly more efficient than moving the data off of the disk for processing. Think about a massive search problem. If every disk has the ability to search it’s own local contents and only return the result, then the overall system and search become much more efficient. This concept is known as Computational Storage.
At this point, I’m sure most will stop reading in anticipation of a sales pitch. However, in this case, I aim to disappoint. Instead, I hope to dust off my seldom used engineering degree and convince a few of you that the payoff is worth the time invested in continuing the read.
We set up an experiment here at NGD Systems to test the efficiency of an x86 CPU, an nVidia GPU, and Computational Storage SSD’s, in running neural networks aimed at object recognition and detection. The goal was to determine which of the three hardware sets were most power efficient and processed the images with the greatest degree of accuracy.
In figure 1 below you can see that a camera records video from a live baseball game at 30fps onto NGD Systems Computational Storage NVMe SSD’s (CSD’s).
Figure 1 – Image Tracking and Object Detection Setup
We decided to run 5 different object recognition and detection algorithms on each of the 3 different processing complexes (CPU, GPU, Computational Storage). Keep in mind that all of these are complex neural networks and/or Correlation Filters. They are also prime examples of the types of applications that might be run in Edge compute environments. Our goal, as previously stated, was to determine both accuracy as well as energy efficiency with each approach. In my next post, I’ll identify the algorithms we used, as well as share some real world results.
I’m sure the results we achieved in this PoC will surprise you the same way they did us.