Big Data: Collect it All, Sort it Later?

Apr 18 2019

Scott Shadley

Technology industry analyst Charles Araujo best summarizes the early approach to big data as “Collect it all, sort it out later”. The focus was (and has continued to be for many companies) a competition of collection and storage of petabyte-scale data with little to no emphasis on the “and then what” question. Besides the obvious problem of data having little to no intrinsic value on its own, the fact is that data is not a finite entity that a company can just capture and store. Data is growing and, as Charles Araujo explains, IT and business leaders are finding they must shift from a focus on operational and transformative outcomes to examine the value of the data itself, and AI initiatives required to make sense of it. Yes, collect and store the data, AND leverage that data for your organization.

Key to optimizing data is the ability to: i) organize the data in a meaningful way; ii) pull only the relevant data to answer the question at hand; and iii) set aside remaining data for other purposes down the road. In this context, AI is not just the latest technology tool – it is a way to create business value by enabling real-time analytics. AI initiatives are not only valuable to automate activity but to create business models, identify key data points, and transform an organization or its processes and technical approaches.

Computational Storage is not just a different approach to moving (or not moving) data. It is an intelligent way to find meaningful data (sorted by in-situ-processing), transfer only relevant data, and make use of it. This approach maximizes efficiency, reducing power consumption and operational costs. This “sort and send” approach enables real-time data applications that are fast, comprehensive, and meaningful.

Peter Burris, Wikibon Research, provides the following insight:

“The difference between a business and a digital business is the degree to which data is used as an asset. In a digital business, data absolutely is used as a differentiating asset for creating and keeping customers.

We look at the challenges of what does it mean to use data differently, how to capture it differently, which is a lot of what IoT is about. We look at how to turn it into business value, which is a lot of what big data and these advanced analytics like artificial intelligence (AI), machine learning and deep learning are all about. And then finally, how to create the next generation of applications that actually act on behalf of the brand with a fair degree of autonomy, which is what we call “systems of agency” are all about. And then ultimately how cloud and historical infrastructure are going to come together and be optimized to support all those requirements.”

It will be interesting to watch the industry evolve, and use cases for AI and Machine Learning in particular, as we move from “data dump and store” to practices that recognize differentiated data value, the creation of meaningful metadata, and the role that minimizing the transfer of data can play in that equation. Computational Storage is bringing intelligence to storage. See you at AI and Big Data Expo in London. And, I look forward to your comments/questions at Scott.Shadley@ngdsystems.com

Data overload image.

355 Goddard, Suite 200
Irvine, CA 92618

For Sales Inquiries:

sales@ngdsystems.com

For General Inquiries:

info@ngdsystems.com