Working of Big Data and AI in a connected way

By ridhigrg |Email | Oct 22, 2019 | 1197 Views

How is artificial intelligence - and its prominent discipline, machine learning - helping deliver better business insights from big data? Let's examine some ways - and peek at what's next for AI and big data analysis

Big data isn't quite the term de rigueur that it was a few years ago, but that doesn't mean it went anywhere. If anything, big data has just been getting bigger.

That once might have been considered a significant challenge. But now, it's increasingly viewed as a desired state, specifically in organizations that are experimenting with and implementing machine learning and other AI disciplines.

"AI and ML are now giving us new opportunities to use the big data that we already had, as well as unleash a whole lot of new use cases with new data types," says Glenn Gruber.

How AI fits with big data
There's a reciprocal relationship between big data and AI: The latter depends heavily on the former for success, while also helping organizations unlock the potential in their data stores in ways that were previously cumbersome or impossible.

"Today, we want as much [data] as we can get - not only to drive better insight into business problems we're trying to solve but because the more data we put through the machine learning models, the better they get," Gruber says. "It's a virtuous cycle in that way."

6 ways AI fuels better insights
1. AI is creating new methods for analyzing data
One of the fundamental business problems of big data could sometimes be summarized with a simple question: Now what? As in: We've got all this stuff (that's the technical term for it) and plenty more of it coming - so what do we do with it? In the once-deafening buzz around big data, it wasn't always easy to hear the answers to that question.

Moreover, answering that question - or deriving insights from your data - usually required a lot of manual effort. AI is creating new methods for doing so. In a sense, AI and ML are the new methods, broadly speaking.

"Historically, when it comes to analyzing data, engineers have had to use a query or SQL (a list of queries). But as the importance of data continues to grow, a multitude of ways to get insights have emerged. AI is the next step to query/SQL," says Steven Mih, CEO at Alluxio. "What used to be statistical models now has converged with computer science and has become AI and machine learning."

2. Data analytics is becoming less labor-intensive
As a result, managing and analyzing data depends less on time-consuming manual effort than in the past. People still play a vital role in data management and analytics, but processes that might have taken days or weeks (or longer) are picking up speed thanks to AI.

"AI and ML are tools that help a company analyze their data more quickly and efficiently than what could be done [solely] by employees," says Sue Clark, senior CTO architect at Sungard AS.

In other words, insights and decisions can happen faster. Moreover, IT can apply similar principles - using AI technologies to reduce manual, labor-intensive burdens and increase speed - to the back-end stuff that, let's face it, few outsides of IT want to hear about.

"The real-time nature of data insights, coupled with the fact that it exists everywhere now - siloed across different racks, regions, and clouds - means that companies are having to evolve from the traditional methods of managing and analyzing [data]," Mih from Alluxio says. That's where AI comes in. "Gone are the days of data engineers manually copying data around, again and again, delivering datasets weeks after a data scientist requests it."

3. Humans still matter plenty
Like others, Elif Tutuk, associate VP of Qlik Research, sees AI and ML as powerful levers when it comes to big data.

"AI and machine learning, among other emerging technologies, are critical to helping businesses have a more holistic view of all of that data, providing them with a way to make connections between key data sets," Tutuk says. But, she adds, it's not a matter of cutting out human intelligence and insight.

"Businesses need to combine the power of human intuition with machine intelligence to augment these technologies - or augmented intelligence. More specifically, an AI system needs to learn from data, as well as from humans, in order to be able to fulfill its function," Tutuk says.

"Businesses that successfully combined the power of human and technology are able to expand who has access to key insights from analytics beyond data scientists and business analysts while saving time and reducing potential bias that may result from business users interpreting data. This results in more efficient business operations, quicker insights gleaned from data and ultimately increased enterprise productivity."

Source: HOB