Dataconomy
  • News
    • Artificial Intelligence
    • Cybersecurity
    • DeFi & Blockchain
    • Finance
    • Gaming
    • Startups
    • Tech
  • Industry
  • Research
  • Resources
    • Articles
    • Guides
    • Case Studies
    • Glossary
    • Whitepapers
  • Newsletter
  • + More
    • Conversations
    • Events
    • About
      • About
      • Contact
      • Imprint
      • Legal & Privacy
      • Partner With Us
Subscribe
No Result
View All Result
  • AI
  • Tech
  • Cybersecurity
  • Finance
  • DeFi & Blockchain
  • Startups
  • Gaming
Dataconomy
  • News
    • Artificial Intelligence
    • Cybersecurity
    • DeFi & Blockchain
    • Finance
    • Gaming
    • Startups
    • Tech
  • Industry
  • Research
  • Resources
    • Articles
    • Guides
    • Case Studies
    • Glossary
    • Whitepapers
  • Newsletter
  • + More
    • Conversations
    • Events
    • About
      • About
      • Contact
      • Imprint
      • Legal & Privacy
      • Partner With Us
Subscribe
No Result
View All Result
Dataconomy
No Result
View All Result

Hadoop Evolved: How Industries Are Being Transformed By Big Data

byVikram Bhalchandra
May 14, 2018
in Artificial Intelligence

A 50-year-old man runs on the treadmill and receives an alert on his Apple Watch. The message tells him to get off immediately because his pulse is abnormally high, which puts him at risk of a heart attack.

Such a scenario is not far off thanks to Pontem, a platform that takes input from devices like Apple Watch and Fitbit and uses cloud-based data, machine learning and cognitive processing to decide when such alerts are warranted. For the end user, this could be a lifesaver. For the developer, this is the latest example of the evolution of big data with real world implications. A large part of the evolution of such platforms can be directly tied to the maturing of the Hadoop ecosystem.

Once merely a tool to manage big data, Hadoop has emerged as the foundation of industry-specific solutions. Adapting Hadoop for this purpose, however, requires a specific approach suited to each industry.

Stay Ahead of the Curve!

Don't miss out on the latest insights, trends, and analysis in the world of data, technology, and startups. Subscribe to our newsletter and get exclusive content delivered straight to your inbox.

Transformation in action: Financial services and manufacturing

In addition to healthcare, financial services and manufacturing are areas in which companies employ Hadoop to manage, store and analyze data. For instance, in financial services, big data is utilized for advanced AI and machine learning models that help users manage credit risk more effectively.

Credit risk (i.e. the probability of a borrower’s failure to keep up with payments) is a major concern for businesses. Credit risk management, which attempts to handicap the probability of that occurrence, has been a major headache for financial services companies. Despite this being the cornerstone around which the whole banking industry has evolved, the complexity and evolving nature of financial services today has put significant stress on the traditional credit risk models. Availability of big data in multiple formats through platforms like Hadoop has helped companies create advanced credit risk models – taking into account variables and factors that were never part of the traditional credit risk frameworks.

Big data allows for new models to be built. There’s a huge amount of customer data available – including spending behavior, online browsing and customer payment – that can help financial institutions make better decisions. Here, Hadoop’s ability to manipulate and sort unstructured data can be applied to a specific function. The ability to create large data lakes on Hadoop with significant protection and data security with tools like Kogni has led to these industry models evolving to the next level today.

Hadoop, which is the base layer in many of these multi-layered industry big data solutions, has also evolved over the years to provide flexibility and scale to manage big data. The ability of the platform to break down big data into manageable chunks and run smaller jobs in parallel – using low cost hardware, internal fault tolerance and self-healing – has made massive scalability at manageable costs possible across hundreds or thousands of servers in a Hadoop cluster. The management of big data through massive infrastructure is now a thing of the past.

Before Hadoop, there were limited means for managing the computation-intensive processes to leverage structured and unstructured data in sophisticated models. Using AI and machine learning, the new complex models built on top of Hadoop data lakes are self-learning and can adapt to changing data patterns, while the overall cost of the solution stays low and scalable.

Another example is in manufacturing. Like other industries, manufacturing is undergoing a digital transformation as sensors and internet connections provide real-time data on operations. For instance, in predictive manufacturing, sensors can detect early anomalies in a production run, therefore preventing the creation and subsequent waste of thousands of defective products. These IoT sensors and edge devices (including auditory sensors) are connected to software that is able to push data in real-time into the cloud to be ingested in Hadoop data lakes for on-demand analytics. There is often a deep learning/AI aspect to the analytics layers built on top of these data lakes that provides self-learning and evolving data analytics capabilities. A recent survey from LNS Research showed that 80 percent of large manufacturers are implementing or plan to implement such technologies in the near future.

How to harness Hadoop for your industry

Hadoop isn’t a magic solution. It’s a platform that you can harness to help surface relevant data to aid your particular industry. These solutions, however, require mastery of Hadoop technology combined with custom knowledge and expertise within the specific industry. GE and American Express have been at the forefront to build such custom industry big data solutions – leveraging big data and Hadoop capabilities while bringing the industry expertise from inside.

The best way to create these solutions is to employ a “layered” approach. At the foundation is the Hadoop data ingestion layer, followed by the algorithm layer that has models specifically built for the industry. On top of that is an even more specific layer. Each component can be customized to the industry or use case to provide maximum ROI.

What’s exciting is that custom models can be used for small and medium-sized organizations in the same way as they are with large enterprises. This is truly the democratization of big data.

Like this article? Subscribe to our weekly newsletter to never miss out!

Tags: USA

Related Posts

ChatGPT adds Instant Checkout with Agentic Commerce Protocol

ChatGPT adds Instant Checkout with Agentic Commerce Protocol

September 30, 2025
California enacts SB 53 AI transparency law

California enacts SB 53 AI transparency law

September 30, 2025
Anthropic releases Claude Sonnet 4.5 with advanced coding and agent capabilities

Anthropic releases Claude Sonnet 4.5 with advanced coding and agent capabilities

September 30, 2025
CESA: 51% of Japanese game firms use AI in development

CESA: 51% of Japanese game firms use AI in development

September 29, 2025
South Korea funds LG Exaone 4.0, SKT A.X for AI sovereignty

South Korea funds LG Exaone 4.0, SKT A.X for AI sovereignty

September 29, 2025
Medicare WISeR pilot uses AI for service approvals in 6 states

Medicare WISeR pilot uses AI for service approvals in 6 states

September 29, 2025
Please login to join discussion

LATEST NEWS

ChatGPT adds Instant Checkout with Agentic Commerce Protocol

California enacts SB 53 AI transparency law

YouTube settles Trump lawsuit for $24.5 million

EA sold to Saudi-backed group for $55 billion

Cross-Chain is the new competitive edge: Building secure, interoperable systems in the Web3 era

Anthropic releases Claude Sonnet 4.5 with advanced coding and agent capabilities

Dataconomy

COPYRIGHT © DATACONOMY MEDIA GMBH, ALL RIGHTS RESERVED.

  • About
  • Imprint
  • Contact
  • Legal & Privacy

Follow Us

  • News
    • Artificial Intelligence
    • Cybersecurity
    • DeFi & Blockchain
    • Finance
    • Gaming
    • Startups
    • Tech
  • Industry
  • Research
  • Resources
    • Articles
    • Guides
    • Case Studies
    • Glossary
    • Whitepapers
  • Newsletter
  • + More
    • Conversations
    • Events
    • About
      • About
      • Contact
      • Imprint
      • Legal & Privacy
      • Partner With Us
No Result
View All Result
Subscribe

This website uses cookies. By continuing to use this website you are giving consent to cookies being used. Visit our Privacy Policy.