Enterprise Hadoop vendors MapR just demonstrated that their data platform is capable of ingesting over 100 million data points a second. These impressive performance results were unveiled at the sold-out Tableau Conference in Washington, using OpenTSDB on the MapR Distribution including the in-Hadoop NoSQL database, MapR-DB.
Using only four nodes of a 10-node cluster, the MapR team accelerated OpenTSDB performance by 1,000 times. In the coming years, data processing at these speeds will become increasingly necessary. The growing interest in real-time analytics and the explosion of Internet of Things applications means data processing often needs to happen at breakneck speeds.
In an interview with Dataconomy about Hadoop’s role in the Internet of Things, MapR’s Chief Marketing Officer explained: “What’s required is a platform that can scale very quickly. We’re not just asking ‘Can the system handle terabytes or petabytes?’, but ‘Can it handle millions or billions small, individual files?’ A hundred million files is not that large from an Internet of Things perspective, so these systems need to scale to a billion, or even a trillion files. That is the area that MapR has provided a platform for from the very beginning.”
“The ability to combine deep predictive analytics with real time capabilities is an absolute requirement. So an integrated, in-Hadoop database is a key feature.”
Cisco estimates that there will be approximately 50 billion connected devices by 2020. This unprecedented level of data generation will push the boundaries of existing data platforms. Ingestion speeds like this stand MapR in good stead for the coming explosion of data that the Internet of Things will bring.
(Image credit: Shashi Bellamkonda)
