Pivotal, the San Francisco-based software and services provider has announced its partnership with the AMPLab at UC Berkeley to support the in-memory Tachyon project. They aim to bolster the data lake technology with an architecture that builds upon disk-based storage with memory-centric processing frameworks.
Being led by Haoyuan Li, a PhD candidate at the UC Berkeley AMPLab, Tachyon is a memory-centric, fault-tolerant distributed file system that allows for reliable data exchange at in-memory speed across cluster frameworks, explains their Blog. Tachyon will be a central data exchange layer for the Pivotal Big Data Suite.
Till date this project is the fastest-growing project in all of AMPLab history, ahead of Mesos and Spark. It is currently a GitHub project developed under the Apache License.
EMC’s Chief Technology Officer John Roese said, ”EMC is excited to join with Pivotal in supporting and fostering the rapidly innovating Big Data ecosystem. The next wave of real-time analytics will be made possible by technologies such as Apache Spark and Tachyon, in combination with innovative storage products such as EMC’s recent DSSD acquisition.”
Pivotal which is a spin-out and joint venture of EMC Corporation and VMware has pledged resources to Tachyon’s development and stands as the number one corporate contributor to the Tachyon code base.
Read more here.
(Image credit: Tachyon)
