Dataconomy
  • News
  • AI
  • Big Data
  • Machine Learning
  • Trends
    • Blockchain
    • Cybersecurity
    • FinTech
    • Gaming
    • Internet of Things
    • Startups
    • Whitepapers
  • Industry
    • Energy & Environment
    • Finance
    • Healthcare
    • Industrial Goods & Services
    • Marketing & Sales
    • Retail & Consumer
    • Technology & IT
    • Transportation & Logistics
  • Events
  • About
    • About Us
    • Contact
    • Imprint
    • Legal & Privacy
    • Newsletter
    • Partner With Us
    • Writers wanted
Subscribe
No Result
View All Result
Dataconomy
  • News
  • AI
  • Big Data
  • Machine Learning
  • Trends
    • Blockchain
    • Cybersecurity
    • FinTech
    • Gaming
    • Internet of Things
    • Startups
    • Whitepapers
  • Industry
    • Energy & Environment
    • Finance
    • Healthcare
    • Industrial Goods & Services
    • Marketing & Sales
    • Retail & Consumer
    • Technology & IT
    • Transportation & Logistics
  • Events
  • About
    • About Us
    • Contact
    • Imprint
    • Legal & Privacy
    • Newsletter
    • Partner With Us
    • Writers wanted
Subscribe
No Result
View All Result
Dataconomy
No Result
View All Result

EBay Brings Out Open-Sourced Kylin, Built from Scratch, Which Taps Distributed Processing and the HBase Data Store to Accelerate Analytics

by Eileen McNulty
October 24, 2014
in News
Home News
Share on FacebookShare on TwitterShare on LinkedInShare on WhatsAppShare on e-mail

Ebay’s latest offering to the open-source community is designed to accelerate analytics on Hadoop and allow the use of SQL-compatible tools, while providing an SQL interface and multi-dimensional analysis (OLAP) on Hadoop to support extremely large datasets.

Dubbed, Kylin, according to an eBay blog post, this is what it essentially helps with:

“When data becomes bigger, the pre-calculation processing becomes impossible – even with powerful hardware. However, with the benefit of Hadoop’s distributed computing power, calculation jobs can leverage hundreds of thousands of nodes. This allows Kylin to perform these calculations in parallel and merge the final result, thereby significantly reducing the processing time.”

It is currently used in production by various business units at eBay. “Our largest use case is the analysis of 12+ billion source records generating 14+ TB cubes. Its 90% query latency is less than 5 seconds. Now, our use cases target analysts and business users, who can access analytics and get results through the Tableau dashboard very easily – no more Hive query, shell command, and so on,” the blog explains.

Ebay has also proposed Kylin as an Apache Incubator project.


Join the Partisia Blockchain Hackathon, design the future, gain new skills, and win!


The Platform offers the following features for big data analytics:

  • Extremely fast OLAP engine at scale: Kylin is designed to reduce query latency on Hadoop for 10+ billion rows of data.
  • ANSI SQL on Hadoop: Kylin supports most ANSI SQL query functions in its ANSI SQL on Hadoop interface.
  • Interactive query capability: Users can interact with Hadoop data via Kylin at sub-second latency – better than Hive queries for the same dataset.
  • MOLAP cube query serving on billions of rows: Users can define a data model and pre-build in Kylin with more than 10+ billions of raw data records.
  • Seamless integration with BI Tools: Kylin currently offers integration with business intelligence tools such as Tableau and third-party applications.
  • Open-source ODBC driver: Kylin’s ODBC driver is built from scratch and works very well with Tableau. We have open-sourced the driver to the community as well.

Derric Harris of GigaOm wonders how Kylin would fare against “the next-generation versions of Hive, Spark SQL and other options for SQL analysis in Hadoop that have emerged as a result of the YARN resource manager available in the latest versions of Apache Hadoop.” He believes that it might be slower but more scalable than in-memory option while being another option for Hadoop users who are running earlier versions of the software.

Other technologies over the past 30 years have have leveraged the same theory on which Kylin is based to accelerate analytics including methods to ‘store pre-calculated results to serve analysis queries, generate each level’s cuboids (referencing cuboid typology) with all possible combinations of dimensions, and calculate all metrics at different levels.’

Read more here.

(Image credit: Flickr)

Tags: eBayHBaseKylinOLAP

Related Posts

How did ChatGPT passed an MBA exam

How did ChatGPT passed an MBA exam?

February 2, 2023
Google code red: ChatGPT and You.com like AI-powered tools threatening the search engine. Moreover, latest Apple Search rumors increased the danger.

Google code red: ChatGPT, You.com and rumors of Apple Search challenge the dominance of search giant

February 2, 2023
T-Mobile data breach 2023 explained: Learn how did the leak happen and explore T-Mobile data breach history. It is not the first time of the company

T-Mobile data breach 2023: The telecom giant got hacked eight times in the last six years

January 20, 2023
Microsoft layoffs 2023: Amazon job cuts that affect 11,000 employees explained. Big tech layoffs continue... Learn why and what will happen next.

Microsoft layoffs will affect more than 11,000 employees

January 18, 2023
Medibank Data Breach Class Action: Compensation can reach up to $20,000 per person

Medibank Data Breach Class Action: Compensation can reach up to $20,000 per person

January 16, 2023
What is DoNotPay AI Lawyer? The world's first robot lawyer ready to give $1 million to represent you. How does it work? Keep reading.

DoNotPay AI lawyer is ready to give $1 million for any case in US

January 12, 2023

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

LATEST ARTICLES

Cyberpsychology: The psychological underpinnings of cybersecurity risks

ChatGPT Plus: How does the paid version work?

AI Text Classifier: OpenAI’s ChatGPT detector indicates AI-generated text

A journey worth taking: Shifting from BPM to DPA

BuzzFeed ChatGPT integration: Buzzfeed stock surges after the OpenAI deal

Adversarial machine learning 101: A new cybersecurity frontier

Dataconomy

COPYRIGHT © DATACONOMY MEDIA GMBH, ALL RIGHTS RESERVED.

  • About
  • Imprint
  • Contact
  • Legal & Privacy
  • Partnership
  • Writers wanted

Follow Us

  • News
  • AI
  • Big Data
  • Machine Learning
  • Trends
    • Blockchain
    • Cybersecurity
    • FinTech
    • Gaming
    • Internet of Things
    • Startups
    • Whitepapers
  • Industry
    • Energy & Environment
    • Finance
    • Healthcare
    • Industrial Goods & Services
    • Marketing & Sales
    • Retail & Consumer
    • Technology & IT
    • Transportation & Logistics
  • Events
  • About
    • About Us
    • Contact
    • Imprint
    • Legal & Privacy
    • Newsletter
    • Partner With Us
    • Writers wanted
No Result
View All Result
Subscribe

This website uses cookies. By continuing to use this website you are giving consent to cookies being used. Visit our Privacy Policy.