Airbnb open sourced a web-based data exploration and SQL query tool for Hadoop called Airpal, built on Facebook’s PrestoDB, that enables query execution to facilitate data analysis, Thursday.
Making the announcement, James Mayfield, product lead at Airbnb explains that working with SQL for ‘exploration and investigation’ is not easy.
“Remembering how a query was written, copying and pasting from the command line, and running multiple terminal windows can slow down analysis and be frustrating. Additionally, when diverse teams are using SQL for analytics, the learning curve can be steep for beginners, so good UI tools can help drive adoption and promote knowledge sharing,” he wrote.
Airpal was deployed for internal use, about a year ago and since then, Mayfield notes, it has had over a third of all employees issue a query.
Airbnb lists out the key features of Airpal as follows:
- optional access controls for users
- ability to search and find tables
- see metadata, partitions, schemas, and sample rows
- write queries in an easy-to-read editor
- submit queries through a web interface
- track query progress
- get the results back through the browser as a CSV
- create new Hive table based on the results of a query
- save queries once written
- searchable history of all queries run within the tool
Software engineer Andy Kramolisch pointed out that earlier Airbnb utilized Amazon‘s Redshift cloud data warehouse for its speed but eventually it proved less user friendly and required managing and replicating data from Hive.
(Featured image credit: Airbnb)
