Dataconomy
  • News
  • AI
  • Big Data
  • Machine Learning
  • Trends
    • Blockchain
    • Cybersecurity
    • FinTech
    • Gaming
    • Internet of Things
    • Startups
    • Whitepapers
  • Industry
    • Energy & Environment
    • Finance
    • Healthcare
    • Industrial Goods & Services
    • Marketing & Sales
    • Retail & Consumer
    • Technology & IT
    • Transportation & Logistics
  • Events
  • About
    • About Us
    • Contact
    • Imprint
    • Legal & Privacy
    • Newsletter
    • Partner With Us
    • Writers wanted
Subscribe
No Result
View All Result
Dataconomy
  • News
  • AI
  • Big Data
  • Machine Learning
  • Trends
    • Blockchain
    • Cybersecurity
    • FinTech
    • Gaming
    • Internet of Things
    • Startups
    • Whitepapers
  • Industry
    • Energy & Environment
    • Finance
    • Healthcare
    • Industrial Goods & Services
    • Marketing & Sales
    • Retail & Consumer
    • Technology & IT
    • Transportation & Logistics
  • Events
  • About
    • About Us
    • Contact
    • Imprint
    • Legal & Privacy
    • Newsletter
    • Partner With Us
    • Writers wanted
Subscribe
No Result
View All Result
Dataconomy
No Result
View All Result

Twitter Outline Home-Grown Analytics Architecture TSAR

by Eileen McNulty
June 30, 2014
in News
Home News
Share on FacebookShare on TwitterShare on LinkedInShare on WhatsAppShare on e-mail

Twitter Outline Home-Grown Analytics Architecture TSAR

Twitter has detailed its home-grown, real-time analytics system TSAR (Time Series AggregatoR) in a blog post. The system is focused on automating and aggregating data collection, as well as integrating the various components Twitter uses, such as Hadoop and Storm.

In their data collection and processing pipeline, Twitter are using a range of different solutions (Hadoop, MySQL, NoSQL), which all process the data in different ways, and use different languages to do it. TSAR takes the legwork out of getting these systems to talk to each other.

The blog post by Anirudh Todi, TSAR’s key design principles are:

  • “Hybrid computation. Process every event twice — in real time, and then again (at a later time) in a batch job. The double processing is orchestrated using Summingbird. This hybrid model confers all the advantages of batch (stability, reproducibility) and streaming (recency) computation.

  • Separation of event production from event aggregation. The first processing stage extracts events from source data; in this example, TSAR parses Tweet impression events out of log files deposited by web and mobile clients. The second processing stage buckets and aggregates events. While the “event production” stage differs from application to application, TSAR standardizes and manages the “aggregation” stage.

  • Unified data schema. The data schema for a TSAR service is specified in a datastore-independent way. TSAR maps the schema onto diverse datastores and transforms the data as necessary when the schema evolves.

  • Integrated service toolkit. TSAR integrates with other essential services that provide data processing, data warehousing, query capability, observability, and alerting, automatically configuring and orchestrating its components.”

TSAR was built on top of the Summingbird system, a high-level extraction library which paired the batch processing capabilities of Hadoop with the real-time powers of Storm. TSAR builds upon these, making it easier for the different technologies to communicate back and forth.


Join the Partisia Blockchain Hackathon, design the future, gain new skills, and win!


With over 500 million tweets created each day, it’s understandable that Twitter developed their own robust technology to orchestrate their multi-faceted system.

Read more here.
(Image credit: Blog post)

Follow @DataconomyMedia


Interested in more content like this? Sign up to our newsletter, and you wont miss a thing!

[mc4wp_form]

 

Tags: Twitter

Related Posts

AI Asmongold video: In the Athene AI Show, a Twitch streamer's funny deepfake revealed and people love it. So how did this happen? Keep reading and find out.

AI Asmongold may have been one of the very first examples of AI streamers

February 6, 2023
How did ChatGPT passed an MBA exam

How did ChatGPT passed an MBA exam?

February 2, 2023
Google code red: ChatGPT and You.com like AI-powered tools threatening the search engine. Moreover, latest Apple Search rumors increased the danger.

Google code red: ChatGPT, You.com and rumors of Apple Search challenge the dominance of search giant

February 2, 2023
T-Mobile data breach 2023 explained: Learn how did the leak happen and explore T-Mobile data breach history. It is not the first time of the company

T-Mobile data breach 2023: The telecom giant got hacked eight times in the last six years

January 20, 2023
Microsoft layoffs 2023: Amazon job cuts that affect 11,000 employees explained. Big tech layoffs continue... Learn why and what will happen next.

Microsoft layoffs will affect more than 11,000 employees

January 18, 2023
Medibank Data Breach Class Action: Compensation can reach up to $20,000 per person

Medibank Data Breach Class Action: Compensation can reach up to $20,000 per person

January 16, 2023

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

LATEST ARTICLES

Google unveils its experimental conversational AI service Bard

Achieving data resilience with StaaS

AI Asmongold may have been one of the very first examples of AI streamers

Mastering the art of efficiency through business process transformation

Google starts testing its ChatGPT rival AI chatbot called Apprentice Bard

How AI improves education with personalized learning at scale and other new capabilities

Dataconomy

COPYRIGHT © DATACONOMY MEDIA GMBH, ALL RIGHTS RESERVED.

  • About
  • Imprint
  • Contact
  • Legal & Privacy
  • Partnership
  • Writers wanted

Follow Us

  • News
  • AI
  • Big Data
  • Machine Learning
  • Trends
    • Blockchain
    • Cybersecurity
    • FinTech
    • Gaming
    • Internet of Things
    • Startups
    • Whitepapers
  • Industry
    • Energy & Environment
    • Finance
    • Healthcare
    • Industrial Goods & Services
    • Marketing & Sales
    • Retail & Consumer
    • Technology & IT
    • Transportation & Logistics
  • Events
  • About
    • About Us
    • Contact
    • Imprint
    • Legal & Privacy
    • Newsletter
    • Partner With Us
    • Writers wanted
No Result
View All Result
Subscribe

This website uses cookies. By continuing to use this website you are giving consent to cookies being used. Visit our Privacy Policy.