Data processing today is done in form of pipelines which include various steps like aggregation, sanitization, filtering and finally generating insights by applying various statistical models. Amazon Kinesis is a platform to build pipelines for streaming data at the scale of terabytes per hour. Parts of the Kinesis platform are
Bitcoin is currently trading at over $1250 and if you are someone who invested a grand in bitcoins back in 2011, your investments are potentially worth over $600K. The most valuable contribution of the bitcoin community is not in the financial returns itself, but in the introduction of blockchain technology.
Oftentimes while analyzing big data we have a need to make checks on pieces of data like number of items in the dataset, number of unique items, and their occurrence frequency. Hash tables or Hash sets are usually employed for this purpose. But when the dataset becomes so enormous that