Emerging technological breakthroughs enter our consciousness promising a bright future, but more often than not they follow Gartner’s well plotted hype cycle – from the peak of inflated expectation through the trough of disillusionment, before mainstream adoption materialises. Some technologies, however, take on a new meaning after a prolonged maturity period.

One of these technologies is analytics. While as a word it has been around for the past three centuries – its origins lie in mathematics and statistics – , it has been reinvented with the onset of the big data revolution which has manifested itself over the course of last decade.

Every two days now, we create as much information across the world as we did from the dawn of civilization up until 2003. The massive growth of data coupled with cheaper storage and computing leads to a powerful value engine in the industry. This means that today it is possible to not only derive insights from consumer/buyer behaviour but also to pump intelligence into machines and various business operations for gain extreme productivity gains. All this is driving the need for better tools to process the data and more innovative analytics business applications.

New capabilities in collecting, processing and analysing massive amounts of data turned analytics from a traditional capability into a disruptive technology that is changing the world today.

Big Data Increasingly Finds Its Place in Unexplored Domains

Today big data applications find their place in practically every industry and new applications are being explored in every functional area within each industry. The maturity of big data solutions in data intensive industries like finance, retail and telecoms etc. are approaching the peak of inflated expectations on the hype curve, while traditional data deprived industries like manufacturing, infrastructure etc. are fast catching them up.

Analytics Is at an Inflection Point Today Where Technology Capabilities Enable Us to Build Solutions Beyond Finding Insights and Helping Decision Making

Disruptive technologies often find applications beyond their original scope. Today telecommunications offer not just voice, but data and real-time presence through video, enabling businesses to communicate in multiple other ways.

I think analytics is one such disruptive technology. When I started exploring opportunities for building my analytics business at Cyient, I did not look at existing solutions for any of the challenges I found with my clients. That area has already been thoroughly exploited by the industry.

I believe that data is the raw material of business, just like physical material, money and people, which are essential for building business. This approach helped me figure out opportunities to solve traditional problems better with the power of analytics.

In one such instance, my team were working with one of our aerospace clients where we were spending close to half a million dollars every year on a testing activity – which is expected to grow by over 10% every year. Testing does not add direct monetary value to the client, though it helps in delivering an impeccable product or service. We have come up with an innovative solution to automate this testing activity by using the power of analytics. The effort saved in eliminating manual testing will leads to savings of over million dollars in the next two years.

How to Use Analytics for Solving Non-Analytical Problems

We have also been supporting one of the leading aircraft OEMs in the aftermarket services. Our client receives tens of thousands of aircraft maintenance events from over 170 airline operators across the globe every month. The data often contains multiple formats and textual remarks with a wide variety of jargon. Nomenclature and descriptions may also vary for specific parts and events depending on the region from which the data is sourced.

Aftermarket data management activity requires strong domain understanding to analyse maintenance events and build a complete and consistent database. We also need to analyse the data to enrich it with missing information. For example, some key decisions like whether the maintenance operation performed was chargeable to the OEM or not can be derived only after a thorough analysis of the event. Each aircraft contains over 60,000 parts and some minute part names are often not reported properly by operators. Our engineers need to analyse the data provided by operators and decide the atomic part of the aircraft on which the maintenance action was performed.

While we do this with the help of our domain expert engineers, we also spend a substantial amount of effort validating the output by another group of senior engineers in order to ensure quality.

We wanted to avoid spending high quality talent on things like validating data given the fact that the cost of this unproductive activity would reach the million dollar mark within a couple of years.

I found an opportunity in our historical data. We have hundreds of thousands of validated records processed over the years. I prepared a business case to build a validation system using the power of analytics to automate the process of validating thousands of maintenance records every month.

We profiled the historical data in order to create stories for each of the maintenance events available to us. For example, when a bird strikes an aircraft, there are certain data points that have to be recorded including which part of the aircraft was hit and what the impact observed by the pilot was. When you can profile the data from thousands of such records for the same event, you can build a more informed story about the possible implications of an event. When you have such stories available for all the common events, you can validate fresh records and situations by comparing them with the database of stories you already have.

We used classification techniques, association rules and a variety of text mining techniques to build this validation engine. We also built an application that contains the validation engine, facilitating the upload of hundreds of records simultaneously by users and validating them in a matter of minutes.

With thousands of new records being fed into the engine for validation, we are currently working on building a machine learning algorithm to help the engine learn from existing data and to improve its overall effectiveness.

Future Outlook

While big data solutions started solving problems in diverse domains like medical, agriculture and functional areas like security, location intelligence etc., they are often used for gaining intelligence and insights from data to support better decision making. Solving traditional, monotonous and non-analytical problems like testing with big data technologies is all set to take off very soon.

Photo by NEC Corporation of America with Creative Commons license.