Real Time Data Analytics: The good, the bad and the hard place

Data analytics has probably been around from the time of invention of abacus and has improved as centuries passed by. Population of the planet and the complexity of the problem increased exponentially since then. Solving a problem in batches and deciding on the corrective action is often not enough.

Enter real time analytics! The main goal for has been to solve problems quickly as they happen, or even better, before they happen. Today we all want to use real time data analytics to foresee and solve problems for our business. But how exactly is the current industry faring?

The Good

Real time data analytics have come a long way in the recent decade. Especially with the advent of big data and the general industry inclination on data driven decision-making. While there has been ongoing debate about instinct vs insight in business, the importance of data has risen exponentially with the introduction of big data.


The best part is that technologies and tools are easily available today. Tools have real time data analytics capability for any business is right at our fingertips. Starting from open source solutions, which are absolutely free (combination of Kafka-Druid etc.) to enterprise grade solutions (New Relic, Splunk), business with patience and/or the fund to afford it are no short of choice.

The Bad

Real time analytics is yet to be efficiently utilized to its full potential. We often see businesses using real time analytics tools to barely monitor a data source such as data center or websites. While the idea of using real time analytics fascinates business decision makers, often the technical team of an organization does not align with the business teams to fulfil the promise of real time data.


Often, the technology stack used (open source components to build real time analytics tools are often not very convenient for rapid development) or the tools that are used (charges for enterprise grade tools are out of reach for small and medium business) provide the challenge.

The Hard Place

Stuck between a rock and hard place. That how small and medium businesses often end up working with tools with limited features. Eventually they lose to bigger organizations in terms of agility.


Building a custom open source based analytical tool requires money, time, effort and the know-how. On the other hand often tools like Splunk (lowest price $900/GB) or New Relic (separate pricing for data and hosting) puts these tools out of the reach of these organizations.

Conclusion

The good news is that several companies in recent times have been trying to bring affordable solution to this space. The likes of IQLECT (approx. $0.93/GB), DataDog ($15/host/month) and Scout ($10/host/month) are making decent effort to provide an alternative to small and medium businesses.

Picture credit: Freepik

Leave a Reply

Your email address will not be published. Required fields are marked *