The rapid emergence of self-service BI tools and what it means for BI developers

The rapid emergence of self-service BI tools and what it means for BI developers

Two years back, I was in New Delhi shopping for my wedding. I was taking the metro back home from the Karol Bagh market. While in the queue for the metro tickets, I noticed something new. An automated ticket vending machine. Yes, instead of queuing up to the regular ticket window, a person can walk up to this automated kiosk, enter the details and pay however they wish, cash, card, or UPI and they receive their ticket.

As I was gazing at this machine, considerably saving both the DMRC staff and the commuters a lot of time, it struck me. For every machine installed, at least one person’s job as a ticket vendor has been put in jeopardy.

Since we now have a machine doing the conventional task of a human with better efficiency, the human element is irrelevant.

Robot

Automation of simple, mundane processes is a great boon to all of us. Well, most of us. There is a flip side to this. The human factor getting irrelevant in the jobs that can be automated.

And this simply does not stop at a simple ticket vendor. Take a look around you. We have an array of smart products at our disposal which not only make our day-to-day lives efficient but are also limiting our human interaction with actual humans.

Think about it, we have driver-less cars, automatic washing machines, and vacuum cleaners, digital assistants in our cellphones just to name a few. What else could we possibly need from the technology? If we keep aside the moral implications of the poor bloke who just lost his job at the automobile painting factory to a robotic arm, the overall scenario is pretty good.

Now I am in no way a conspiracy theorist. I do not take the movie ‘Terminator’ seriously and harbor a belief that machines will one day become our cruel overlords. But they do seem to be giving us good competition. There is an increasing trend nowadays to automate every single task out there for the sake of simplicity & effectiveness.

It is not just the non-technical jobs that are at the risk of extinction here. Even we as solutions providers, developers, and consultants need to be aware of this changing dynamic as well.

Consider a more personal example. I have majorly been an enterprise reports developer for the majority of my career as a Business Intelligence consultant. It has been my job to add the visual element to the raw data of the world’s largest companies and help them make sense of it. They love the outcomes, though only some can understand the technical mumbo-jumbo that happens behind the scenes. And quite frankly, they couldn’t care less about the tech that got them the results. If the results are good enough, all is well.

Back then, I’d been working on Microsoft’s SSRS product, which is a simple enough product to get the job done. Of course, I’ve worked on much more powerful tools, such as the IBM Cognos. But SSRS can easily get your basic job done. The entire project went well and since we were dealing with were major tabular reports sprinkled with a hint of pie and bar charts, SSRS did a pretty good job.

Fast forward one year. We had a new client in a new domain but faced a similar set of challenges. We need to take the data, analyze and visualize it.

This time around though, the client wanted interactive dashboards with real-time data and visuals with a jazz-factor (client’s words, not mine). We decided to take the help of Microsoft’s other offering, meant especially as a visualization tool, Power BI.

When I fired up Power BI, it seems quite a lot different than SSRS at the first glance itself. Quite frankly, it felt like MS Excel on steroids. But the difference became evident when I started working on it. I noticed that it took me around 20% of the time to create a visual in Power BI compared to SSRS. And guess what? Power BI is not just faster, it is pretty simple in letting the user create the visuals. It’s more of a simple drag and drop action.

This was quite disturbing for me, as a developer. But on the other hand, as a user, I was quite impressed. It seemed like a no-nonsense tool for the user to create meaningful visuals and dashboards, with little to no need to get into the technical mumbo-jumbo, in no time at all.

And then it hit me. This Power BI tool is the automated ticket vending kiosk to my human ticket vendor.

If the user can itself create stunning, meaningful visuals with a little bit of training, what does he need me for? Isn’t my job as a reports developer in the complex reporting tools kind of obsolete now?

Well obviously visualization tools are not as quite as powerful as conventional report development tools, but license costs are also a major issue here. Since the conventional reporting tools offer you a plethora of services, they’re pretty expensive. And then you do need the specialized resources to work on them. If that was not enough, you’d still have to train your team on the complex tool to take over the day-to-day activities from the development team.

Dashboard

But with a simple self-service BI tool, such as Power BI, Tableau, or Qlikview, you can get the basic services that you need at a much more competitive price range, leaving you with ample budget to up-skill yourself and your team on the tool. Thereby eliminating the need for the specialized developer as the middleman.

So what does this mean? Is this the curtain call for report developers & data analysts like me?

Fortunately not. There is a glimmer of hope. With the increased reliance on the self-service BI tools, there has been an increased reliance on the data modelers. Since the data in the presentation layer is only as good as it is in the functional layer, the need for specialists is being shifted from the visual aspect to the semantic aspect.

What this means for us as developers is that, due to the constant disruption by automation and ease-of-use of the latest tools and technologies, we have to be on our toes and all times. There is a constant need to up-skill and cross-skill yourself to stay relevant in the ever-changing technology landscape.

It kind of sounds like being a jack of all trades and a master of none, but in the solutions and service landscape, increasing the breadth of your scope is sometimes more important than increasing the depth. Specializations are now focusing more on the process rather than the tools.

All in all, the rapid emergence of self-service BI tools is out to get all our conventional jobs. But on the flip side, a lot of avenues are opening up with every new feature or service that is coming in our way. If only we could manage to stay afloat and steer carefully in the rapids, our boat will manage to safely find the port. Up-skilling, and more importantly cross-skilling, is the need of the hour. We can only stay and shape the ongoing course of the technology world if we stay relevant.