With SAP chewing up Business Objects and IBM swallowing Cognos whole, you might think that the BI market has reached the end-game stage, with just a few giants left standing (IBM, SAP, SAS, Oracle and Microsoft) and one or two as-yet-uneaten critters, like MicroStrategy, flitting around in the shadows. Dr Fern Halper and I believe otherwise.
As the old guard disappears, we see it being superseded by a new set of energetic companies with new approaches to BI. Most such companies have come to our notice very recently and they are all innovative in their approach. Particularly they emphasize new analytical approaches, new and better ways to illustrate data, and quite often, a real-time or near real-time approach to BI.
QlikView
QlikView, from QlikTech, falls into the first category. It offers a new analytical approach. In fact, imho, QlikView has dug a hole for the once admired “data cube” and is gradually burying it.
Data cubes, and even multi-dimensional cubes (multi-dimensional data structures) were once the “new new thing” in BI. They were the much vaunted child of OLAP (On-Line Analytical Processing). The idea was simple enough, you grabbed a “subject area” (or set up a data mart) and built multi-dimensional data structures that allowed you to analyze data in terms of any of the dimensions you had specified.
The problem OLAP products solved was that relational databases held data in 2 dimensional tables and didn’t provide easy ways to build and analyze 3 dimensional or multidimensional data structures. Add this to the fact that one of the most important BI needs for a company was to analyze sales figures, which had at least 3 distinct dimensions (of sales totals, time and geography) and the attraction of the OLAP cube became obvious.
So what does QlikView do that’s new?
First of all, it doesn’t hold data in cubes or similar OLAP structures, it holds data within a specially designed in-memory structure that is far faster to access. It is faster to access for two reasons:
- It is in memory, not on disk (see this posting for more on the memory/disk trend).
- It is designed to provide fast access to all data by any given item in any table.
The way that QlikView works is that you extract data from one or more databases (even very large databases) and it builds its data structure in memory on a server using the associated schema information from the databases.
The power is in that data structure. It means that QlikView can build any multi-dimensional view into that data in a fraction of a second. You want to see sales by month by product by state, OK. How about weekly sales by discount scheme by city by customer age, OK. Or maybe product by customer, by marital status by popularity, OK. With QlikView you can analyze on any data item combined with any other data items that links to it, in a fraction of a second.
QlikView also has the advantage that it is really easy to use. You can get the gist of it by going to the web site and trying the demo. One of the realities of the old data cube was that users had to have a reasonably good understanding of what they were doing in order to decide which cubes to build.
With QlikView you don’t think in cubes at all. You can literally play with the data and, to be honest, you could have no idea that you’re carrying out multidimensional analysis. You are just navigating through the data, discovering interesting information as you go. And if you want dials or pie charts or bar charts or tables of data, you can have them. It’s all ticks in boxes.
QlikView doesn’t require “knowledge workers” to use it and it provides results dramatically quickly. It is game changing. It is going to bury the data cube.
Note: Dr Fern Halper and I collaborate on the topic of BI. You’ll find her blog here. Fern’s comments on QlikView are here: Is This the Death of the Data Cube? (continued)
Subscribe to HaveMacWillBlog in a reader
I was stupefied by your comments so I test drive the software and also contacted my mentor Ralph Kimball
and this what I got from him.
”
While I have no doubt that QlikView employs clever, appealing technology, their claims fall into the category I call “objection removers”. In other words, they sweep away all your troubles in one dramatic step in hopes that you will buy their product before you start thinking about the larger picture.
What stand do they take on
· Local data control (i.e. staging) both of the original extract as well as the delivered BI payload
· Cleaning
· Deduplicating
· Conforming across multiple original sources
· Establishing durable surrogate keys
· Processing slowly changing dimensions (some technologies are extremely sensitive to Type 1 changes for instance)
· Handling late arriving fact data as well as dimension data
· Drilling down anywhere, not just declared hierarchies
· Multi-valued dimensions
· Ragged hierarchies
Also do their claims support a multi-vendor environment using other BI ttols?
You might want to read an article I wrote on this issue of objection removers. Please see http://www.intelligententerprise.com/showArticle.jhtml?articleID=167100313 .
Finally, I believe there is no way to avoid a specification step somewhere in the use of any BI tool. Somewhere, you have to map the source data into the user interface of the BI tool. Every time I have looked at this step, I have concluded that the work to do this correctly is as much work as the cube or star schema building that they claim to avoid!
Good luck,
Ralph”
So I contacted the vendor and they tried to dodge the questions and refer us to their documentation on incremental loading and saving incremental files and reuse them, so I asked for proof of concept regarding these points, the magnitude of programming involved is huge so we decided to go with our old fashioned data warehouse design, before committing to this type of claim do more research and ask for proof of concept.
Sam Moayedi
Sam,
It’s hard to argue with Ralph Kimball on the academic virtues of data warehousing, but that appears to be exactly the feedback he gave you. If you asked me how to mow your lawn and I asked you 20 questions about the meaning of grass and how to effectively grow it evenly and slowly such that less mowing would be required, would you go back to the Toro and tell them they sold you a mower you didn’t need?
BI is not easy and NO vendor will solve all of your problems. Qliktech is trying to solve a few of them, and is doing it better than most because their solution breaks the age-old paradigm of needing large databases. Think about it for a minute…how scary is that to all the academics and BI vendors that based their entire careers on database management? It’s very scary and their first reaction is to talk about process maturity, data cleansing and other BI disciplines that Qliktech never claimed to solve for you in the first place. Believe me, I spent 15 years on warehouses trying to deliver on the BI promise before I decided that one tool (or one solution) can not do it all. Warehouses have their place, big BI has its place, and now data discovery and in-memory analytics finally has its place.
Here’s the real value - try Qlikview and find out for yourself WHERE it fits into your BI stack and solution set. Don’t let an author, vendor or consultant whose salary depends on databases tell you that it can’t be done without massive warehouses. It can, and it does not replace the warehouse as a BI deliverable. It just allows you to handle parts of your BI solution in a faster, more intelligent, user friendly, agile and meaningful way (read: $$$ savings).
Just like every other paradigm shift in the IT industry, most of the existing experts in legacy solutions were late adopters, and this won’t be any different.
Brad
BradP
I’m inclined to agree with you on this. Quite clearly Qlikview makes no contribution to the whole gamut of Master Data Management, and as far as I can tell, it was never meant to. It kicks in at the point where you have data that you want to query which is stored in some database with a schema - which is the same place that a data cube product kicks in.
You can thus go through the whole datawarehouse exercise and create subject databases from the warehouse and use it on those. Alternatively you can replicate production datbases or subsets of them and group them together and use it on those.
Sam’s response to my article doesn’t make much sense to me.
Leave A Reply