Skip to main content
OpenI for Digital Health

How is OpenI Different from JPivot?

Last week during a client meeting to evaluate a new project, a question came up — “If JPivot does all the MDX generation/parsing,   provides the UI components for charts and grahps and tables — what is it that OpenI does that’s different from JPivot?”

A legitimate question indeed.

First off — this is not attempt to trivialize what JPivot does. We wouldn’t be here without JPivot – period.  This post is more about what OpenI does to improve certain JPivot functionality and what OpenI adds in terms of its own features to deliver OpenI as a complete web application.  Here’s my assessment:

• JPivot is a component (JAR file), OpenI is a complete deployable web application (WAR). JPivot is one of the many components that OpenI uses
• JPivot provides the following key features
o UI components for tables and charts using OLAP data, and specifying reporting parameters
o MDX query generation based on UI events like drill up/down, filter, sort, etc.
o Representing the results from an OLAP server in an object model
• OpenI makes the following significant improvements to JPivot
o JPivot is built for open source OLAP server Mondrian. It CAN NOT communicate with Microsoft OLAP server out-of-the-box. OpenI extends JPivot to enable this feature.
o JPivot’s native UI is not so clean and pretty archaic (See screenshots here – http://jpivot.sourceforge.net/temp-N101F1.html). OpenI improves on this quite a bit (see OpenI screenshots here – http://sourceforge.net/project/screenshots.php?group_id=142873)
• OpenI adds following key features for a complete BI app that are not in JPivot
o Security – either via its own user management, or by integrating to existing OLAP or custom authentication. Enabling restrictions to analyses based on user permissions
o Dashboard – Jpivot doesn’t have one, OpenI has complete dashboard creation and management
o Create/Save reports– Jpivot doesn’t have a way to persist reports. OpenI provides a well defined XML structure (.analysis files) that works as a report definition language (RDL) to save and manage reports. Also supports the notion of public vs private reports
o Navigation – OpenI provides a full file-explorer like UI to navigate through multiple analyses and manage them in folders
o Report customization by dragging/dropping attributes to columns, rows, and filters
o Tabbed view of tables and charts
o Provides results of an OLAP drillthrough as a text file dump (JPivot can’t), or publish result set to a custom web API
o Explore data feature – lets you “eyeball” data in a cube without having to create individual reports, very handy for exploratory analysis
o Concept of “projects” to enable multi-tenant reporting, i.e. same application can serve multiple clients. Responsys leverages this to have multiple client accounts served separately from a single web application instance
o Administration UI to manage accounts, application, data sources, and many other tasks that you’d otherwise have to do by hand-editing a configuration file
  • JPivot is a component (JAR file) that you can embed in a J2EE application, OpenI is a complete deployable web application (WAR). JPivot is one of the many components that OpenI uses
  • JPivot provides the following key features:
    • UI components for tables and charts using OLAP data, and specifying reporting parameters
    • MDX query generation based on UI events like drill up/down, filter, sort, etc.
    • Representing the results from an OLAP server in an object model
  • OpenI makes the following significant improvements to JPivot
    • JPivot is built for open source OLAP server Mondrian. It CAN NOT communicate with Microsoft OLAP server out-of-the-box. OpenI extends JPivot to enable this feature so users can report on data from both Microsoft Analysis Services (all 3 versions – 2000, 2005, and 2008)
  • OpenI adds following key features for a complete BI app that are not in JPivot
    • Security – either via its own user management, or by integrating to existing OLAP or custom authentication. Enabling restrictions to analyses based on user permissions
    • Dashboard – Jpivot doesn’t have one, OpenI has complete dashboard creation and management
    • Create/Save reports– Jpivot doesn’t have a way to persist reports. OpenI provides a well defined XML structure (.analysis files) that works as a report definition language (RDL) to save and manage reports. Also supports the notion of public vs private reports
    • Navigation – OpenI provides a full file-explorer like UI to navigate through multiple analyses and manage them in folders
    • Report customization by dragging/dropping attributes to columns, rows, and filters
    • Tabbed view of tables and charts
    • Provides results of an OLAP drillthrough as a text file dump (JPivot can’t), or publish result set to a custom web API
    • Explore data feature – lets you “eyeball” data in a cube without having to create individual reports, very handy for exploratory analysis
    • Concept of “projects” to enable multi-tenant reporting, i.e. same application can serve multiple clients. An on-demand  can leverages this to have multiple client accounts served separately from a single web application instance
    • Administration UI to manage accounts, application, data sources, and many other tasks that you’d otherwise have to do by hand-editing a configuration file
While thinking about JPivot, it is important to realize it is performing 2 key, yet different, aspects of dealing with OLAP data – one is UI, and the other more understated part is the bit about MDX generation, parsing, and providing an object model for OLAP result set. If you look at how JasperReports and Pentaho deal with OLAP data, they leverage JPivot as well. This poses a problem that if we want to improve or replace our UI layer with something else, it is not as clear cut since the layers are not clearly separated. Ideally, we would like to have the option of replacing the UI portion of JPivot, while preserving its MDX generation/parsing features.
Julian Hyde, the project lead of Mondrian project, has started a new project Olap4J that builds on this concept, so that could be the next step in this evolution, which we are considering as we plan the roadmap for OpenI 3.0. Stay tuned for more on that.
Sandeep
Uncategorized

OpenI Goes Commercial

This week I completed my tenure as an employee at Responsys and started my new venture OpenI — a company that provides open source business intelligence software and services to businesses that want to be data-driven in their operational strategy.
I guess you can call me a serial entrepreneur now, since OpenI will be my fourth startup — last one being Loyalty Matrix, which was acquired by Responsys in 2007. I am happy to say that the marketing analytics technology we built at Loyalty Matrix found a way to express itself as Responsys’s own analytics product Interact Insight. It was interesting to see the formal structures it requires in a more established company to release a product — valuable lessons that I’ll surely apply in future product releases. It is also great that Responsys will remain a client of OpenI, so that we can advance this technology in a mutually beneficial fashion (and also that OpenI has a few clients from the get go :-).
OpenI will partner with Codemandu, a software development company in Kathmandu, Nepal that has provided the engineering help for OpenI in the past. Codemandu will help us deliver support and integration work for our clients. So — if you have software projects in business intelligence, reporting, and/or analytics (or know of someone who does) — we are here for you 🙂 Basically, if you are an on-demand company that stores transactional data for your customers, we can help you build an on-demand analytics product based on OpenI — something you can private-label and up-sell to your customers.
So, needless to say — next couple of months are going to be crazy, and pretty exciting. Personally, I have a lot of pent-up ideas on making BI more accessible and actionable, and we will be toying around with these ideas in OpenI. And given the nature of open source, these experimentations will happen in public domain — and so you’ll see some fun stuff appear on this blog and OpenI site.
The BI landscape has definitely evolved since OpenI started back in 2005. Most of the big guys (Busienss Objects, Hyperion, Cognos, SPSS) have been acquired by even bigger guys (SAP, Oracle, IBM). On the open source BI side, Pentaho and JasperSoft have done a remarkable job in leading the sector. Plus there has been a great deal of movement in on-demand BI as well – with Swivel, GoodData, and PivotLink, and also at desktop level with Tableau. We will definitely give our best shot to stand on the shoulders of these giants and raise the bar a bit differently.
I recall Sting (lead singer of The Police, for the benefit of our younger readers) say this in a Rolling Stone interview once when asked about his unique singing voice — something like “Nobody can sing like me — I’m not saying that I have the best voice in Rock ‘n Roll, it’s more like someone can sing better or worse, but they can’t sing exactly like me”
So, this I can say — OpenI will be unique in its approach to BI. Stay tuned..
cheers,
Sandeep

Dear OpenI Community:

I am very happy to announce that we are finally forming an official commercial structure around OpenI to provide license, support, and integration services.

OpenI will still remain fully open source, and this site and the related platform at sourceforge.net will continue to function just the same way they have always functioned. However, for those businesses and organizations that need a commercial support structure and/or need a dedicated team to work on integrating or customizing OpenI for their internal projects or product development, this commercial structure enables us to provide such services.

Plus it enables those of us who have worked on this project since its inception a way to make a living doing what we love.

When we started this open source project back in 2005, we couldn’t have guessed where we are today. We started OpenI as an open source project at my last company Loyalty Matrix, where we built a commercial marketing analtyics product on top of OpenI. Loyalty Matrix was acquired by Responsys in 2007. I am happy to say that the marketing analytics technology we built at Loyalty Matrix found a way to express itself as Responsys’s own analytics product Interact Insight. It was interesting to see the formal structures it requires in a more established company to release a product — valuable lessons that I’ll surely apply in future product releases. It is also great that Responsys will remain a client of OpenI, so that we can advance this technology in a mutually beneficial fashion (and also that OpenI has a few clients from the get go :- ).

OpenI will partner with Codemandu, a software development company in Kathmandu, Nepal that has provided the engineering help for OpenI to-date. Codemandu will help us deliver support and integration work for our clients. So — if you have software projects in business intelligence, reporting, and/or analytics (or know of someone who does) — we are here for you :- ) Basically, if you are an on-demand company that stores transactional data for your customers, we can help you build an on-demand analytics product based on OpenI — something you can private-label and up-sell to your customers.

So, needless to say — next couple of months are going to be crazy, and pretty exciting. Personally, I have a lot of pent-up ideas on making BI more accessible and actionable, and we will be toying around with these ideas in OpenI. And given the nature of open source, these experimentations will happen in public domain — and so you’ll keep seeing some fun new stuff appear on this site.

The BI landscape has definitely evolved since OpenI started back in 2005. Most of the big guys (Busienss Objects, Hyperion, Cognos, SPSS) have been acquired by even bigger guys (SAP, Oracle, IBM). On the open source BI side, Pentaho and JasperSoft have done a remarkable job in leading the sector. Plus there has been a great deal of movement in on-demand BI as well – with Swivel, GoodData, and PivotLink, and also at desktop level with Tableau. We will definitely give our best shot to stand on the shoulders of these giants and raise the bar a bit differently.

I recall Sting (lead singer of The Police, for the benefit of our younger readers) say this in a Rolling Stone interview once when asked about his unique singing voice — something like “Nobody can sing like me — I’m not saying that I have the best voice in Rock ‘n Roll, it’s more like someone can sing better or worse, but they can’t sing exactly like me”

So, this I can say — OpenI will be unique in its approach to BI. Stay tuned..

cheers,

Sandeep
Project Lead, OpenI.Org

Uncategorized

OpenI 2.0 RC1 is Out

Today, we have the release candidate RC1 of OpenI 2.0 available for download. There is also a demo available at http://demo.openi.org/openi (login is openi2/openi2)

We look forward to hearing your feedback on improving this release as we work on further testing of this version to get to general release. Please note that we are changing the license of OpenI to GPL v2 from this release since we feel that GPL is a much better option for us at the moment as it enables us to include other GPL-based components in our distribution.

Here’re some more details on the release:

Files (available at: https://sourceforge.net/project/showfiles.php?group_id=142873&package_id=287874&release_id=680493):

  • openi-2.0-RC1-server.zip
    • This is the complete distribution containing all the components necessary to run OpenI with a single click. This is in response to the feedback from our community about simplifying our installation process. We are quite happy with the result, and would love to hear your feedback.
  • openi-2.0-RC1-tomcat.zip contains binary build specific to Tomcat 6.0x
  • openi-2.0-RC1-jboss.zip contains binary build specific to Jboss 4.2.2 (and can be used for other j2ee servers as well)
  • openi-2.0-RC1-src.zip contains full source code to make your own custom builds
As always, we look forward to hearing your feedback.
best,
Sandeep
Project Lead, OpenI.Org
Uncategorized

Deconstructing Hans Rosling’s Latest TED Talk

First off — Hans Rosling is an inspiration to us all in the business of analytics and data visualization. Not only this story is extremely relevant, but the way he shows the numbers — there is a lot to learn. I will make an attempt here to deconstruct his latest TED talk in terms of what a good BI tool show do, and also how this is a great use case of how great BI users behave:

BI Features Used by Hans Rosling:

  • The most prominent is the use of Time as a special type of “dimension”. The tool knows that Time will support the concept of a Play button. This is still very novel — most BI tool, OpenI included, treat Time as any other dimension — you can drill up, drill down, set date filters, or date ranges — but that’s about it.  Taking a lesson from here, what we should do instead is that the moment there is a Time dimension, user should have the option to “superimpose” Time in “Play” mode within a given analysis — this should result in a Video Player like slider widget appear at the bottom of the analysis with a big old Play/Pause button next to it
  • Notice how he first presents the data bubbles in dual-axis graph and then transitions it over to a map view. This makes the concept of “background canvas” a dynamic entity for presenting data. How many other choices a user can have (in addition to dual-axis and map overlay) to use as the context in which the data should be presented
  • He keeps only 1 attribute per axis – country in X-axis, and % of population with HIV on Y-axis, and everything else (gender, per capital income, etc.) is treated as a filter (in OLAP speak). This keeps the visual very clear on its message. I have often struggled with OLAP based analyses, which have multiple dimensions on each axis, which makes sense sometimes in the table view, but the chart-view is completely horrid. Single data attribute per axis is a way to address that
  • When it comes to drilling further into data, he basically clicks on a country bubble — and it can either split by income groups, or only the specific country goes on a time play motion while others stay the some, etc. — the key for me here is that drilling down is best done at the visual level — somewhere on the chart/graph itself the user should be able to isolate a data group (in this case a country bubble), and have a choice on drilling down or move it back and forth in time

Hans Rosling as a BI User/Presenter

  • Emotion, emotion, emotion… he is so far away from the stereotype of a statistician making a presentation. He cares about what he’s presenting. The numbers are real people — they get sick, and they can either get better or they can die.. you can feel that empathy as he presents. 
  • Al Gore did this first (that I can recall) in The Inconvenient Truth when he brought a crane ladder to hoist him up so he can point to the tallest bar in the chart that he is showing. Maybe a bit too melodramatic — but it drives the point, and also makes a more visceral connection with the data. Hans Rosling stands on top of a table at the beginning of the presentation to explain the different numbers he is presenting, and the audience is at once connected and engaged
  • His bringing of the long metal pole to point to the numbers instead of your generic laser pointer (“I have solidified the laser beam”) is another way to get more personal and physical to show how involved he is
  • Ultimately he has leverages the BI tool to make a presentation, to tell a compelling story. Earlier in my career, we worked on a feature with another BI tool that automatically generated powerpoints from its charts.  Yes, it was pretty crude, and didn’t really work that well usabilitywise — but the point is, this was definitely a feature aimed at helping users build a story off the various charts and grahps and analyses. People want to tell a story — the BI tool should help them do that.

Ultimately, watching Mr. Rosling is definitely inspirational — I can only hope that OpenI will one day does the things he’s shown us in this presentation. I’m sure we will get there in due time, but it is the spirit in which BI tools are used, and their ultimate message.. that’s the important thing to keep in mind as we move the product forward.

Uncategorized,

OpenI 2.0 Beta is Released!

Dear OpenI Community:

I am very happy to announce that today we released the beta version of OpenI 2.0. I want to thank our entire development team for all the hard work they have put in for this release.

Please download it from sourceforge.net and try it out. We look forward to hearing your feedback on improving this release as we work on further testing of this version to get to general release.

There is also a demo available at http://demo.openi.org/openi (login is openi2/openi2)

New features and enhancements in OpenI 2.0 Beta:

  • Create New Project: Create new project option is added in the preferences menu that creates a new project by selecting a new template which can be either the duplicate of the current project or a new template. It also allows the users to define the category, project logo etc. for the project being created.
  • Multiple Dashboards: an enhancement in the dashboard user interface which allows the user to create multiple project dashboards. this feature is available for the admin user only.
  • Purge Files: this feature allows the admin user to purge the older files, the user is prompted the file type ( i.e file extension ) and the date , the files with that type and older than the given date will be purged

New Features from OpenI 2.0 Alpha:

  • Completely re-written based on Spring, Java Server Faces and AJAX
  • New enhanced UI layout, completely “ajaxified”
  • New Drag and Drop Navigator UI to move attributes across rows, columns, and filters
  • Enhanced repository system implemented. Now project contents can be deployed anywhere. No need to have project contents within the “webapps” folder of j2ee servers
  • Application can be deployed into any J2EE server
  • QA feature added to validate the MDX statements for all analyses within a project with a single command
  • Manage Feed feature enhancement – now contains split rows feature and better CSV file parser
  • Default analysis for project – you can now specify default analysis for a project which is the first analysis displayed to a user immediately after user login
  • Customizable chart series color (for non pie type chart only) – Now you can define custom color palettes for chart
  • Excel export enhancement – OpenI now generates excel binary file which embeds chart in file itself (previously it contained a web link to the chart, which could result in broken image links)
  • Dashboard UI enhancement – Better UI that enables displaying both table and chart view option
  • Better File Manager – shows file in explorer style tree
  • User/Role management (requires custom build from source) : now users/roles management can be done from OpenI

Plus there are gobs of bug fixes. Check out the release  notes for more details.

Please pass the word around. And as always, we look forward to hearing from you.

cheers,

Sandeep

Sandeep Giri
Project Lead, OpenI.Org

Uncategorized

Business Intelligence for Startups

A product’s progress is measured not by features, but by user experience.  

You can add a lot of value to your product by removing features.

Spending 6 to 12 months with a 4-5 person team to develop the first release of your product is way too long. You can’t wait that long to get market feedback.

These were just a few golden nuggets of wisdom in Dave McClure‘s presentation yesterday at SD Forum’s BI SIG meeting. At one point, Dave said — “I come from an engineering background, I used to write code.. and I can’t believe that I’m saying this to you — but all the architecture, algorithms and engineering behind your product don’t matter as much as how strong your marketing is, which starts with user experience.. we all tend to develop a lot of cool features which the user never ends up discovering” — reminded me of that tree that fell in the middle of a big forest (did it really fall?)

Much of Dave’s talk may seem like web analytics 101. He’s got a (cleverly named) “Startup Metrics for Pirates” called AARRR (stands for Acquisition, Activation, Retention, Referral, and Revenue) to measure your overall success. Then as a startup, you focus on improving these metrics one step at a time (here is his detailed presentation):

  1. Make a Good Product: Activation and Retention
  2. Market the Product: Acquisition and Referral
  3. Make Money: Revenue and Profitability

Personally, I get the step 2 and 3, but it is the first step where a great many of us falter because it is not so easy as it sounds.

Part of making a great product is to have inspiration, vision, and an intense passion to solve a particular problem — and then you hope that there are millions of others out in the market who see the problem with the same high priority as you do — and then also hope that your solution beats the competition by being in a completely different league, either by being the first of its kind, or by introducing a completely superior technology, er, user experience.

So can you do all that in 3 months or less? Seems like you have to, if you want to survive in the web 2.0 world (or whichever version we are on these days)

This clearly does not apply to everything. It only applies to a certain class of products/services where customers can experience the
product on the web, and where the quality of user experience can
be measured (even if it is a model of lead generation online and fulfillment being offline)

So if you are building a new product/service today — can you fit it into this model? Well, one way to try it out is to strip away all the features until you are left with just one or two that are your core differentiators, and see if you can create a version of it that can be experienced completely online. But the point is, if you can do this, you have a framework to measure success a lot earlier in the game. One key edge that startups generally have over bigger organizations is the speed of their product development iteration. How soon can you release a feature to your customers, get feedback and go through a series of subsequent iterations and releases?

Dave’s “Pirate Metrics” are about tracking the effectiveness of these iterations, but first you have to configure your business practice so this model can fit. Not a bad thing if you can.

Uncategorized

BI 2.0, Next Generation BI, and Everythig New and Improved

Fellow blogger Bhupendra Khanal has an interesting post that mentions the challenges associated with BI 2.0/Information 2.0 (he also plugs our very own OpenI, which is much appreciated — Bhupendra, may OpenI karma come back to you thousand-fold :-)Software industry, not unlike any other, contains a lot of hype and probably sometimes even more so with all this 2.0 buzz, which probably seems cool to the industry insiders, but is definitely confusing to the market.

Take BI 2.0 (or Information 2.0) for example – what in the world does it mean? Well, turns out, at the end of the day, to most BI vendors, it means more fancy charts and graphs and dashboards, except this time they’ll have rounded corners, larger fonts with brighter colors, and maybe a fit of Flash and/or Ajax thrown in for a good measure to demonstrate live interactivity.

All this is fine and well, but all this is also pure BS if you are not helping your user make better decisions, or informing them of something new.

If BI 2.0 or Information 2.0 is to be seen as the “next generation” (it seems you cant’ escape these cliches), then it needs to go beyond charts/graphs/dashboard paradigm. BI applications and tools need to be rooted in the knowledge worker’s workflow – and should be cognizant of the types of decisions that need supporting.

BI needs to be aware of the domain context – i.e. which industry are you supporting? which area – marketing, finance, operations, research..? Because without this, the best BI can do is to provide nice visuals and hope and pray that the user knows how to translates them into intelligence and action.

But software can be better than that if is stops being lazy. And that’s my hope with our work in OpenI. We certainly started in the charts/graphs/dashboards paradigm, so we are as guilty as anybody. But as they say in any 12-step plan, “acceptance” is the first step — and now, we are moving towards a future of BI software that caters to the root need for intelligence — i.e. not only that you see your data clearly, but you also see it in your specific business context, and get immediate option to act upon it.

For e.g. a marketing analytics BI application – once it incorporates the data about customers and marketing campaigns and resulting purchases — should not wait for a user to define dashboards and reports, but rather already provide a suite of analyses that answer the most typical marketer’s questions – i.e. how effective are my campaigns , who are my best customers/prospects, and what tactics work the best for individual customer segments? And don’t stop there btw — if you have identified some new and interesting customer segments – it should integrate with an online campaign manager tool to immediately launch new campaigns; or publish the list of your most valuable customers to your e-commerce engine or call center platform which know how to treat them in a special way; etc. etc.

Similar scenarios can be applied to other industries and domains too. We as BI software developers just need to imagine differently. And that’s what 2.0, new version, or next generation is all about – next level of imagination.

Uncategorized

Taking BI Beyond Charts and Graphs

I attended a talk at the monthly BI SIG meeting at SDForum by Christian Marcazzo from Spotfire, now a part of Tibco. I have long admired Spotfire’s innovations on data visualization front, so I was curious how they see BI from the whole Human-Computer Interaction (HCI) aspect, and couple of things stood out.

First – if we look at consumer-centric data applications (Zillow, Google Finance, etc.) and compare their interfaces to more traditional enterprise BI applications, it’s amazing to see how the latter just doesn’t even attempt to look good.

Why is that?

Because enterprise BI app developers aren’t under the same pressure to seduce their users like consumer data apps. Zillow, Google Finance, et al live and die by the community they create, so for them, user experience in paramount, and it shows. Most BI apps, on the other hand, are almost developed under the assumption that users are under a “thou shalt always use this BI software” executive order, and as such don’t have much leverage in rejecting a software based on poor or sub-optimal user experience.

So they begrudgingly use the BI software for its least interesting/effective use – churn out one report after another. The BI app basically becomes a report production factory.

That brings me to my second point – for BI to be more than charts/graphs/dashboards, it needs to be part of the user’s workflow. Now the term “workflow” means a lot of different thing to differnt people, and has recently become a popular box in BI markitechture diagrams – but to me, it basically means that BI app needs to know the various contexts under which its users are using it, and provide a way to add intelligence/insight to the process. BI app by itself should almost be invisible.

Zillow users think of themselves as a home buyers/sellers, not a real estate data analysts. Typical Google Finance users are checking out their portfolio and evaluating stocks, but don’t think of themselves as financial analysts. So, why in the world BI applications are hell bent to think of their users are data analyst first, rather than understanding the specific tasks they are trying to accomplish more intelligently?

That’s where workflow comes in. BI app needs to understand the nuances of the business domains their users are in, make intelligence available in their task workflow where it’s needed, and provide a clear way to act upon that intelligence. Too often we think of BI as a separate app where a user will do analysis, and then the users will jump to other apps where they can take actions — that’s now how users see the world. And without understanding the users, BI apps can’t really provide intelligence.

It’s time to turn this model around. BI apps should think more like mashups –pull data from any “public” repository with REST like API’s, make anlayses available to share and tweak, and make the resulting insights be integrateable to other apps. The more lines get blurred between BI apps and the rest, more successful its adoption is going to be.

Uncategorized