Microsoft has a proof of concept built on SharePoint – Looking Glass (must stress: It’s a proof of concept (PoC), no mention of an actual working solution yet and Microsoft PoCs often have a dash of smoke and mirrors about them):

But wafting the smoke aside and stepping around the mirrors, the concept is sound. There are plenty of tools on the Internet for visualising trends from social media sites like Twitter and Facebook. (Twistori is a pet favourite of mine for simply observing the world in a conversation) But few go beyond visual analytics. This video explores how to integrate ways that enable you to act on the trends uncovered, what happens next. Interesting stuff. Bet the Microsoft CRM team just love it!…

So much for publishing soundbites during the recent SharePoint 2009 Conference. What can I say, I got distracted Here’s a delayed one.

Microsoft announced a new product at the conference that has been going under the codename Project Gemini – PowerPivot

PowerPivot is being released as an add-on component to Excel 2010 (licensing not available at time of writing). It’s purpose: analysing massive sets of data using familiar tools. It brings business intelligence (BI) into Excel. Historically, to do such large scale analysis has required specialist tools.

PowerPivot enables incredibly fast filtering and sorting of spreadsheet data extending to 100 million rows. That’s a pretty big dataset for Excel to handle. PowerPivot includes some nifty compression algorithms and the working data set is read only. There are features to enable you to edit related tables that feed into it. With SharePoint 2010 you will be able to display the content and analysis in web parts for browser-only scenarios. And whilst its title suggests it’s a giant PivotTable, PowerPivot is not your traditional Excel pivot table. You can have multiple slices based on related tables to cross-analyse the data. Here’s a couple of images taken from the conference:

In the image above there is the main pivot table (selected in blue) summarising and filtering total purchases by selected continents. To the left and top left of it you can see two slices that are being used to further filter the data by genre and rating

In the image above you can see one of the new functions that are included with PowerPivot, it is creating a sum by matching values in a related table. It’s hard to visualise how different this is to the traditional Excel pivot tables and formulas. One example given during the talk was that PowerPivot could enable historical comparison analysis such as comparing accounting information across financial years.

This is an interesting move for Microsoft as we enter an era where massive amounts of data are being created and shared across the Internet. Finding easy ways to visualise such quantities of data is a hot topic. Microsoft is not the ony one coming up with new tools…

Some reading on how massive amounts of data is challenging conventional wisdom:

• Recent Innovations in method (The Technium, December 2004)
• The Google Way of Science – replacing the hypothesis (The Technium, June 2008)
• The Future is Big Data in the Cloud (GigaOM, October 2009)

Side note: Whilst I can see where Microsoft got the name from, I can’t help but keep calling it PivotPoint instead. Blame SharePoint, PowerPoint and PerformancePoint for that

Conditional formatting is being used within Excel to help visualise the data. You can see immediately that black cars are selling as well as red, despite having a lower target. The traffic lights highlight problems with stock levels. In this case, there isn’t enough stock to meet the target for red cars. But there are plenty of yellow cars to sell. The savvy sales person would come up with a spin to make yellow the new black, red, blue…

So how do you set this up?

Configuring Excel Services

If you have MOSS 2007 Enterprise Edition installed, your Shared Services Administration screen should look as follows:

To use Excel Web Access, the minimum you need to do is add the location of your spreadsheet as a ‘Trusted File Location’. A trusted file location can be a SharePoint document library (format http://), web site URL (format http://) or file share UNC (format \\servername\fileshare). For each trusted file location, you can configure additional settings such calculation behaviour (cache lifetime, manual or automatic recalculation), access to external data sources (none by default) and user-defined functions (disabled by default).

Setting up the Excel spreadsheet

To prepare an Excel spreadsheet for display in MOSS 2007, you simply need to do two things:

• Create a named range for the data you want to display
• Store the spreadsheet in a trusted file location (see ‘Setting up Excel Services’)

To create a named range, select the data you want to display and, from the Office ribbon, select the Formulas tab and click ‘Define Name’. Enter a name for your range (ideally one word and easy to remember). To get ready for the next step, count the number of rows and columns that form your named range

This means only the named range will be displayed inside Excel Web Access – all calculations and input fields can be tucked away elsewhere within the worksheet. You can even get clever and create multiple named ranges – one for an input form, to allow users to manipulate the data, and a second one to display the results, based on values in the input form.

Using Excel Web Access

To display your Excel spreadsheet in MOSS 2007, you need to edit a web part page and add the Excel Web Access web part. Then modify the web part properties:

• Enter the location of the spreadsheet (workbook)
• Enter the named range within the spreadsheet (named item)
• Specify the number of rows and columns you want to display

There are all sorts of extra things you can do, depending on how interactive you want the web part to be. You can choose the type of toolbar you want and whether or not people can recalculate the workbook. In the example above, I’ve switched everything off and have automatic recalculation every 5 minutes during the session. I also recommend sizing the web part to fit rather than leaving MOSS to its own devices.

Gotchas

Spot the mistakes I have made in the (recent) past

1. If you want to change the title of the web part, you must first uncheck the box ‘Autogenerate Web Part Title’ in the web part properties (under ‘Title and Toolbar’ heading)
2. If you add rows and/or columns within the named range of the spreadsheet, make sure you adjust the number of rows and columns defined in the web part properties, or the bottom of your named range will disappear from view. You may also need to resize the web part to fit
3. If you move/copy the site to a new environment, make sure you configure the Excel Services settings for the new environment (i.e. configure the Trusted File Location)

Excel Web Access introduces all sorts of possibilities, thanks to the new data visualisation features introduced in Excel 2007 combined with Excel’s ability to connect to external data sources, such as business applications and online services. Displaying the summary information inside a web part eliminates the need to open the spreadsheet in the full Excel application and makes it possible to display Excel data alongside other related information sources.

MOSS 2007 Enterprise Edition could get even more interesting if/when Visio Web Access joins the party (mentioned at the Office Developer Conference earlier this year.) For example, Visio and Excel together make it possible to create a fully interactive web-based business scorecard, complete with strategy map. The challenge remains that competition for delivering web-based data visualisations is heating up, thanks to the likes of Google and Zoho.

[Update: 07 May 08] Please check the first comment from Mark Miller, highlighting some of the limitations with the current version of Excel Services. (I should have highlighed it is a v1 feature.)

Filed in Library under: SharePoint, Microsoft BI

Technorati tags: SharePoint, MOSS 2007, Excel

Installation

Can be fussy, especially when installing with SQL Server 2005. You will need to download and install PTSLite.exe to pass the OLEDB provider requirement, along with ADOMD.NET and MSXML4 (SQL Svr 2005 is up to MSXML 6). For the sensible souls who actually do read manuals, the readme file included on the CD does explain all this – it’s the first link that appears when you install the CD, ignore it at your peril…

Make sure SharePoint is already installed and the virtual server extended. Because I was building a demo, I left SharePoint Portal Server (SPS) on the default WMDSE database but most installations would put SPS on SQL Server. When checking Google for blogs on BSM, I came across one (here) that suggests you cannot install SPS with SQL Server 2005 Developer edition.

Once that’s all done, installing the Builder and Scorecard should be straightforward. My installation failed the hardware requirements test (although the machine meets the system requirements listed for BSM), but it doesn’t prevent you from installing. Any other failures (that will occur if you don’t get OLEDB and friends installed first) will prevent you from continuing the installation.

Setting up a scorecard

To get a very basic scorecard published… (the following assumes you have some familiarity with the BSM Builder tool… which could be quite an assumption, because the UI isn’t as intuitive as it could be.)

• In SharePoint (Windows SharePoint Services or SharePoint Portal Server), create a site where the scorecard will be published to
• Open up BSM Builder and create a new workspace.
• Create an indicator (new definition), select 3 levels. Edit it and assign an image (traffic light) for each of 3 values (red, amber, green) – images are available with the sample folder that is installed with BSM. Publish indicator to server
• Create a KPI and give it a fixed value (actual & target). Edit the target and assign the indicator you just created. Publish KPI to server
• Create another 2 KPIs that would be subs to the first one (e.g. first one = profit, second one = profit value, third one = profit margin)
• Create a scorecard. Add the KPIs to it (add all 3, then indent the two sub-KPIs under the main one by clicking the arrow in the toolbar). Edit the default scorecard view, and add in the columns (actual and target) for the KPI. Publish scorecard to server
• Deploy scorecard to SharePoint. You will be prompted to enter a page name – call it anything (it will create a web part page for the scorecard). You will then be prompted to connect to SharePoint – enter URL of the site created. You’ll need to also select a library – the default ‘shared documents’ will do
• Hey presto, one scorecard published and deployed.

To go beyond basic, create a data source to use instead of entering fixed values. For a simple scorecard, use ODBC and connect to an Excel spreadsheet (see below for a sample connection string). For more advanced scorecards, you will be connecting to one or more databases. For more advanced filtering capabilities (i.e. multi-dimensional), use ADOMD.NET to connect to an OLAP cube.

Sample Excel connection string: (replace c:\test.xls with your actual filename)

Driver={Microsoft Excel Driver (*.xls)};DBQ=c:\test.xls;ReadOnly=True;

This string didn’t work with folders (i.e. file needs to be in root of drive) so there is probably a better one to use, but it does the job for building a simple demo.

This information and other bits and bobs are posted on the web site here.