SharePoint 2010 includes service applications for managing information that traditionally has been stuck in spreadsheets. The challenge can be choosing which tool best fits your needs. Here’s a rough guide, based on only editing and viewing data in a web browser. From the user’s perspective, no Office client required…

…however, first a licensing note. SharePoint Lists are available in all versions of SharePoint 2010. Excel Web App is available for all versions of SharePoint 2010 but requires Office 2010 licenses to use (you don’t need the software, just the license). InfoPath Forms Services, Access Services and Excel Services all require SharePoint 2010 Enterprise Edition.

Second: This post is just covering services to store and manage structured or semi-structured data typically found in Office . It’s not covering the analytics side, i.e. PerformancePoint and Excel Services, and it’s not covering integrated application features, i.e. Business Connectivity Services or External Lists.

And off we go…

The short version:

• Use SharePoint lists for simple applications with not toooo many columns
• Use InfoPath web forms if you’re creating a lot of columns of the type ‘Multiple Lines’…
• Use Excel Web Apps, if you’re licensed, if you are trying to make a list behave just like a spreadsheet. You know you’re going to need Freeze Panes at some point…
• Use Access Services if you’re creating a lot of columns, need more functions to calculate, more queries to filter and more items to view than a list can possibly handle… and you’re on a budget that excludes more expensive alternatives

Now for a bit more info:

*
Note this refers to the Excel web application that displays the full Excel spreadsheet, not Excel web parts that display a named range of data from a spreadsheet.

Table explanation: X = requirement can be met by the feature.  / = sort of, but another feature can do more or better.  – = not applicable (Excel Web App doesn’t use views to display content). Blank = no, requirement cannot be met by the feature.

Lists

Easiest to use and most common choice for replacing spreadsheets for managing information. Don’t forget lists include calendars, tasks, links, contacts etc.  You can create calculated columns using common functions from Excel, including ‘If’ statements. And you can have multiple different views to manage the data. Different people can edit the list at the same time provided they are editing different items. And finally, if there’s a process involved, you can integrate a workflow to manage the data.

The limits:

• You can only have so many columns before lists become unusable. You have a single form for editing individual items and all columns will be displayed on it. You can use Datasheet views instead to display a limited set of columns. But note: Datasheet views won’t work for any columns containing Managed Metadata, they will be read only.
• According to Microsoft capacity planning guidelines, you can store up to 30,000,000 items in a list. However all views must return less than 5,000 items. And in reality, the number is often far lower before performance degrades.

InfoPath

InfoPath is used to create forms containing semi-structured data, i.e a mix of numbers, calculations, and large clumps of text. The sort of files that could have been created in either Excel or Word because neither really fits the purpose.  With InfoPath you can create web forms for people to fill in and automatically submit to a SharePoint library. You can have multiple views within the form and control which view is displayed when a user opens the form. And you can choose to promote certain properties into SharePoint columns for quick access without opening the full form.

Pros:

• Managing semi-structured data, including lots of text.
• Integrated workflows to manage updates to the form

Cons:

• You can only edit one item in a web form. If you need to edit lots of items at once, you are unlikely to be including lots of text and InfoPath wouldn’t be the right tool.
• The same Microsoft capacity planning guidelines for lists also apply to InfoPath forms libraries

Excel Web App

With SharePoint Server 2010, you can integrate Office Web Applications. They are similar, but not quite twinned, with the Office Web Apps available with Windows Live Online.  You store the spreadsheet in a SharePoint library and open it in the browser instead of the full Excel desktop client.  You don’t get full Excel functionality but it should be more than sufficient for editing content.  Excel trumps lists in that you can use that popular feature – Freeze Panes – to scroll across lots of columns without forgetting which item you are viewing.

Pros:

• Freeze panes
• Bigger range of functions for calculations
• Excel data visualisations can provide dashboard-style analytics

Limits

• In the web-browser, only one person can edit the entire spreadsheet at a time.
• Creating views is a manual effort, it is still just a spreadsheet.

Haven’t found a definitive answer as to how large an Excel spreadsheet can grow and still be opened in the browser.

Access

Access, that client database tool loved by users and loathed by IT usually in roughly equal proportions, has joined the world of SharePoint. You can now create web databases in Access and publish them as a SharePoint site. Forms and reports can be edited and analysed using just a web browser.

Pros:

• You can go column-crazy and choose how many appear in the different forms available within Access, including single item view, multiple items and datasheet views <- side note: lists use Access Services built-in, even in SharePoint 2010 Foundation, to display datasheet views.
• Whilst you don’t get workflow integrated, the query capability offers much more compared to configuring list views enabling different forms to be used for different stages of data update. (You just can’t email reminders automatically when data needs updating).
• Increased capacity compared to lists. Lists can store 30 million items and still be supported but no more than 5,000 items will be displayed in any view. There is no defined limit to the number of records that can be stored in an Access web database, the default is 500,000. But Access queries can contain up to 200,000 items, with a default value of 50,000. But don’t get too excited – see performance note below.
• Better option than Access client databases in networked mode. Access Services in SharePoint uses far better caching mechanisms to lock/unlock items and you don’t need to compact every 5 minutes.

Limits

• Less flexible permissions compared to lists. With lists, you can set permissions per site, list and item. With Access Services, you set permissions per site only.  You can lock columns to read-only in forms but everyone is affected. There’s no concept of a ‘super’ user from the content perspective. Changes to the database schema can only be made by people with Full Control or Design permissions to the site.  And anyone who can view the site can open it in Access 2010 if they have Access installed on their computer. And they can export the table contents to Excel…
• Consider it a replacement only for lists, spreadsheets and simple databases. Lots of limited functionality, such as not being able to do table joins, you have to use field lookups instead (DBAs may howl). Queries can only get so complex and you may resort to doing calculated columns within the table (DBAs – don’t bother howling, it’s no different to a calculated column in a SharePoint list – which is what the table becomes when it is published to SharePoint).

Whilst scale is way beyond SharePoint lists, it’s nowhere near fully-fledged databases (SQL or NoSQL) with web front-ends. The challenge is that it can do enough for users that they inevitably want that other 20% of requirements you said no to.

From a performance perspective, users will experience a significant delay (more than 10 seconds) waiting for a form or report to open if the underlying table or query contains more than 1,000 items. Possibly less – it’s comparable to the delay opening a normal SharePoint list of equivalent size.  The difference is that the delay is due to caching in the background. Once the form opens, editing and filtering is nearly as quick as if working in the full Access client, even if the form contains 10,000+ items. This is where it trumps lists.  If you’re displaying more than 1,000 items, aim for fewer forms and use the Datasheet form which enables filtering without refreshing the query – in effect, you’re viewing a list but without the limit of 5,000 items. May publish more guidelines in a separate blog post.

And there you have it, a rough guide to the different ways you can store and manage data in SharePoint.

References

• SharePoint Editions comparison – Microsoft
• Capacity Planning Guidelines – blog post to be published

Not a million of references for this one, most is based on experience tinkering with SharePoint over the past 12 years. It’s come a long way from that document management add-on to Exchange I first saw in 1999

Whilst most of my software-specific consulting activities revolve around the luscious SharePoint, I’m always on the lookout to bring Excel to the party. Excel 2007 introduces some new visualization capabilities to make it easier and quicker to analyze large data sets without requiring specialist tools. If you haven’t seen what Excel can do, here’s a brief overview of some of the new features. They can be found on the Home tab in the ribbon – Conditional Formatting.

A simple example – some data about product sales. Here’s a traditional approach: a table of data and a chart visualising the results.

Now let’s see what we can do with Excel 2007.

Data bars

Data bars are great for showing quick comparisons between data without needing to create a separate bar chart. In the example above, we can see that we are selling more nuggets than anything else despite the budget suggesting widgets would be the best seller. To set up data bars, simply select the columns and, from the Home tab in the Office ribbon, select Conditional Formatting and choose Data Bars. Then select the color you want to use.

Icon sets

Icon sets are key performance indicators (KPIs). Great for showing if the data represents good, bad or indifferent results.

Tip: The default behavior for icon sets is to display green when the value is above 66%, amber when it is above 33% otherwise red. In just about every case, you will want to change the behaviour. To do so, in the ribbon go to Conditional Formatting and select Manage Rules. For the icon set you want to modify, click Edit rules and make your changes. In this example, I want a green light if the sales variance is 1 or above (i.e. met or exceeded target), amber if sales are within 20% of budget, otherwise it’s a red light for the sales team.

Icon sets have two major shortcomings. First, there are only 3 indicators of performance. In the above example, widget sales are far closer to hitting the amber mark than gizmos, and gadgets only just reached amber. An alternative feature to use is called ‘Color Scales’. Not as pretty, but gives a better range of indicators:

Here you can see gadgets are amber, but widgets are nearly there too. Gizmos are deep red – beyond saving.

Back to relying on charts

The second shortcoming with icon sets as key performance indicators is that they give no indication about trends. Sales of nuggets look great, but is it expected or a surprise? In the image below, I have create line charts for each product and stripped away all the chart gumpf to convert them into ‘spark lines’. Now I can see that widgets have had a sudden drop whilst nuggets have had a sudden spike. And gizmos seem to be heading to oblivion.

Compare this information to the original example – It contains exactly the same data but you can instantly interpret a lot more from the information and it takes up less screen space.

For more information about designing useful visualisations, including spark lines and bullet graphs, see previous blog post: Designing Dashboards

Related blog posts:

• Dashboard Design
• Excel Web Access 101

Filed in library under: Microsoft Office; Microsoft BI

Technorati tags: Excel

Excel Services is a new feature introduced with Microsoft Office SharePoint Server 2007 (MOSS 2007) Enterprise Edition. It enables you to, amongst other things, display information from an Excel 2007 spreadsheet inside an Excel Web Access web part. Why is this useful? There are all sorts of reasons, but the best by far is to use Excel 2007 to visualise data. You can build a nifty web-based performance management solution without needing specialist BI tools. Take the following simple example (click to view larger):

Conditional formatting is being used within Excel to help visualise the data. You can see immediately that black cars are selling as well as red, despite having a lower target. The traffic lights highlight problems with stock levels. In this case, there isn’t enough stock to meet the target for red cars. But there are plenty of yellow cars to sell. The savvy sales person would come up with a spin to make yellow the new black, red, blue…

So how do you set this up?

Configuring Excel Services

If you have MOSS 2007 Enterprise Edition installed, your Shared Services Administration screen should look as follows:

To use Excel Web Access, the minimum you need to do is add the location of your spreadsheet as a ‘Trusted File Location’. A trusted file location can be a SharePoint document library (format http://), web site URL (format http://) or file share UNC (format \\servername\fileshare). For each trusted file location, you can configure additional settings such calculation behaviour (cache lifetime, manual or automatic recalculation), access to external data sources (none by default) and user-defined functions (disabled by default).

Setting up the Excel spreadsheet

To prepare an Excel spreadsheet for display in MOSS 2007, you simply need to do two things:

• Create a named range for the data you want to display
• Store the spreadsheet in a trusted file location (see ‘Setting up Excel Services’)

To create a named range, select the data you want to display and, from the Office ribbon, select the Formulas tab and click ‘Define Name’. Enter a name for your range (ideally one word and easy to remember). To get ready for the next step, count the number of rows and columns that form your named range

This means only the named range will be displayed inside Excel Web Access – all calculations and input fields can be tucked away elsewhere within the worksheet. You can even get clever and create multiple named ranges – one for an input form, to allow users to manipulate the data, and a second one to display the results, based on values in the input form.

Using Excel Web Access

To display your Excel spreadsheet in MOSS 2007, you need to edit a web part page and add the Excel Web Access web part. Then modify the web part properties:

• Enter the location of the spreadsheet (workbook)
• Enter the named range within the spreadsheet (named item)
• Specify the number of rows and columns you want to display

There are all sorts of extra things you can do, depending on how interactive you want the web part to be. You can choose the type of toolbar you want and whether or not people can recalculate the workbook. In the example above, I’ve switched everything off and have automatic recalculation every 5 minutes during the session. I also recommend sizing the web part to fit rather than leaving MOSS to its own devices.

Gotchas

Spot the mistakes I have made in the (recent) past

1. If you want to change the title of the web part, you must first uncheck the box ‘Autogenerate Web Part Title’ in the web part properties (under ‘Title and Toolbar’ heading)
2. If you add rows and/or columns within the named range of the spreadsheet, make sure you adjust the number of rows and columns defined in the web part properties, or the bottom of your named range will disappear from view. You may also need to resize the web part to fit
3. If you move/copy the site to a new environment, make sure you configure the Excel Services settings for the new environment (i.e. configure the Trusted File Location)

Excel Web Access introduces all sorts of possibilities, thanks to the new data visualisation features introduced in Excel 2007 combined with Excel’s ability to connect to external data sources, such as business applications and online services. Displaying the summary information inside a web part eliminates the need to open the spreadsheet in the full Excel application and makes it possible to display Excel data alongside other related information sources.

MOSS 2007 Enterprise Edition could get even more interesting if/when Visio Web Access joins the party (mentioned at the Office Developer Conference earlier this year.) For example, Visio and Excel together make it possible to create a fully interactive web-based business scorecard, complete with strategy map. The challenge remains that competition for delivering web-based data visualisations is heating up, thanks to the likes of Google and Zoho.

[Update: 07 May 08] Please check the first comment from Mark Miller, highlighting some of the limitations with the current version of Excel Services. (I should have highlighed it is a v1 feature.)

Filed in Library under: SharePoint, Microsoft BI

Technorati tags: SharePoint, MOSS 2007, Excel