Ignoring the methodologies used to calculate this and whether the findings are correct or not, the question I wanted to discuss was – if visitor counts are inflated, does it matter??

First of all, the absolute numbers.  Your web analytics tool says you had 1.5m visitors.  Maybe you only had 0.5m.  To me, this doesn’t matter.  If you are a publisher who is focussed still on the number of eyeballs that view your content for selling to advertisers, then yes, you would like to report the higher number.  But in terms of web analysis, the actual number of visitors to your website doesn’t matter, it is the trend over time that matters and with the level of visitor inflation remaining consistent, this trend should still hold true.

What about frequency of visit, whether the average number of visits per visitor or the proportion of visitor who make 1 visit, 2-3 visits, 4-6 visits or 7+ visits?  Well if visitor counts are inflated then these numbers are very inaccurate.  Let’s look at the data for Feb ’10 for very.co.uk, the new online department store in the UK, from Ad Planner.

Ad Planner claims there were 3.1m unique visitors based on cookies for very.co.uk in Feb, 1.2m actual unique visitors to the website with these people having made 4.6m visits.  First of all, the suggestion here is that the visitor count for very.co.uk was inflated by 2.6 times in Feb (but we are still ignoring whether this is accurate or not).  The interesting thing however is that the average number of visits per visitor could be either 1.48 or 3.83 depending on which visitor count is accurate (assuming either is).  That is a big difference.  Just imagine what the difference is for those proportions too.  And all this is the type of difference that would mean you should have very different business strategies.  Visitor counts being inflated may just matter after all…

I was lucky enough to be at a presentation by Avinash recently where a big topic of discussion was campaign attribution.  One of the points he raised was that if the number of visits to conversion for an ecommerce website is 1 or 2, visitor based attribution is fairly irrelevant.  It is only when the visitor makes multiple visits prior to making the purchase that visitor based campaign attribution becomes relevant.  But if visitor counts are inflated, the reported number of visits to conversion is very likely to be under reported and suddenly the behaviour of your website visitors is quite different to what you may think it is.

So visitor level campaign attribution could be important after all, based on the logic from Avinash, whatever the data for visits to conversion may say.  Well yes but no.  The idea of visitor counts being inflated is due to visitors using multiple devices to access a website and also some level of cookie deletion.  And what it means when it comes to visitor level campaign attribution is that you are only recording a proportion of the visits that led to that conversion.

It would mean that whatever campaign attribution method you may use – last click (can we now say this is generally agreed to be less useful), first click, even weighting, proportional weighting – well they all only count some of the visits leading to the conversion so the data and the conclusions drawn from the data are incorrect.  The conversion for the visitor may be recorded on their 2nd visit, the first being via a generic search term and the second being via an affiliate.  Simplistic example but this would still lead to various combinations of value assigned to the different campaigns depending on the attribution model.

What might be missing (if visitor counts are inflated) are those other visits by that visitor prior to the purchase – with these visits maybe coming via an organic generic search term, a link on twitter and also those two visits from paid brand search terms.  What all this might just possibly mean is that the data that is being used to determine budget allocation for the next year based on the carefully researched campaign attribution method just might not be that useful after all.

All of this is of course just hypothetical.  Various claims have been made that visitor counts are inflated but it doesn’t appear yet that this is universally agreed.  Personally I can imagine that as people use their work computer, home computer and mobile phone to access websites, that reported visitor counts are a little higher than actual fact.  And if they are, the above are a few ways in which the data that is being relied upon to make business decisions may be a little flawed, meaning those decisions that are being made might end up being flawed as well.  And this matters.

Related articles by Zemanta

• How unique is a unique visitor? (vator.tv)
• Your unique visitor figures are 2-4 times too high (blogstorm.co.uk)
• Biometrics firm confirms: User counts for websites are 2-4 times too high (venturebeat.com)

The last 9 months with Logan Tod has been very interesting.  Learning how to use HBX and Omniture SiteCatalyst and increasing my knowledge of Google Analytics.  Focusing my attention on looking for insights in the data which can be used to recommend improvements to websites or marketing.  Broading my experience with the range of websites that I have worked on and the nature of the work.

I still like to keep it simple though.  I think with the vast amount of web analytics data available for any website, it is easy to get lost in it.  And if you dive in too deep too quickly, it can be difficult to find anything as you are surrounded by too many pieces of data.  I think it is best to start simple, with the key actions or conversion points on the site, the top line metrics.  An understanding of these will generally lead to fairly basic questions about the performance of the website.

But answering these questions will lead to more and increasingly more complex detailed questions about performance and an understanding of the factors that truly are impacting on the performance and success or otherwise of your website and business.  The data itself and an exploration of it can lead you to map out what is important to know in order to make improvements.

One of the more frustrating parts of learning HBX and SiteCatalyst was the discovery of the basic reports that I feel are missing.  In particular, there is not a simple report that gives visits (or even responses) by traffic source.  Having to try and pull data from various sources into a single excel report (as the alternative) is painful with question marks over the accuracy.  SiteCatalyst has the option of Unified Sources via a Vista Rule, this needs to be mandatory for anyone using SiteCatalyst and I wish it came out of the box.  While I have various workaround for HBX, I finally came up with a version using multiple segments where I am satisfied with the accuracy, but no where near the ideal solution.  All tools that I have used need to work harder on getting the balance right between reporting performance and allowing analysis.  I know analysis leads to insights leads to recommendations leads to improvements leads to more money but simple performance reporting highlights where and when the analysis should occur.

Anyway, likely more on that in the future.  I might need to be careful or it could be too easy to write about the philosophy of web analytics ahead of the practicalities of it.  And I need to get a new simple WordPress theme.

The next Monday I started my new life as a Customer Insight Consultant with Logan Tod.  Over the past two months I have been using Omniture (including Discover), HBX, Google Analytics and Sitestat as well as other online data tools.  Whatever the tool, the intention is always the same, to use the data to make better decisions, decisions that will lead to a more successful performance for the company.

I am working across a range of clients performing a range of tasks.  While part of my time is spent running your typical weekly reports, I am also working on some interesting projects, using my knowledge of web analytics to make recommendations for actions to take that will improve the performance of these websites and businesses.

I have been exposed to more e-commerce clients than in the past and have definitely already improved my understanding of what elements can reduce the conversion rate for a website.  The same principles can be applied to non e-commerce websites, looking at the conversion rates for these sites however the conversion is defined.

It is also interesting using a range of tools and getting to know the strengths and weaknesses of each. While all tools are using a similar methodology and reporting on similar metrics, it is surprising just how different they can be.  Early signs is that there is no best tool, rather that they are all good (and all frustrating) for different reasons.

I have also been busy learning how to be the President of my local Australian Rules Football team in London.  Recently this somehow meant I had to learn how to create websites – a week and a half of work later and I had put together the new Putney Magpies website.  Unfortunately it doesn’t appear to work as intended across all browsers and screen resolutions but it is a starting point.

Anyway, they may only be intentions at this point, but they are to write a post on web analytics (not website development) each week – hopefully more interesting and useful now given the experiences I am having.

Related articles by Zemanta

• John Lewis Direct boosts consumer interaction
• Yahoo! Analytics Rolling Out
• Omniture Launches Analytics-Driven Site Search
• Web Analytics Association Releases 2009 Outlook Survey Data (searchenginewatch.com)

Perhaps unsurprisingly, they are facing many of the same issues as those reported by large organisations with e-commerce websites.  The key issues are:

• no clear objectives from stakeholders
• lack of time
• lack of support of internal stakeholders
• insufficient knowledge of how to use web analytics data to make business decisions.

Given this, I aimed my presentation at addressing these issues, hopefully providing practical suggestions for actions that can taken to make web analytics more actionable within their organisations.  This presentation that I gave can be found below.  While it was directed towards universities, I believe the suggestions would be relevant for any organisation.

Most metrics, when viewed at daily level, contain a form of daily seasonality.  This is most clear in metrics such as visits, page views or sales which are absolute numbers.  There is a re-occuring pattern throughout the week with peaks and troughs on the same day/s each week.  An example of this pattern can be seen in Figure 1 below.

While this makes any chart pretty to look at, it makes it difficult to really identify trends or spikes in the data.  Is a data point high because there was a spike or because it was a Monday?  It is school holidays but should the number of visits on that Sat really be that low?  And of course, what day did we start to see traffic decline from and how much of a change is it really?

Figure 1

A common method used to remove daily seasonality is to smooth the line out using a moving average.  As it is a weekly pattern, a seven point moving average should lead to a nice smooth line.  Unfortunately, as can be seen in Figure 2, this means you get a nice smooth line, hiding most of those interesting spikes and step changes and general data trends.  You can see overall trends but you cannot pinpoint particular days when a change occurred.  It is also difficult to clearly identify a change immediately, as each day only contributes one seventh to each data point.

Figure 2

What I advise doing instead is to remove the daily seasonality from each data point, resulting in a line that is unaffected by what day of the week it is.  Using this method means that it is clear to see if the performance each day was good or bad. For example, in Figure 3, it can be seen that the relatively worst day for visits was actually the 25th Aug, even though visits for that day were higher than for other days during the reported period.  The technique for removing daily seasonality can be applied each day, meaning that you can identify and react to a change in performance immediately.

Figure 3

The difficulty then is in calculating the daily seasonality across a week.  This can be done properly using SPSS or a similar tool but I use a quick hack workaround in Excel that, while not 100% accurate, gets the job done.  The steps to calculate daily seasonality for a metric (using the examples of visits) are as follows, with the example displayed in Figure 4:

1. Extract historical daily visits data.  You will need at least 6 weeks, more if the period includes a known number of factors that could impact on traffic e.g. school holidays, public holidays, product releases, marketing campaigns, etc.
2. Reorder the data so that each column contains a single week and each row contains only data for a particular day of the week.
3. Recreate this table so but replace the visits for each day with the % that visits for that day contributed to total visits for that week.
4. Add two more columns to calculate the mean and median for each row of data.
5. Delete all weeks which contain days which don’t reflect the general pattern.  In this example, weeks 5 and 6 were deleted.  At this point, the mean and the median should be relatively similar for each day of the week.
6. The daily seasonality pattern is achieved by multiplying the daily mean by 7.

Figure 4

This daily seasonality pattern can then be used for removing daily seasonality for that metric for any day.  Simply divide the value for each day by the relevant daily seasonality in order to remove it.  I generally do this using a vlookup against the day of the week for each date.

Going back to the reason for web analytics, you can use this technique to clean data so that you can instantly identify good and bad days, whether this is historical data or just for the preceding day.  If you are using this for historical data, you can identify the interesting days to investigate further (play with by segmenting).  If you are using on an on-going basis, you can see instantly what performance was like for the previous day and if need be, investigate and react to a change accordingly.

Currently, in order to be able to do this sort of analysis, you need to extract the data into Excel.  Hopefully one day, web analytics tools will allows you to upload a daily seasonality pattern for a metric so that you can display the daily data with this seasonality removed.  And my dream is of a tool that would incorporate the ability to automatically create the pattern for any selected metric (with manual over rides for tweaking of course).

The other key use that I have found for a daily seasonality pattern is it can be used in forecasting daily traffic levels.  If you are able to forecast what the week’s traffic should be, this can easily be multiplied out using the daily seasonality pattern to forecast traffic at a daily level.

A copy of the Excel file containing all the data, charts and formulae used in the examples above can be downloaded here – Daily Seasonality File.

Social media consists of online conversations. The reason for measuring these conversations in some way (and the reason for all web analytics) is to collect data that can be used to make informed and therefore better business decisions.

Now if the data is not used or is not useful, there is no point in doing the measurement. This is not to say that the only use for measuring social media is to calculate an ROI in terms of sales (or equivalent conversions for non-transactional websites). But I firmly believe there is value in collecting whatever data you can collect, acknowledging it is incomplete and will always be incomplete but then using it as a guide in the decisions you are making.

As with any spend within an organisation, the first step should be to define objectives, what you would like to get in return for your spend. Simply tying social media to immediate sales, while easiest, is likely to underestimate the contribution it is making to your company. But not recognising that there is a need to justify the resource allocation (spend, time, etc) on social media in terms of how it is helping a company to meet its overall business objectives, will put at risk the existence of those resources.

The compromise may be a Balanced Scorecard that contains a mix of metrics with associated targets based on what you are trying to achieve with your complete marketing campaign, of which social media is one element. These metrics could include revenue, cost of providing customer support (targeting a decline), brand awareness across the internet, visitor participation on corporate blog, website engagement, ideas generated through customer feedback, etc.

This Balanced Scorecard would show the value and impact of social media to a company beyond the basic ‘how much money did it make us’. The metrics used would be tied to business objectives so they are relevant and meaningful to management. Each metric would have a goal so that performance can be clearly evaluated. If goals are not being met, then that is a trigger to say that action needs to be taken in order to bridge the gap. And any decisions made would of course be a more informed decision using the data that has been collected through the measurement of your social media efforts.

I guess the point of all this is to say that people need to be thinking about how to evaluate the performance of their social media strategy from the initial stages, this is not an afterthought that can be tacked on at the last second.

*******

This was an article I wrote for one of the Amplified 08 newsletters, a network of network event that is occurring on the 27th Nov in London.  For more discussions around calculating the ROI of social media, I recommend having a read of a post by Jason Falls (along with all 93 comments) or the post by Clare Munn (which contains links to other relevant blog posts).  A lot of my thinking around this was influenced by the comments on Jason’s post.  From that, I agree that social media must be more than a straight forward ROI calculation but I still feel that it needs to be measured and tracked over time in some way, hence my suggestion of a Balanced Scorecard approach.

I have thought of an alternative view of a report which I believe would be a useful tool in evaluating the effectiveness of a page. This reporting view does not exist in Sitestat or Google Analytics and I suspect it probably does not exist within other web analytics packages either (please let me know if I am wrong). On the assumption it does not, can I please request that someone designs this reporting view for me.

Quickly first, some current methods of approximating the effectiveness of a page are:

• rating of the page by a sample of visitors (there can be issues with sample size and potential bias?)
• actions taken on a page (but what if there was no way to interact with the page?)
• time on page (only valid if didn’t exit from page but even then, was the visitor actually viewing the page the whole time?)
• actions taken by visitor during rest of visit or within specified time period (did viewing that page actually influence future actions?)

Another method of judging the effectiveness of a page is to base it on what the visitor did next. This can be done using a website navigation report (Navigation Summary in GA or Clickpath Explorer in Sitestat) or a Site Overlay. However it is currently only really practical and useful within a multi-page process e.g. the 3rd page of a 5 step checkout process. At this point, you know where the visitor should have come from and what they should be doing next – therefore can see if visitors are behaving as you expect.

For most other pages on the website, what is happening looks something like this diagram. There are a range of pages viewed immediately before and after the current page with one option in each category being ‘Entered Site’ or ‘Exited Site’. What can’t be easily seen is which of the previous pages link up to which of the next pages. And it is this kind of detail which could be useful in evaluating the performance of a webpage.

Imagine that your (small) site had been visited 9 times with the following being the clickpaths on your site.

1. A -> B -> X
2. A -> B -> X -> C
3. A -> B -> X -> B
4. A -> B -> X -> A
5. A -> B -> X -> X
6. A -> C -> X -> C
7. A -> C -> X -> B
8. A -> X -> A
9. X

Using the current reports for the page X, you would be able to see the following breakdowns for the previous and next page viewed:

Previous pages

• B – 50%
• C – 20%
• Entry – 20%
• A – 10%
• X – 10%

Next pages

• Exit -30%
• A – 20%
• B – 20%
• C – 20%
• X – 10%

This report is nice but doesn’t really tell you that much. Visitors came from a variety of pages and left to a variety of pages. However, if the next pages could be grouped, an alternative view of the ‘Next pages’ could be shown as follows:

Next Pages

• Exit – 30%
• Previous page – 30%
• Previously viewed page – 10%
• Current page – 10%
• New page – 20%
• Specified page – 0% (no page specified)
• Internal search results – 0%

This report now shows that the visitor went on the view a new page only 20% of the time, information that was not readily available in the current reports. The visitor returned to their previous page 30% of the time, possibly they are clicking on the back button due to unappealing content or because there is no clear indication of where to navigate to next on that page. For expected breakdown would vary for different types of web pages but this report could be useful in evaluating the performance of pretty much any page on a website.

Of course, for any report to be really useful, it needs to be segmented. Besides the usual segmentation options (traffic source, etc), I can imagine this report would be improved by first of all eliminating bounces visits for that page and then segmenting on whether the page was the landing page or not. Also, assuming there was an option for specifying a page or set of pages viewed next, this could be useful in understanding if the visitor clicked through to either target pages or the homepage from the page being evaluated.

While it is hard to definitely say without the report being in front of me, I believe a report of this nature would be useful in evaluating the performance of a webpage. Most importantly, I believe the report would help in making business decisions that lead to a company achieving its business objectives. And given that is (or at least should be) the requirement for a web analytics report existing, I think this would be a useful addition to the reporting suite for any web analytics vendor.

His first recommendation is to innovate your way through the downturn in the economy by being creative (not gimmicky) and focusing on increasing the value to the customer.  The second recommendation is to become more efficient with your marketing spend, ensuring every dollar is optimised for results.

While I am biased, I believe this makes the role of the web analyst even more important.  If you are to know if the changes you are making are working for your business, you need to be able to understand and interpret the data.  It is great to have people who can create efficient and/or innovative marketing campaigns but it is just as important to have people who can accurately tell you what worked best, which marketing campaign delivered the best ROI and had the highest conversion rates and/or satisfaction levels.  With this knowledge, a business has a chance of not just surviving but of growing during this financial crisis.

Most of the issues I faced were due to the page content not matching the page expectation.  When a user clicks on a link, they generally have an idea of what they should be seeing next, based on the name or context of the link.  A failure to meet this expectation will often lead to the visitor leaving the site – I only stuck around and tried harder to find the information I was after due to having that voucher.

My experience with the STA Travel website

My idea was to take a local tour, somewhere in the UK and so I selected ‘tours & treks’ under the tours menu at the top.  I wasn’t really after an ‘Adventure Trip’ as per the heading of this page but assumed this is a generic name for all tours and ignored.  I was interested in Europe and chose one of the three options to click on for Europe (all lead the same place).  Scrolling down this page, I found the section for England, Ireland and Scotland and clicked on the single link provided to search through this tours.

So far, the experience was all reasonable.  It took me three clicks to get to a point where I had basically told STA Travel that I would be interested in tours for the UK.

Unfortunately, the page presented (eventually, very slow loading) is a search page, requesting that the visitor advise STA Travel of the continent, country and tour duration they are interested in.  Given I have already given the first two pieces of information, this is a little frustrating.  There is also a set of boxes, one for each continent.  I guess an easy way to bypass the search box would be click on one of these boxes?  No, this just takes you to the page for that continent (in my case, the page I was previously viewing) with this page containing a set of links back to the search page.

As I had a voucher, I forgave STA Travel for requesting information I had already given it (I would love to see the exit rate for this page) and requested to see tours in the United Kingdom of up to 1 week.  I would have preferred to specify weekend trips of not more than 3 nights but did not have the option.  This returned 27 tours (not in any particular order that I could identify) with the first 10 being displayed on the first page of results.  If you are clever, you will realise that you can sort this list by clicking on any heading (I wasn’t that clever and only discovered this while writing this post).

Within this list of tours, it was surprising to find for Oktoberfest (Germany), Rome, Prague and Paris.  Also surprising were the 8 day tours.  As I could not specify when I wanted to go on a tour, the Christmas and New Year tours were valid responses, just not relevant to me at all.  Of the 27 tours, only 8 were for the UK, of less than a week and were not a Christmas/New Year tour.  Given I had the voucher, I did what I suspect many would not do (this should be fairly obvious in the web analytics data using funnels or navigation reports) and stuck around to look further.

A little issue, when clicking on the next button for the next page of tour results, your view stayed at the bottom of the list of results.  In an industry where the cost of switching is negligible (unless you have a gift voucher), the settings should be adjusted so that the visitor is viewing the top results for that page without having to scroll to the top of the page.

I have not travelled around Scotland (the University of Aberdeen is very pretty though) and the two sentence short blurb sounded interesting so I selected ‘more info’ for the tour ‘Skye High’.  This took me to a page that displayed the exact same blurb and four buttons, two for ‘call us’ and two for ‘email enquiry’, all for ‘enquire about this tour’.  No information about tour dates, itinerary, what you get for your money and definitely no option to ‘book now’.  Given that STA Travel has a checkout engine for booking flights online; if after all this pain you still wanted to book a trip through them, why don’t they let you give them your money.

Should I add that i didn’t get a response when I tried emailing through an enquiry.  Or that there is no option to book Eurostar online if you are not a student or under 26.  And that the experience with looking up ski holidays can be worse than that above – the page doesn’t display properly in Firefox (you are happy to ignore what proportion of your visitors?) and I got the line ‘your chosen criteria returned no results’ (when I clicked on the link for ‘more’ under the section ‘On the Piste’).

How to use web analytics to identify and help fix these issues

Most of these issues are obvious without web analytics.  However with web analytics, you can quantify the scale of the issue.  The key metric to be looking at is the overall site conversion rate, what proportion of visits result in a sale or a request for more information from the visitor.  I can very confidently say that this metric is currently lower than it should be.

The key reports that should be run are

• exit page reports to understand if people are leaving from pages that aren’t natural exit points (indicating they are frustrated and giving up).
• funnel/clickpath reports to understand if people are not navigating through the website in the sequence they are expected to.
• page views per visit and frequency reports to see if they have high numbers.  This is what I would expect to see if people are having to search through the website for the information they are after.  The key question is whether visitors keep on trying until they do find that information and make a purchase or if they give up and go to a competitor instead.

With some clever calculations, it is possible to estimate the amount of revenue that is being lost as a result of people giving up on a site.  This can be used to make a business case for investing in the website improvements.  Web analytics also makes it easy to measure the improvements due to any changes that are made to the website.  If multivariate testing is used, you can even prove in advance that the change will make a difference and the scale of the improvement, at minimal risk.

STA Travel needs to immediately look at their conversion rate and understand where people are dropping off (giving up on) their website.  The proportion of visitors using alternative browsers to IE should also be looked at, no major company can be happy with a website that cannot be viewed properly by around 40% of their visitors. I hope these issues are already being pointed out by internal web analysts at STA Travel.  And that if they need assistance, they are talking to their WebTrends consultants (as that is the tool they are using).  I am happy of course to offer my assistance as well but I will be charging for any future advice.

My advice is to start simple and to talk small steps from there.  Don’t worry initially about actionable insights, they will come with time.  Instead, just get your company used to looking at web metrics.  Create a one page report containing basic metrics and start sending it out to key internal stakeholders.  Include some comments but again, starting simple, these need only be what has happened that week.

Based on experience, once departments and managers start receiving data, they will start asking questions.  Set yourself a target of answering 1 to 3 of these questions each week, focusing initially on those questions that you can answer (including the answers in the weekly report).

Over time, as experience grows, the questions will get more complicated and the answer will become more useful to the business, they will turn into actionable insights.  The report that is sent out will evolve as well, as people understand which metric they need to see and which are only nice to know.  KPIs will be defined and used to measure the success of the business.  The weekly report will contain recommendations on how the business can improve and this information will be used.

Just keep in mind that this won’t all happen overnight so don’t expect it to.  if you try to do it all at once, it likely won’t work.  Take small steps, educate the people around you and you can make a real difference to any organisation that you work for.

For suggestions on which metrics to start with, try reading Your First Metrics – Part 1 and Your First Metrics – Part 2.