Listening today to comments that Germany’s goals were just counter attacks, that England had more shots, more shots on target, more possession and therefore hadn’t been comprehensively outplayed made me think. Having watched the game, (and I freely admit it is not my sport, I don’t have a complete appreciation of the complexities of it and I was a little biased) I had the impression that Germany were running all over England. Not all game, England were very good for the last 10 min of the first half but outside of that… So what about these key footballing statistics, why didn’t they represent my impression of the game?

My thinking led me to the realisation that common football statistic like shots on target and % possession are actually Football Metrics 1.0. What is needed to clearly represent the difference between the two sides is Football Metrics 2.0. And for the first of these Football Metrics 2.0, can I propose a calculated metric of Goal Scoring Potential (GSP).

Watching the games, you see so many balls kicked softly directly to the keeper due to the pressure on the player with minimal chance of a goal being scored. But that goes down as a shot on target, a top KPI under Football Metrics 1.0. So many corners are kicked into the box, the vast majority are cleared but each has the potential to be headed in. On the flip side, what about the amazing runs leading to a pass to a player with an open goal, except the pass is just a few centimetres wide resulting in a shot not even being possible, such a good chance but not even a shot at goal registered under those Football Metrics 1.0. But this all changes with Football Metrics 2.0.

My proposal is for each potential opportunity to score being graded on a five point scale between A and E reflecting the probability that a goal would result from that opportunity. Grade A opportunities are where you are around 90% likely to score, down to grade E where you would score a goal in only about 1 in 20 times given a similar situation (obviously not a linear relationship between the grades and I admit I am still working this out). Clearly penalties are grade A while corners are grade E, unless a really good cross in which case they might rise to grade D. Then with a number of points assigned to each grade, it is a simple matter to multiply opportunities by points for that grade, sum up the total end up with the GSP for a team.

Without watching the whole game again myself and doing this properly, I would expect that the GSP for Germany would be at least double that of England, reflecting their superiority and dominance. It would reflect the numerous opportunities (grade C?) that Germany had but didn’t take while England had very few chances for goals that didn’t rely on that 1 in 20 shot going in (and actually did in two cases).

So how is this relevant for web analytics? Well, are the metrics that you are reporting on and trying to improve really a good reflection of the performance of your website and/or marketing? When you have a good week, do you rate more highly on these metrics than when you have a poor week? And if not, are these metrics actually meaningful?  So next time your senior football loving manager comes up to you and says the website needs more pageviews, ask them if they think England (or other sporting team) would always win if they just had 55% possession of the ball.

One last thought though – when the siren blows, only one metric really matters in football and that is the number of goals scored. It is the same for web analytics, define all the KPIs and context metrics you want and analyse the hell out of them – but at the end of the day, are you making money?

Ignoring the methodologies used to calculate this and whether the findings are correct or not, the question I wanted to discuss was – if visitor counts are inflated, does it matter??

First of all, the absolute numbers.  Your web analytics tool says you had 1.5m visitors.  Maybe you only had 0.5m.  To me, this doesn’t matter.  If you are a publisher who is focussed still on the number of eyeballs that view your content for selling to advertisers, then yes, you would like to report the higher number.  But in terms of web analysis, the actual number of visitors to your website doesn’t matter, it is the trend over time that matters and with the level of visitor inflation remaining consistent, this trend should still hold true.

What about frequency of visit, whether the average number of visits per visitor or the proportion of visitor who make 1 visit, 2-3 visits, 4-6 visits or 7+ visits?  Well if visitor counts are inflated then these numbers are very inaccurate.  Let’s look at the data for Feb ’10 for very.co.uk, the new online department store in the UK, from Ad Planner.

Ad Planner claims there were 3.1m unique visitors based on cookies for very.co.uk in Feb, 1.2m actual unique visitors to the website with these people having made 4.6m visits.  First of all, the suggestion here is that the visitor count for very.co.uk was inflated by 2.6 times in Feb (but we are still ignoring whether this is accurate or not).  The interesting thing however is that the average number of visits per visitor could be either 1.48 or 3.83 depending on which visitor count is accurate (assuming either is).  That is a big difference.  Just imagine what the difference is for those proportions too.  And all this is the type of difference that would mean you should have very different business strategies.  Visitor counts being inflated may just matter after all…

I was lucky enough to be at a presentation by Avinash recently where a big topic of discussion was campaign attribution.  One of the points he raised was that if the number of visits to conversion for an ecommerce website is 1 or 2, visitor based attribution is fairly irrelevant.  It is only when the visitor makes multiple visits prior to making the purchase that visitor based campaign attribution becomes relevant.  But if visitor counts are inflated, the reported number of visits to conversion is very likely to be under reported and suddenly the behaviour of your website visitors is quite different to what you may think it is.

So visitor level campaign attribution could be important after all, based on the logic from Avinash, whatever the data for visits to conversion may say.  Well yes but no.  The idea of visitor counts being inflated is due to visitors using multiple devices to access a website and also some level of cookie deletion.  And what it means when it comes to visitor level campaign attribution is that you are only recording a proportion of the visits that led to that conversion.

It would mean that whatever campaign attribution method you may use – last click (can we now say this is generally agreed to be less useful), first click, even weighting, proportional weighting – well they all only count some of the visits leading to the conversion so the data and the conclusions drawn from the data are incorrect.  The conversion for the visitor may be recorded on their 2nd visit, the first being via a generic search term and the second being via an affiliate.  Simplistic example but this would still lead to various combinations of value assigned to the different campaigns depending on the attribution model.

What might be missing (if visitor counts are inflated) are those other visits by that visitor prior to the purchase – with these visits maybe coming via an organic generic search term, a link on twitter and also those two visits from paid brand search terms.  What all this might just possibly mean is that the data that is being used to determine budget allocation for the next year based on the carefully researched campaign attribution method just might not be that useful after all.

All of this is of course just hypothetical.  Various claims have been made that visitor counts are inflated but it doesn’t appear yet that this is universally agreed.  Personally I can imagine that as people use their work computer, home computer and mobile phone to access websites, that reported visitor counts are a little higher than actual fact.  And if they are, the above are a few ways in which the data that is being relied upon to make business decisions may be a little flawed, meaning those decisions that are being made might end up being flawed as well.  And this matters.

Related articles by Zemanta

• How unique is a unique visitor? (vator.tv)
• Your unique visitor figures are 2-4 times too high (blogstorm.co.uk)
• Biometrics firm confirms: User counts for websites are 2-4 times too high (venturebeat.com)

This required me to think about how web pages can be grouped or categorised.  While I am sure there are many ways to categorise web pages, the method I came up with was based on the purpose of the web page.  This means the same method can be used to evaluate the performance of, and calculate a value for, all pages within each category.  I am sure I will evolve this categorisation and evaluation methodology over time but it has given me a nice place to start.

The different categories of web pages that I have identified are:

Information

These are destination pages containing information that may be of interest to visitors. As there is no action to take on these pages, it can be very difficult to evaluate whether the page is performing well or not. Given the objective for the page is for the visitor to read the content, success can be defined as the visitors who spend at least XX minutes on the page or who exit the website from that page (assuming this is because they have found and read the information they were after).

Navigation

These pages contain links through to other internet pages, whether within the site or through to 3^rd party websites. These pages do not contain any direct information and instead exist purely to direct visitors onwards. Success is defined based on the visitor clicking on one of the specific links contained within this page (not the general navigation links).

Transitional

These pages are just one in a series of pages. It may be a set of pages containing information or one stage in a checkout process. Either way, the intention is for the visitor to arrive on this page from the previous step and then to proceed to the next stage in the process. Success is defined by the visitor going through to the next stage in this process.

Transactional

These pages contain one or more actions that it is desired the visitor takes. These could be anything from downloading a file to submitting a form to adding a product to their basket. Success is defined by the visitor taking at least one of the desired actions.

Interactive

These pages allow the visitor to interact with them in some way. It could be to watch a video or to use a calculator. Success is defined by the visitor interacting with the element on the webpage. Note that this just defines whether the web page is good at getting the visitor to interact with the element, not how well the actual element is performing.

Multiple Categories

It is quite possible for a web page to fall into multiple categories. An example of this would be the Travelex UK homepage. This is a navigational page that visitors can interact with and (ideally for Travelex) click on a button to add the currency exchange to their basket. The performance of this page can be measured in different ways to understand how it is performing as a navigation page, as a transactional page and as an interactive page. It is possible for a page to perform well in one category but not so well in a different category.

My advice is to start simple and to talk small steps from there.  Don’t worry initially about actionable insights, they will come with time.  Instead, just get your company used to looking at web metrics.  Create a one page report containing basic metrics and start sending it out to key internal stakeholders.  Include some comments but again, starting simple, these need only be what has happened that week.

Based on experience, once departments and managers start receiving data, they will start asking questions.  Set yourself a target of answering 1 to 3 of these questions each week, focusing initially on those questions that you can answer (including the answers in the weekly report).

Over time, as experience grows, the questions will get more complicated and the answer will become more useful to the business, they will turn into actionable insights.  The report that is sent out will evolve as well, as people understand which metric they need to see and which are only nice to know.  KPIs will be defined and used to measure the success of the business.  The weekly report will contain recommendations on how the business can improve and this information will be used.

Just keep in mind that this won’t all happen overnight so don’t expect it to.  if you try to do it all at once, it likely won’t work.  Take small steps, educate the people around you and you can make a real difference to any organisation that you work for.

For suggestions on which metrics to start with, try reading Your First Metrics – Part 1 and Your First Metrics – Part 2.

An easy way to narrow down the range of options is to focus first on elements that can actually be influenced.  There is no point making recommendations if they are just going to be ignored.  So if you know that the main navigation is not going to be reviewed for the next six months but that management will consider adjusting the buttons on the homepage, look first at the homepage.  If the team in charge of display advertising is dismissive of web analytics data, spend your time helping the ppc team.

There is no purpose in developing a plan or set of recommendations that are just going to sit on the shelf, a web analyst’s time is just too valuable.  Use that time instead to make recommendations that can be implemented and that will lead to improvements for the business.  When the situation changes, maybe due to different people in teams, additional budget allocation for site development or just recognition of the skills and values added by the web analyst, then time can be spent focusing on those elements that initially had to be put to one side.

That is not to say totally ignore anything that you cannot influence, just that there is only so much time in the day and that this time should be used wisely.  All recommendations that are put forward should come with a sound business case and strong logic, but there are times when this is just not going to be good enough.  So when deciding what to work on, don’t choose on the basis on where the business could benefit the most, choose where the business will benefit the most.

This was related to social media through the examples I used, especially with my suggestions on which visitor interactions STA Travel and Bebo might be focusing on in evaluating the performance of a particular social media element within their websites.  While the methodology of how to actually calculate and use engagement is still being argued about in the web analytics world, I suggested a process that can be followed for anyone who wishes to start using engagement in evaluating their own websites.

My presentation at SocialMediaCamp London 2008

After finding BarCamp very interesting last weekend , I managed to score a ticket at the last minute to SocialMediaCamp London which was held yesterday.  The crowd was very different to BarCamp with a wide mix of professions and interests and it was great to know people from such a wide variety of areas are really interested in discussing social media on the internet.

After the debacle of my attempted presentation at BarCamp when 2 people turned up (one was a friend), I decided to learn from my mistakes and lure more people to this presentation.  Accordingly, the title for my presentation ‘Measuring Engagement of Social Media websites in the Web 2.0 world’ was stacked full of buzzwords.  It appeared to work as the room (advertised as holding 20 people) ended up containing around 30, leaving me a little nervous.  Thankfully everything went well and everyone seemed to appreciate the points that I was making.

To download a copy of the presentation, please click here.  Let me know if there are any questions regarding what I was speaking about.  I was very excited to discover that a couple of people were actually writing a post for the SocialMediaCamp blog during my presentation on what I was talking about.  To read more about my presentation (and some of the other talks given yesterday), please check out the SocialMediaCamp London website.

Other presentations

While all of the other presentations that I listened to yesterday were interesting, I got some very useful ideas from two of them.

Julius Solaris gave a presentation on how to use LinkedIn to get a better job and to promote yourself.  I had thought I was using LinkedIn properly already with around 60 contacts but I was wrong.  I will now be accepting any invites to connect as well as joining appropriate groups and trying to promote myself as an authority on web analytics.

Tom Whitwell (assistant editor of online at The Times) talked to us about headlines and how important it is to get them right in order to attract people to reading your content.  The trick is not to be clever or funny but to try and summarise the content (article or blog post) within the heading.  Again, I will be trying to make use of these tips and will stop trying to be clever with my presentations.

These presentations are now both available for download if anyone wants to see what they were missing out on.  As they are presentations, I was going to talk around the written points but let me know if you have any questions regarding the points I was making.

Why should You care about web analytics?

The idea behind my first presentation was to provide reasons to developers (I had expected the majority of attendees at BarCamp to be developers) why they too should care about web analytics and how it could help them in their jobs and thus why they should spend give implementing the code a high priority.  Unfortunately, all the developers cared more about other topics…

But the key point for web analysts, in order to report on something, it needs to be measured.  And it usually falls to others in the organisation to add the tags that capture these measurements.  So if you want to do your job effectively, you need to persuade others that they get benefits out of it too.

To download file, please click here

A practical application of web analytics

The second presentation that I put together was a case study using a website that I recently reviewed for a friend’s website Human Rights TV.  The case study goes through the process I followed in order to analyse this website and the web analytics data captured on it (using Google Analytics).  I included key findings from the investigation and the recommendations that I made for improving the performance of the website as a result of these findings.

While this is a bit of a plug for myself by showing the work I can do, it should be useful in seeing how to go about analysing a website, what to look for in the data and what kind of recommendations can come out of a website review.

To download file, please click here

Clearly I disagree with this line of thought. I dislike the assumption that the web analyst (i.e. me) brings nothing to the party and that their their role has been reduced to simply extracting data from various systems and then mass producing reports and presentations. But more importantly, it leads to the belief that web analytics is easy and that all you have to do is look at a few reports, a few numbers to get to a solution.

I find web analytics very different from this. I do have core reports and core metrics that I will look at for every website to get a feel for what is going on. But when I see numbers that are off, I can’t tell you the answer straight away. Instead it is a trigger to look at more reports, and in multiple date and page configurations in order to understand what is going on be able to provide some recommended actions/solutions. Being able to recognise when a number is off, that is an art. Knowing which reports to look at next in order to understand what is going on and what data/page configurations to try with those reports, another art. I do not believe there is or will ever be a rule book which can tell you how to do this and as such, it is not a science.

So how do you learn this art? Is it possible to learn it? Yes, of course you can. It comes with experience, playing constantly with reports and data and thinking a lot about web analytics and reading a lot of articles about web analytics. Of course reading Occum’s Razor is the most important thing you can do… It helps to have numerical reasoning skills. I don’t think statistic skills are essential beyond the basics such as knowing there is a bell curve and alternative ways of defining an average but am sure that these skills would be useful. Creative thinking is very important too, you need to be able to conceive of potential causes of the problem before you can look for them. After that, a lot of it just comes down to developing a feel for the numbers and a good sense of where the answers may hide.

I have learnt to understand the benefit of buzzwords and jargon, that it is a simple way of conveying a complex idea with a single word or phrase – engagment, web 2.0, social networking. But it only works properly when both parties have the same understanding of the word or phrase. And I very much get the feeling that this is not the case with engagement and indeed with most buzzwords – as much as people like to pretend that they get it.

So what is engagement really – or what do I think it is? Is it just another term that people can throw around to pretend they know what they are talking about or is it something that people should really be thinking about or focusing on. My thoughts on it are below – but I am not going to claim that they are definitely right and I do know for a fact that I am not trendy (I don’t have the glasses).

A lot of people say engagement is vital for a website but why, what is so important about it? How does having people ‘engaged’ with your website make any difference to your bottom line?

To me, having people engaged with your website means you are using numbers and metrics to approximate the number or proportion of people who visited your website and were interested in the content. There is only a small step from being interested in your website to being satisfied with your website. So engagement is really a measure relating to satisfaction.

And why is this important, why do we want satisfied customers? Well satisfied customers are more likely to return. And to tell their friends about websites they found that were useful/good. They have entered the realms of that most valued commodity, the loyal customer. There are plenty of studies that show the profit that can be made from the loyal customer compared to the acquisition costs of luring new customers.

So I guess what I am trying to say here is that when people are saying engagement, they are really talking about satisfaction and loyalty. Which does mean that engagement is not a new buzzword or concept but instead it is something that has been around for as long as businesses have been. The difference between now and then is that in the online world, we can measure it.

Engagement is an attempt to use quantitative measures to understand satisfaction (using data that we already have), as opposed to having to rely on qualitative studies (data that we have to make an effort to acquire). Engagement is important and it should be used as an indicator of the performance of a website. Now the hard part is finding an accurate/useful way of measuring engagement…