L3 Analytics carries the tagline of “Making it simpler to understand and improve your online business performance” and I aim to live up to that objective.  I intend to work with companies of all sizes, focusing on recommending actions which directly add value to their business as well as giving them the tools and knowledge they need to get value from web analytics themselves.  I will also be happy to work with other agencies who are under resourced with their web analytics, either assisting in-house with their web analytics requirements or acting on their behalf with their clients.

I do intend to write on a more regular basis in the future but on the L3 Analytics blog, so please follow me over there to read more about my thoughts on web analytics and the online world.

The approach that I am developing towards evaluating marketing performance can be used across both online and offline marketing.  It is intended to be used to optimise the financial return that is gained from investing in marketing via multiple channels , taking into account the performance of individual channels and their combined impact on business performance.

The future is campaign attribution??

Working with online businesses, the common approach is to use campaign attribution to allocate revenue to the different marketing channels that are being used. (I am going to keep it simple here by only referring to ecommerce websites that have revenue as their goal although the same principles can be applied to non-ecommerce websites using alternative conversion actions).  Whichever campaign attribute technique is used (first touch, last touch, weighted attribution, etc), it provides a revenue figure that each marketing channel can claim credit for earning.  With this data, the business owner or marketer can calculate the ROI against marketing spend by channel and to understand what the impact would be if marketing spend for a particular channel was increased or cut.

There are known issues right now with this approach due to limitations in the various models.  However web analytics and other marketing management tools are developing more sophisticated techniques for attributing revenue to the different touch points.

This is naturally incredibly valuable information as long it reflects reality and I just don’t think it does. It can only be based on known online touch points prior to a purchase being made online and this leaves out so many factors:

• Offline marketing influences are not captured as these cannot be tracked online (except in the use of vanity URLs or similarly identifying features)
• Non marketing influences such as a friend’s recommendation or product reviews are not captured
• Only online touch points on the same computer used to make the purchase are captured, when people could easily have researched the purchase on a different computer
• Purchases made offline are not captured but these could also have been impacted by online marketing and should also be used in any ROI calculations

Beyond all this, marketing campaign attribution relies on being able to convert a person’s buying decision into simple numbers, to calculate the weighting each touch point had on that decision.  I don’t believe this is actually possible as people would not be able to accurately say themselves what motivated them to make a purchase – did you book that holiday 15% due to an email, 25% due to a link on an affiliate marketing site and 60% due to a paid search link?  This is further complicated by the interaction between different information sources and the requirement to develop a model that works with all people (maybe some people are impacted by the first touch point and others by the last touch point).

What are business owners and marketers really trying to achieve?

My philosophy about all this is still in development (and I totally reserve the right to change my mind based on feedback from others) but my current recommendation is to lose the belief that you can accurately or usefully attribute revenue between different marketing channels.  Instead I think you should evaluate the performance of marketing at two levels:

• Overall business performance
• Individual marketing channels based on success actions

The real question that business owners and marketers need the answer to is what to do for the next campaign, not how much revenue did each marketing channel make in the last campaign.  While that information would be great to have in order to plan future campaigns, I don’t believe it is possible to calculate.  Instead the focus should be on gathering useful information that is accurate and that will allow for the optimisation of future marketing campaigns.

Evaluate overall business performance

As a web analyst, I could list off several pages of potential metrics for understanding performance across different marketing sources and several might even be valid KPIs.  But at the end of the day, only one number is going to matter and that is profit.

The measure of success of a marketing campaign is quite simply whether the incremental profit generated was greater than the incremental marketing spend (including salary costs for people working on the campaigns) during the defined time period.  A simple way of looking at things maybe but it is what the CEO is going to do.

This simplistic approach to evaluating the performance of marketing covers both online and offline marketing and also purchases made through websites or in store.  As the public is using multiple channels to research your products and then makes the purchase via whichever method is most convenient (or offers best value), this approach covers all angles and most importantly covers all customer behaviour.

Note that it does rely on a decent forecast being made of what revenue would be if nothing changed, whether this means no marketing at all or no extra investment in additional marketing channels.

This approach may also just be the best way of evaluating how different combinations of marketing channels perform.  While there are numerous statistical and econometrical approaches that can be used, the business bottom line is the one true test.  Different combinations of marketing investment across multiple marketing channels can be compared between regions or different date ranges in order to determine the optimal combination.

Evaluate individual marketing channels based on success actions

Being unable to attribute revenue to marketing channels does not mean that their performance should not be measured or evaluated as successful or not.  But instead of linking performance purely to purchases and thus revenue/profit, each channel should be evaluated based on predefined success actions.  These are specific to each marketing channel based on what you are trying to achieve with the investment in that channel.  This applies to both online and offline marketing channels.

The success actions can be linked to website behaviour, other online behaviour or any offline behaviour.  Examples of potential success actions include:

• Website visits from a particular traffic source
• Website entries on a particular landing page
• A level of engagement with the website (e.g. view at least two product pages) for traffic from a particular traffic source
• In store voucher redemptions
• Use of a particular hashtag on twitter
• Increased brand awareness as measured by a survey
• Increased sales of a particular product, either online or in store
• Increased sales within a particular store

These success actions need to be defined in advance as part of the marketing planning process, both what the actions are and at what level can the performance of that marketing channel be considered a success given the investment in it.  It is this information that is used to review the performance by marketing channel and to decide on future investment in it.

Learn from experience

So if you want to achieve excellence in using multiple marketing channels, my advice is to measure performance, compare against forecasts and use what you have learnt to improve future marketing campaigns.

Do not try and split your revenue between marketing channels, it is just not possible to do accurately.  Instead evaluate individual marketing channels against success actions relevant to what you are trying to achieve with that marketing channel.

Be prepared to experiment with different channels, all the old ones and the new ones.  But look to the bottom line to understand which combinations work best.  Use the data gained from previous marketing campaigns to improve and optimise your future marketing campaigns.

This post is part of the #JUMPchallenge, a blogging competition designed to raise awareness of how to join up online and offline marketing, launched to support Econsultancy’s JUMP event.

Listening today to comments that Germany’s goals were just counter attacks, that England had more shots, more shots on target, more possession and therefore hadn’t been comprehensively outplayed made me think. Having watched the game, (and I freely admit it is not my sport, I don’t have a complete appreciation of the complexities of it and I was a little biased) I had the impression that Germany were running all over England. Not all game, England were very good for the last 10 min of the first half but outside of that… So what about these key footballing statistics, why didn’t they represent my impression of the game?

My thinking led me to the realisation that common football statistic like shots on target and % possession are actually Football Metrics 1.0. What is needed to clearly represent the difference between the two sides is Football Metrics 2.0. And for the first of these Football Metrics 2.0, can I propose a calculated metric of Goal Scoring Potential (GSP).

Watching the games, you see so many balls kicked softly directly to the keeper due to the pressure on the player with minimal chance of a goal being scored. But that goes down as a shot on target, a top KPI under Football Metrics 1.0. So many corners are kicked into the box, the vast majority are cleared but each has the potential to be headed in. On the flip side, what about the amazing runs leading to a pass to a player with an open goal, except the pass is just a few centimetres wide resulting in a shot not even being possible, such a good chance but not even a shot at goal registered under those Football Metrics 1.0. But this all changes with Football Metrics 2.0.

My proposal is for each potential opportunity to score being graded on a five point scale between A and E reflecting the probability that a goal would result from that opportunity. Grade A opportunities are where you are around 90% likely to score, down to grade E where you would score a goal in only about 1 in 20 times given a similar situation (obviously not a linear relationship between the grades and I admit I am still working this out). Clearly penalties are grade A while corners are grade E, unless a really good cross in which case they might rise to grade D. Then with a number of points assigned to each grade, it is a simple matter to multiply opportunities by points for that grade, sum up the total end up with the GSP for a team.

Without watching the whole game again myself and doing this properly, I would expect that the GSP for Germany would be at least double that of England, reflecting their superiority and dominance. It would reflect the numerous opportunities (grade C?) that Germany had but didn’t take while England had very few chances for goals that didn’t rely on that 1 in 20 shot going in (and actually did in two cases).

So how is this relevant for web analytics? Well, are the metrics that you are reporting on and trying to improve really a good reflection of the performance of your website and/or marketing? When you have a good week, do you rate more highly on these metrics than when you have a poor week? And if not, are these metrics actually meaningful?  So next time your senior football loving manager comes up to you and says the website needs more pageviews, ask them if they think England (or other sporting team) would always win if they just had 55% possession of the ball.

One last thought though – when the siren blows, only one metric really matters in football and that is the number of goals scored. It is the same for web analytics, define all the KPIs and context metrics you want and analyse the hell out of them – but at the end of the day, are you making money?

Ignoring the methodologies used to calculate this and whether the findings are correct or not, the question I wanted to discuss was – if visitor counts are inflated, does it matter??

First of all, the absolute numbers.  Your web analytics tool says you had 1.5m visitors.  Maybe you only had 0.5m.  To me, this doesn’t matter.  If you are a publisher who is focussed still on the number of eyeballs that view your content for selling to advertisers, then yes, you would like to report the higher number.  But in terms of web analysis, the actual number of visitors to your website doesn’t matter, it is the trend over time that matters and with the level of visitor inflation remaining consistent, this trend should still hold true.

What about frequency of visit, whether the average number of visits per visitor or the proportion of visitor who make 1 visit, 2-3 visits, 4-6 visits or 7+ visits?  Well if visitor counts are inflated then these numbers are very inaccurate.  Let’s look at the data for Feb ’10 for very.co.uk, the new online department store in the UK, from Ad Planner.

Ad Planner claims there were 3.1m unique visitors based on cookies for very.co.uk in Feb, 1.2m actual unique visitors to the website with these people having made 4.6m visits.  First of all, the suggestion here is that the visitor count for very.co.uk was inflated by 2.6 times in Feb (but we are still ignoring whether this is accurate or not).  The interesting thing however is that the average number of visits per visitor could be either 1.48 or 3.83 depending on which visitor count is accurate (assuming either is).  That is a big difference.  Just imagine what the difference is for those proportions too.  And all this is the type of difference that would mean you should have very different business strategies.  Visitor counts being inflated may just matter after all…

I was lucky enough to be at a presentation by Avinash recently where a big topic of discussion was campaign attribution.  One of the points he raised was that if the number of visits to conversion for an ecommerce website is 1 or 2, visitor based attribution is fairly irrelevant.  It is only when the visitor makes multiple visits prior to making the purchase that visitor based campaign attribution becomes relevant.  But if visitor counts are inflated, the reported number of visits to conversion is very likely to be under reported and suddenly the behaviour of your website visitors is quite different to what you may think it is.

So visitor level campaign attribution could be important after all, based on the logic from Avinash, whatever the data for visits to conversion may say.  Well yes but no.  The idea of visitor counts being inflated is due to visitors using multiple devices to access a website and also some level of cookie deletion.  And what it means when it comes to visitor level campaign attribution is that you are only recording a proportion of the visits that led to that conversion.

It would mean that whatever campaign attribution method you may use – last click (can we now say this is generally agreed to be less useful), first click, even weighting, proportional weighting – well they all only count some of the visits leading to the conversion so the data and the conclusions drawn from the data are incorrect.  The conversion for the visitor may be recorded on their 2nd visit, the first being via a generic search term and the second being via an affiliate.  Simplistic example but this would still lead to various combinations of value assigned to the different campaigns depending on the attribution model.

What might be missing (if visitor counts are inflated) are those other visits by that visitor prior to the purchase – with these visits maybe coming via an organic generic search term, a link on twitter and also those two visits from paid brand search terms.  What all this might just possibly mean is that the data that is being used to determine budget allocation for the next year based on the carefully researched campaign attribution method just might not be that useful after all.

All of this is of course just hypothetical.  Various claims have been made that visitor counts are inflated but it doesn’t appear yet that this is universally agreed.  Personally I can imagine that as people use their work computer, home computer and mobile phone to access websites, that reported visitor counts are a little higher than actual fact.  And if they are, the above are a few ways in which the data that is being relied upon to make business decisions may be a little flawed, meaning those decisions that are being made might end up being flawed as well.  And this matters.

Related articles by Zemanta

• How unique is a unique visitor? (vator.tv)
• Your unique visitor figures are 2-4 times too high (blogstorm.co.uk)
• Biometrics firm confirms: User counts for websites are 2-4 times too high (venturebeat.com)

The key limitation, if I have understood things correctly, is that you currently can’t use segmentation within the API.  Which is fine when you are getting general numbers out but not when you need to create a segmented dashboard across a dozen metrics or so.  It also means you cannot get visits to groups of pages, e.g. visits which saw a product page.  However, there are a couple of workarounds.

For the first issue, you need to return to the old system of creating a profile per data segment.  So, as I have recently done, create a profile for New Visits, another for UK Visitors, one for Paid Search and so on.  Then, with your segments already created, you can easily extract top line numbers from each profile and combine to create that segmented automated report.

The second issue can be resolved through the use of goals.  Two key points to remember are goals can be created based on Head Match or Regular Expressions for page names and that they can only be triggered once per visit.  Given this, the number of goal conversions is suddenly equivalent to the number of visits in which a group of pages was viewed e.g. set up a goal for view a product page and the number of goal conversions is the number of visits in which a product page was viewed.

The API has not yet been updated with the upgrade to 20 goals so you can currently only use the API on the first 4 goals but hopefully that update won’t be far away.  Along of course, with the ability to access segmented data and also to extract the numbers from within funnel visualisations.  In the meantime, I hope these two tips are helpful.

Due to cookie deletion and multiple computer usage, it is difficult to get a true picture of the split between people who have never seen a website before and those who have.  However, recording whether the visitor had a cookie from this website previously does at least give an indication of this new/returning split.

What I like to be able to do is to segment out new visitors for a time period (week or month) and examine their behaviour on the website compared to visitors who had visited previously.  The new visitor segment should include all visits during that time period by these visitors, not just their initial visit.

Frustratingly, this information is usually not available as default in a web analytics tool unless you can segment at visitor level.  However, as long as you have one of the four metrics from New and Returning Visits or Visitors, you can calculate the other three.  And most tools will give at least one number.   As examples:

• Google Analytics gives New Visits and Return Visits
• SiteCatalyst provides Return Visits
• HBX contains Returning Visitors

The key to this is knowing that the first time a site is visited, that is both a new visit and a new visitor.  And as any subsequent visits by these people will be reported as a return visit, the number of new visits equals the number of new visitors.

With that logic in mind, it is simple to calculate all four metrics once you have a single one.  For example, assume that the tool available is SiteCatalyst (without access to visitor level segmentation via Data Warehouse or Discover):

• The number of Return Visits is available but none of the other three metrics
• Total Visits minus Return Visits gives New Visits
• New Visits equals New Visitors
• Total Unique Visitors minus New Visitors gives Return Visitors

And now it is easy to calculate the proportion of Visits that were New or Returning or to calculate the proportion of Visitors that were New or Returning.

The same principle can be applied to Google Analytics:

• New and Returning Visits is available (note that this metric is visits, not visitors as it is titled in the report)
• New Visits equals New Visitors
• Total Unique Visitors minus New Visitors gives Return Visitors

Of course, these numbers don’t mean that much on their own but do become more useful when trended over time or across different segments.

An interesting thing to look at can be the split in New and Returning Visitors for different time periods – day, week and month.  This can indicate the scale of the issue with cookie deletion, but more on that another time.

The last 9 months with Logan Tod has been very interesting.  Learning how to use HBX and Omniture SiteCatalyst and increasing my knowledge of Google Analytics.  Focusing my attention on looking for insights in the data which can be used to recommend improvements to websites or marketing.  Broading my experience with the range of websites that I have worked on and the nature of the work.

I still like to keep it simple though.  I think with the vast amount of web analytics data available for any website, it is easy to get lost in it.  And if you dive in too deep too quickly, it can be difficult to find anything as you are surrounded by too many pieces of data.  I think it is best to start simple, with the key actions or conversion points on the site, the top line metrics.  An understanding of these will generally lead to fairly basic questions about the performance of the website.

But answering these questions will lead to more and increasingly more complex detailed questions about performance and an understanding of the factors that truly are impacting on the performance and success or otherwise of your website and business.  The data itself and an exploration of it can lead you to map out what is important to know in order to make improvements.

One of the more frustrating parts of learning HBX and SiteCatalyst was the discovery of the basic reports that I feel are missing.  In particular, there is not a simple report that gives visits (or even responses) by traffic source.  Having to try and pull data from various sources into a single excel report (as the alternative) is painful with question marks over the accuracy.  SiteCatalyst has the option of Unified Sources via a Vista Rule, this needs to be mandatory for anyone using SiteCatalyst and I wish it came out of the box.  While I have various workaround for HBX, I finally came up with a version using multiple segments where I am satisfied with the accuracy, but no where near the ideal solution.  All tools that I have used need to work harder on getting the balance right between reporting performance and allowing analysis.  I know analysis leads to insights leads to recommendations leads to improvements leads to more money but simple performance reporting highlights where and when the analysis should occur.

Anyway, likely more on that in the future.  I might need to be careful or it could be too easy to write about the philosophy of web analytics ahead of the practicalities of it.  And I need to get a new simple WordPress theme.

The next Monday I started my new life as a Customer Insight Consultant with Logan Tod.  Over the past two months I have been using Omniture (including Discover), HBX, Google Analytics and Sitestat as well as other online data tools.  Whatever the tool, the intention is always the same, to use the data to make better decisions, decisions that will lead to a more successful performance for the company.

I am working across a range of clients performing a range of tasks.  While part of my time is spent running your typical weekly reports, I am also working on some interesting projects, using my knowledge of web analytics to make recommendations for actions to take that will improve the performance of these websites and businesses.

I have been exposed to more e-commerce clients than in the past and have definitely already improved my understanding of what elements can reduce the conversion rate for a website.  The same principles can be applied to non e-commerce websites, looking at the conversion rates for these sites however the conversion is defined.

It is also interesting using a range of tools and getting to know the strengths and weaknesses of each. While all tools are using a similar methodology and reporting on similar metrics, it is surprising just how different they can be.  Early signs is that there is no best tool, rather that they are all good (and all frustrating) for different reasons.

I have also been busy learning how to be the President of my local Australian Rules Football team in London.  Recently this somehow meant I had to learn how to create websites – a week and a half of work later and I had put together the new Putney Magpies website.  Unfortunately it doesn’t appear to work as intended across all browsers and screen resolutions but it is a starting point.

Anyway, they may only be intentions at this point, but they are to write a post on web analytics (not website development) each week – hopefully more interesting and useful now given the experiences I am having.

Related articles by Zemanta

• John Lewis Direct boosts consumer interaction
• Yahoo! Analytics Rolling Out
• Omniture Launches Analytics-Driven Site Search
• Web Analytics Association Releases 2009 Outlook Survey Data (searchenginewatch.com)

Perhaps unsurprisingly, they are facing many of the same issues as those reported by large organisations with e-commerce websites.  The key issues are:

• no clear objectives from stakeholders
• lack of time
• lack of support of internal stakeholders
• insufficient knowledge of how to use web analytics data to make business decisions.

Given this, I aimed my presentation at addressing these issues, hopefully providing practical suggestions for actions that can taken to make web analytics more actionable within their organisations.  This presentation that I gave can be found below.  While it was directed towards universities, I believe the suggestions would be relevant for any organisation.

Most metrics, when viewed at daily level, contain a form of daily seasonality.  This is most clear in metrics such as visits, page views or sales which are absolute numbers.  There is a re-occuring pattern throughout the week with peaks and troughs on the same day/s each week.  An example of this pattern can be seen in Figure 1 below.

While this makes any chart pretty to look at, it makes it difficult to really identify trends or spikes in the data.  Is a data point high because there was a spike or because it was a Monday?  It is school holidays but should the number of visits on that Sat really be that low?  And of course, what day did we start to see traffic decline from and how much of a change is it really?

Figure 1

A common method used to remove daily seasonality is to smooth the line out using a moving average.  As it is a weekly pattern, a seven point moving average should lead to a nice smooth line.  Unfortunately, as can be seen in Figure 2, this means you get a nice smooth line, hiding most of those interesting spikes and step changes and general data trends.  You can see overall trends but you cannot pinpoint particular days when a change occurred.  It is also difficult to clearly identify a change immediately, as each day only contributes one seventh to each data point.

Figure 2

What I advise doing instead is to remove the daily seasonality from each data point, resulting in a line that is unaffected by what day of the week it is.  Using this method means that it is clear to see if the performance each day was good or bad. For example, in Figure 3, it can be seen that the relatively worst day for visits was actually the 25th Aug, even though visits for that day were higher than for other days during the reported period.  The technique for removing daily seasonality can be applied each day, meaning that you can identify and react to a change in performance immediately.

Figure 3

The difficulty then is in calculating the daily seasonality across a week.  This can be done properly using SPSS or a similar tool but I use a quick hack workaround in Excel that, while not 100% accurate, gets the job done.  The steps to calculate daily seasonality for a metric (using the examples of visits) are as follows, with the example displayed in Figure 4:

1. Extract historical daily visits data.  You will need at least 6 weeks, more if the period includes a known number of factors that could impact on traffic e.g. school holidays, public holidays, product releases, marketing campaigns, etc.
2. Reorder the data so that each column contains a single week and each row contains only data for a particular day of the week.
3. Recreate this table so but replace the visits for each day with the % that visits for that day contributed to total visits for that week.
4. Add two more columns to calculate the mean and median for each row of data.
5. Delete all weeks which contain days which don’t reflect the general pattern.  In this example, weeks 5 and 6 were deleted.  At this point, the mean and the median should be relatively similar for each day of the week.
6. The daily seasonality pattern is achieved by multiplying the daily mean by 7.

Figure 4

This daily seasonality pattern can then be used for removing daily seasonality for that metric for any day.  Simply divide the value for each day by the relevant daily seasonality in order to remove it.  I generally do this using a vlookup against the day of the week for each date.

Going back to the reason for web analytics, you can use this technique to clean data so that you can instantly identify good and bad days, whether this is historical data or just for the preceding day.  If you are using this for historical data, you can identify the interesting days to investigate further (play with by segmenting).  If you are using on an on-going basis, you can see instantly what performance was like for the previous day and if need be, investigate and react to a change accordingly.

Currently, in order to be able to do this sort of analysis, you need to extract the data into Excel.  Hopefully one day, web analytics tools will allows you to upload a daily seasonality pattern for a metric so that you can display the daily data with this seasonality removed.  And my dream is of a tool that would incorporate the ability to automatically create the pattern for any selected metric (with manual over rides for tweaking of course).

The other key use that I have found for a daily seasonality pattern is it can be used in forecasting daily traffic levels.  If you are able to forecast what the week’s traffic should be, this can easily be multiplied out using the daily seasonality pattern to forecast traffic at a daily level.

A copy of the Excel file containing all the data, charts and formulae used in the examples above can be downloaded here – Daily Seasonality File.