Most metrics, when viewed at daily level, contain a form of daily seasonality.  This is most clear in metrics such as visits, page views or sales which are absolute numbers.  There is a re-occuring pattern throughout the week with peaks and troughs on the same day/s each week.  An example of this pattern can be seen in Figure 1 below.

While this makes any chart pretty to look at, it makes it difficult to really identify trends or spikes in the data.  Is a data point high because there was a spike or because it was a Monday?  It is school holidays but should the number of visits on that Sat really be that low?  And of course, what day did we start to see traffic decline from and how much of a change is it really?

Figure 1

A common method used to remove daily seasonality is to smooth the line out using a moving average.  As it is a weekly pattern, a seven point moving average should lead to a nice smooth line.  Unfortunately, as can be seen in Figure 2, this means you get a nice smooth line, hiding most of those interesting spikes and step changes and general data trends.  You can see overall trends but you cannot pinpoint particular days when a change occurred.  It is also difficult to clearly identify a change immediately, as each day only contributes one seventh to each data point.

Figure 2

What I advise doing instead is to remove the daily seasonality from each data point, resulting in a line that is unaffected by what day of the week it is.  Using this method means that it is clear to see if the performance each day was good or bad. For example, in Figure 3, it can be seen that the relatively worst day for visits was actually the 25th Aug, even though visits for that day were higher than for other days during the reported period.  The technique for removing daily seasonality can be applied each day, meaning that you can identify and react to a change in performance immediately.

Figure 3

The difficulty then is in calculating the daily seasonality across a week.  This can be done properly using SPSS or a similar tool but I use a quick hack workaround in Excel that, while not 100% accurate, gets the job done.  The steps to calculate daily seasonality for a metric (using the examples of visits) are as follows, with the example displayed in Figure 4:

1. Extract historical daily visits data.  You will need at least 6 weeks, more if the period includes a known number of factors that could impact on traffic e.g. school holidays, public holidays, product releases, marketing campaigns, etc.
2. Reorder the data so that each column contains a single week and each row contains only data for a particular day of the week.
3. Recreate this table so but replace the visits for each day with the % that visits for that day contributed to total visits for that week.
4. Add two more columns to calculate the mean and median for each row of data.
5. Delete all weeks which contain days which don’t reflect the general pattern.  In this example, weeks 5 and 6 were deleted.  At this point, the mean and the median should be relatively similar for each day of the week.
6. The daily seasonality pattern is achieved by multiplying the daily mean by 7.

Figure 4

This daily seasonality pattern can then be used for removing daily seasonality for that metric for any day.  Simply divide the value for each day by the relevant daily seasonality in order to remove it.  I generally do this using a vlookup against the day of the week for each date.

Going back to the reason for web analytics, you can use this technique to clean data so that you can instantly identify good and bad days, whether this is historical data or just for the preceding day.  If you are using this for historical data, you can identify the interesting days to investigate further (play with by segmenting).  If you are using on an on-going basis, you can see instantly what performance was like for the previous day and if need be, investigate and react to a change accordingly.

Currently, in order to be able to do this sort of analysis, you need to extract the data into Excel.  Hopefully one day, web analytics tools will allows you to upload a daily seasonality pattern for a metric so that you can display the daily data with this seasonality removed.  And my dream is of a tool that would incorporate the ability to automatically create the pattern for any selected metric (with manual over rides for tweaking of course).

The other key use that I have found for a daily seasonality pattern is it can be used in forecasting daily traffic levels.  If you are able to forecast what the week’s traffic should be, this can easily be multiplied out using the daily seasonality pattern to forecast traffic at a daily level.

A copy of the Excel file containing all the data, charts and formulae used in the examples above can be downloaded here – Daily Seasonality File.

There are a lot of different factors that impact on traffic levels for a website.  Common big factors are public holidays and school holidays with traffic always affected on these days.  Typically there would be a significant decline and, while it is possible that these days were actually positive for certain websites, there is a clear correlation between these days and traffic levels.  What this means is that it is not traffic for that website that is being impacted but the number of people using the internet (and their behaviour) is different for these days of the year.

A factor that leads to a similar change in user behaviour but that is harder to understand, difficult to explain and impossible to predict is the weather.  I have had numerous discussion trying to explain to management that the reason that traffic was down for a day or weekend was simply because the weather was nice.  People sometimes want a complicated explanation when the real answer can be quite simple.  My theory behind this change in behaviour is that on warm sunny days, more people are going to be outside enjoying the day and doing so for a longer period.  A lengthy and expensive piece of market research should prove this theory – or you try could looking outside on a sunny day.

There are two really tricky aspects to defining the exact impact from the weather, these being the varying weather conditions across a country (let alone the world) and an inconsistent level of change to traffic from similar changes in the weather.

For the first, it is not a significant issue in the UK or similar sized countries as the weather can be judged to be relatively consistent across the whole country.  Yes there is a large variation across the UK but simply taking London temperature will give an idea of hot, warm, mild or cold for the country.  For larger countries like the US or Australia it is more difficult as a no single population centre cannot be used to represent the whole country.  In these situations, I would recommend taking a weighted average across 3 to 5 locations to get a feel for overall weather conditions for that day (or segment the traffic for different regional areas).

The second inconsistency relates to an inability to create a formula that relates temperate and traffic.  Similar weather conditions can result in different levels of traffic, all other factors being equal.  This can be explained using logical thinking to explain the variations in website performance and how it is impacted by the weather.  What I have seen is that the decline from a hot weekend is always higher for the first hot weekend of spring or a late sunny weekend (as it was in London this weekend).  The simple explanation for this is that more people are going to spend time outdoors and away from their computers when it is the first hot weekend after the winter or potentially the last hot weekend of the year.

If you are doing weekly or monthly performance reporting, keep in mind the option of including the temperature when reporting on the warmer months.  The best site I found for historical weather data is the Weather Underground website (given current media reporting, the name is a little unfortunate) – an example here is historical data for Sept for London.  The only issue is that temperatures are in Fahrenheit but this can easily be adjusted in Excel.

• the number of people, use unique visitors
• traffic levels, use visits
• quantity of content consumed, use events

Caveat: there are certain times when you want to group people on a certain criteria.  Examples of this include wanting to know the number of people who accessed a certain site section or who arrived via a particular traffic source.  However people can view multiple site section and access a site via multiple traffic sources.  Therefore be aware that the sum of people via traffic sources, site sections and other similar situations will be more than the actual number of unique visitors to a website. 

If you want to evenly attribute a total between different options, you might need to choose a lower level metric.  The important thing in this case is to choose the metric which is independent for what you are looking at.  For example, a unique visitor may have multiple entry types but each visit only has a single entry type.  Unique visitors and visits can view multiple site sections but events are each from only a single site section.

Using the example of page views per visit, while the average for a site might be 3.50, in actual fact some visits would have involved a single page view while others would have involved 7, 8, 9 page views or even more. A frequency table can be developed showing the number of visits that contained each number of page views. While useful, this doesn’t give an easy overview of user behaviour that can be trended over time, which is the stage we are trying to get to. So we need to simplify the frequency table.

The way to do this is to create three or more groups within the frequency table with the simpliest option being of course just three. These three groups are commonly described as light, medium and heavy. The groupings can be defined any which way appears best. This is not a situation where one rule fits all, the definition needs to be customised to the situation.

A nice way of doing it is to split the groups fairly evenly. Alternatively, just use what appears to be the most logical manner of splitting the groups based on desired user behaviour. For page views per visit, you could define light as visits containing 1 page view, medium as 2 to 4 page views and heavy as 5 + page views.

This gets you the number of light visits, medium visits and heavy visits – an easy overview of user behaviour that can be trended over time. However, it can be taken to another level of usefulness simply by converting the absolute numbers into proportions. This means you can see very easily an increasing trend in the proportion of heavy visits over time – a metric that could be used as a KPI and suddenly we are in a very useful place.

Metrics that I have referred to previously that can easily be treated in this manner are frequency (visits per unique visitor), page views per visit and duration per visit. However the principles can be applied to any situation where each case of one metric can be linked to a variable number of instances of another metric e.g. each unique visitor can have a variable number of visits during each time period.

For the sake of this example, let’s assume that it is the number of visits to your site which has declined over the previous week. And knowing you would have quickly scanned your other key metrics for more information, let’s assume that they don’t provide helpful information (unique visitors and pageviews also dropped 10%, no change in the proportion of traffic from different traffic sources).

Given this, a good place to look for an explanation of a change in weekly data is at your daily data. This may quite quickly show the exact day that values changed (whether this is up or down) providing insight into the cause of the change.

I am an analyst with Excel consistly open. Whenever I go to manipulate data, I transfer it from whatever data source it can be found in (Sitestat these days of course) into Excel where I can easily play with it. I have created some basic examples in Excel using simple numbers to highlight alternative situations.

Your Basic Data

The first step is to run your visits data at daily level so that they can be examined. By themselves, the numbers don’t mean much, they need to have some context added to them. I have added additional data showing the week on week change for each day during the last week as well as charting the two weeks data. If nothing had changed, you would get data and a chart as per below, with this sample data following a fairly standard internet day of the week pattern.

Change 1 – Trending Down (or Up)

But for these examples, visits had dropped, down by 10% against the previous week. You need to know why visits have dropped from 877,000 two weeks ago to 789,300 last week, a decline of 87,700 visits or (in useful terms) a decline of 10.0%. In this first example, there has been a consistent decline during the course of the week. Every day was down 10% against the equivalent day in the previous week. In reality of course, there will be day to day variations but you can always see a fairly similar level of change during the week.

Change 2 – The Step Change

A trend like that above won’t often be seen for a big change, say > 5%. It is more common when you are looking at a 1% to 3% change. While these are small numbers, over a number of weeks they do add it. I have seen that trend most often leading into Summer or Winter as days get longer/shorter and hotter/cooler, with these conditions impacting on overall internet usage.

Instead what you will see is a sudden step change where a metric suddenly changes to a new level. If you are looking at absolute numbers, there will be a six to eight consecutive days where each day has a similar week on week change until the numbers settle down at their new level. If you are looking at a metric that is a ratio or percentage with a less obvious day of week pattern, the change can be seen even more clearly, for example the frequency may change from being constantly around 2.50 to being constantly around 2.85.

Note: I will show in a future post a simple way to remove the day of week seasonality from metrics like visits so that trends can be more easily seen.

In the example on the left, there was a clear step change on the Thurs (day 11) with visits dropping by 20% every day from there. There is commonly only a partial change on the first day of a step change with the factor that caused the change commencing from part way through a day (change to website or online marketing).

The beauty of a step change is that it can allow you to pinpoint the exact day (and sometimes the approximate time of day) that something happened that is impacting the performance of your website. Through this, it is easier to identify what was the cause of the change. Additionally, if a business case is required, it is not difficult to calculate the impact of this change.

It should be kept in mind though that if a permanant step change occurs part way through a week, the weekly numbers will decline (or improve) further the next week when they have a complete week at this new level. I used the term permanant step change as the most common step change is temporary, occuring every school holidays.

Change 3 – The Spike

Another common cause of a weekly change in a metric is a sudden one to three day spike in the metric, either up or down. In the example on the right, there was a one day negative spike in visits on the Thursday which resulted in total visits being down 10% week on week.

In a way though, despite their impact, spikes are less important. As be seen, every other day of the week had exactly the same visits as the previous week. There is no reason to believe that the big drop in visits will be repeated. The cause of this drop in visits should be investigated but as it has already corrected itself, there is no reason to believe it will happen again.

Potential causes of spikes are servers crashes where the website either goes down or everything is fine but data wasn’t being captured, spam producing fake traffic, online marketing dropping off possibly due to budget temporarily running out, public holidays, etc.

Summary

A vital part of understanding what could have caused a change in a metric for a week (or a month) is looking at the data on a daily basis. By comparing each day against the same day in the previous week (or if need be, an alternative baseline week) and by looking at the data visually, it is possible to get a feel for when the change occured. This aids in identifying the cause of the change as well as understanding whether it is something to be concerned for the future or not.

My first belief is that a time period of longer than a month should just not be used. I am reliant on trending in order to understand performance and for periods of greater than a month, there are too many factors affecting the numbers for them to be that meaningful. Instead, when required to produce a performance review over a longer time span, data should be trended at monthly level and, if need be, average monthly numbers can be used.

This just leaves daily, weekly and monthly time periods. All are useful but they do provide different levels of information.

Monthly Data

Web analytics cannot operate in a silo and since most companies have all their financial reporting produced on a monthly basis, web performance reporting should be available in the same time period. Beyond this, the strength of monthly data is that it provides big numbers over a lengthy period of time meaning good or poor performance is more visible/obvious.

However the problem with months is that they contain different numbers of days and different days of the week, with both of these being basic factors that impact on reported performance during a time period. The simple solution is knowing and understanding these impacts and being able to explain how they affect the numbers to your managers.

Weekly Data

This should be the basic time period used by any web analyst. Their first task on a Monday morning (or whatever the first day is for a new week for their company) should be to click a button to update their weekly reports and to cast their eye over the performance for the week to see if anything looks interesting. If any numbers look high or low, they should then be spending a couple of hours (or whoever long it takes) running ad hoc reports in order to understand why.

I feel that looking at a fairly complete set of metrics on a weekly basis strikes a nice balance between looking at data too often and not often enough. If the business is only looking at their data in detail once a month, they are simply not able to react fast enough in case of something changing. However, there is a danger in overanalysing data and being too sensitive to random fluctuations. There is no point in an analyst spending a large chunk of every day examining the web traffic in detail for the previous day if there were no changes to the website or marketing.

Daily Data

So having said that, why do I advise daily data as being useful? While I believe in a set of reports at weekly and monthly level, at daily level I would have only one or two metrics, potentially segmented, that are looked at. The obvious one is total interactions (usually page views) for the day, compared against a forecast total or, if that is not available, against the same day in the previous week. If the percentage change is within a certain range, move on with your work for the day. But if something has happened on that day, as should be immediately obvious just by looking at this one number, this is a clear alert that the data for that day needs to be interrogated further.

So what does everyone else compare against?

For weekly or monthly data, the commonly used comparison periods are the previous period and the same period in the previous year. By comparing against the previous period, you can quickly see if things got better or worse or even just stayed mostly the same.

I am less sure what a comparison against the same period in the previous year is meant to show. This may have been good in traditional industries but the internet changes too fast for this to be useful in my mind. Your website or traffic mix will change and suddenly you are constantly around 20% off last year every single week, making that comparison just additional numbers on the report.

So what should I compare against then?

The point of a comparison is to aid in understanding if the performance for that time period was good or bad. Therefore, logically, the best comparison to be made is against the expected or forecast performance for that period.

This forecast should have been created using the data from recent periods and using the data from last year, taking into account known or expected website and traffic mix changes. Once you start setting goals through forecasts and targets, you start becoming accountable.

Now this is slightly scary, it is so much easier to say ‘hey, we are up 20% on last year, we must be going well’ (ignoring how you were 40% up two months ago). But when you are admit that ‘hey, we are down 1.5% against where I thought we would be’, you can start to investigate the reasons why and see what is and isn’t working.

Anything else?

While I don’t like comparing against the same period last year, the trend or % change for the same period last year can be useful. This can show you if a change this year is merely a seasonal effect or something that should be investigated further.

Targets are slightly different from forecasts and not as useful for short term comparisons. They are where you want to be longer term, something to keep an eye on but not a call for action if not at that level immediately. And like everything else, they should be one of the factors that are used in developing the forecasts.

Summary

Give your data company. In order to provide it with a meaningful existence, it should be compared against other numbers. The most useful comparison is against a forecast as this is the true measure of whether performance was above or below expectations. And the forecast should draw on the knowledge of all other numbers that could be compared against.

Note: I will be writing some posts about forecasting in the future. If you need a forecast to understand performance, you need to know how to forecast (ideally with some degree of accuracy).