The Clickstream
Clickstream data is the latest innovation in keyword research. It allows us access to far better and more reliable keyword data, as we can see the entire path that a user has taken rather than just an individual click on any given term.
Wondering what the Clickstream is? Read our article What is Clickstream data?
Up until now the only way to get search data from Google was via the Adwords API or through services which scrape the Google Keyword Planner directly.
The Adwords API involves having to build an application that meets Google's requirements and usually means A LOT of development time and resources! Adwords API data (direct or scraped) also comes with its own issues, outlined by our article on why the Planner isn't fit for purpose, Replacing the Google Keyword Planner. These issues have only been getting worse over time and we are seeing increasingly odd results in the Google Planner...
So if we can't trust the Planner what do we do?
Clickstream data is collected (with permission) from a range of different applications that people install on their computers and browsers (usually the free ones). It is then anonymised and resold to third parties.
We are talking terrabytes upon terrabytes of data, which means big data crunching algorithms and databases need to be used to extract the juice.
There are few keyword tools which currently use raw search data pulled from the Clickstream, mainly due to its expense, and the technical limitations mean that there just aren't any API's that give you access to this kind of data.
This makes life difficult and means having to rely either on second hand scraped Google data or the keywords Google chooses to give out via the Adwords API. Even when that data is demonstably faulty.
The benefits of Clickstream data
Clickstream data gives us access to Google searches without going through Google. It means we don't have to rely on believing just what Google allows users to see. Instead we have a completely separate data source which is based on panels of actual users.
Right now we have updated our search data with Clickstream keyword searches, meaning we have keywords taken directly from the Clickstream.
Being Clickstream data there is a lot more information within it than just keywords and searches. It also gives us the ability to filter out spam more effectively so we can have a high level of confidence in this third-party data.
We’re currently working on bringing you even more kinds of data from within the Clickstream, such as identifying which keywords are search repetition (how often a keyword is searched for by the same user). All of this takes a huge amount of processing power and time, so as we solve the technical hurdles you'll be seeing even more data in the tool.
You can see the Clickstream data right now in the Wordtracker Keyword tool under the Google (Raw) data source, and it's also available via the Wordtracker API 2.0.
Yes that’s right, there is now Clickstream data available via our new API. So finally everyone has a cost effective way of tapping into this data and getting round the restrictions of relying directly on Google data.
Even better Amazon, YouTube and eBay data
We’re working hard on developing the best techniques possible for creating the most accurate data sets. So we're not only bringing you the data sources we can, but also presenting them as close to the truth as possible. A big part of that is in how we scale the data. Something every keyword tool has to do, is scale their sample data up to be representative.
This is because even though we have high quality keyword data, it’s only ever going to capture a percentage of searches. Those searches are then scaled up to be representative of the total number. In other words, as no one has a record every search ever done, you have to make sure your data properly represents the proportion of searches you have captured and that it is properly scaled.
We have our own proprietary scaling technology which we have built. Other tools that deal in raw data will have their own. We have the advantage of course of always having maintained our own Wordtracker keyword database. This means we have a massive headstart in terms of both technology and in-house knowledge about how to handle these sorts of datasets. Including removing spam and making data representative.
We have improved the scaling across our data sets and you’ll see the new data scaling coming into the data shortly. This does mean that the continuity between the new, scaled data and older data will be disrupted, so for those people tracking product popularity with our data you may see some jumping around between the last and this month as the new data scaling is implemented.
Wondering what else we’ve been working on?
We're not just stopping at market leading data. We've got some great new changes coming to list management within the tool, a brand new reports builder and a really exciting new and completely free, product that we'll be letting everyone have access to soon :) So watch out for further details as we'll be releasing lots over the next week.
In the meantime if you havn't yet got access to Wordtracker don't forget to try out the Wordtracker Keyword tool with our 7-day free trial and get started with that Clickstream data for yourself.
