Google Analytics Architecture – A look under the hood to explore APIs

jsnGoogle Analytics0 Comments

Google Analytics is a really powerful and feature rich platform, which provides you with a wide-range of reports straight out of the box. Simply adding the JavaScript snippet with the tracking code on your pages fuels a growing list of reports that you can explore users, their onsite behaviour, acquisition information and more. However, to take full advantage of the platform and start it running “on steroids” we have to take a deeper look under the hood and explore the available APIs & tools.

In order to do that, let’s first take a look at Google Analytics’s Architecture

Diagram courtesy of Google

As you can see the platform has 4 main components:
Collection where the different methods for user-interaction data collection live
Configuration that allows you to manage how the data in the platform is processed
Processing where the user-interaction data is processed with the configuration data.
Reporting where you can access all of the processed data.

In the most basic example, you would have the analytics.js tracker on your pages, which would pump collected hits to the Logs. On the Configuration side of things, you would structure the data in Properties and Views, apply a couple of filters, Custom Channel Groupings etc and configure the access rights for the users via the Web Interface. That would form the Configuration Data that would influence the Processing that takes place. After that, you will be able to access the reports with populated data via the Web Interface again, which uses the Query Engine behind the scenes.

Quite simple, yet powerful. However, what if you need to collect data not only from the web, but from an offline environment like a CRM or an instore till? On top of that, you need to maintain a list of a dozen Accounts with a dozen Properties which have hundreds of views where multiple users from the organisation have a different level of access? And if that is not enough, you then need to export the raw data outside of analytics to either do some more advanced analysis and segmentation or link the data to an external system. In that case, you need something a bit more scalable where you are programmatically interacting with the platform.

In the next few lines, I will highlight a protocol and 2 APIs that are often used to tackle problems like the mentioned above, providing an overview, example use and further reading.

The Measurement Protocol

The Measurement Protocol is the mechanism that is used to construct the hits generated by the different libraries and SDKs. Once the hits are constructed with the predefined parameters, an HTTP request is made to send raw user data directly to the Google Analytics Servers. Aside from that, you can also use the Measurement Protocol to construct hits yourself and send them from any environment, considering the guidelines and documentation. That allows you to enrich your data considerably and combine offline hits with online hits for example.

An example use cases are:
1. Send data from devices that are connected to the internet like in-store tills, kiosks, etc.
2. Tie online to offline behavior by sending additional data from a CRM for example
3. Configure server-side tracking, instead of client-side.
4. Email tracking

Resources:
Protocol Reference
Parameters Reference
Developer Guide and Documentation
Tool – Hit Builder
GitHub – Send data to GA from the server with PHP (Really usefull for server-side transaction tracking on eCommerce platforms built on PHP)

The Management API
This API is all about making boring administrative tasks exciting by automation. Using the API, you can programmatically access the Configuration Data to manage large accounts, create/update/delete users setting various access levels, import external data and more and more. Its all about scalability and making things easier. Depending on the type of application you are building, there are plenty of quick-start resources for languages such as PHP, Java, Python and JavaScript.

There are some limitations of course and you have to consider that you are allowed no more than 50,000 requests per project per day as well as no more than 10 queries per second (QPS) per IP address. However, there is a batch processing feautre which can help reducing those requests.

An example uses cases are:
1. Manage users and their permissions for an Account (add/delete/update)
2. Build configurations for data import on a scheduled basis
3. Manage links between platforms

Resources:
Google Analytics Account Hierarchy
API Overview and Documentation
Limits & Quotas
Batch Processing

The Core Reporting API
The Core Reporting API on the other side, allows you to export the already collected, filtered and structured data. Using the API you have access to the powerful reporting infrastructure built by Google that servers Google Analytics. You get access to all of the available data and you can export dimensions and metrics, have calculations based on those made for you on the fly, apply segmentation on the fly and more. However, all that power has its limitations and you have to be mindful as sampling can be applied which follows the general Google Analytics thresholds for free and paid accounts.

The level of limitations listed above in the Management API section also apply here. Similarly batching is also an option.

An example use cases are:
1. Export data to Google Sheets for further analysis. There is a really powerful Add-on developed by Phil Walton for this, which is linked down in the resources
2. Export data to an external platform to mix the data from Google Analytics
3. Automate frequent reports

Resources:
API Overview and Documentation
API Reference Guide
Dimensions and Metrics Explorer
Sampling and Sampling thresholds
Batch Processing
Google Spreadsheet Add-on

Conclustion and additional resources
I hope the overview of Google Analytics as a platform, its inner workings, and the discussed features provided a good quick start for anyone looking to push the platform to its limits. Additionally, here are some additional resources and tools that I found useful.

Google Analytics developer tools
API Explorer

Share this Post

Leave a Reply

Your email address will not be published. Required fields are marked *