Skip to content

Data collection with pipe.js

This document is superseded

With the release of peach-collector.js we strongly recommend to use the new library as described in this document.

Although the use of the old pipe.js collection script is deprecated, this document is here in support of those still using it.

The purpose of this tutorial is to show you how to add pipe.js, and use it in order to collect data depending on client informations.

How this works

The snippet we identify as loader is the small JavaScript below, which you will need to add to all your pages to start collecting data. The loader is designed to buffer the collected data without slowing down the browser when loading the page. First, the loader defines the _pipe function as a simple queue and stores the calls into a JavaScript array. Thus, it allows you to start pushing collect data even if the pipe library has not finished to load. Then, once loaded, the _pipe function is replaced by a callable object which will directly send the data to the collection APIs using the preferred method of the browser.

Also in order to track users and collect consistent data, two cookies are set by the collect server for a user. One of them never expires (cookie_id), thus it stays inside the browser and allows to uniquely identify the browser over time. The second (session_id) expires when closing the browser. Therefore, it can be used to identify uniquely the session of the user.

Most European countries require website providers to inform their visitors about Cookie usage, any kind of tracking or data collection on their website. Make sure you comply to legal regulations in place in your country before using the Pipe library on your webpages.

Prerequisites

To be able to complete this tutorial, you will need a site key provided by the EBU that matches your account informations. That site key can be created following this link (if you encounter a 403 Forbidden error, you may need to log in or create an account on EBU's platform). For the meaning of this tutorial, you can use zzebu00000000017 as site key.

Production site key

To be able to create the site key for production you will need access to https://ebu.io/peach. If you need an account, feel free to contact us so we can manage it with you.

How to integrate pipe.js lib

To start collecting data, include the loader in the header of the pages:

<script>

!function(){window.EBUPipeQName="_pipe","_pipe"in window||(window._pipe=function(){
window._pipe.q.push(arguments)},window._pipe.q=[]),window._pipe.startTime=(new Date).getTime();
var e=document.createElement("script");e.src="https://peach-static.ebu.io/pipe-5.x.min.js",
e.async=!0;var i=document.getElementsByTagName("script")[0];i.parentNode.insertBefore(e,i)}();

_pipe('enableTracking');
_pipe('register', 'zzebu00000000017');
_pipe('collect', 'pageview');

</script>

And you are all done! (pretty much).

In more details, some points regarding the loader you should be aware of before starting :

  • This is the setup if you want to use the version of pipe.js matching the new recommended data format.
  • You may also link a given version, for example : pipe-5.0.1.min.js
  • If you want to use the previous data model, use pipe-latest.min.js instead of pipe-5.x.min.js.
  • The method _pipe('enableTracking'); tells pipe that you've asked your user for consent and pipe is allowed to collect data.
  • The method _pipe('register', 'zzebu00000000017'); registers pipe regarding your domain name and enables it. You will not be able to collect anything until the register method will be completed. Although, all collect methods call will be queued, and processed at the moment pipe will be enabled and registered. Then data will be sent.
  • _pipe('collect', 'pageview'); is a basic collect call. The former parameter will be considered as the event, the latter as action. The third argument is either a string describing the content's id or an object with additional variables to be collected, which is the most frequent case.
  • For more information on how to start with pipe JS (integration debugging etc) you can read the Pipe JS Getting started

Info

You can use our hosted library but define your own collection servers or your local server for testing purposes. You need to set it before registration.

Two ways of doing it: _pipe('setEndpoint', 'localhost:8080'); or _pipe('setEndpoint', ['localhost:8080','localhost:8081']);

You need to use a local server (MAMP, Intellij built in server etc.) to run the example. Else you may experience some issues with links to external files.

Send data to collect APIs

Now that our pipe.js is registered and ready to use, let's try to add a simple button to start collecting data.

First create a simple button element

<input id="button" type="button" value="Test" onclick="testSend()"/>

Now simply create the function that will send some data to your newly registered pipe.js

function testSend() {
  _pipe('collect', 'test_click');
}

If you open your browser console and check the network tab, you should be able to see a POST request sent to http://pipe-collect.ebu.io/v3/collect?s=zzebu00000000017' (POST "http://pipe-collect.ebu.io/v1/collect?s=zzebu00000000017&e=test_click" for previous version) with success and the request payload with all collected data.

In case you have a 405 HTTP error on OPTIONS request, it is most likely because the site key is not registered correctly. Check here that the site key appears and is enabled.

Collect data from a player

For the purpose of this tutorial we will demonstrate how to get data from basic html5 player, and send them to pipe collect API. Of course, you will most likely use your own player with other constraints and needs, but this is really just to have a quick and working overview of the process.

First thing to do, create a basic html document and add your video element

<video id="simple-player" data-media-id="" data-data-source="" controls="true" playsinline>
    <source id="player-source" src="">
</video>

Then, make sure you properly initialized the library as explained in the section above.

Now, let's start capturing events from the player. We got a video element in the DOM and pipe.js registered. So we will simply add the event listeners for the video we want to collect informations from.

var player = document.getElementById("simple-player");

player.addEventListener("play", function (evt) {
    playing(evt);
});

function playing(evt) {
    console.log("play", evt);

    var currentPlayer = evt.target;

    // gather data related to the event, and send it to collect endpoint
    var data = {
        id: currentPlayer.dataset.mediaId, // get attribute data-media-id
        "props" : {
                "playback_position_s": currentPlayer.currentTime,    
                "page_start_timestamp": localTimeUTC
        },

        "context" : {    
            "id": currentPlayer.dataset.dataSource,
        }
    };
    _pipe("collect", "media_play", data);
}

/**
* Loads a media source into the player, and keep track ot the mediaId
* Parameter dataSource allows to specify the recommendationId
*/
function loadMedia(mediaId, mediaSource, dataSource){
  getPlayer().dataset.mediaId=mediaId; // set data-media-id
  document.getElementById("player-source").src=mediaSource;
  getPlayer().dataset.dataSource=dataSource; 
  getPlayer().load();
}

The context object provides information about the event. We use it to track the source of the video. When you want to track performance of algorithms, it is very important to understand when videos are watched from a recommendation list, and when not. Specific events are also needed (see Data Format) to track display of recommendation list, or when the users selects an item. You'll learn more in the [recommendation section].(./tuto-get-recommendation.md)

As you can see, it is nothing more than basic javascript. Whenever a client will play the video, the corresponding event will be fired and we will then get the right, formatted data.

Info

You can see this in action in the demo page.

Data Collected

By using pipe.js, you get more data about the client automatically. This is a sample of what we receive with the code above :

{
        "site_key":"zzebu00000000017",
        "session_start_timestamp":1521530761208,
        "user_id":"userId123456789",
        "collect_timestamp":1521533846267,
        "peach_schema_version":"1.0.0",
        "client":{
            "name":"Safari",
            "type":"web",
            "version":"11.0.3",
            "device":{
                "timezone":-1,
                "screen_size":"2560x1440",
                "type":"desktop",
                "language":"en-US"
            },
            "os":{
                "version":"10.12.6",
                "name":"Mac OS X"
            },
            "id":"a2286d55-a02e-b0a3-56cf-e6e35c8f3a80"
        },
        "sent_timestamp":1521533846012,
        "event":{
            "event_timestamp":1521533846012,
            "context":{
                "id":"recommendation123"
            },
            "props":{
                "page_start_timestamp":1521533838769,
                "playback_position_s":0
            },
            "type":"media_play",
            "id":"42"
        }
    }

Additional Documentation

Some documents to know more about pipe :

  • pipe.js documentation: More details on available informations, options, and methods using pipe.js.
  • Recommended data format (new).
  • Since version 3.x device type is automatically inferred. A test page shows you how your current device is classified.
  • Documentation for previous data collection can be found here. Follow it only if your integration was done with pipe.js below 3.0.0 and you do not want to migrate to current version.

Final notes

If you followed this tutorial entirely you should now be able to collect with pipe.js, following our best practices. You noticed anything missing, wrong or not working? Do not hesitate to contact us and we will be pleased to answer your request.

The code for this tutorial is available in the repository. Also you can access a slightly enhanced version of the page directly here (for pipe.js before 3.0, here).