Setup algorithms to get recommendations

The purpose of this tutorial is to demonstrate how to setup your own algorithm's notebook in PEACH so you can understand how the data pipeline works and finally get appropriate recommendations.

Prerequisites

To be able to complete this tutorial, you will need an access to PEACH (if you encounter a 403 Forbidden error, you may need to log in or create an account on EBU's platform).

About notebooks

First you need to set up your notebook engine here. You should see a big green button to start your engine, just click on it and wait for the page to reload.

Notebook start

If everything is ok you should see this page :

Notebook page

Now your notebook engine should be running and you can access it clicking on the Open your notebooks button (you will need the password just above for next step to log in so just copy/paste it).

If you reached this point of the tutorial you should now have access to Jupyter, well done! The Jupyter Notebook (click here for more informations) is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and explanatory text.

Go to ./shared folder first. Here are Python notebooks we already use for recommendations algorithms. You can create a new Python notebook, or just use the example one wich is tuto_test.ipynb for the purpose of this tutorial.

This is it for this part. You should now have a running notebook engine and a python notebook that we can reference later on.

Write and setup a task

To start with tasks, you should be aware that a task is very similar to a CRON process which runs your algorithms repeatedly.

The list of available tasks is accessible via the menu below EBU PROCESSING DATAPIPE / Tasks.

Tasks page

You can there set a new task with only a name for the task and selecting the algorithm you want to run as a CRON.

For the purpose of this tutorial we will not use it but they are really similar to algorithms, that we will cover more in depth in the next part if you need more informations.

Write a simple algorithm and setup an experiment

Algorithm

As with tasks, it is best to remember what corresponds to an algorithm here. The purpose of algorithms is to bind functions created in Jupyter notebooks to callable methods in order to provide recommendations to your users.

Here is the link to algorithms page.

Algorithms page

You can find here a simple example named tuto_test which is using the Python notebook we created previously. However you can also create a new algorithm if you added some functions that you want to bind to callable functions.

To add a new algorithm, click the blue button Add an algorithm below the main table. You should now see the creation algorithm form. There is nothing more simple than setting an algorithm :

  • Set a new name for it
  • Set a version number if needed (default is 1)
  • Choose the notebook which contains the function you want to use
  • The Method field below should fill in automatically with the available methods inside the selected notebook. Choose one there that the algorithm should use (you can also select the complete notebook by checking the box below)

Just save and this is it, you just set up an algorithm using a python notebook.

Experiment

Experiments are used to A/B test or prepare any test on your algorithms and provide data to your endpoints.

Experiments are available following this link.

Experiments page

Again you can find a simple tuto_test experiment if you want to have a look on an existing working one. But we will go more in depth on the experiment creation process.

To add a new experiment, click the blue button Add an experiment below the main table. The process of adding a new experiment is as follow :

  • Set a new name for it
  • Set a version number if needed (default is 1)
  • Set Test type (default is none)
  • Set Test Branch Assignment (default is Random)
  • Set Experiment Components. This is the most important part. You have to add new components, knowing that a component is an algorithm which will count for a certain percentage (that you decide by yourself) in the experiment calculation
  • Set a Fallback Algorithm in case we can not reach the experiment (default is empty)

Now save the form and you will create your own experiment. You are now all set to create an endpoint reachable by any client which will return recommendations.

Set an endpoint and assign an experiment

This time we are getting more concrete with the help of API endpoints. Use the endpoints to create REST APIs to retrieve data and recommendations from your systems and websites.

API Endpoints are available following this link.

API endpoint page

You retreive there all the currently available API endpoints that we already use. For the purpose of this tutorial we created the tuto_test endpoint which is using the experiment we created in the part before. You can have a look on how it is setup (to see the result of the test endpoint, you can go to http://peach.ebu.io/api/v1/chrts/tuto_test).

To add a new API endpoint, as usual click the blue button Add a new endpoint below the main table. In the API endpoint creation form :

  • Name your API endpoint
  • Choose an URL for the endpoint you are defining (suffix part)
  • Select the experiment that will be bound to the endpoint

And that's it! To be able to see that data returned by your brand new endpoint you can type the url you previously filled in directly in your browser navigation bar. Or you can also use curl http://peach.ebu.io/api/v1/chrts/tuto_test for example in your command line tool to view the API endpoint returned data. Save your endpoint and then you should be able to see data results at the url you filled earlier on.

You may need to wait some minutes so that the endpoint will be available and running

Setup a fallback algorithm

You may need to setup a fallback algorithm that will be used in case main algorithm will return empty result. For example a trending algorithm is a really good fallback algorithm in case the main recommendation algorithm using user informations does not return anything.

Setup a fallback endpoint consists in 2 simple steps :

  • Create an algorithm based on a python notebook method that will do some basic work to display data anyway
  • Add a new experiment which will be bound to this algorithm

And that is all you have to do. All is already setup to fallback to the endpoint if needed so you do not have much to worry about.

Retrieve and display recommendations

Now that we are all done regarding our algorithms and that the endpoint is set, we can finally get our recommendations! To do so, we already created a complete tutorial that will simply show you how to retrieve data through your endpoint and display them to the final user. Just follow this link and start recommend!