Welcome on board
The recommendation system provides a convenient way to collect data. It's also easy to deliver recommendations for each user.
For now, only "play", "pause", "page view", "recommendation loaded" and "recommendation displayed" are sent. More information on event data format here.
Events are collected one by one (collect API) or by batch (batch ingest).
Both collect are made via REST endpoints.
The collect API let the user browser send (AJAX call) event that was triggered by the user. Whereas the batch ingest collect several events in each API call (A broadcaster might prefer to collect event by batch for instance send event in a mobile app only when connected through a WIFI).
- REST endpoint are hosted on a Python webserver ( Flask)
- For scaling and fault tolerance reason the Python code push data to RabbitMQ (instead of writing directly in HDFS [ HaDoop File System)). Flume is in charge of taking messages from RabbitMQ queue and store it in HDFS
A recommendation can be queried via a corresponding endpoint. The part of the algorithm which is executed should return a list of recommendations given user id and, optionally, some additional parameters. This part usually uses the results of the task computation stored in Redis (i.e. the model).