How to define recommendation API endpoints
Recommendation endpoint executes corresponding algorithms and serves the result to client applications (web and mobile). At a glance recommendation endpoint is a way to bound particular url to one or many algorithms which produce recommendation according received parameters.
The recommendation endpoints can be also seen as simple analogy to AWS Lambda functions. Basically it is required to implement corresponding function (handler) in the notebook, define endpoint configuration and push it to the GitLab repository. Afterwards Endpoint Scheduler will make updated version available on Recommendation API (the process usually takes around a couple of minutes).
Major features include:
- Custom arguments by URL query params
- A/B testing based on user, client or session
- Fallback algorithm to be used in case of error in the main one
- Recommendation token to evaluate algorithm performance
- Monitoring and common metrics out of the box
- Input parameters validation
- Debug flags to troubleshoot issues
To be able to complete this tutorial, you will need to be familiar with PEACH Lab environment.
Endpoints are configured by the same mechanism as tasks using PEACH configuration files. It can be either peach.yaml or any other yaml file inside peach.conf folder in the root of your Git repository.
Let's define a simple function:
def hello_world(): return 'Hello, world'
To register it as an endpoint the following configuration is required:
codops: default endpoints: hello-world: url: /hello-world components: hello-world-basic: notebook: <your_notebook>.ipynb method: hello_world
Similar to the tasks it is required to provide the codops, name and url for the endpoint and set of algorithms under components attribute (in most of the cases there will be only one component). For each component required fields are notebook and method
By default only cell with definition of the given method would be imported. If you want the full notebook to be imported - use
full_notebook: true setting here
After notebook with algorithm and corresponding configuration is pushed to default branch (usually master, unless you have defined another default branch in Gitlab) of the git repository the endpoint should be deployed and available in couple of minutes.
If you want your code to unlock execution loop (to not block a worker while waiting for IO) or use some async libraries - you can make your function asynchronous by defining it with
async def .... Then in the code you can use
Requests to endpoints can be made with either
POST requests and parameter can be sent via query parameters, headers and for
POST request - via body. See examples for details
Endpoints with arguments
Similarly to tasks, endpoints allows optional arguments:
endpoints: dependencies: - orjson==3.3.1 - pyarrow no_args: url: /test_no_args components: test: notebook: notebooks/zzebu/techebu/related_publication_labse.ipynb method: related_publications test_args_1: url: /test_args_1 components: test: notebook: notebooks/zzebu/techebu/related_publication_labse.ipynb method: related_publications args: user: f57246b6-1848-4b1a-a8a2-d8469a050025 item_id: 35421 message: "used for testing only" my_dict: item1: 'message' second_dict: sub_item1: 8734 sub_item2: - 'test' - 23.45
We defined here 2 endpoints, with same code but different arguments. Thus, it allows to conveniently reuse the same code for different endpoints.
Adding a healthcheck on your endpoint
You can define custom healthchecks to your endpoint, made by an outside servers. This can be defined in the peach.conf file:
codops: zzebu endpoints: my-recommendations: url: /my_recommendations components: main: notebook: ... method: ... healthcheck: alias: personal-episodes # how your endpoint should be displayed on the dashboard interval: 2 # do a check every 2 minutes parameters: u: 00000000-0000-0000-0000-000000000000 size: 40 method: GET apdex_t: 0.5 # https://updown.uservoice.com/knowledgebase/articles/915588-what-is-apdex custom_headers: - Token: xyz dashboard: zzebu # to which dashboard the check should belong
This will make sure that there will be a GET request made to https://peach.ebu.io/api/v1/zzebu/my_recommendations?u=00000000-0000-0000-0000-000000000000&size=40 made every 2 minutes, sending the
Token: xyz in headers. This check with details of the statuses will be added to
zzebu dashboard status page
You can benefit from validating input parameters with a help of Python type hints. If you provide type hints to your entrypoint function - the endpoint will validate if the type of input fields is matching expected types and convert them if necessary (by default all parameters are passed as
from pydantic import conint from pydantic.typing import Literal def recommend(mode: Literal['audio', 'video'], size: conint(gt=0, le=50), factor: float = 0.5, test: bool = False): ...
Annotating your function with these type hints will:
- check if
modeargument is either
- check if
sizecan be converted to
(0, 50]range. In your code it will already be int
- checks if
factorcan be converted to
floattype and converts it (if passed, otherwise default value is used)
- check if
testcan be converted to
boolvalue, convert it (if passed). In your code it will be a
- Return the response with
HTTP 400status if any of the validations failed
By default input validation with type hints is disabled. To enable it - you need to set
true in your
peach.conf file (on the same level as defining
method, so you can only enable validation for the desired algorithm components)
In addition to the types used in example, it is possible to use any native or pydantic validations
Please, check out corresponding notebook with more detailed examples covering endpoint parametes, A/B tests and fallbacks from tutorials repository here.