AI Model Insights – Custom connectors

Overview

This document details the technical specification for a REST/JSON wrapper around an existing machine learning prediction function to be used with Chatterbox Labs’ AI Model Insights platform.

Out of the box, default connectors are provided to Google Cloud, AWS, Microsoft Azure and IBM Watson and a connector can be auto generated with an OpenAPI spec.  There is no customization needed for these standard entry points.

There is a further scenario to address: You have a machine learning prediction function, however it is not yet wrapped in JSON/REST

In this scenario, a very simple wrapper should be created.  As you are starting from a blank canvas, it makes sense to use the default JSON specification as this will accelerate the development of the wrapper.  Typically, we see the time required to build the initial wrapper as in the region of 1 – 2 hours.

Before reading this document please read the AI Model Insights User Guide, and if appropriate, the AI Model Insights Developer Guide.

General Principles

Whilst the prediction functions for image, mixed data and text differ slightly as they contain different data types, they all have some general principles in common:

  • The REST wrapper is simply a wrapper around your existing predict function, there is no new functionality needed
  • Typically, the endpoints operate in batch mode – they take multiple data points as input, and return multiple predictions as output.  Under the hood your predict function may pass all of these data points into your model in one go, or simply iterate through the datapoints one by one
  • The important information that needs to be returned for each data point is the prediction score
  • All endpoints use POST

Text Prediction Endpoints

Input: A map with the key ‘text’.  The value of this key should be a list of text datapoints to be predicted against

Output: A list of maps.  Each map should have as keys the classes in the model, and as values the prediction scores associated with that class.  If the model does not have classes (such as a regression model) a placeholder key such as ‘prediction’ should be used in the map.

Example POST Input:

{
    "text": ["text contents of data point 1", "text contents of data point 2", "text contents of data point 3"]
}

Example Output:

[
    {"class_a":-0.890348729,"class_b":0.67463927743},
    {"class_a":-0.764738254,"class_b":0.75638494794},
    {"class_a":-0.593348572,"class_b":0.43048573952}
]

Mixed Data Prediction Endpoints

Input: A map with the key ‘payload’.  The value of this key should be a list of datapoints.  Each datapoint is itself a list.  Please note that the order of this list matters – the order of the variables should match the order that the model is expecting.

Output: A list of maps. Each map should have as keys the classes in the model, and as values the prediction scores associated with that class. If the model does not have classes (such as a regression model) a placeholder key such as ‘prediction’ should be used in the map.

Example POST Input:

{
    "payload": [
    	["63","1","3","145","233","1","0","150","0","2.3","0","0","1"],
    	["27","1","3","145","233","1","0","150","0","2.3","0","0","1"],
    	["85","1","3","145","233","1","0","150","0","2.3","0","0","1"]
    	]
}

Example Output:

[
    {
        "class_a": 0.92,
        "class_b": 0.08
    },
    {
        " class_a ": 0.84,
        " class_b ": 0.16
    },
    {
        " class_a ": 0.86,
        " class_b ": 0.14
    }
]

Image Prediction Endpoints

Input: A map with the key ‘images’.  The value of this key should be a list of base64 encoded images. 

Output: A list of maps.  Each map should have as keys the classes in the model, and as values the prediction scores associated with that class.  If the model does not have classes (such as a regression model) a placeholder key such as ‘prediction’ should be used in the map.

Example POST Input:

{
    "images": ["/9j/4AAQSkZJRgABA ", "/9j/4AAQSkZJRgABA ", "/9j/4AAQSkZJRgABA "]
}

Example Output:

[
    {"class_a":-0.890348729,"class_b":0.67463927743},
    {"class_a":-0.764738254,"class_b":0.75638494794},
    {"class_a":-0.593348572,"class_b":0.43048573952}
]

Get in Touch