Document Overview
This document details the integration points for Chatterbox Labs’ AI Model Insights software product. The intended readers of this document are developers looking to deploy the software inside an existing software platform. Business users should look to the AI Model Insights page.
Prerequisites
Prior to using AI Model Insights software, you should have a trained machine learning system which has a predict function that takes input and returns a score as output. You must also have at least 1 test data point.
Deployment
Firstly, it is important to distinguish between the deployment of Chatterbox Labs’ AI Model Insights software and the deployment of your existing trained machine learning models.
For the deployment of your own trained machine learning models, each organization will have their own process, however a typical and recommended approach is for these models to be deployed within a Docker container, with this container exposing a predict function.
Chatterbox Labs’ software runs on the Java Virtual Machine and as such can be deployed as a JVM software dependency, a standalone executable JAR or via a self-contained Docker container.
A Docker container is the preferred mechanism of deployment; it contains the standalone executable jar and manages the configuration of Chatterbox Labs’ software.
Accessing the software
Your organization will be provided with secure credentials to Chatterbox Labs’ online repositories. These credentials will give you access to the Docker registry, downloads for standalone jars and a Maven style repo for the software dependencies.
Connecting via JSON
Whether deployed via Docker or the standalone jar, the software exposes JSON endpoints for your application to connect to. See Appendix A for more details.
Connecting to External AI systems
Once your application connects to Chatterbox Labs’ software, this software in turn needs to interrogate your machine learning model (the External AI System in the diagram above). Default connectors to cloud providers are supplied (see AI Model Insights User Guide for details), a connector can be auto generated with an OpenAPI spec (see the OpenAPI documentation), or a custom REST or gRPC connector can be created (see the custom connectors documentation).
Hardware Requirements
AI Model Insights sits as a layer on top of an existing ML model therefore the underlying machine learning model will have its own resource requirements.
Typical minimum requirements for AI Model Insights are:
- CPU: Quad Core @ > 2GHz
- Memory: 16GB
- Software: JDK 11 or Docker
Logging
Logging happens via SLF4J with Logback.
Appendix A. REST/JSON Interfaces
Explain Endpoints
POST /api/v2/explain/text
input:
{
"connector": {
"predictor": "co.chatterbox.connectors.rest.GenericText",
"params": {
"endpoint": "http://localhost/predict",
"label": "class_name",
"payloadKey": "text"
}
},
"data": [
" A long string to be explained "
],
"distance": "signed",
"batch-size": 100
}
returns:
[{
"text": {
"str": "A long string to be explained",
"predicted": 0.6281051635742188
},
"explanations": [
{
"id": "0",
"ranges": [
{
"lower": 20,
"upper": 29
}
],
"tags": [
"WORD:VERB"
],
"ablated": [
"explained"
],
"score": 8.187423057429893,
"parent": ""
},
{
"id": "1",
"ranges": [
{
"lower": 2,
"upper": 6
}
],
"tags": [
"WORD:ADJ"
],
"ablated": [
"long"
],
"score": 1.5704364544042868,
"parent": ""
},
{
"id": "2",
"ranges": [
{
"lower": 7,
"upper": 13
}
],
"tags": [
"WORD:NOUN"
],
"ablated": [
"string"
],
"score": -8.35124143483302,
"parent": ""
}
]
}]
POST /api/v2/explain/mixed
input:
{
"connector": {
"predictor": "co.chatterbox.connectors.rest.GenericMixed",
"params": {
"endpoint": "http://localhost/predict",
"label": "class_name",
"payloadKey": "mixed"
}
},
"data": [
[
"5.2",
"A"
],
[
"4.9",
"B"
]
],
"domain": [
{
"name": "num col",
"type": "numerical",
"min": 0,
"max": 10
},
{
"name": "cat col",
"type": "categorical",
"distinct": [
"A",
"B",
"C"
]
}
],
"batch-size": 100
}
returns:
[ [0.0, 1.0] ]
POST /api/v2/explain/image
input:
{
"connector": {
"predictor": "co.chatterbox.connectors.rest.GenericImage",
"params": {
"endpoint": "http://localhost/predict",
"label": "class_name",
"payloadKey": "images"
}
},
"data": [
"/9j/4AAQSkZJRgABAQAAAQABAAD ..."
],
"iters": 100,
"generate-heatmap": false,
"mask-size": 0,
"mask-probability": 0,
"predictor-max": 0,
"batch-size": 100
}
returns:
[{"pixel-score": [0.0, 1.0, 0.9]}]
To get the heatmap image instead of raw scores add the param generate-heatmap
{
"connector": {
"predictor": "co.chatterbox.connectors.rest.GenericImage",
"params": {
"endpoint": "http://localhost/predict",
"label": "class_name",
"payloadKey": "images"
}
},
"data": [
"/9j/4AAQSkZJRgABAQAAAQABAAD ..."
],
"iters": 100,
"generate-heatmap": true,
"mask-size": 0,
"mask-probability": 0,
"predictor-max": 0,
"batch-size": 100
}
returns
[{
"pixel-score": [0.0, 1.0, 0.9],
"heatmap-relative": "iVBORw0KGgoAAAA",
"heatmap-absolute": "AAADICAYAAAAKhR"
}]
Trace Endpoint
POST /api/v2/trace
input:
{
"data": [
[
"Ok lar, it's so early!"
]
],
"train-data": [
[
"Ok lar... Joking wif u oni..."
],
[
"U dun say so early hor... U c already then say..."
],
[
"I HAVE A DATE ON SUNDAY WITH WILL!!"
]
],
"data-types": [
"text"
],
"limit": 3
}
returns:
[
[
{
"index": 1,
"distance": 0.11661005
},
{
"index": 0,
"distance": 0.18568748
},
{
"index": 2,
"distance": 0.18970925
}
]
]
Actions Endpoint
POST /api/v2/actions/mixed
input
{
"connector": {
"predictor": "co.chatterbox.connectors.rest.GenericMixed",
"params": {
"endpoint": "http://localhost/predict",
"label": "class_name",
"payloadKey": "mixed"
}
},
"data": [
[
"5.2",
"A"
],
[
"4.9",
"B"
]
],
"domain": [
{
"name": "num col",
"type": "numerical",
"min": 0,
"max": 10
},
{
"name": "cat col",
"type": "categorical",
"distinct": [
"A",
"B",
"C"
]
}
],
"frozen": [
"some col",
"another col"
],
"threshold": 0.5,
"batch-size": 100
}
returns
[ [ { "action": [ 0 ], "features": {}, "prediction": 0, "prediction-distance": 0, "feature-distance": 0 } ] ]
Fairness Endpoint
POST /api/v2/fairness/legacy
Input:
{
"connector": {
"predictor": "co.chatterbox.connectors.rest.GenericMixed",
"params": {
"endpoint": "http://localhost/predict",
"label": "class_name",
"payloadKey": "mixed"
}
},
"data": [
[
"5.2",
"A"
],
[
"4.9",
"B"
]
],
"feat-changes": {
"1": "C"
},
"threshold": 0.5,
"batch-size": 100
}
returns:
{
"raw-scores":
[
{ "original-score": 0,
"changed-score": 0,
"score-diff": 0,
"flipped-class": true }
],
"mean-across-all": 0,
"flipped-mean": 0,
"non-flipped-mean": 0,
"num-flipped-rows": 0
}
Vulnerabilities Endpoint
POST /api/v2/vulnerabilities/mixed
Input:
{
"connector": {
"predictor": "co.chatterbox.connectors.rest.GenericMixed",
"params": {
"endpoint": "http://localhost/predict",
"label": "class_name",
"payloadKey": "mixed"
}
},
"data": [
[
"5.2",
"A"
],
[
"4.9",
"B"
]
],
"domain": [
{
"name": "num col",
"type": "numerical",
"min": 0,
"max": 10
},
{
"name": "cat col",
"type": "categorical",
"distinct": [
"A",
"B",
"C"
]
}
],
"frozen": [
"some col",
"another col"
],
"threshold": 0.5,
"batch-size": 100
}
Returns:
[ { "vulns": [ { "feat-idx": 0, "feat-vulns": [ { "code": 0, "name": "string", "outcome": {} } ] } ], "actions": [ { "action": [ 0 ], "features": {}, "prediction": 0, "prediction-distance": 0, "feature-distance": 0 } ] } ]
Connectors Endpoint
GET /api/v2/connectors
returns:
{
"plugin-path": "string",
"connectors": {
"co.chatterbox.connectors.rest.GenericText": {
"endpoint": "localhost",
"label": "spam",
"payloadKey": "image"
}
}
}