JavaScript frontend SDK

The JavaScript SDK enables you to use IntelliProves features with ease from within your front end.
Requirements
JavaScript front-end project
This SDK only provides tools for front-end applications. Refer to our  NodeJS SDK﻿ for JavaScript back-ends. 
Tested on
Vue.JS 2
Vue.JS 3
React
Since our SDK does not use any framework-specific code, it should be compatible with most common JS web frameworks.
Recommendations
We recommend the use of TypeScript as it provides type hints and a better developer experience. However, our SDK works with CommonJS as well.
Installation
NPM
You can find our SDK on the node package manager repository for easy installation:
﻿https://www.npmjs.com/package/intelliprove-sdk﻿
Shell
npm add intelliprove-streaming-sdk
npm add intelliprove-streaming-sdk
﻿
Updating SDK
Shell
npm update intelliprove-streaming-sdk
npm update intelliprove-streaming-sdk
﻿
CDN
If you are not using an NPM-compatible package manager or just prefer using a CDN, you have multiple options. The most popular ones are:
jsDelivr
﻿https://cdn.jsdelivr.net/npm/intelliprove-streaming-sdk@latest﻿
unpkg
﻿https://unpkg.com/intelliprove-streaming-sdk@latest﻿
Self-hosting a fixed version
If preferred you can also self-host our SDK by downloading our intelliprove-sdk.js file from a CDN or NPM.
However, this is not recommended as updates will require you to manually re-download the latest file and update it on your own server(s).
Authentication
More info on our API authentication can be found in our API docs. The action token﻿, which you will use in your frontend to perform authentication as shown below, should be requested from your backend using your API key.
Functionality
Attach SDK
Attach the SDK to your current application.
The attach method finds the user's available cameras, opens a camera stream, and previews the stream in the provided video object.
Parameters:
videoElementId: string - HTML Element ID of the video element you want to attach to
cameraId: string | null - OPTIONAL: A camera id if a specific camera is preferred
streamWidth: MediaResolution | null - OPTIONAL: Custom width for video stream
streamHeight: MediaResolution | null - OPTIONAL: Custom height for video stream
JS
import IntelliProveStreamingSDK from "intelliprove-streaming-sdk";

const VIDEO_ELEMENT_ID = ""
const ACTION_TOKEN = "";

const intelliprove = new IntelliProveStreamingSDK(ACTION_TOKEN);
await intelliprove.attach(VIDEO_ELEMENT_ID);
import IntelliProveStreamingSDK from "intelliprove-streaming-sdk";

const VIDEO_ELEMENT_ID = ""
const ACTION_TOKEN = "";

const intelliprove = new IntelliProveStreamingSDK(ACTION_TOKEN);
await intelliprove.attach(VIDEO_ELEMENT_ID);
﻿
Start measurement
This method starts the measurement, i.e. performs the quality check and records a video. This corresponds to steps 2 and 3 found in our Integration guidelines﻿.
Keep in mind, the attach method needs to be successful before calling start.

When calling start, first, an image from the camera stream will be captured and the quality check will be performed on the frame. 
If the quality check fails, a QualityError will be thrown with details on why the measurement cannot be started. More info on the quality check score and error type can be found here﻿.
If the quality check succeeds, the SDK will create a new measurement and start recording a video of the subject.
Parameters:
length: number - The length of the recording
chunkSize: number - The size of each video chunk when recording, default 500 ms.
streamingMetadata: StreamingMetadata - An optional object with information about the subject and performer. See the models section below for more info
Exceptions: (doc﻿)
QualityError
IntelliProveAPIError
IntelliProveSDKError
JS
import IntelliProveStreamingSDK from "intelliprove-streaming-sdk";

const VIDEO_ELEMENT_ID = ""
const ACTION_TOKEN = "";

const intelliprove = new IntelliProveStreamingSDK(ACTION_TOKEN);
await intelliprove.attach(VIDEO_ELEMENT_ID);

await intelliprove.start(20000); // Start a recording for 20s => 20 000ms
import IntelliProveStreamingSDK from "intelliprove-streaming-sdk";

const VIDEO_ELEMENT_ID = ""
const ACTION_TOKEN = "";

const intelliprove = new IntelliProveStreamingSDK(ACTION_TOKEN);
await intelliprove.attach(VIDEO_ELEMENT_ID);

await intelliprove.start(20000); // Start a recording for 20s => 20 000ms
﻿
When a measurement is started with the start method, the stopCallback method will automatically be called when the video is uploaded and processing has started. More info on the stopCallback here﻿.
JS
import IntelliProveStreamingSDK from "intelliprove-streaming-sdk";

const VIDEO_ELEMENT_ID = ""
const ACTION_TOKEN = "";

const intelliprove = new IntelliProveStreamingSDK(ACTION_TOKEN);
await intelliprove.attach(VIDEO_ELEMENT_ID);

await intelliprove.start(20000); // Start a recording for 20s => 20 000ms

intelliprove.stopCallback = async (stream) => {
  // Get results here
}
import IntelliProveStreamingSDK from "intelliprove-streaming-sdk";

const VIDEO_ELEMENT_ID = ""
const ACTION_TOKEN = "";

const intelliprove = new IntelliProveStreamingSDK(ACTION_TOKEN);
await intelliprove.attach(VIDEO_ELEMENT_ID);

await intelliprove.start(20000); // Start a recording for 20s => 20 000ms

intelliprove.stopCallback = async (stream) => {
  // Get results here
}
﻿
Finish measurement
Finishing a measurement means stopping the recording and waiting for the results. Once available, the results of the processed video are returned as a BiomarkersResponse﻿ object. 
In case we detect the video input requirements﻿ are not met, an EngineValidationError﻿ is returned. This could be related, for example, with the subject moving too much or looking away during from the camera the measurement.
Exceptions:
﻿IntelliProveAPIError﻿﻿
﻿IntelliProveSDKError﻿﻿
﻿
JS
import IntelliProveStreamingSDK from "intelliprove-streaming-sdk";

const VIDEO_ELEMENT_ID = ""
const ACTION_TOKEN = "";

const intelliprove = new IntelliProveStreamingSDK(ACTION_TOKEN);
await intelliprove.attach(VIDEO_ELEMENT_ID);

await intelliprove.start(20000); // Start a recording for 20s => 20 000ms

intelliprove.stopCallback = async (stream) => {
  const result = await intelliprove.finish();
  // Do something with results or handle ValidationError
}
import IntelliProveStreamingSDK from "intelliprove-streaming-sdk";

const VIDEO_ELEMENT_ID = ""
const ACTION_TOKEN = "";

const intelliprove = new IntelliProveStreamingSDK(ACTION_TOKEN);
await intelliprove.attach(VIDEO_ELEMENT_ID);

await intelliprove.start(20000); // Start a recording for 20s => 20 000ms

intelliprove.stopCallback = async (stream) => {
  const result = await intelliprove.finish();
  // Do something with results or handle ValidationError
}
﻿
Cancel measurement
Cancel the current recording before it has been finished and reset the SDK.
Exceptions:
﻿IntelliProveAPIError﻿﻿
﻿
JS
import IntelliProveStreamingSDK from "intelliprove-streaming-sdk";

const VIDEO_ELEMENT_ID = ""
const ACTION_TOKEN = "";

const intelliprove = new IntelliProveStreamingSDK(ACTION_TOKEN);
await intelliprove.attach(VIDEO_ELEMENT_ID);

await intelliprove.start(20000); // Start a recording for 20s => 20 000ms
await intelliprove.cancel();
import IntelliProveStreamingSDK from "intelliprove-streaming-sdk";

const VIDEO_ELEMENT_ID = ""
const ACTION_TOKEN = "";

const intelliprove = new IntelliProveStreamingSDK(ACTION_TOKEN);
await intelliprove.attach(VIDEO_ELEMENT_ID);

await intelliprove.start(20000); // Start a recording for 20s => 20 000ms
await intelliprove.cancel();
﻿
Get results for existing measurement
In case there is a need to manually retrieve the results for an existing measurement (referenced by UUID), the getResults method can be used. The results of the processed video are returned as a BiomarkersResponse﻿ object. 
In case we detected something wrong with the recorded video, an EngineValidationError﻿ is returned. This could be related, for example, with the subject moving too much or looking away during the measurement.
Parameters:
uuid: string - The UUID of the finished measurement
Exceptions:
﻿IntelliProveAPIError﻿﻿
﻿IntelliProveSDKError﻿﻿
﻿
JS
import IntelliProveStreamingSDK from "intelliprove-streaming-sdk";

const ACTION_TOKEN = "";

const intelliprove = new IntelliProveStreamingSDK(ACTION_TOKEN);

const uuid = '<MEASUREMENT_UUID>'
const result = await intelliprove.getResults(uuid);

import IntelliProveStreamingSDK from "intelliprove-streaming-sdk";

const ACTION_TOKEN = "";

const intelliprove = new IntelliProveStreamingSDK(ACTION_TOKEN);

const uuid = '<MEASUREMENT_UUID>'
const result = await intelliprove.getResults(uuid);
﻿
Select next camera
If the user has multiple cameras, you can cycle through them using the nextCamera method. When the final camera has been reached and nextCamera is called again, the first camera will be selected again.
JS
import IntelliProveStreamingSDK from "intelliprove-streaming-sdk";

const VIDEO_ELEMENT_ID = ""
const ACTION_TOKEN = "";

const intelliprove = new IntelliProveStreamingSDK(ACTION_TOKEN);
await intelliprove.attach(VIDEO_ELEMENT_ID);
await intelliprove.nextCamera();
import IntelliProveStreamingSDK from "intelliprove-streaming-sdk";

const VIDEO_ELEMENT_ID = ""
const ACTION_TOKEN = "";

const intelliprove = new IntelliProveStreamingSDK(ACTION_TOKEN);
await intelliprove.attach(VIDEO_ELEMENT_ID);
await intelliprove.nextCamera();
﻿
﻿
Models
Those are simple representations of the models accessible through the SDK and their helper functions.
All models have a .json() method to convert it to a normal JS object in case this is preferred.
QualityResponse
Describes the quality of an image.
JS
class QualityResponse {
    score: number;
    error_type: number;
    
    prompt: string | null;
    signature: string | null;
}
class QualityResponse {
    score: number;
    error_type: number;
    
    prompt: string | null;
    signature: string | null;
}
﻿
Helpers
success(): boolean - Has the quality check succeeded
BiomarkersResponse
Holds the results of a processed video after a measurement.
JS
class BiomarkersResponse {
    epoch: string;
    timestamp: Date;
    heart_rate: string;
    
    respiratory_rate: string | null;
    heart_rate_variability: string | null;
    ans_balance: string | null;
    resonant_breathing_score: string | null;
    morning_readiness: string | null;
    acute_mental_stress_score: string | null;
    mental_health_risk: string | null;
}
class BiomarkersResponse {
    epoch: string;
    timestamp: Date;
    heart_rate: string;
    
    respiratory_rate: string | null;
    heart_rate_variability: string | null;
    ans_balance: string | null;
    resonant_breathing_score: string | null;
    morning_readiness: string | null;
    acute_mental_stress_score: string | null;
    mental_health_risk: string | null;
}
﻿
StreamingMetadata
Holds metadata info about a stream's subject and performer.
JS
class StreamingMetadata {
    performer: string | null;
    patient: string | null;
}
class StreamingMetadata {
    performer: string | null;
    patient: string | null;
}
﻿
StreamStatus
Holds information about the current measurement stream. StreamInfo objects are used as the parameter for the stopCallback and dataCallback
remaining is the amount of time the recording has left. It can be null if the recording is finished or has not begun yet.
JS
class StreamInfo {
    uuid: string;
    done: boolean;
    expires_in: number;
    remaining: number | null;
}
class StreamInfo {
    uuid: string;
    done: boolean;
    expires_in: number;
    remaining: number | null;
}
﻿
Helpers
isExpired(): boolean - Has the current stream expired
MediaResolution
This model is used for retrieving a user's camera. If preferred you can provide custom instances of MediaResolution as arguments to our SDKs attach method.
JS
class MediaResolution {
    min: number;
    max: number;
    ideal: number;
}
class MediaResolution {
    min: number;
    max: number;
    ideal: number;
}
﻿
UnprocessableVideoResponse
This is a class containing information about a problem with the video or stream. Refer to the Result outcomes﻿ for more information about the different types of errors.
Contains information on why a recording or video has failed to process. 
JS
class UnprocessableVideoResponse {
    errorCode: number;
    errorType: string;
    errorDescription: string;
}
class UnprocessableVideoResponse {
    errorCode: number;
    errorType: string;
    errorDescription: string;
}
﻿
﻿
Callbacks
We provide the option to set callbacks that get called after certain actions are completed by the SDK.
Stop callback
Gets called when the recording stops and processing is started. It provides the stream status as a StreamStatus object as a parameter.
JS
sdk.stopCallback = async (streamInfo) => {
    // Log the stream info
    console.log(streamInfo);

    // Fetch results and log them to the console
    const biomarkers = await intelliprove.getResults();
    console.log(biomarkers);
}
sdk.stopCallback = async (streamInfo) => {
    // Log the stream info
    console.log(streamInfo);

    // Fetch results and log them to the console
    const biomarkers = await intelliprove.getResults();
    console.log(biomarkers);
}
﻿
Data callback
Gets called each time a new chunk of video is recorded. This will happen multiple times during a measurement. As parameters, it provides the stream's status as a StreamInfo object and a data Blob object. This could be handy when you want to do some custom actions with the recording or save the video yourself.
JS
sdk.dataCallback = async (streamInfo, data) => {
    // Log the stream info
    console.log(streamInfo);
    
    // Do some action with the binary in the data Blob
}
sdk.dataCallback = async (streamInfo, data) => {
    // Log the stream info
    console.log(streamInfo);
    
    // Do some action with the binary in the data Blob
}
﻿
﻿
Exceptions
Our SDK throws 3 types of exceptions. All of our exceptions inherit from the base JavaScript Error class.
QualityError
Thrown when the quality check failed
It contains information about why the quality check failed
JS
class QualityError extends Error {
    quality: QualityResponse;
    constructor(msg: string, qr: QualityResponse);
}
class QualityError extends Error {
    quality: QualityResponse;
    constructor(msg: string, qr: QualityResponse);
}
﻿
IntelliProveAPIError
Thrown when a request to our API has failed or is invalid
Includes a message about what failed
Includes the HTTP status code
Eg. Invalid Authentication (401)
JS
class IntelliProveAPIError extends Error {
    status_code: number;
    constructor(msg: string, status_code: number);
}
class IntelliProveAPIError extends Error {
    status_code: number;
    constructor(msg: string, status_code: number);
}
﻿
IntelliProveSDKError
Thrown when there is an issue within the SDK
Includes a message about what went wrong
Includes ErrorType
Eg. Your internet connection dropped while streaming
JS
const ErrorType = {
  Connection: 0,
  Configuration: 1,
  SDK: 2,
  UI: 3,
  Other: 99
}

class IntelliProveSDKError extends Error {
    error_type: number;  // ErrorType enum as number
    constructor(msg: string, error_type?: number);
}
const ErrorType = {
  Connection: 0,
  Configuration: 1,
  SDK: 2,
  UI: 3,
  Other: 99
}

class IntelliProveSDKError extends Error {
    error_type: number;  // ErrorType enum as number
    constructor(msg: string, error_type?: number);
}
﻿
﻿
Examples
Complete example
Provided with a valid action token and a valid video element ID, this code snippet runs from start to finish and prints out the results of the measurement to the console. In total, it takes around 20-30 seconds to run.
JS
import IntelliProveSDK from "intelliprove-streaming-sdk";

// You need a video HTML element with the a unique id
// For example: the id 'video'
const VIDEO_ELEMENT_ID = "video"

// Your intelliprove token
const ACTION_TOKEN = "my-action-token";

// Create an instance of our SDK and attach it to the video object with the id provided above
const intelliprove = new IntelliProveSDK(ACTION_TOKEN);
await intelliprove.attach(VIDEO_ELEMENT_ID);

// Assign a callback to handle data when the recording stops or is finished
sdk.stopCallback = async (streamStatus) => {
    // Log the stream info
    console.log(streamStatus);

    // Fetch results from recording
    const biomarkers = await intelliprove.finish();
    console.log(biomarkers);

    // Detach the video stream from your video element
    await intelliprove.detach();
}

try {
    // Start the recording with the default settings
    await sdk.start();
} catch (error) {
    if (error instanceof QualityError) {
        alert(`User prompt: ${error.quality.message}`);
    } else if (error instanceof IntelliProveAPIError) {
        alert(`${error.message}: ${error.status_code}`);
    } else if (error instanceof IntelliProveSDKError) {
        alert(error.message);
    }
}
import IntelliProveSDK from "intelliprove-streaming-sdk";

// You need a video HTML element with the a unique id
// For example: the id 'video'
const VIDEO_ELEMENT_ID = "video"

// Your intelliprove token
const ACTION_TOKEN = "my-action-token";

// Create an instance of our SDK and attach it to the video object with the id provided above
const intelliprove = new IntelliProveSDK(ACTION_TOKEN);
await intelliprove.attach(VIDEO_ELEMENT_ID);

// Assign a callback to handle data when the recording stops or is finished
sdk.stopCallback = async (streamStatus) => {
    // Log the stream info
    console.log(streamStatus);

    // Fetch results from recording
    const biomarkers = await intelliprove.finish();
    console.log(biomarkers);

    // Detach the video stream from your video element
    await intelliprove.detach();
}

try {
    // Start the recording with the default settings
    await sdk.start();
} catch (error) {
    if (error instanceof QualityError) {
        alert(`User prompt: ${error.quality.message}`);
    } else if (error instanceof IntelliProveAPIError) {
        alert(`${error.message}: ${error.status_code}`);
    } else if (error instanceof IntelliProveSDKError) {
        alert(error.message);
    }
}
﻿
Fully working web app
The file below is a fully working example of how you could use the Intelliprove Streaming SDK on a web page. Download the file and open it in your favorite browser to try it out.
working-example.html
﻿
Persisting the video recording
This example shows you how you can use the Intelliprove Streaming SDK to do a measurement and save the video for later playback using the callbacks we provide.
﻿
JS
import IntelliProveSDK from 'intelliprove-streaming-sdk';

const VIDEO_ELEMENT_ID = "video"
const ACTION_TOKEN = "my-action-token";

const intelliprove = new IntelliProveSDK(ACTION_TOKEN);
await intelliprove.attach(VIDEO_ELEMENT_ID);

// Array of blobs
var videoChunks = [];


// Set the dataCallback
sdk.dataCallback = async (streamStatus, data) => {
  videoChunks.push(data);
}

sdk.stopCallback = async (streamStatus) => {
    // Fetch results
    const biomarkers = await intelliprove.finish();
    console.log(biomarkers);

    // Detach the video stream from your video element
    await intelliprove.detach();
    
    // merge the seperate blobs into one blob file
    const video = new Blob(videoChunks);
}

// Start a 20s recording with the default settings
await sdk.start(20000);
import IntelliProveSDK from 'intelliprove-streaming-sdk';

const VIDEO_ELEMENT_ID = "video"
const ACTION_TOKEN = "my-action-token";

const intelliprove = new IntelliProveSDK(ACTION_TOKEN);
await intelliprove.attach(VIDEO_ELEMENT_ID);

// Array of blobs
var videoChunks = [];


// Set the dataCallback
sdk.dataCallback = async (streamStatus, data) => {
  videoChunks.push(data);
}

sdk.stopCallback = async (streamStatus) => {
    // Fetch results
    const biomarkers = await intelliprove.finish();
    console.log(biomarkers);

    // Detach the video stream from your video element
    await intelliprove.detach();
    
    // merge the seperate blobs into one blob file
    const video = new Blob(videoChunks);
}

// Start a 20s recording with the default settings
await sdk.start(20000);
﻿
﻿
Overview
Python SDK
JavaScript frontend SDK

Requirements

Tested on

Recommendations

Installation

NPM

Updating SDK

CDN

Self-hosting a fixed version

Authentication

Functionality

Attach SDK

Start measurement

Finish measurement

Cancel measurement

Get results for existing measurement

Select next camera

Models

QualityResponse

BiomarkersResponse

StreamingMetadata

StreamStatus

MediaResolution

UnprocessableVideoResponse

﻿

Callbacks

Stop callback

Data callback

Exceptions

QualityError

IntelliProveAPIError

IntelliProveSDKError

﻿

Examples

Complete example

Fully working web app

Persisting the video recording
