The JavaScript SDK enables you to use IntelliProves features with ease and without the need for your own back-end.

Requirements

• JavaScript front-end project

This SDK only provides tools for front-end applications. Our SDK for JavaScript back-ends can be found here: https://docs.intelliprove.com/sdk/nodejs-sdk

Tested on

• Vue.JS 2
• Vue.JS 3
• React

Since our SDK does not use any framework specific code, it should be compatible with most common JS web frameworks.

Recommendations

We recommend the use of TypeScript as it provides type hints and a better developer experience. However our SDK works as CommonJS as well.

Installation

You can find our SDK on the node package manager repository for easy installation:

https://www.npmjs.com/package/intelliprove-sdk

npm add intelliprove-streaming-sdk

Updating SDK

npm update intelliprove-streaming-sdk

Authentication

More info on our API authentication can be found here.



Functionality

Attach SDK

Attach the SDK to your current application. The attach method finds the users available cameras, opens a camera stream and previews the stream in the provided video object.

Parameters:

• cameraId: string | null - OPTIONAL: A camera id if a specific camera is prefered

import IntelliProveStreamingSDK from "intelliprove-streaming-sdk";

const VIDEO_ELEMENT_ID = ""
const ACTION_TOKEN = "";

const intelliprove = new IntelliProveSDK(VIDEO_ELEMENT_ID, ACTION_TOKEN);
await intelliprove.attach();

Start measurement

The attach method needs to be successfull before calling start.

After calling start, an image from the camera stream will be captured and the quality check will be performed on the frame.

• If the quality check fails, an QualityError will be thrown with details on why the measurement cannot be started. More info on the quality check score and error type can be found here.
• If the quality check succeeds, the SDK will create a new measurement and start recording a video of the subject.

Parameters:

• length: number - The length of the recording
• chunkSize: number - The size of each video chunk when recording, default 500 ms.
• streamingMetadata: StreamingMetadata - An optional object with information about the subject and perfomer. See the models section bellow for more info

import IntelliProveStreamingSDK from "intelliprove-streaming-sdk";

const VIDEO_ELEMENT_ID = ""
const ACTION_TOKEN = "";

const intelliprove = new IntelliProveSDK(VIDEO_ELEMENT_ID, ACTION_TOKEN);
await intelliprove.attach();

intelliprove.start(20000); // Start a recording for 20s => 20 000ms

When a measurement is started with the start method, the stopCallback method will automatically be called when the video is uploaded and processing has started. More info on the stopCallback here.

import IntelliProveStreamingSDK from "intelliprove-streaming-sdk";

const VIDEO_ELEMENT_ID = ""
const ACTION_TOKEN = "";

const intelliprove = new IntelliProveSDK(VIDEO_ELEMENT_ID, ACTION_TOKEN);
await intelliprove.attach();

intelliprove.stopCallback = async (stream) => {
  // Get results here
}

Finish measurement

Finishing a measurement means stopping the recording and waiting for the results. The results of the processed video are returned as a Biomarkers object.

import IntelliProveStreamingSDK from "intelliprove-streaming-sdk";

const VIDEO_ELEMENT_ID = ""
const ACTION_TOKEN = "";

const intelliprove = new IntelliProveSDK(VIDEO_ELEMENT_ID, ACTION_TOKEN);
await intelliprove.attach();

intelliprove.stopCallback = async (stream) => {
  const result = intelliprove.finish()
  // Do something with results
}

Cancel measurement

Cancel the current recording before it has been finished and reset the sdk.

import IntelliProveStreamingSDK from "intelliprove-streaming-sdk";

const VIDEO_ELEMENT_ID = ""
const ACTION_TOKEN = "";

const intelliprove = new IntelliProveSDK(VIDEO_ELEMENT_ID, ACTION_TOKEN);
await intelliprove.attach();

intelliprove.start(20000); // Start a recording for 20s => 20 000ms
intelliprove.cancel();

Get results for existing measurement

In case some wants to manually retrieve the results for an existing measurement (referenced by UUID), the getResults method can be used. The results of the processed video are returned as a Biomarkers object.

Parameters:

• uuid: string - The UUID returned by the upload method

import IntelliProveStreamingSDK from "intelliprove-streaming-sdk";

const VIDEO_ELEMENT_ID = ""
const ACTION_TOKEN = "";

const intelliprove = new IntelliProveSDK(VIDEO_ELEMENT_ID, ACTION_TOKEN);

const uuid = '<MEASUREMENT_UUID>'
const result = await intelliprove.getResults(uuid);

Select next camera

If the user has multiple cameras, you can cycle through them using the nextCamera method. When the final camera has been reached and nextCamera is called again, the first camera will be selected again.

import IntelliProveStreamingSDK from "intelliprove-streaming-sdk";

const VIDEO_ELEMENT_ID = ""
const ACTION_TOKEN = "";

const intelliprove = new IntelliProveSDK(VIDEO_ELEMENT_ID, ACTION_TOKEN);
await intelliprove.attach();

await intelliprove.nextCamera();



Models

Those are simple representations of the models accessible through the SDK and their helper functions. All models have a .json() method to convert it to a normal JS object in case this is preferred.

QualityResponse

Describes the quality of an image.

class QualityResponse {
    score: number;
    error_type: number;
    message: string;
    signature: string;
}

Helpers

• success(): boolean - Has the quality check succeeded

BiomarkersResponse

Holds the results of a processed video after a measurement.

class BiomarkersResponse {
    epoch: string;
    timestamp: Date;
    heart_rate: string;
    respiratory_rate: string;
    heart_rate_variability: string;
    ans_balance: string;
    resonant_breathing_score: string;
    morning_readiness: string;
    acute_mental_stress_score: string;
    mental_health_risk: string;
}

StreamingMetadata

Holds metadata info about a streams subject and performer.

class StreamingMetadata {
    performer: string | null;
    patient: string | null;
}

StreamStatus

Holds information about the current measurements stream. StreamInfo objects are used as the parameter for the stopCallback and dataCallback

class StreamInfo {
    uuid: string;
    done: boolean;
    expires_in: number;
}

Helpers

• isExpired(): boolean - Has current stream expired or not.



Callbacks

We provide the option to set callbacks that get called after certain actions are completed by the SDK.

Stop callback

Gets called when the recording stops and processing is started. It provides the stream status as a StreamStatus object as a parameter.

sdk.stopCallback = async (streamInfo) => {
    // Log the stream info
    console.log(streamInfo);

    // Fetch results and log them to the console
    const biomarkers = await intelliprove.getResults();
    console.log(biomarkers);
}

Data callback

Gets called each time a new chunk of video is recorded. This will happen multiple times during a measurement. As parameters it provides the streams status as a StreamInfo object and a data Blob object. This could be handy when you want to do some custom actions with the recording or save the video yourself.

sdk.dataCallback = async (streamInfo, data) => {
    // Log the stream info
    console.log(streamInfo);
    
    // Do some action with the binary in the data Blob
}



Exceptions

Our SDK throws 3 types of exceptions. All of our exceptions inherit from the base JavaScript Error class.

• QualityError
  • Thrown when the quality check failed
  • It contains information about who the quality check failed
• IntelliProveAPIError
  • Thrown when a request to our API has failed or is invalid
  • Includes a message about what failed
  • Includes the HTTP status code
  • Eg. Invalid Authentication (401)
• IntelliProveSDKError
  • Thrown when there is an issue withing the SDK
  • Includes a message about what went wrong
  • Includes ErrorType
  • Eg. Your internet connection dropped while streaming



ErrorType

This is a list of the possible values the error_type field in IntelliProveSDKError

const ErrorType = {
  Connection: 0,
  Configuration: 1,
  SDK: 2,
  UI: 3,
  Other: 99
}

Examples

import IntelliProveStreamingSDK from "intelliprove-streaming-sdk";

// You need a video HTML element with the a unique id
// For example: the id 'video'
const VIDEO_ELEMENT_ID = "video"

// Your intelliprove token
const ACTION_TOKEN = "my-action-token";

// Create an instance of our SDK and attach it to the video object with the id provided above
const intelliprove = new IntelliProveSDK(VIDEO_ELEMENT_ID, ACTION_TOKEN);
await intelliprove.attach();

// Assign a callback to handle data when the recording stops or is finished
sdk.stopCallback = async (streamStatus) => {
    // Log the stream info
    console.log(streamStatus);

    // Fetch results from recording
    const biomarkers = await intelliprove.finish();
    console.log(biomarkers);

    // Detach the video stream from your video element
    await intelliprove.detach();
}

// Start the recording with the default settings
sdk.start();

import IntelliProveStreamingSDK from "intelliprove-streaming-sdk";

const VIDEO_ELEMENT_ID = "video"
const ACTION_TOKEN = "my-action-token";

const intelliprove = new IntelliProveSDK(VIDEO_ELEMENT_ID, ACTION_TOKEN);
await intelliprove.attach();

// Array of blobs
var videoChunks = [];


// Set the dataCallback
sdk.dataCallback = async (streamStatus, data) => {
  videoChunks.push(data);
}

sdk.stopCallback = async (streamStatus) => {
    // Fetch results
    const biomarkers = await intelliprove.finish();
    console.log(biomarkers);

    // Detach the video stream from your video element
    await intelliprove.detach();
    
    // merge the seperate blobs into one blob file
    const video = new Blob(videoChunks);
}

// Start a 20s recording with the default settings
await sdk.start(20000);