Supported Runtime Engines

Live Sense SDK supports multiple instances for executing the Live Sense models. Exact support depends upon the device and model being used.

NNEngine

NNEngine represents the library that will be used to execute the ML model. The following libraries are supported:

• TensorFlow Lite (TFLite)

TFLite is the default NNEngine and can run on most any device that meets the minimum requirements.

Each NNEngine may have multiple runtimes available. If none are specified, then the SDK will select the best runtime available for the current NNEngine.

Engine Runtime Support

Model Runtime Support

Default Prioritization

Through experimentation and analysis of accuracy and inference time, a flow has been devised to prioritize the available options for engine and processor for each model. This flow is described in the diagram below.

The checks are performed internally to come up with the best possible configuration.

Usage

In order to use this feature and let the Live Sense SDK decide on the best possible option, the models should be initialized without specifying any options.

// Initialize RoadBasicsModel with configuration decided by the SDK
RoadBasicsModel roadBasicsModel = new RoadBasicsModel(new RoadBasicsModel.Options());

Explicitly Specify Configuration

If you do not want to use the priority decided by the Live Sense SDK, you can overwrite it.

getAvailableModelConfigurations() method will return the supported options (combination of engine and processor) for any model provided. This is ordered based on the priority described above.

To specify the engine and processor explicitly from one of the supported options on which the model should run, provide that option during the model initialization.

// Setting Road Basics to run on GPU

Map<String, List<ModelConfig>> modelConfigs = RoadBasicsModel.getAvailableModelConfigurations();
ModelConfig dayVariantConfig = modelConfigs.get("day").get(0);
RoadBasicsModel.Options rbOptions = new RoadBasicsModel.Options(dayVariantConfig);
rbOptions.setVariantId("day");
RoadBasicsModel roadBasicsModel = new RoadBasicsModel(rbOptions);