Virtual Try-on API

Bambuser product: Video Consultation

Virtual try-on adds capabilities to virtually apply products onto the shopper. During a virtual try-on session many different product and properties can be changed to find the best set of products for the shopper to buy.

All communication and data passing between agent and shopper for the virtual try-on session is handled under the hood by Bambuser.

Methods

createProvider()

Permission: callsWidget:virtualTryOnProvider.write
Context: Main app

Add a virtual try-on provider that can virtually render the currently applied products in virtual try-on session onto the shopper

Parameters

• vendorId - Unique text string identifying the provider within the Bambuser platform, should prefferably be in the format com.your-company.vtoProviderName

Return value

A promise whose fulfillment handler will receive an instance of a Virtual Try-on Provider.

Example

const { virtualTryOnApi } = await bambuserAppFramework.getContext();

const vtoProvider = await virtualTryOnApi.createProvider({
  vendorId: 'com.example.myVtoProvider',
});

getProducts()

Permission: callsWidget:virtualTryOn.read / callsAgentTool:virtualTryOn.read
Context: All

Get all products that currently exists in the virtual try-on session

Return value

A promise whose fulfillment handler will receive an array of product representations for all the products that currently exists in the virtual try-on session

Example

const { virtualTryOnApi } = await bambuserAppFramework.getContext();

const products = await virtualTryOnApi.getProducts()
console.log(products);
// [{ 
//   sku,
//   locale,
//   currency,
//   defaultVariationIndex,
//   description,
//   virtualTryOn?, // See 'provide-vto-data' event listener
//   variations: [{
//     sku,
//     name,
//     subtitle?,
//     url?,
//     imageUrls,
//     price?,
//     virtualTryOn?, // See 'provide-vto-data event listener
//     ..., // See https://bambuser.com/docs/calls/provide-product-data for additional possible properties a product may have
//   }]
// }, ...]

removeProduct()

Permission: callsWidget:virtualTryOn.write / callsAgentTool:virtualTryOn.write
Context: All

Remove a product from the virtual try-on session

Parameters

• sku - The product or variation sku of the product and all its variations that should be removed

Example

const { virtualTryOnApi } = await bambuserAppFramework.getContext();

const product = ...; // Product from 'product-added' or getProducts()

await virtualTryOnApi.removeProduct({
  sku: product.sku,
});

apply()

Permission: callsWidget:virtualTryOn.write / callsAgentTool:virtualTryOn.write
Context: All

Add or replace an item that should be applied on the shopper. When this is the first applied item the virtual try-on session will start or resume

Parameters

• item.id - The id representing the applied item, usually a product variation sku
• item.* - Any other properyies that should be applied, ex. pattern
• replaceId - The id the provided item should replace, ex. when swiching between a product's variations

Example

const { virtualTryOnApi } = await bambuserAppFramework.getContext();

await virtualTryOnApi.apply({
  item: {
    id: 'lipAbcRed', // Required
    pattern: 'matte',
    someCustomProp: 'someCustomProp123',
  },
  replaceId: 'libAbcGreen', // Optional, use to replace already applied variation from same product
});

getAppliedItems()

Permission: callsWidget:virtualTryOn.read / callsAgentTool:virtualTryOn.read
Context: Main app, Tool view

Get all the items that currently are applied on shopper in the virtual try-on session

Return value

A promise whose fulfillment handler will receive an array of all items that currently are applied in the virtual try-on session

Example

const { virtualTryOnApi } = await bambuserAppFramework.getContext();

const items = await virtualTryOnApi.getAppliedItems();
console.log(items);
// [{
//   id,
//   pattern,
//   ..., // General product vto data 
//   ..., // Specfific vto data for variation
//   ..., // Any custom props that was provided in apply
// }]

removeAppliedItem()

Permission: callsWidget:virtualTryOn.write / callsAgentTool:virtualTryOn.write
Context: Main app, Tool view

Remove an applied item from shopper in the current virtual try-on session. When this is the last applied item the virtual try-on session will pause

Example

const { virtualTryOnApi } = await bambuserAppFramework.getContext();

await virtualTryOnApi.removeAppliedItem({ sku })

removeAllAppliedItems()

Permission: callsWidget:virtualTryOn.write / callsAgentTool:virtualTryOn.write
Context: Main app, Tool view

Remove all applied items from shopper in virtual try-on session. When this is called the virtual try-on session will pause

Example

const { virtualTryOnApi } = await bambuserAppFramework.getContext();

await virtualTryOnApi.removeAllAppliedItems()

Events

product-added

Permission: callsWidget:virtualTryOn.read / callsAgentTool:virtualTryOn.read
Context: Main app, Tool view

The product-added event is fired when a product is added to the virtual try-on session

Event properties

• product Product representation for the product and all its variations that was added to the virtual try-on session

Example

const { virtualTryOnApi } = await bambuserAppFramework.getContext();

virtualTryOnApi.on('product-added', async ({ product }) => {

  console.log(product);
  // { 
  //   sku,
  //   locale,
  //   currency,
  //   defaultVariationIndex,
  //   description,
  //   virtualTryOn?, // See 'provide-vto-data' event listener
  //   variations: [{
  //     sku,
  //     name,
  //     subtitle?,
  //     url?,
  //     imageUrls,
  //     price?,
  //     virtualTryOn?, // See 'provide-vto-data event listener
  //     ..., // See https://bambuser.com/docs/calls/provide-product-data for additional possible properties a product may have
  //   }]
  // }

});

product-removed

Permission: callsWidget:virtualTryOn.read / callsAgentTool:virtualTryOn.read
Context: Main app, Tool view

The product-removed event is fired when product is removed from the virtual try-on session

Event properties

• product Product representation for the product and all its variations that was removed from the virtual try-on session

Example

const { virtualTryOnApi } = await bambuserAppFramework.getContext();

virtualTryOnApi.on('product-removed', async ({ product }) => {

  console.log(product);
  // { 
  //   sku,
  //   locale,
  //   currency,
  //   defaultVariationIndex,
  //   description,
  //   virtualTryOn?, // See 'provide-vto-data' event listener
  //   variations: [{
  //     sku,
  //     name,
  //     subtitle?,
  //     url?,
  //     imageUrls,
  //     price?,
  //     virtualTryOn?, // See 'provide-vto-data event listener
  //     ..., // See https://bambuser.com/docs/calls/provide-product-data for additional possible properties a product may have
  //   }]
  // }

});

apply

Permission: callsWidget:virtualTryOn.read / callsAgentTool:virtualTryOn.read
Context: Main app, Tool view

The apply event is fired when anything in the the current set of applied items in the virtual try-on session are changed

Event properties

• items - Array with the current set of applied items in the virtual try-on session

Example

const { virtualTryOnApi } = await bambuserAppFramework.getContext();

virtualTryOnApi.on('apply', async ({ items }) => {
  console.log(items);
  // [{
  //   id,
  //   pattern,
  //   ..., // General product vto data 
  //   ..., // Specfific vto data for variation
  //   ..., // Any custom props that was provided in apply action
  // }]
});

Virtual try-on provider

A Virtual try-on provider adds capabilities to virtually apply products onto the shopper. It modifies the shopper’s camera track that is shown locally to the shopper which is also shared with the agent. The provider can also provide additional data to identify supported products that should be enabled for virtual try-on with the provider.

Virtual try-on provider events

init

Permission: Implicit via Virtual Try-on API
Context: Main app

The init event is fired when the provider should start to initialize, ex. load any additional assets

Event return value

A promise whose fulfillment handler will mark the provider as ready for usage by the Bambuser platform

Example

// ...

vtoProvider.on('init', async () => {
  // Load any additional assets
  // After this async method executes successfully the provider is marked as ready to be used
  // Throw error on failure
});

provide-vto-data

Permission: Implicit via Virtual Try-on API
Context: Main app

The provide-vto-data event is fired when an agent wants to activate virtual try-on during product lookup (via the product hydration / search merchant integration) and the required virtual try-on data wasn't provided. The virtual try-on data that your app provides will be used to indicate if product is enabled for virtual try-on and allow to configure the virtual try-on session like changing current color in the virtual try-on sidebar tool. You must provide the virtual try-on data for all the requested variation skus that you like to support, any omitted variation sku will not be enabled for virtual try-on and not be asked if they are supported or not again

info

When merchant product hydration integration is available a “Try-on” button will be available during product lookup allowing agents to easily activate the product for try-on. It will only be enabled if virtual try-on data is provided via product hydration directly or as a result from this event listener.

tip

The data returned by the event handler will be added in a virtualTryOn property on the product and it’s variations. This may later be accessed on both agent and shopper side with either .getProducts() or .on('product-added', ...) after the product have been added to the virtual try-on session

Event properties

• product - The product hydration representation of the product and all its variations to provide the additional virtual try-on data for

Event return value

A promise whose fulfillment handler will provide any additional virtual try-on data. When no data is provided the product will not be marked as supported by your virtual try-on provider

Example

// ...
vtoProvider.on('provide-vto-data', async ({ product }) => {
  // Send request to your backend to load all product variation data from sku or skus
  //
  // return {
  //   category: "lipstic",
  //   patterns: [{
  //     id: "matte",
  //     label: "Matte",
  //     imageURL: "https://example.com/lipstic-pattern-matte-preview.jpg",
  //   }],
  //   variations: [{
  //     sku: "lipAbcRed",
  //     color: {
  //       label: "Red",
  //       hexValue: "#ff0000", // Color will be displayed for the agent as a choice in a drop-down. It is not used for the actual stream modification.
  //     },
  //     finish: "metalic" // A category of the color, you can filter on this in the UI as an agent
  //     ..., // Any other custom props you like to add to the variation
  //  }, {
  //     sku: "lipAbcPink",
  //     color: {
  //       label: "Pink",
  //       hexValue: "#ff10f0",
  //     },
  //     finish: "metalic"
  //     ..., // Any other custom props you like to add to the variation
  //  }],
  //  ..., // Any other custom general props you like to add
  // };
  return null; // No data, product variation skus will not be enabled for virtual try-on by your app
});

provide-frame

Permission: Implicit via Virtual Try-on API
Context: Main app

The provide-frame event is fired for each camera frame that should be modified with the virtual try-on layer. This will only be fired when shopper camera is enabled, virtual try-on session is active and a new camera frame is ready to be processed. When a new camera frame becomes available and previous callback hasn’t completed yet such frame will be discarded and output camera frame rate be degraded

info

The current approach is to transfer video frames as ImageBitmaps to support lower versions of browsers, ex. iOS 15+

Event properties

• bitmap - The original ImageBitmap frame from shopper's camera

Event return value

A promise whose fulfillment handler will provide the modified frame as an ImageBitmap

Example

// ...

vtoProvider.on('provide-frame', async ({ bitmap }) => {
  // Do your GPU/CPU WASM virtual try-on thing
  // Return result as an ImageBitmap
});

pause

Permission: Implicit via Virtual Try-on API
Context: Main app

The pause event is fired when you should stop doing any processing that may be running ex. in background

Example

// ...

vtoProvider.on('pause', () => {
  // Stop doing any background processing that you may have running
});

resume

Permission: Implicit via Virtual Try-on API
Context: Main app

The resume event is fired when you should resume processing, ex. after pause

Example

// ...

vtoProvider.on('resume', () => {
  // Resume doing any background processing that you may want to be running
});

dispose

Permission: Implicit via Virtual Try-on API
Context: Main app

The resume event is fired when the virtual try-on provider will not be used by Bambuser anymore, ex. after call has ended

Example

// ...

vtoProvider.on('dispose', () => {
  // Stop doing any processing that you may have running
  // Teardown SDK
  // Release any other things tied to this virtual try-on provider as it will not be used anymore
});