@pexip/media-control

Abstract away all the quirks of handling media devices

A library that will handle media by wrapping enumerateDevices and getUserMedia extending both functions with additional logic.

Install

npm install @pexip/media-control

Table of Contents

• Introduction
• Installation
• Usage

Introduction

This library is meant to extend the mediaDevices api methods such as getUserMedia, enumerateDevices and events on media streams and tracks.

The final goal is to have a library that gives a stable and robust way to use these methods while guaranteeing that the developer has better control over which devices are delivered, fail, or exist.

Installation

The package is published to Github registry scoped with @pexip. Access right to @pexip is needed in order to install the package.

Here is the steps to setup Github Personal Access Token for npm/yarn in order to add the package.

1. A Github Personal Access Token is needed to be setup in order to get the @pexip/media-control package. Follow Github's doc to create a Personal Access Token.

2. When the Personal Access Token is ready, you can set it up for npm or yarn via .npmrc. By adding your personal access token to your ~/.npmrc file, edit the ~/.npmrc file for your project to include the following line, replacing TOKEN with your personal access token. Create a new ~/.npmrc file if one doesn't exist.

   //npm.pkg.github.com/:_authToken=TOKEN

3. Add a local .npmrc file to your repository with the following lines to tell the package manager, npm or yarn where the registry is for @pexip scoped packages.

   @pexip:registry=https://npm.pkg.github.com

4. Now it is ready to add any @pexip/PACKAGE_NAME package to your repository.

   # Use yarn
   yarn add @pexip/media-control
   
   # Or use npm
   npm install @pexip/media-control

P.S.: there is no polyfill added to this package, the user needs to add polyfill to the application, i.e. core-js. We leverage on @babel/plugin-transform-runtime and @babel/runtime to accomplish this.

Usage

Get user media

Get stream with audio and video without specifying devices, let the browser choose the right devices.

import {getUserMedia} from '@pexip/media-control';

const stream = await getUserMedia({audio: true, video: true});

Only get stream with video

import {getUserMedia} from '@pexip/media-control';

// neglecting audioInput is the same as setting it explicitly to false
const stream = await getUserMedia({video: true});

Get stream with the specific device only but any for audio.

import {getUserMedia} from 'media-control';

const videoInputWanted = {
  deviceId: '0abbc454-71e5-4f7e-9c8b-9de36f1334c3',
  kind: 'videoinput',
  label: 'Camera',
};

const stream = await getUserMedia({
  audio: true,
  video: {device: {exact: videoInputWanted}},
});

Get stream with specific video input and fallback if it's not available and the specific audio input only, no fallback

import {getUserMedia} from 'media-control';

const audioInput = {
  deviceId: 'a373b235-3ebf-40c7-b619-a491e05e35e6',
  label: "Rick's Microphone",
  kind: 'audioinput',
};
const videoInput = {
  deviceId: '0abbc454-71e5-4f7e-9c8b-9de36f1334c3',
  label: "Morty's Camera",
  kind: 'videoinput',
};

const stream = await getUserMedia({
  audio: {device: {exact: audioInput}},
  video: videoInput,
});

Get stream with specific devices and fallback when they are not available

import {getUserMedia} from 'media-control';

const audioInput = {
  deviceId: 'a373b235-3ebf-40c7-b619-a491e05e35e6',
  label: "Rick's Microphone",
  kind: 'audioinput',
};
const videoInput = {
  deviceId: '0abbc454-71e5-4f7e-9c8b-9de36f1334c3',
  label: "Morty's Camera",
  kind: 'videoinput',
};

const stream = await getUserMedia({
  audio: audioInput,
  video: videoInput,
});

The following requests both audio and video without any specific requirements:

import {getUserMedia} from 'media-control';

const stream = await getUserMedia({audio: true, video: true});

The following expresses a preference for 1280x720 camera resolution:

import {getUserMedia} from 'media-control';

const stream = await getUserMedia({
  audio: true,
  video: {
    width: 1280,
    height: 720,
  },
});

The following expresses a preference for user facing camera:

import {getUserMedia} from 'media-control';

const stream = await getUserMedia({
  audio: true,
  video: {facingMode: 'user'},
});

Subscribe media events

import {MediaEventType, subscribe} from 'media-control';

subscribe((event: MediaEvent) => {
  switch (event.detail.type) {
    case MediaEventType.Mute: // {id: string}
      // id of the device that was muted
      const id = event.detail.id;
      break;
    case MediaEventType.Unmute: // {id: string}
      // id of the device that was unmuted
      const id = event.detail.id;
      break;
    case MediaEventType.Ended: // {id: string}
      // id of the device that ended
      const id = event.detail.id;
      break;
    case MediaEventType.DevicesChanged: // {devices: MediaDeviceInfo[]}
      // devices currently available through enumerateDevices
      const devices = event.detail.devices;
      break;
    case MediaEventType.DevicesFound: // {devices: MediaDeviceInfo[], authorizedDevices: MediaDeviceInfo[]}
      // new devices connected
      const devices = event.detail.devices;
      break;
    case MediaEventType.DevicesLost: // {devices: MediaDeviceInfo[], authorizedDevices: MediaDeviceInfo[]}
      // devices lost from the enumerateDevices list
      const devices = event.detail.devices;
      break;
    case MediaEventType.DeviceLost: // {device: MediaDeviceInfo}
      // device currently in use that was lost from the enumerateDevices list
      const device = event.detail.device;
      break;
    case MediaEventType.DevicesUnauthorized: // {devices: MediaDeviceInfo[], authorizedDevices: MediaDeviceInfo[]}
      // Unauthorized list of devices
      const device = event.detail.devices;
      break;
    case MediaEventType.NoInputDevices:
      // Indicate that there is no input devices from the machine
      break;
    case MediaEventType.Error: // {error: Error}
      const error = event.detail.error;
      break;
  }
});

media-control is meant to extend the mediaDevices api methods such as getUserMedia, enumerateDevices and events on media streams and tracks. The final goal is to have a library that gives a stable and robust way to use these methods while guaranteeing that the developer has better control over which devices are delivered, fail, or exist.

Enumerations

Enumeration	Description
MediaEventType	MediaStreamTrack Events
MediaDeviceKinds	an enum of MediaDeviceKind which can be reused
MediaDeviceFailure	An enum of failure events for media device

Interfaces

Interface	Description
MediaControl	MediaControl Interface
IndexedDevices	-
InputConstraintSet	-
MediaDeviceRequest	The request constraints that applies to getMediaStream
MediaInput	Holds user media input devices
StreamTrackEventHandlers	An object contains the event handlers
InputDevicePermission	-
DisplayMediaOptions	The DisplayMediaStreamOptions dictionary is used to instruct the user agent what sort of MediaStreamTracks may be included in the MediaStream returned by getDisplayMedia.

Type Aliases

Type Alias	Description
DisplayCaptureSurfaceType	The DisplayCaptureSurfaceType enumeration describes the different types of display surface
CursorCaptureConstraint	The CursorCaptureConstraint enumerates the conditions under which the cursor is captured.
IncludeExcludeConstraint	-
DeviceChangedChanges	DeviceChange Event
Events	Event object by MediaEventType
MediaEvent	Custom Media Event
MediaDeviceInfoLike	A striped version of MediaDeviceInfo
InputDeviceConstraint	-
FacingMode	-
MediaStreamTrackLike	Lesser strict type of MediaStreamTrack
Unsubscribe	-

Variables

Variable	Description
DISPLAY_CAPTURE_SURFACE_TYPE	The DisplayCaptureSurfaceType enumeration describes the different types of display surface
CURSOR_CAPTURE_CONSTRAINT	The CursorCaptureConstraint enumerates the conditions under which the cursor is captured.
INCLUDE_EXCLUDE_CONSTRAINT	-
DEVICE_ID_SEPARATOR	-
findAudioInputDevices	Find in the list of devices only the audio input ones
findVideoInputDevices	Find in the list of devices only the video input ones
findAudioOutputDevices	Find in the list of devices only the audio output ones
getUserMedia	Get MediaStream with provided input constraints
setDefaultConstraints	Set default media stream constraints
subscribe	Subscribe media events
isAudioInput	-
isVideoInput	-
isAudioOutput	-

Functions

Function	Description
isExactDeviceConstraint	Check if provided constraint is an exact device constraint
mergeConstraints	Merge base constraints with provided base constraints and another constraints
extractConstrainDevice	-
getValueFromConstrainNumber	-
getFacingModeFromConstraintString	-
resolveMediaDeviceConstraints	Resolve constraints and mediaTrackConstraints by checking the type and try to return the best possible from mediaTrackConstraints.
applyConstraints	Call applyConstraints foreach track accordingly
extractConstraintsWithKeys	Extract the constraints with provided keys
findDeviceFromConstraints	Find device from the device list with provided constraints
relaxInputConstraint	Relax input constraints to enable looking up the device by deviceId as well as label as a fallback as a best effort to get a similar device when possible. If the the device is found we will use the exact constraint otherwise we will use the ideal.
toKey	Convert provided info to key for Map
toMediaDeviceInputKind	-
getDevices	Future proofing: in case we want to alter the values or type returned by enumerateDevices
createTrackDevicesChanges	-
deviceChanged	Unified interface for subscribing DeviceChangedChanges Event
toMediaDeviceInfoLike	Convert MediaStreamTrack to MediaDeviceInfoLike
toMediaDeviceInfo	-
findMediaInputFromMediaStreamTrack	Find the MediaDeviceInfo from provided MediaDeviceInfo[] by comparing with provided MediaStreamTrack
findMediaInputFromStream	Find media input from media stream
isRequestedResolution	-
isRequestedInputDevice	-
isRequestedInputTrack	Compare request and the input to see if the request has been fulfilled
hasRequestingDevice	Check if there is a requesting device with respecting to the current devices
isTrackMutedOrEnded	Check if the track is muted or 'ended'
shouldRequestDevice	Decide if we should send a new gUM request
isStreamingRequestingDevice	-
isStreamingRequestedDevicesBase	-
isStreamingRequestedDevices	Check if provided request has already been fulfilled
findPermissionGrantedDevices	Find in the list of devices which has permissions granted
getInputDevicePermissionState	A wrapper for navigator.permissions.query with fallback to use navigator.mediaDevices.enumerateDevices to guess the PermissionState
findCurrentAudioOutputId	Find current audio output id to be used to set as sinkId
findCurrentVideoInputDeviceIdFromStream	Find videoinput device id in the stream
findDeviceWithDeviceId	Finds device with given deviceId
findDevicesByKind	Find in the list of devices with given MediaDeviceKind
muteStreamTrack	Set MediaStreamTrack['enabled'] according to mute param for the provided stream
stopMediaStream	Stops all tracks in the given stream
areTracksEnabled	Checks that tracks with a given type are enabled in the stream
hasAudioOrVideoInputs	Check if list contains media inputs
hasAudioInputs	Check if list contains audio inputs
hasVideoInputs	Check if list contains video inputs
hasAnyGrantedInput	Check if provided devices have any granted input device
hasAnyInputs	Check if provided devices have any video and audio inputs
hasChangedInput	An utility function to check the input device has been changed
areMultipleFacingModeSupported	An utility function to check if facing mode is supported by interpreting the device label and the API getSupportedConstraints
interpretCurrentFacingMode	Interpret the current facing mode from the provided track, and try to get the mode from settings, or use the label to guess the facing mode when facingMode is not supported from settings
limitSharingMonitorInterface	A fallback implementation of monitorTypeSurfaces limitation when it is not supported by the current browser and the source of the displaySurface is available.
setLogger	-
createStreamTrackEventSubscriptions	Create a MediaStreamTrack's native events subscription
isMediaDeviceInfo	Check if provided is MediaDeviceInfo
isMediaDeviceInfoArray	-
isInputConstraintSet	-
isMediaStreamTrack	Check if provided is MediaStreamTrack
isFacingMode	-
isIndexedDevices	-
isDeviceGranted	Check if provided device is a permission-granted device by inspecting the device label
getDeviceId	Create a device ID from provided device. The device ID here is in the format of deviceId:kind, be default.
reverseDeviceId	Reverse the device ID to its original form i.e. MediaDeviceInfo The device ID here is in the format of groupId:deviceId:kind, be default.
createIndexedDevices	Create an IndexedDevices construct from provided devices which index the device by device ID or kind.
compareDevices	Compare 2 devices to see if they are the same by deviceId or label as a fallback
findDevice	Lookup the device from devices list