Android: Picture-in-picture Mode

This article should give you an idea on how to implement native Android Picture-in-Picture (PiP) feature for conference calls in your VidyoClient-based application.

In fact, all the needed steps are already well documented in Android developers community article, but here we would like to outline what we have changed in our Android sample application in order to make it switch to PiP mode under certain conditions.

You can also refer to Android Kotlin PictureInPicture sample without Vidyo integration if you need toi dive deeper.

Declare PiP support

By default, the system does not automatically support PiP for apps. If you want support PiP in your app, register your video activity in your manifest by setting android:supportsPictureInPicture to true. Also, specify that your activity handles layout configuration changes so that your activity doesn't relaunch when layout changes occur during PiP mode transitions.

<activity
            android:name="com.vidyo.vidyoconnector.ui.MainActivity"
            ...
            android:supportsPictureInPicture="true"
            android:configChanges="screenSize|smallestScreenSize|screenLayout|orientation">
            ...
</activity>

Switch your activity to PiP

Starting with Android 12, you can switch your activity to PiP mode by setting the setAutoEnterEnabled flag to true. With this setting, an activity automatically switches to PiP mode as needed without having to explicitly call enterPictureInPictureMode() in onUserLeaveHint.

Check if PiP is supported on device

The following code will return whether PiP is supported on the current device:

private val isPipSupported by lazy {
    Build.VERSION.SDK_INT >= Build.VERSION_CODES.N &&
        packageManager.hasSystemFeature(PackageManager.FEATURE_PICTURE_IN_PICTURE)
}

Observe PiP

When the MainActivity is created, we have to add a logic to observe the PiP mdoe and track all the actions related to it. Add the following to the onCreate function of MainActivity class:

if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
    checkAndObservePip()
}

And here is how it actually functions:

@RequiresApi(Build.VERSION_CODES.O)
private fun checkAndObservePip(){
    if(isPipSupported) {
        lifecycleScope.launch {
            repeatOnLifecycle(Lifecycle.State.STARTED) {
                trackPipAnimationHintView(ConnectorManager.layout)
            }
        }
        observePipActionsAndParams()
        observeConferenceCallAction()
    }
    logD { "$logTag, checkAndApplyPip isPipSupported: $isPipSupported" }
}

Adding observers to handle PIP actions click event and any update related to PiP actions like mic mute/unmute etc:

@RequiresApi(Build.VERSION_CODES.O)
private fun observePipActionsAndParams(){
    logD { "$logTag, observePipActionsAndParams"}
    pipActionTypeLiveData.observe(this) { 
        viewModel.onPipActionReceived(it) 
    }
    viewModel.pipParamsLiveData.observe(this) { 
        (isMicroPhoneMute, isCameraMute) ->
            doOnPipParamsReceived(isMicroPhoneMute, isCameraMute)
    }
}

Observe Conference Call State to check if user is in PiP mode in conference end then exit from PiP mode:

private fun observeConferenceCallAction(){
    ConnectorManager.conference.conference.collectInScope(lifecycleScope) {
        val isPipActive = viewModel.pipModeActive.value
        logD { "$logTag, observeConferenceCallAction Conference State: ${it.state}, isPipActive: $isPipActive" }
        if (!it.state.isActive && isPipActive == true){
            exitFromPipMode()
        }
    }
}

Define PiP Actions

You can and should define which actions will be available on PiP window:

sealed class PipAction(@DrawableRes val iconResId: Int, @StringRes val titleResId: Int, @PipControlType val controlType: Int){
    object MicrophoneMute :PipAction(R.drawable.ic_microphone_off, R.string.CONFERENCE__contentdesc_mic_muted, CONTROL_TYPE_MICROPHONE_MUTE)
    object MicrophoneUnMute :PipAction(R.drawable.ic_microphone_on, R.string.CONFERENCE__contentdesc_mic_unmuted, CONTROL_TYPE_MICROPHONE_UN_MUTE)
    object CameraMute :PipAction(R.drawable.ic_camera_off, R.string.CONFERENCE__contentdesc_camera_muted, CONTROL_TYPE_CAMERA_MUTE)
    object CameraUnMute :PipAction(R.drawable.ic_camera_on, R.string.CONFERENCE__contentdesc_camera_unmuted, CONTROL_TYPE_CAMERA_UN_MUTE)
    object EndCall :PipAction(R.drawable.ic_call_end_24, R.string.CONFERENCESERVICE__notification_end_call, CONTROL_TYPE_END_CALL)
}

PiP Management

We have encapsulated all PiP mode support management into PipManager.kt file. The main part here is building PiP parameters that will rule how to react on previously defined actions:

@RequiresApi(Build.VERSION_CODES.O)
fun buildPictureInPictureParams(isMicroPhoneMute: Boolean, isCameraMute: Boolean): PictureInPictureParams.Builder {
    logD { "$logTag, buildPictureInPictureParams: isMicroPhoneMute = $isMicroPhoneMute, isCameraMute = $isCameraMute" }

    return if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.S) {
                PictureInPictureParams.Builder()
                // Set action items for the picture-in-picture mode. These are the only custom controls
                // available during the picture-in-picture mode.
                .setActions(createPipActions(isMicroPhoneMute, isCameraMute))
                // Set the aspect ratio of the picture-in-picture mode.
                .setAspectRatio(pipRational)
                // if TRUE, Turn the screen into the picture-in-picture mode if it's hidden by the "Home" button.
                .setAutoEnterEnabled(false)
                // Disables the seamless resize. The seamless resize works great for videos where the
                // content can be arbitrarily scaled, but you can disable this for non-video content so
                // that the picture-in-picture mode is resized with a cross fade animation.
                .setSeamlessResizeEnabled(false)
        } else {
            PictureInPictureParams.Builder()
            // Set action items for the picture-in-picture mode. These are the only custom controls
            // available during the picture-in-picture mode.
            .setActions(createPipActions(isMicroPhoneMute, isCameraMute))
            // Set the aspect ratio of the picture-in-picture mode.
            .setAspectRatio(pipRational)
     }
}

Handle PiP

Here are some outlines on what you should be handling in termsof PiP actions.

Handle PiP mode changes:

override fun onPictureInPictureModeChanged(
        isInPictureInPictureMode: Boolean,
        newConfig: Configuration
    ) {
        logD { "$logTag, onPictureInPictureModeChanged isInPictureInPictureMode: $isInPictureInPictureMode, isPipEnabled: ${appContext.isPipEnabled}" }
        if(appContext.isPipEnabled) {
            viewModel.onPictureInPictureModeChanged(isInPictureInPictureMode)
            doOnPictureInPictureModeChanged(isInPictureInPictureMode)
        }
        super.onPictureInPictureModeChanged(isInPictureInPictureMode, newConfig)
}

Add broadcast receiver to listen to PiP actions:

@SuppressLint("UnspecifiedRegisterReceiverFlag")
private fun doOnPictureInPictureModeChanged(isInPictureInPictureMode: Boolean){
    logD { "$logTag, onPictureInPictureModeChanged isInPictureInPictureMode: $isInPictureInPictureMode" }
    if (isInPictureInPictureMode) {
        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.TIRAMISU) {
            registerReceiver(
                    pipActionReceiver, IntentFilter(ACTION_PIP_CONTROL),
                    Context.RECEIVER_EXPORTED
            )
        } else {
            registerReceiver(pipActionReceiver, IntentFilter(ACTION_PIP_CONTROL))
        }
    } else {
        unregisterPipActionReceiver()
    }
}

Unregister PiP actions receiver when exiting PiP mode (used in Observe PiP):

private fun exitFromPipMode(){
    if (isPipSupported && viewModel.pipModeActive.value == true) {
        unregisterPipActionReceiver()
        apply {
            val startIntent = intent.apply {
                addFlags(Intent.FLAG_ACTIVITY_REORDER_TO_FRONT)
                putExtra(ACTION_PIP_CONTROL, true)
            }
            startActivity(startIntent)
        }
    }
}