35. Video Decode and Encode Operations

Vulkan implementations can expose video decode and encode engines, which are independent from the graphics and compute engines. Video decode and encode is performed by recording video operations and submitting them to video decode and encode queues. Vulkan provides core support for video decode and encode and can support a variety of video codecs through individual extensions built on the core video support.

The subsections below detail the fundamental components and operation of Vulkan video.

35.1. Technical Terminology and Semantics

35.1.1. Video Picture Resources

Video Picture Resources contain format information, can be multidimensional and may have associated metadata. The metadata can include implementation-private details required for the decode or encode operations and application managed color-space related information.

In Vulkan, a Video Picture Resource is represented by a VkImage. The VkImageView, representing the VkImage, is used with the decode operations as Output and Decoded Picture Buffer (DPB), and with the encode operation as Input and Reconstructed Video Picture Resource.

35.1.2. Reference Picture

Video Reference Picture is a Video Picture Resource that can be used in the video decode or encode process to provide predictions of the values of samples in the subsequently decoded or encoded pictures.

35.1.3. Decoded Output Picture

The pixels resulting from the video decoding process are stored in a Decoded Output Picture, represented by a VkImageView. This can be shared with the Encoder Reconstructed or Decoder DPB Video Picture Resources. It can also be used as an input for Video Encode, Graphics, Compute processing, or WSI presentation.

35.1.4. Input Picture to Encode

The primary source of input pixels for the video encoding process is the Input Picture to Encode, represented by a VkImageView. This can be shared with the Encoder Reconstructed or Decoder DPB Video Picture Resources. It can be a direct target of Video Decode, Graphics, Compute processing, or WSI presentation.

35.1.5. Decoded Picture Buffer (DPB)

Previously decoded pictures are used by video codecs to provide predictions of the values of samples in the subsequently decoded pictures. At the decoder, such Video Picture Resources are stored in a Decoded Picture Buffer (DPB) as an indexed set of Reference Pictures.

35.1.6. Reconstructed Pictures

An integral part of the video decoding pipeline is the reconstruction of pictures from the compressed stream. A similar stage exists in the video encoding pipeline as well. Such reconstructed pictures may be used as Reference Pictures for subsequently decoded or encoded pictures. The correct use of such Reference Pictures is driven by the video compression standard, the implementation, and the application-specific use cases.

This specification refers to the collection of the Decoded Picture Buffer and Reconstructed Pictures as Decoded Picture Buffer (DPB) Set, or only, DPB.

35.1.7. Decoded Picture Buffer (DPB) Slot

Decoded Picture Buffer (DPB) Slot represents a single or multi-layer indexed Reference Picture’s entry within the Video Session’s DPB Set. A valid DPB Slot index starts from zero and goes up to the maximum of N - 1, where N is the number of Reference Picture entries requested for a Video Session.

35.1.8. Reference Picture Metadata

The opaque DPB Slot state managed by the implementation may contain Reference Picture Metadata, present when the picture resource associated with the DPB Slot is used as a reference picture in one or more video decode or encode operations.

An implementation or application may have other Picture Metadata related to the Video Picture Resource or the DPB Slot, but such data is outside the scope of this specification.

Note:

The video decode or encode implementation does not maintain internal references to the Reference Pictures, beyond the Reference Picture Metadata. It is the responsibility of the Vulkan Application to create, manage, and destroy, as well as to provide those Video Picture Resources, when required, during the decoding or encoding process.

35.1.9. Color Space Metadata

Color Space Metadata is the additional static or dynamic state associated with a Video Picture Resource specifying the color volume (the color primaries, white point, and luminance range) of the display that was used in mastering the video content. The use of Color Space Metadata is outside the scope of the current version of the video core specification.

35.2. Introduction

This chapter discusses extensions supporting Video Decode or Encode operations. Video Decode and Encode operations are supported by queues with an advertised queue capability of VK_QUEUE_VIDEO_DECODE_BIT_KHR and VK_QUEUE_VIDEO_ENCODE_BIT_KHR, respectively. Video Decode or Encode queue operation support allows for Vulkan applications to cooperate closely with other graphics or compute operations seamlessly and efficiently, therefore improving the overall application performance.

35.2.1. Video Decode Queue

VK_KHR_video_decode_queue adds a video decode queue type bit VK_QUEUE_VIDEO_DECODE_BIT_KHR to VkQueueFlagBits. As in the case of other queue types, an application must use vkGetPhysicalDeviceQueueFamilyProperties to query whether the physical device has support for the Video Decode Queue. When the implementation reports the VK_QUEUE_VIDEO_DECODE_BIT_KHR bit for a queue family, it advertises general support for Vulkan queue operations described in Devices and Queues.

35.2.2. Video Encode Queue

VK_KHR_video_encode_queue adds a video encode queue type bit VK_QUEUE_VIDEO_ENCODE_BIT_KHR to VkQueueFlagBits. As in the case of other queue types, an application must use vkGetPhysicalDeviceQueueFamilyProperties to query whether the physical device has support for the Video Encode Queue. When the implementation reports the VK_QUEUE_VIDEO_ENCODE_BIT_KHR bit for a queue family, it advertises general support for Vulkan queue operations described in Devices and Queues.

The rest of the chapter focuses, specifically, on Video Decode and Encode queue operations.

35.2.3. Video Session

Before performing any video decoding or encoding operations, the application must create a Video Session instance, of type VkVideoSessionKHR. A Video Session instance is an immutable object and supports a single compression standard (for example, H.264, H.265, VP9, AV1, etc.). The implementation uses the VkVideoSessionKHR object to maintain the video state for the video decode or video encode operation. A Video Session instance is created specifically:

  • For a particular video compression standard;

  • For video decoding or video encoding;

  • With maximum supported decoded or encoded picture width/height;

  • With the maximum number of supported DPB or Reconstructed Pictures slots that can be allocated;

  • With the maximum number of Reference Pictures that can be used simultaneously for video decode or encode operations;

  • Codec color and features profile;

  • Color Space format description (not supported with this version of the specification);

VkVideoSessionKHR represents a single video decode or encode stream. For each concurrently used stream, a separate instance of VkVideoSessionKHR is required. After the application has finished with the processing of a stream, it can reuse the Video Session instance for another, provided that the configuration parameters between the two usages are compatible (as determined by the video compression standard in use). Once the VkVideoSessionKHR instance has been created, the video compression standard and profiles, Input / Output / DPB formats, and the settings like the maximum extent cannot be changed.

The values of the following VkVideoSessionKHR parameters can be updated each frame, subject to the restrictions imposed on parameter updates by the video compression standard in use:

The updated parameters must not exceed the maximum limits specified when creating the VkVideoSessionKHR instance.

35.2.4. Video Session Device Memory Heaps

After creating a Video Session instance, and before the object can be used for any of the decode or encode operations, the application must allocate and bind device memory resources to the Video Session object. An implementation may require one or more device memory heaps of different memory types, as reported by the vkGetVideoSessionMemoryRequirementsKHR function, to be bound with the vkBindVideoSessionMemoryKHR function to the Video Session, For more information about the Video Session Device Memory, please refer to the Binding the Session Object Device Memory section, below.

35.2.5. Video Session Parameters

A lot of codec standards require parameters that are in use for the entire video stream. For example, H.264/AVC and HEVC standards require sequence and picture parameter sets (SPS and PPS) that apply to multiple Video Decode and Encode frames, layers, and sub-layers. Vulkan Video uses Video Session Parameters objects to store such standard parameters. The application creates one or more Video Session Parameters Objects against a Video Session, with a set of common Video Parameters that are required for the processing of the video content. During the object creation, the implementation stores the parameters to the created instance. During command buffer recording, it is the responsibility of the application to provide the Video Session Parameters object containing the parameters that are necessary for the processing the portion of the stream under consideration.

35.2.6. Video Picture Subresources

For Video Picture Resources, an application has the option to use single or multi-layer images for image views. The layer to be used during decode or encode operations can be specified when the image view is being created with the VkImageSubresourceRange::baseArrayLayer parameter, and/or within the resource binding operations in command buffer by using the VkVideoPictureResourceKHR::baseArrayLayer parameter.

Note:

Both Video Decode and Encode operations only work with a single layer at the time.

The Image views representing the Input / Output / DPB Video Picture Resources could have been created with sizes bigger than the coded size that is used with Video Decode and Encode operations. This allows for the same Video Picture Resources to be reused when there is a change in the input video content resolution. The effective coded size of the Video Picture Resources used for Video Decode and Encode operations is provided with VkVideoPictureResourceKHR::extent parameter of each resource in use.

Note:

Many codec standards require the coded and Video Picture Resources' sizes to match.

Video Session DPB and Reconstructed Video Picture Resources

The video compression standard chosen may require the use of Reference Pictures. In Vulkan Video, like any other Video Picture Resources, the Reference Pictures are represented with Image Views.

When an application requires Reference Picture Resources, it creates and then associates image views, representing these resources, with Video Session DPB or Reconstructed slots while recording the command buffer.

Decoded output pictures may be used as reference pictures in future video decode operations. The same pictures may be used in texture sampling operations or in the (WSI) presentation pipeline. Representing the DPB’s Video Picture Resources by image views makes it possible to accommodate all these use cases in a “zero-copy” fashion. Also, it provides more fine-grained control of the application over the efficient usage of the DPB and Reconstructed Device Memory Resources.

Video Session DPB and Reconstructed Slot Resource Management

Before Video Picture Resources can be used as Reference Picture Resources, Video Session DPB or Reconstructed Slots must be associated with those resources.

The application allocates a DPB or Reconstructed Slot and associates it with a Video Picture Resource and then sets up the resource as a target of decode or encode operation. After successfully decoding or encoding a picture with the targeted DPB or Reconstructed Slot , in addition to the Reference Picture pixel data, the implementation may generate an opaque Reference Picture Metadata for that video session Slot and its associated Video Picture Resource.

Subsequently, one or more DPB or Reconstructed video session Slots, along with their associated Video Picture Resources, can be used as Reference Picture’s source for the video decode or encode operations.

If Reference Pictures were to be required for decoding or encoding of the video bitstream, the VkVideoSessionCreateInfoKHR::maxReferencePicturesSlotsCount must be set to a value bigger than 0 when the instance of the Video Session object is created.

Up to VkVideoSessionCreateInfoKHR::maxReferencePicturesSlotsCount slots can be activated with Video Picture Resources for a video session and up to VkVideoSessionCreateInfoKHR::maxReferencePicturesActiveCount active slots can be used as DPB or Reconstructed Reference Pictures within a single decode or encode operation.

When the implementation is associating Reference Picture Metadata with the Video Picture Resources themselves, such data must be independent of the Video Session to allow for those Video Picture Resources to be shared with other Video Session instances. All of the Video Session-dependent Reference Picture Metadata must only be associated with the Video Session DPB or Reconstructed Slots.

The application with the help of the implementation is responsible for managing the individual DPB, or Reconstructed Slots that belong to a single Video Session DPB set:

The application also manages the mapping between the codec-specific picture IDs and DPB Slots.

When a Video Picture is decoded and is set as a Reference Picture against a Video Session DPB Slot, or is encoded and a Reconstructed Video Picture Resource is associated with a Video Session DPB Slot then:

When a DPB’s Slot is deactivated, or a different Video Picture Resource is used with the Slot, or the content of the Video Picture Resource is modified, the Reference Picture Metadata associated with the DPB Slot gets invalidated by the implementation. Subsequent attempts to use such, invalidated, DPB Slot as a Reference source would produce undefined results.

Video Session DPB Slot subresources

DPB Reference Picture’s coded width and height can change, dynamically, via VkVideoPictureResourceKHR::extent, and the picture parameters from the codec-specific extensions. When a DPB Slot is activated as a Reference Picture and a decode or encode operation is performed against that slot, the coded extent can be recorded by the implementation to the corresponding DPB Slot’s metadata state. Subsequently, when the Reference Pictures are used with the decoded Output or encoded Input Picture, their coded extent can differ. Decoding or encoding pictures, using picture sizes, different from the previously produced Reference Pictures should be used with care, not to conflict with the codec standard and the implementation’s support for that. It is the responsibility of the application to ensure that valid DPB Set of Reference Pictures are in use, according to the codec standard.

In addition, the Video Picture Resources extent cannot exceed the VkVideoSessionCreateInfoKHR::maxCodedExtent.

Note:

Coding Standards such as VP9 and AV1 allow for images with different sizes to be used as Reference Pictures. Others, like H.264 and H.265, do not support Reference Pictures with different sizes. Using Reference Pictures with incompatible sizes with such standards would render undefined results.

The application is in control of the allocation and use of the system resource

In Vulkan Video, the application has complete control over how and when system resources are used. The Vulkan Video framework provides the following tools to ensure that device and host memory resources are used in an optimal way:

Using DPB and Reconstructed Slot’s Associated Resources

Before a DPB Slot is to become Valid for use with a Reference Picture, it requires memory resources to be bound to it.

Some of the memory resources required for the DPB Slot, are opaquely managed by the implementation and, internally, allocated from the Session’s Device Memory Heaps. The application provides the image resources of one or more Reference Pictures, in the VkVideoBeginCodingInfoKHR::pReferenceSlots as part of the vkCmdBeginVideoCodingKHR command.

If a DPB Slot was already used with an image view, and a new image view or a VK_NULL_HANDLE handle is used with that Slot, then the DPB Slot’s state will be invalidated by the implementation. If a DPB Slot were to be reused with the same image view, the state of the Slot would not change.

Video Session Activating DPB Slot as a Reference

Before a DPB Slot is to be used for a Reference Pictures index, it must be activated. The activation of a DPB Slot is done within the vkCmdDecodeVideoKHR command’s VkVideoDecodeInfoKHR::slotIndex field for the decode operations, and within the vkCmdEncodeVideoKHR command’s VkVideoEncodeInfoKHR::slotIndex field for the encode operations.

While activating a Slot for DPB, it must already have an associated image view, within the VkVideoBeginCodingInfoKHR::pReferenceSlots in the vkCmdBeginVideoCodingKHR command and Device Memory backing of the the image resources must be resident.

When a DPB Slot were to be activated, the VkVideoDecodeInfoKHR::slotIndex for decode, or VkVideoDecodeInfoKHR::slotIndex for encode, must be set to the application’s allocated DPB Slot’s index. When activating a DPB Slot, the application will perform a decode or encode operation against its Slot’s index in order to enable its state as a Valid Picture Reference. If a DPB Slot is activated, but a decode or encode operation is not performed against that Slot’s index, or the decode or encode operation was unsuccessful, then the DPB Slot would remain in the Invalid Picture Reference state (see below the DPB Slot States).

By just providing a Video Picture Resources for a DPB Slot within the VkVideoBeginCodingInfoKHR::pReferenceSlots, and without successfully performing a decode or encode operation against that Slot, the DPB Slot’s state cannot be changed to Valid Picture Reference. If the DPB Slots were already in Valid Picture Reference, and there is no Video Picture Resources associated with the DPB Slot for a decode or encode operation, the state DPB Slot would not change. However, if an application is referring to a valid DPB Slot in its current decode or encode operations, then a valid image view must be provided for that Slot within VkVideoPictureResourceKHR::imageViewBinding for that decode or encode operation.

Video Session Invalidating DPB Slot’s Reference State

When a DPB Slot is invalidated, its state is set to Invalid Picture Reference. Using a DPB Slot as a Reference Picture index for video decode or encode operations while the Slot is in Invalid Picture Reference state would render undefined results.

Video Session DPB Slot States

To help understand the valid use of the Video Session DPB and its resource management, this section aims to explain the different states and correct usage of DPB Slots.

There are four (4) states that a DPB Slot could be in:

  • Picture Reference Unused;

  • Invalid Picture Reference;

  • Updating Picture Reference;

  • Valid Picture Reference;

The different states are outlined within the DPB Slot States and DPB Slot States Flow Diagram below.

All DPB Slot management operations are performed via the VkVideoDecodeInfoKHR::slotIndex or VkVideoEncodeInfoKHR::slotIndex field.

All DPB resource binding, invalidating, and activating Slot management operations are performed, by the implementation, before the decoding or encoding commands, based on the VkVideoDecodeInfoKHR::slotIndex or VkVideoEncodeInfoKHR::slotIndex field and the entries from the VkVideoBeginCodingInfoKHR::pReferenceSlots. The application cannot move a DPB Slot from a Picture Reference Unused to Updating Picture Reference state, implicitly, within a decode or encode command operation. Such a DPB Slot must first be transitioned to an Invalid Picture Reference state using VkVideoDecodeInfoKHR::slotIndex or VkVideoEncodeInfoKHR::slotIndex, as part of a decode command. For more details, see Video Picture Decode Modes.

When using sparse memory resources, it would be acceptable and valid behavior for the application to unbind the memory while the DPB Slot is any of the DPB Slot states, provided the command buffers, in a pending state, do not reference any such Video Picture Resources.

Accessing unbound regions of the sparse memory resources by the decoder or encoder, regardless if those are used as Output, Input, DPB or Reconstructed Video Picture Resources, would render undefined results. The VkPhysicalDeviceSparseProperties::residencyNonResidentStrict property reported by the implementation does not offer guarantees on the behavior of decode or encode operations when it comes to accessing unbound regions. However, both reads and writes are still considered safe and will not affect other resources or populated regions of the image.

Table 37. Video Session DPB Slot States
DPB Slot State Moving to DPB Slot State Exiting DPB Slot State Retain Video Picture Resource Memory

Picture Reference Unused

* Bind Device Memory;
* Reset decode or encode state;
* Invalidate, delete or unbind memory of a Picture Reference associated with Reference DPB Slot

* Activate Reference DPB Slot → Invalid Picture Reference

Application Controlled

Invalid Picture Reference

* Activate Reference DPB Slot;
* Unsuccessful video decode or encode operation;

* Start decode or encode operation with an active Reference DPB Slot target → Updating Picture Reference;
* Updating a Picture Resource outside the decoder or encoder or deleting or removing the memory binding(sparse) → Picture Reference Unused;

Application Controlled

Updating Picture Reference

Start decode or encode operation with an active Reference DPB Slot target;

Decode or encode operation with an active Reference DPB Slot target Completed Successfully → Valid Picture Reference;
Unsuccessful video decode or encode operation → Invalid Picture Reference

Yes

Valid Picture Reference

Video decode or encode operation with an active Reference DPB Slot target Completed Successfully;

* Replace Reference DPB Slot → Invalid Picture Reference;
* Invalidate, delete or unbind memory of a Picture Reference of the Reference DPB Slot → Picture Reference Unused;

Yes

Figure 25. DPB Slot States Flow Diagram

35.3. Video Physical Device Capabilities

35.3.1. Supported Video Codec Operations Enumeration

The structure VkVideoQueueFamilyProperties2KHR may be chained to VkQueueFamilyProperties2 when calling vkGetPhysicalDeviceQueueFamilyProperties2 to retrieve the video codec operations supported for the physical device queue family index.

The VkVideoQueueFamilyProperties2KHR structure is defined as:

// Provided by VK_KHR_video_queue
typedef struct VkVideoQueueFamilyProperties2KHR {
    VkStructureType                  sType;
    void*                            pNext;
    VkVideoCodecOperationFlagsKHR    videoCodecOperations;
} VkVideoQueueFamilyProperties2KHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • videoCodecOperations is a bitmask of VkVideoCodecOperationFlagBitsKHR specifying supported video codec operation(s).

Valid Usage (Implicit)
  • VUID-VkVideoQueueFamilyProperties2KHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_QUEUE_FAMILY_PROPERTIES_2_KHR

  • VUID-VkVideoQueueFamilyProperties2KHR-videoCodecOperations-parameter
    videoCodecOperations must be a valid combination of VkVideoCodecOperationFlagBitsKHR values

  • VUID-VkVideoQueueFamilyProperties2KHR-videoCodecOperations-requiredbitmask
    videoCodecOperations must not be 0

The codec operations are defined with the VkVideoCodecOperationFlagBitsKHR enum:

// Provided by VK_KHR_video_queue
typedef enum VkVideoCodecOperationFlagBitsKHR {
    VK_VIDEO_CODEC_OPERATION_INVALID_BIT_KHR = 0,
} VkVideoCodecOperationFlagBitsKHR;

Each decode or encode codec-specific extension extends this enumeration with the appropriate bit corresponding to the extension’s codec operation:

  • VK_VIDEO_CODEC_OPERATION_INVALID_BIT_KHR - No video operations are supported for this queue family.

// Provided by VK_KHR_video_queue
typedef VkFlags VkVideoCodecOperationFlagsKHR;

VkVideoCodecOperationFlagsKHR is a bitmask type for setting a mask of zero or more VkVideoCodecOperationFlagBitsKHR.

35.3.2. Video Profiles

A video profile is defined by VkVideoProfileKHR structure as:

// Provided by VK_KHR_video_queue
typedef struct VkVideoProfileKHR {
    VkStructureType                     sType;
    void*                               pNext;
    VkVideoCodecOperationFlagBitsKHR    videoCodecOperation;
    VkVideoChromaSubsamplingFlagsKHR    chromaSubsampling;
    VkVideoComponentBitDepthFlagsKHR    lumaBitDepth;
    VkVideoComponentBitDepthFlagsKHR    chromaBitDepth;
} VkVideoProfileKHR;
Valid Usage (Implicit)
  • VUID-VkVideoProfileKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_PROFILE_KHR

  • VUID-VkVideoProfileKHR-videoCodecOperation-parameter
    videoCodecOperation must be a valid VkVideoCodecOperationFlagBitsKHR value

  • VUID-VkVideoProfileKHR-chromaSubsampling-parameter
    chromaSubsampling must be a valid combination of VkVideoChromaSubsamplingFlagBitsKHR values

  • VUID-VkVideoProfileKHR-chromaSubsampling-requiredbitmask
    chromaSubsampling must not be 0

  • VUID-VkVideoProfileKHR-lumaBitDepth-parameter
    lumaBitDepth must be a valid combination of VkVideoComponentBitDepthFlagBitsKHR values

  • VUID-VkVideoProfileKHR-lumaBitDepth-requiredbitmask
    lumaBitDepth must not be 0

  • VUID-VkVideoProfileKHR-chromaBitDepth-parameter
    chromaBitDepth must be a valid combination of VkVideoComponentBitDepthFlagBitsKHR values

  • VUID-VkVideoProfileKHR-chromaBitDepth-requiredbitmask
    chromaBitDepth must not be 0

The video format chroma subsampling is defined with the following enums:

// Provided by VK_KHR_video_queue
typedef enum VkVideoChromaSubsamplingFlagBitsKHR {
    VK_VIDEO_CHROMA_SUBSAMPLING_INVALID_BIT_KHR = 0,
    VK_VIDEO_CHROMA_SUBSAMPLING_MONOCHROME_BIT_KHR = 0x00000001,
    VK_VIDEO_CHROMA_SUBSAMPLING_420_BIT_KHR = 0x00000002,
    VK_VIDEO_CHROMA_SUBSAMPLING_422_BIT_KHR = 0x00000004,
    VK_VIDEO_CHROMA_SUBSAMPLING_444_BIT_KHR = 0x00000008,
} VkVideoChromaSubsamplingFlagBitsKHR;
  • VK_VIDEO_CHROMA_SUBSAMPLING_MONOCHROME_BIT_KHR - the format is monochrome.

  • VK_VIDEO_CHROMA_SUBSAMPLING_420_BIT_KHR - the format is 4:2:0 chroma subsampled. The two chroma components are each subsampled at a factor of 2 both horizontally and vertically.

  • VK_VIDEO_CHROMA_SUBSAMPLING_422_BIT_KHR - the format is 4:2:2 chroma subsampled. The two chroma components are sampled at half the sample rate of luma. The horizontal chroma resolution is halved.

  • VK_VIDEO_CHROMA_SUBSAMPLING_444_BIT_KHR - the format is 4:4:4 chroma sampled. Each of the three YCbCr components have the same sample rate, thus there is no chroma subsampling.

// Provided by VK_KHR_video_queue
typedef VkFlags VkVideoChromaSubsamplingFlagsKHR;

VkVideoChromaSubsamplingFlagsKHR is a bitmask type for setting a mask of zero or more VkVideoChromaSubsamplingFlagBitsKHR.

The video format component bit depth is defined with the following enums:

// Provided by VK_KHR_video_queue
typedef enum VkVideoComponentBitDepthFlagBitsKHR {
    VK_VIDEO_COMPONENT_BIT_DEPTH_INVALID_KHR = 0,
    VK_VIDEO_COMPONENT_BIT_DEPTH_8_BIT_KHR = 0x00000001,
    VK_VIDEO_COMPONENT_BIT_DEPTH_10_BIT_KHR = 0x00000004,
    VK_VIDEO_COMPONENT_BIT_DEPTH_12_BIT_KHR = 0x00000010,
} VkVideoComponentBitDepthFlagBitsKHR;
  • VK_VIDEO_COMPONENT_BIT_DEPTH_8_BIT_KHR - the format component bit depth is 8 bits.

  • VK_VIDEO_COMPONENT_BIT_DEPTH_10_BIT_KHR - the format component bit depth is 10 bits.

  • VK_VIDEO_COMPONENT_BIT_DEPTH_12_BIT_KHR - the format component bit depth is 12 bits.

// Provided by VK_KHR_video_queue
typedef VkFlags VkVideoComponentBitDepthFlagsKHR;

VkVideoComponentBitDepthFlagsKHR is a bitmask type for setting a mask of zero or more VkVideoComponentBitDepthFlagBitsKHR.

A video profile is provided when querying capabilities or image formats for video using vkGetPhysicalDeviceVideoCapabilitiesKHR or vkGetPhysicalDeviceVideoFormatPropertiesKHR, respectively. A video profile is also provided when creating resources (images, video sessions, etc.) used by video queues. Each instance of VkVideoProfileKHR must chain a codec-operation specific video profile extension structure, corresponding to the codec-operation specified in VkVideoProfileKHR::videoCodecOperation. Additional information is provided in each codec-operation-specific video extension.

35.3.3. Supported Video Decode or Encode Capabilities

To query video decode or encode capabilities for a specific codec, call:

// Provided by VK_KHR_video_queue
VkResult vkGetPhysicalDeviceVideoCapabilitiesKHR(
    VkPhysicalDevice                            physicalDevice,
    const VkVideoProfileKHR*                    pVideoProfile,
    VkVideoCapabilitiesKHR*                     pCapabilities);
  • physicalDevice is the physical device whose video decode or encode capabilities will be queried.

  • pVideoProfile is a pointer to a VkVideoProfileKHR structure with a chained codec-operation specific video profile structure.

  • pCapabilities is a pointer to a VkVideoCapabilitiesKHR structure in which the capabilities are returned.

Valid Usage (Implicit)
  • VUID-vkGetPhysicalDeviceVideoCapabilitiesKHR-physicalDevice-parameter
    physicalDevice must be a valid VkPhysicalDevice handle

  • VUID-vkGetPhysicalDeviceVideoCapabilitiesKHR-pVideoProfile-parameter
    pVideoProfile must be a valid pointer to a valid VkVideoProfileKHR structure

  • VUID-vkGetPhysicalDeviceVideoCapabilitiesKHR-pCapabilities-parameter
    pCapabilities must be a valid pointer to a VkVideoCapabilitiesKHR structure

Return Codes
Success
  • VK_SUCCESS

Failure
  • VK_ERROR_EXTENSION_NOT_PRESENT

  • VK_ERROR_INITIALIZATION_FAILED

  • VK_ERROR_FEATURE_NOT_PRESENT

  • VK_ERROR_FORMAT_NOT_SUPPORTED

If pVideoProfile and chained codec-operation specific profile is not supported, VK_ERROR_FORMAT_NOT_SUPPORTED is returned.

Otherwise, the implementation will fill pCapabilities with capabilities associated with this video profile.

The VkVideoCapabilitiesKHR structure is defined as:

// Provided by VK_KHR_video_queue
typedef struct VkVideoCapabilitiesKHR {
    VkStructureType                sType;
    void*                          pNext;
    VkVideoCapabilitiesFlagsKHR    capabilityFlags;
    VkDeviceSize                   minBitstreamBufferOffsetAlignment;
    VkDeviceSize                   minBitstreamBufferSizeAlignment;
    VkExtent2D                     videoPictureExtentGranularity;
    VkExtent2D                     minExtent;
    VkExtent2D                     maxExtent;
    uint32_t                       maxReferencePicturesSlotsCount;
    uint32_t                       maxReferencePicturesActiveCount;
} VkVideoCapabilitiesKHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • capabilityFlags is a bitmask of VkVideoCapabilitiesFlagBitsKHR specifying capability flags.

  • minBitstreamBufferOffsetAlignment is the minimum alignment for the input or output bitstream buffer offset.

  • minBitstreamBufferSizeAlignment is the minimum alignment for the input or output bitstream buffer size

  • videoPictureExtentGranularity is the minimum size alignment of the extent with the required padding for the decoded or encoded video images.

  • minExtent is the minimum width and height of the decoded or encoded video.

  • maxExtent is the maximum width and height of the decoded or encoded video.

  • maxReferencePicturesSlotsCount is the maximum number of DPB Slots supported by the implementation for a single video session instance.

  • maxReferencePicturesActiveCount is the maximum slots that can be used as Reference Pictures with a single decode or encode operation.

Valid Usage (Implicit)
  • VUID-VkVideoCapabilitiesKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_CAPABILITIES_KHR

  • VUID-VkVideoCapabilitiesKHR-pNext-pNext
    pNext must be NULL

The VkVideoCapabilitiesKHR flags are defined with the following enumeration:

// Provided by VK_KHR_video_queue
typedef enum VkVideoCapabilitiesFlagBitsKHR {
    VK_VIDEO_CAPABILITIES_PROTECTED_CONTENT_BIT_KHR = 0x00000001,
    VK_VIDEO_CAPABILITIES_SEPARATE_REFERENCE_IMAGES_BIT_KHR = 0x00000002,
} VkVideoCapabilitiesFlagBitsKHR;
  • VK_VIDEO_CAPABILITIES_PROTECTED_CONTENT_BIT_KHR - the decode or encode session supports protected content.

  • VK_VIDEO_CAPABILITIES_SEPARATE_REFERENCE_IMAGES_BIT_KHR - the DPB or Reconstructed Video Picture Resources for the video session may be created as a separate VkImage for each DPB picture. If not supported, the DPB must be created as single multi-layered image where each layer represents one of the DPB Video Picture Resources.

// Provided by VK_KHR_video_queue
typedef VkFlags VkVideoCapabilitiesFlagsKHR;

VkVideoCapabilitiesFlagsKHR is a bitmask type for setting a mask of zero or more VkVideoCapabilitiesFlagBitsKHR.

35.3.4. Enumeration of Supported Video Output, Input and DPB Formats

To enumerate the supported output, input and DPB image formats for a specific codec operation and video profile, call:

// Provided by VK_KHR_video_queue
VkResult vkGetPhysicalDeviceVideoFormatPropertiesKHR(
    VkPhysicalDevice                            physicalDevice,
    const VkPhysicalDeviceVideoFormatInfoKHR*   pVideoFormatInfo,
    uint32_t*                                   pVideoFormatPropertyCount,
    VkVideoFormatPropertiesKHR*                 pVideoFormatProperties);
  • physicalDevice is the physical device being queried.

  • pVideoFormatInfo is a pointer to a VkPhysicalDeviceVideoFormatInfoKHR structure specifying the codec and video profile for which information is returned.

  • pVideoFormatPropertyCount is a pointer to an integer related to the number of video format properties available or queried, as described below.

  • pVideoFormatProperties is a pointer to an array of VkVideoFormatPropertiesKHR structures in which supported formats are returned.

If pVideoFormatProperties is NULL, then the number of video format properties supported for the given physicalDevice is returned in pVideoFormatPropertyCount. Otherwise, pVideoFormatPropertyCount must point to a variable set by the user to the number of elements in the pVideoFormatProperties array, and on return the variable is overwritten with the number of values actually written to pVideoFormatProperties. If the value of pVideoFormatPropertyCount is less than the number of video format properties supported, at most pVideoFormatPropertyCount values will be written to pVideoFormatProperties, and VK_INCOMPLETE will be returned instead of VK_SUCCESS, to indicate that not all the available values were returned.

Valid Usage
  • VUID-vkGetPhysicalDeviceVideoFormatPropertiesKHR-imageUsage-04844
    The imageUsage enum of VkPhysicalDeviceVideoFormatInfoKHR must contain at least one of the following video image usage bit(s): VK_IMAGE_USAGE_VIDEO_DECODE_DST_BIT_KHR, VK_IMAGE_USAGE_VIDEO_DECODE_DPB_BIT_KHR, VK_IMAGE_USAGE_VIDEO_ENCODE_SRC_BIT_KHR, or VK_IMAGE_USAGE_VIDEO_ENCODE_DPB_BIT_KHR.

Note:

For most use cases, only decode or encode related usage flags are going to be specified. For a use case such as transcode, if the image were to be shared between decode and encode session(s), then both decode and encode related usage flags can be set.

Valid Usage (Implicit)
  • VUID-vkGetPhysicalDeviceVideoFormatPropertiesKHR-physicalDevice-parameter
    physicalDevice must be a valid VkPhysicalDevice handle

  • VUID-vkGetPhysicalDeviceVideoFormatPropertiesKHR-pVideoFormatInfo-parameter
    pVideoFormatInfo must be a valid pointer to a valid VkPhysicalDeviceVideoFormatInfoKHR structure

  • VUID-vkGetPhysicalDeviceVideoFormatPropertiesKHR-pVideoFormatPropertyCount-parameter
    pVideoFormatPropertyCount must be a valid pointer to a uint32_t value

  • VUID-vkGetPhysicalDeviceVideoFormatPropertiesKHR-pVideoFormatProperties-parameter
    If the value referenced by pVideoFormatPropertyCount is not 0, and pVideoFormatProperties is not NULL, pVideoFormatProperties must be a valid pointer to an array of pVideoFormatPropertyCount VkVideoFormatPropertiesKHR structures

Return Codes
Success
  • VK_SUCCESS

  • VK_INCOMPLETE

Failure
  • VK_ERROR_EXTENSION_NOT_PRESENT

  • VK_ERROR_INITIALIZATION_FAILED

  • VK_ERROR_FORMAT_NOT_SUPPORTED

The VkPhysicalDeviceVideoFormatInfoKHR input structure is defined as:

// Provided by VK_KHR_video_queue
typedef struct VkPhysicalDeviceVideoFormatInfoKHR {
    VkStructureType              sType;
    void*                        pNext;
    VkImageUsageFlags            imageUsage;
    const VkVideoProfilesKHR*    pVideoProfiles;
} VkPhysicalDeviceVideoFormatInfoKHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • imageUsage is a bitmask of VkImageUsageFlagBits specifying intended video image usages.

  • pVideoProfiles is a pointer to a VkVideoProfilesKHR structure providing the video profile(s) of video session(s) that will use the image. For most use cases, the image is used by a single video session and a single video profile is provided. For a use case such as transcode, where a decode session output image may be used as encode input for one or more encode sessions, multiple video profiles representing the video sessions that will share the image may be provided.

Valid Usage (Implicit)
  • VUID-VkPhysicalDeviceVideoFormatInfoKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_VIDEO_FORMAT_INFO_KHR

  • VUID-VkPhysicalDeviceVideoFormatInfoKHR-pNext-pNext
    pNext must be NULL

The VkVideoProfilesKHR structure is defined as:

// Provided by VK_KHR_video_queue
typedef struct VkVideoProfilesKHR {
    VkStructureType             sType;
    void*                       pNext;
    uint32_t                    profileCount;
    const VkVideoProfileKHR*    pProfiles;
} VkVideoProfilesKHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • profileCount is an integer which holds the number of video profiles included in pProfiles.

  • pProfiles is a pointer to an array of VkVideoProfileKHR structures. Each VkVideoProfileKHR structure must chain the corresponding codec-operation specific extension video profile structure.

Valid Usage (Implicit)
  • VUID-VkVideoProfilesKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_PROFILES_KHR

  • VUID-VkVideoProfilesKHR-pProfiles-parameter
    pProfiles must be a valid pointer to a valid VkVideoProfileKHR structure

// Provided by VK_KHR_video_queue
typedef struct VkVideoFormatPropertiesKHR {
    VkStructureType    sType;
    void*              pNext;
    VkFormat           format;
} VkVideoFormatPropertiesKHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • format is one of the supported formats reported by the implementation.

If the pVideoProfiles provided in input structure pVideoFormatInfo are not supported, VK_ERROR_FORMAT_NOT_SUPPORTED is returned. If the implementation requires an opaque video decode or encode DPB, then when querying with the corresponding video decode or encode DPB image usage in imageUsage, only one image format is returned: VK_FORMAT_UNDEFINED.

Valid Usage (Implicit)
  • VUID-VkVideoFormatPropertiesKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_FORMAT_PROPERTIES_KHR

  • VUID-VkVideoFormatPropertiesKHR-pNext-pNext
    pNext must be NULL

Before creating an image, the application should obtain the supported image creation parameters by querying with vkGetPhysicalDeviceFormatProperties2 or vkGetPhysicalDeviceImageFormatProperties2 using one of the reported pImageFormats and adding VkVideoProfilesKHR to the pNext chain of VkFormatProperties2.

35.4. Video Session Objects

35.4.1. Video Session

Video session objects are abstracted and represented by VkVideoSessionKHR handles:

// Provided by VK_KHR_video_queue
VK_DEFINE_NON_DISPATCHABLE_HANDLE(VkVideoSessionKHR)
Creating a Video Session

To create a video session object, call:

// Provided by VK_KHR_video_queue
VkResult vkCreateVideoSessionKHR(
    VkDevice                                    device,
    const VkVideoSessionCreateInfoKHR*          pCreateInfo,
    const VkAllocationCallbacks*                pAllocator,
    VkVideoSessionKHR*                          pVideoSession);
  • device is the logical device that creates the decode or encode session object.

  • pCreateInfo is a pointer to a VkVideoSessionCreateInfoKHR structure containing parameters specifying the creation of the decode or encode session.

  • pAllocator controls host memory allocation as described in the Memory Allocation chapter.

  • pVideoSession is a pointer to a VkVideoSessionKHR structure specifying the decode or encode video session object which will be created by this function when it returns VK_SUCCESS

Valid Usage (Implicit)
  • VUID-vkCreateVideoSessionKHR-device-parameter
    device must be a valid VkDevice handle

  • VUID-vkCreateVideoSessionKHR-pCreateInfo-parameter
    pCreateInfo must be a valid pointer to a valid VkVideoSessionCreateInfoKHR structure

  • VUID-vkCreateVideoSessionKHR-pAllocator-parameter
    If pAllocator is not NULL, pAllocator must be a valid pointer to a valid VkAllocationCallbacks structure

  • VUID-vkCreateVideoSessionKHR-pVideoSession-parameter
    pVideoSession must be a valid pointer to a VkVideoSessionKHR handle

Return Codes
Success
  • VK_SUCCESS

Failure
  • VK_ERROR_OUT_OF_HOST_MEMORY

  • VK_ERROR_OUT_OF_DEVICE_MEMORY

  • VK_ERROR_INITIALIZATION_FAILED

  • VK_ERROR_INCOMPATIBLE_DRIVER

  • VK_ERROR_FEATURE_NOT_PRESENT

The VkVideoSessionCreateInfoKHR structure is defined as:

// Provided by VK_KHR_video_queue
typedef struct VkVideoSessionCreateInfoKHR {
    VkStructureType                 sType;
    const void*                     pNext;
    uint32_t                        queueFamilyIndex;
    VkVideoSessionCreateFlagsKHR    flags;
    const VkVideoProfileKHR*        pVideoProfile;
    VkFormat                        pictureFormat;
    VkExtent2D                      maxCodedExtent;
    VkFormat                        referencePicturesFormat;
    uint32_t                        maxReferencePicturesSlotsCount;
    uint32_t                        maxReferencePicturesActiveCount;
} VkVideoSessionCreateInfoKHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • queueFamilyIndex is the queue family of the created video session.

  • flags is a bitmask of VkVideoSessionCreateFlagBitsKHR specifying creation flags.

  • pVideoProfile is a pointer to a VkVideoProfileKHR structure.

  • pictureFormat is the format of the image views representing decoded Output or encoded Input pictures.

  • maxCodedExtent is the maximum width and height of the coded pictures that this instance will be able to support.

  • referencePicturesFormat is the format of the DPB image views representing the Reference Pictures.

  • maxReferencePicturesSlotsCount is the maximum number of DPB Slots that can be activated with associated Video Picture Resources for the created video session.

  • maxReferencePicturesActiveCount is the maximum number of active DPB Slots that can be used as Dpb or Reconstructed Reference Pictures within a single decode or encode operation for the created video session.

Valid Usage
  • VUID-VkVideoSessionCreateInfoKHR-pVideoProfile-04845
    pVideoProfile must be a pointer to a valid VkVideoProfileKHR structure whose pNext chain must include a valid codec-specific profile structure.

  • VUID-VkVideoSessionCreateInfoKHR-maxReferencePicturesSlotsCount-04846
    If Reference Pictures are required for use with the created video session, the maxReferencePicturesSlotsCount must be set to a value bigger than 0.

  • VUID-VkVideoSessionCreateInfoKHR-maxReferencePicturesSlotsCount-04847
    maxReferencePicturesSlotsCount cannot exceed the implementation reported VkVideoCapabilitiesKHR::maxReferencePicturesSlotsCount.

  • VUID-VkVideoSessionCreateInfoKHR-maxReferencePicturesActiveCount-04848
    If Reference Pictures are required for use with the created video session, the maxReferencePicturesActiveCount must be set to a value bigger than 0.

  • VUID-VkVideoSessionCreateInfoKHR-maxReferencePicturesActiveCount-04849
    maxReferencePicturesActiveCount cannot exceed the implementation reported VkVideoCapabilitiesKHR::maxReferencePicturesActiveCount.

  • VUID-VkVideoSessionCreateInfoKHR-maxReferencePicturesActiveCount-04850
    maxReferencePicturesActiveCount cannot exceed the maxReferencePicturesSlotsCount.

  • VUID-VkVideoSessionCreateInfoKHR-maxCodedExtent-04851
    maxCodedExtent cannot be smaller than VkVideoCapabilitiesKHR::minExtent and bigger than VkVideoCapabilitiesKHR::maxExtent.

  • VUID-VkVideoSessionCreateInfoKHR-referencePicturesFormat-04852
    referencePicturesFormat must be one of the supported formats in VkVideoFormatPropertiesKHR format returned by the vkGetPhysicalDeviceVideoFormatPropertiesKHR when the VkPhysicalDeviceVideoFormatInfoKHR imageUsage contains VK_IMAGE_USAGE_VIDEO_DECODE_DPB_BIT_KHR or VK_IMAGE_USAGE_VIDEO_ENCODE_DPB_BIT_KHR depending on the session codec operation.

  • VUID-VkVideoSessionCreateInfoKHR-pictureFormat-04853
    pictureFormat for decode output must be one of the supported formats in VkVideoFormatPropertiesKHR format returned by the vkGetPhysicalDeviceVideoFormatPropertiesKHR when the VkPhysicalDeviceVideoFormatInfoKHR imageUsage contains VK_IMAGE_USAGE_VIDEO_DECODE_DST_BIT_KHR.

  • VUID-VkVideoSessionCreateInfoKHR-pictureFormat-04854
    pictureFormat targeting encode operations must be one of the supported formats in VkVideoFormatPropertiesKHR format returned by the vkGetPhysicalDeviceVideoFormatPropertiesKHR when the VkPhysicalDeviceVideoFormatInfoKHR imageUsage contains VK_IMAGE_USAGE_VIDEO_ENCODE_SRC_BIT_KHR.

Valid Usage (Implicit)
  • VUID-VkVideoSessionCreateInfoKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_SESSION_CREATE_INFO_KHR

  • VUID-VkVideoSessionCreateInfoKHR-pNext-pNext
    pNext must be NULL

  • VUID-VkVideoSessionCreateInfoKHR-flags-parameter
    flags must be a valid combination of VkVideoSessionCreateFlagBitsKHR values

  • VUID-VkVideoSessionCreateInfoKHR-pVideoProfile-parameter
    pVideoProfile must be a valid pointer to a valid VkVideoProfileKHR structure

  • VUID-VkVideoSessionCreateInfoKHR-pictureFormat-parameter
    pictureFormat must be a valid VkFormat value

  • VUID-VkVideoSessionCreateInfoKHR-referencePicturesFormat-parameter
    referencePicturesFormat must be a valid VkFormat value

The decode or encode session creation flags defined with the following enums:

// Provided by VK_KHR_video_queue
typedef enum VkVideoSessionCreateFlagBitsKHR {
    VK_VIDEO_SESSION_CREATE_DEFAULT_KHR = 0,
    VK_VIDEO_SESSION_CREATE_PROTECTED_CONTENT_BIT_KHR = 0x00000001,
} VkVideoSessionCreateFlagBitsKHR;
  • VK_VIDEO_SESSION_CREATE_PROTECTED_CONTENT_BIT_KHR - create the video session for use with protected video content

// Provided by VK_KHR_video_queue
typedef VkFlags VkVideoSessionCreateFlagsKHR;

VkVideoSessionCreateFlagsKHR is a bitmask type for setting a mask of zero or more VkVideoSessionCreateFlagBitsKHR.

35.4.2. Destroying a Video Session

To destroy a decode session object, call:

// Provided by VK_KHR_video_queue
void vkDestroyVideoSessionKHR(
    VkDevice                                    device,
    VkVideoSessionKHR                           videoSession,
    const VkAllocationCallbacks*                pAllocator);
  • device is the device that was used for the creation of the video session.

  • videoSession is the decode or encode video session to be destroyed.

  • pAllocator controls host memory allocation as described in the Memory Allocation chapter.

Valid Usage (Implicit)
  • VUID-vkDestroyVideoSessionKHR-device-parameter
    device must be a valid VkDevice handle

  • VUID-vkDestroyVideoSessionKHR-videoSession-parameter
    videoSession must be a valid VkVideoSessionKHR handle

  • VUID-vkDestroyVideoSessionKHR-pAllocator-parameter
    If pAllocator is not NULL, pAllocator must be a valid pointer to a valid VkAllocationCallbacks structure

  • VUID-vkDestroyVideoSessionKHR-videoSession-parent
    videoSession must have been created, allocated, or retrieved from device

35.4.3. Video Session Memory Resource Management

Obtaining the Video Session Object Device Memory Requirements

To get memory requirements for a video session, call:

// Provided by VK_KHR_video_queue
VkResult vkGetVideoSessionMemoryRequirementsKHR(
    VkDevice                                    device,
    VkVideoSessionKHR                           videoSession,
    uint32_t*                                   pVideoSessionMemoryRequirementsCount,
    VkVideoGetMemoryPropertiesKHR*              pVideoSessionMemoryRequirements);
  • device is the logical device that owns the video session.

  • videoSession is the video session to query.

  • pVideoSessionMemoryRequirementsCount is a pointer to an integer related to the number of memory heap requirements available or queried, as described below.

  • pVideoSessionMemoryRequirements is NULL or a pointer to an array of VkVideoGetMemoryPropertiesKHR structures in which the memory heap requirements of the video session are returned.

If pVideoSessionMemoryRequirements is NULL, then the number of memory heap types required for the video session is returned in pVideoSessionMemoryRequirementsCount. Otherwise, pVideoSessionMemoryRequirementsCount must point to a variable set by the user with the number of elements in the pVideoSessionMemoryRequirements array, and on return the variable is overwritten with the number of formats actually written to pVideoSessionMemoryRequirements. If pVideoSessionMemoryRequirementsCount is less than the number of memory heap types required for the video session, then at most pVideoSessionMemoryRequirementsCount elements will be written to pVideoSessionMemoryRequirements, and VK_INCOMPLETE will be returned, instead of VK_SUCCESS, to indicate that not all required memory heap types were returned.

Valid Usage (Implicit)
  • VUID-vkGetVideoSessionMemoryRequirementsKHR-device-parameter
    device must be a valid VkDevice handle

  • VUID-vkGetVideoSessionMemoryRequirementsKHR-videoSession-parameter
    videoSession must be a valid VkVideoSessionKHR handle

  • VUID-vkGetVideoSessionMemoryRequirementsKHR-pVideoSessionMemoryRequirementsCount-parameter
    pVideoSessionMemoryRequirementsCount must be a valid pointer to a uint32_t value

  • VUID-vkGetVideoSessionMemoryRequirementsKHR-pVideoSessionMemoryRequirements-parameter
    If the value referenced by pVideoSessionMemoryRequirementsCount is not 0, and pVideoSessionMemoryRequirements is not NULL, pVideoSessionMemoryRequirements must be a valid pointer to an array of pVideoSessionMemoryRequirementsCount VkVideoGetMemoryPropertiesKHR structures

  • VUID-vkGetVideoSessionMemoryRequirementsKHR-videoSession-parent
    videoSession must have been created, allocated, or retrieved from device

Return Codes
Success
  • VK_SUCCESS

  • VK_INCOMPLETE

Failure
  • VK_ERROR_INITIALIZATION_FAILED

The VkVideoGetMemoryPropertiesKHR structure is defined as:

// Provided by VK_KHR_video_queue
typedef struct VkVideoGetMemoryPropertiesKHR {
    VkStructureType           sType;
    const void*               pNext;
    uint32_t                  memoryBindIndex;
    VkMemoryRequirements2*    pMemoryRequirements;
} VkVideoGetMemoryPropertiesKHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • memoryBindIndex is the memory bind index of the memory heap type described by the information returned in pMemoryRequirements.

  • pMemoryRequirements is a pointer to a VkMemoryRequirements2 structure in which the requested memory heap requirements for the heap with index memoryBindIndex are returned.

Valid Usage (Implicit)
  • VUID-VkVideoGetMemoryPropertiesKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_GET_MEMORY_PROPERTIES_KHR

  • VUID-VkVideoGetMemoryPropertiesKHR-pNext-pNext
    pNext must be NULL

  • VUID-VkVideoGetMemoryPropertiesKHR-pMemoryRequirements-parameter
    pMemoryRequirements must be a valid pointer to a VkMemoryRequirements2 structure

Binding the Session Object Device Memory

To attach memory to a video session object, call:

// Provided by VK_KHR_video_queue
VkResult vkBindVideoSessionMemoryKHR(
    VkDevice                                    device,
    VkVideoSessionKHR                           videoSession,
    uint32_t                                    videoSessionBindMemoryCount,
    const VkVideoBindMemoryKHR*                 pVideoSessionBindMemories);
  • device is the logical device that owns the video session’s memory.

  • videoSession is the video session to be bound with device memory.

  • videoSessionBindMemoryCount is the number of pVideoSessionBindMemories to be bound.

  • pVideoSessionBindMemories is a pointer to an array of VkVideoBindMemoryKHR structures specifying memory regions to be bound to a device memory heap.

Valid Usage (Implicit)
  • VUID-vkBindVideoSessionMemoryKHR-device-parameter
    device must be a valid VkDevice handle

  • VUID-vkBindVideoSessionMemoryKHR-videoSession-parameter
    videoSession must be a valid VkVideoSessionKHR handle

  • VUID-vkBindVideoSessionMemoryKHR-pVideoSessionBindMemories-parameter
    pVideoSessionBindMemories must be a valid pointer to an array of videoSessionBindMemoryCount valid VkVideoBindMemoryKHR structures

  • VUID-vkBindVideoSessionMemoryKHR-videoSessionBindMemoryCount-arraylength
    videoSessionBindMemoryCount must be greater than 0

  • VUID-vkBindVideoSessionMemoryKHR-videoSession-parent
    videoSession must have been created, allocated, or retrieved from device

Return Codes
Success
  • VK_SUCCESS

Failure
  • VK_ERROR_OUT_OF_HOST_MEMORY

  • VK_ERROR_OUT_OF_DEVICE_MEMORY

  • VK_ERROR_INITIALIZATION_FAILED

The VkVideoBindMemoryKHR structure is defined as:

// Provided by VK_KHR_video_queue
typedef struct VkVideoBindMemoryKHR {
    VkStructureType    sType;
    const void*        pNext;
    uint32_t           memoryBindIndex;
    VkDeviceMemory     memory;
    VkDeviceSize       memoryOffset;
    VkDeviceSize       memorySize;
} VkVideoBindMemoryKHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • memoryBindIndex is the index of the device memory heap returned in VkVideoGetMemoryPropertiesKHR::memoryBindIndex from vkGetVideoSessionMemoryRequirementsKHR.

  • memory is the allocated device memory to be bound to the video session’s heap with index memoryBindIndex.

  • memoryOffset is the start offset of the region of memory which is to be bound.

  • memorySize is the size in bytes of the region of memory, starting from memoryOffset bytes, to be bound.

Valid Usage (Implicit)
  • VUID-VkVideoBindMemoryKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_BIND_MEMORY_KHR

  • VUID-VkVideoBindMemoryKHR-pNext-pNext
    pNext must be NULL

  • VUID-VkVideoBindMemoryKHR-memory-parameter
    memory must be a valid VkDeviceMemory handle

35.4.4. Video Session Parameters

This specification supports several classes of preprocessed parameters stored in Video Session Parameters objects. The Video Session Parameters objects reduces the number of parameters being dispatched and then processed by the implementation while recording video command buffers.

35.4.5. Creating Video Session Parameters

Video session parameter objects are represented by VkVideoSessionParametersKHR handles:

// Provided by VK_KHR_video_queue
VK_DEFINE_NON_DISPATCHABLE_HANDLE(VkVideoSessionParametersKHR)

To create a video session parameters object, call:

// Provided by VK_KHR_video_queue
VkResult vkCreateVideoSessionParametersKHR(
    VkDevice                                    device,
    const VkVideoSessionParametersCreateInfoKHR* pCreateInfo,
    const VkAllocationCallbacks*                pAllocator,
    VkVideoSessionParametersKHR*                pVideoSessionParameters);
  • device is the logical device that was used for the creation of the video session object.

  • pCreateInfo is a pointer to VkVideoSessionParametersCreateInfoKHR structure specifying the video session parameters.

  • pAllocator controls host memory allocation as described in the Memory Allocation chapter.

  • pVideoSessionParameters is a pointer to a VkVideoSessionParametersKHR handle in which the video session parameters object is returned.

Valid Usage (Implicit)
  • VUID-vkCreateVideoSessionParametersKHR-device-parameter
    device must be a valid VkDevice handle

  • VUID-vkCreateVideoSessionParametersKHR-pCreateInfo-parameter
    pCreateInfo must be a valid pointer to a valid VkVideoSessionParametersCreateInfoKHR structure

  • VUID-vkCreateVideoSessionParametersKHR-pAllocator-parameter
    If pAllocator is not NULL, pAllocator must be a valid pointer to a valid VkAllocationCallbacks structure

  • VUID-vkCreateVideoSessionParametersKHR-pVideoSessionParameters-parameter
    pVideoSessionParameters must be a valid pointer to a VkVideoSessionParametersKHR handle

Return Codes
Success
  • VK_SUCCESS

Failure
  • VK_ERROR_OUT_OF_HOST_MEMORY

  • VK_ERROR_OUT_OF_DEVICE_MEMORY

  • VK_ERROR_TOO_MANY_OBJECTS

The VkVideoSessionParametersCreateInfoKHR structure is defined as:

// Provided by VK_KHR_video_queue
typedef struct VkVideoSessionParametersCreateInfoKHR {
    VkStructureType                sType;
    const void*                    pNext;
    VkVideoSessionParametersKHR    videoSessionParametersTemplate;
    VkVideoSessionKHR              videoSession;
} VkVideoSessionParametersCreateInfoKHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • videoSessionParametersTemplate is VK_NULL_HANDLE or a valid handle to a VkVideoSessionParametersKHR object. If this parameter represents a valid handle, then the underlying Video Session Parameters object will be used as a template for constructing the new video session parameters object. All of the template object’s current parameters will be inherited by the new object in such a case. Optionally, some of the template’s parameters can be updated or new parameters added to the newly constructed object via the extension-specific parameters.

  • videoSession is the video session object against which the video session parameters object is going to be created.

Valid Usage
  • VUID-VkVideoSessionParametersCreateInfoKHR-videoSessionParametersTemplate-04855
    If videoSessionParametersTemplate represents a valid handle, it must have been created against videoSession.

Valid Usage (Implicit)
  • VUID-VkVideoSessionParametersCreateInfoKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_SESSION_PARAMETERS_CREATE_INFO_KHR

  • VUID-VkVideoSessionParametersCreateInfoKHR-pNext-pNext
    pNext must be NULL

  • VUID-VkVideoSessionParametersCreateInfoKHR-videoSessionParametersTemplate-parameter
    videoSessionParametersTemplate must be a valid VkVideoSessionParametersKHR handle

  • VUID-VkVideoSessionParametersCreateInfoKHR-videoSession-parameter
    videoSession must be a valid VkVideoSessionKHR handle

  • VUID-VkVideoSessionParametersCreateInfoKHR-videoSessionParametersTemplate-parent
    videoSessionParametersTemplate must have been created, allocated, or retrieved from videoSession

  • VUID-VkVideoSessionParametersCreateInfoKHR-commonparent
    Both of videoSession, and videoSessionParametersTemplate must have been created, allocated, or retrieved from the same VkDevice

35.4.6. Updating the parameters of the Video Session Parameters object

To update, add, or remove video session parameters state, call:

// Provided by VK_KHR_video_queue
VkResult vkUpdateVideoSessionParametersKHR(
    VkDevice                                    device,
    VkVideoSessionParametersKHR                 videoSessionParameters,
    const VkVideoSessionParametersUpdateInfoKHR* pUpdateInfo);
  • device is the logical device that was used for the creation of the video session object.

  • videoSessionParameters is the video session object that is going to be updated.

  • pUpdateInfo is a pointer to a VkVideoSessionParametersUpdateInfoKHR structure containing the session parameters update information.

Valid Usage (Implicit)
  • VUID-vkUpdateVideoSessionParametersKHR-device-parameter
    device must be a valid VkDevice handle

  • VUID-vkUpdateVideoSessionParametersKHR-videoSessionParameters-parameter
    videoSessionParameters must be a valid VkVideoSessionParametersKHR handle

  • VUID-vkUpdateVideoSessionParametersKHR-pUpdateInfo-parameter
    pUpdateInfo must be a valid pointer to a valid VkVideoSessionParametersUpdateInfoKHR structure

Return Codes
Success
  • VK_SUCCESS

Failure
  • VK_ERROR_INITIALIZATION_FAILED

  • VK_ERROR_TOO_MANY_OBJECTS

The VkVideoSessionParametersUpdateInfoKHR structure is defined as:

// Provided by VK_KHR_video_queue
typedef struct VkVideoSessionParametersUpdateInfoKHR {
    VkStructureType    sType;
    const void*        pNext;
    uint32_t           updateSequenceCount;
} VkVideoSessionParametersUpdateInfoKHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • updateSequenceCount is the sequence number of the object update with parameters, starting from 1 and incrementing the value by one with each subsequent update.

Valid Usage (Implicit)
  • VUID-VkVideoSessionParametersUpdateInfoKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_SESSION_PARAMETERS_UPDATE_INFO_KHR

  • VUID-VkVideoSessionParametersUpdateInfoKHR-pNext-pNext
    pNext must be NULL

35.4.7. Destroying Video Session Parameters

To destroy a video session object, call:

// Provided by VK_KHR_video_queue
void vkDestroyVideoSessionParametersKHR(
    VkDevice                                    device,
    VkVideoSessionParametersKHR                 videoSessionParameters,
    const VkAllocationCallbacks*                pAllocator);
  • device is the device the video session was created with.

  • videoSessionParameters is the video session parameters object to be destroyed.

  • pAllocator controls host memory allocation as described in the Memory Allocation chapter.

Valid Usage (Implicit)
  • VUID-vkDestroyVideoSessionParametersKHR-device-parameter
    device must be a valid VkDevice handle

  • VUID-vkDestroyVideoSessionParametersKHR-videoSessionParameters-parameter
    videoSessionParameters must be a valid VkVideoSessionParametersKHR handle

  • VUID-vkDestroyVideoSessionParametersKHR-pAllocator-parameter
    If pAllocator is not NULL, pAllocator must be a valid pointer to a valid VkAllocationCallbacks structure

35.4.8. Video Encode and Decode commands

To start video decode or encode operations, call:

// Provided by VK_KHR_video_queue
void vkCmdBeginVideoCodingKHR(
    VkCommandBuffer                             commandBuffer,
    const VkVideoBeginCodingInfoKHR*            pBeginInfo);
  • commandBuffer is the command buffer to be used when recording commands for the video decode or encode operations.

  • pBeginInfo is a pointer to a VkVideoBeginCodingInfoKHR structure.

Valid Usage (Implicit)
  • VUID-vkCmdBeginVideoCodingKHR-commandBuffer-parameter
    commandBuffer must be a valid VkCommandBuffer handle

  • VUID-vkCmdBeginVideoCodingKHR-pBeginInfo-parameter
    pBeginInfo must be a valid pointer to a valid VkVideoBeginCodingInfoKHR structure

  • VUID-vkCmdBeginVideoCodingKHR-commandBuffer-recording
    commandBuffer must be in the recording state

  • VUID-vkCmdBeginVideoCodingKHR-commandBuffer-cmdpool
    The VkCommandPool that commandBuffer was allocated from must support decode, or encode operations

  • VUID-vkCmdBeginVideoCodingKHR-renderpass
    This command must only be called outside of a render pass instance

  • VUID-vkCmdBeginVideoCodingKHR-bufferlevel
    commandBuffer must be a primary VkCommandBuffer

Host Synchronization
  • Host access to the VkCommandPool that commandBuffer was allocated from must be externally synchronized

Command Properties
Command Buffer Levels Render Pass Scope Supported Queue Types

Primary

Outside

Decode
Encode

The VkVideoBeginCodingInfoKHR structure is defined as:

// Provided by VK_KHR_video_queue
typedef struct VkVideoBeginCodingInfoKHR {
    VkStructureType                       sType;
    const void*                           pNext;
    VkVideoBeginCodingFlagsKHR            flags;
    VkVideoCodingQualityPresetFlagsKHR    codecQualityPreset;
    VkVideoSessionKHR                     videoSession;
    VkVideoSessionParametersKHR           videoSessionParameters;
    uint32_t                              referenceSlotCount;
    const VkVideoReferenceSlotKHR*        pReferenceSlots;
} VkVideoBeginCodingInfoKHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • flags is reserved for future use.

  • codecQualityPreset is a bitmask of VkVideoCodingQualityPresetFlagBitsKHR specifying the Video Decode or Encode quality preset.

  • videoSession is the video session object to be bound for the processing of the video commands.

  • videoSessionParameters is VK_NULL_HANDLE or a handle of a VkVideoSessionParametersKHR object to be used for the processing of the video commands. If VK_NULL_HANDLE, then no video session parameters apply to this command buffer context.

  • referenceSlotCount is the number of reference slot entries provided in pReferenceSlots.

  • pReferenceSlots is a pointer to an array of VkVideoReferenceSlotKHR structures specifying reference slots, used within the video command context between this vkCmdBeginVideoCodingKHR command and the vkCmdEndVideoCodingKHR commmand that follows. Each reference slot provides a slot index and the VkVideoPictureResourceKHR specifying the reference picture resource bound to this slot index. A slot index must not appear more than once in pReferenceSlots in a given command.

Valid Usage
  • VUID-VkVideoBeginCodingInfoKHR-referenceSlotCount-04856
    VkVideoBeginCodingInfoKHR::referenceSlotCount must not exceed the value specified in VkVideoSessionCreateInfoKHR::maxReferencePicturesSlotsCount when creating the video session object that is being provided in videoSession.

  • VUID-VkVideoBeginCodingInfoKHR-videoSessionParameters-04857
    If videoSessionParameters is not VK_NULL_HANDLE, it must have been created using videoSession as a parent object.

Valid Usage (Implicit)
  • VUID-VkVideoBeginCodingInfoKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_BEGIN_CODING_INFO_KHR

  • VUID-VkVideoBeginCodingInfoKHR-pNext-pNext
    pNext must be NULL

  • VUID-VkVideoBeginCodingInfoKHR-flags-zerobitmask
    flags must be 0

  • VUID-VkVideoBeginCodingInfoKHR-codecQualityPreset-parameter
    codecQualityPreset must be a valid combination of VkVideoCodingQualityPresetFlagBitsKHR values

  • VUID-VkVideoBeginCodingInfoKHR-codecQualityPreset-requiredbitmask
    codecQualityPreset must not be 0

  • VUID-VkVideoBeginCodingInfoKHR-videoSession-parameter
    videoSession must be a valid VkVideoSessionKHR handle

  • VUID-VkVideoBeginCodingInfoKHR-videoSessionParameters-parameter
    If videoSessionParameters is not VK_NULL_HANDLE, videoSessionParameters must be a valid VkVideoSessionParametersKHR handle

  • VUID-VkVideoBeginCodingInfoKHR-pReferenceSlots-parameter
    pReferenceSlots must be a valid pointer to an array of referenceSlotCount valid VkVideoReferenceSlotKHR structures

  • VUID-VkVideoBeginCodingInfoKHR-referenceSlotCount-arraylength
    referenceSlotCount must be greater than 0

  • VUID-VkVideoBeginCodingInfoKHR-videoSessionParameters-parent
    If videoSessionParameters is a valid handle, it must have been created, allocated, or retrieved from videoSession

  • VUID-VkVideoBeginCodingInfoKHR-commonparent
    Both of videoSession, and videoSessionParameters that are valid handles of non-ignored parameters must have been created, allocated, or retrieved from the same VkDevice

The decode preset types are defined with the following:

// Provided by VK_KHR_video_queue
typedef enum VkVideoCodingQualityPresetFlagBitsKHR {
    VK_VIDEO_CODING_QUALITY_PRESET_DEFAULT_BIT_KHR = 0,
    VK_VIDEO_CODING_QUALITY_PRESET_NORMAL_BIT_KHR = 0x00000001,
    VK_VIDEO_CODING_QUALITY_PRESET_POWER_BIT_KHR = 0x00000002,
    VK_VIDEO_CODING_QUALITY_PRESET_QUALITY_BIT_KHR = 0x00000004,
} VkVideoCodingQualityPresetFlagBitsKHR;
  • VK_VIDEO_CODING_QUALITY_PRESET_NORMAL_BIT_KHR defines normal decode case.

  • VK_VIDEO_CODING_QUALITY_PRESET_POWER_BIT_KHR defines power efficient case.

  • VK_VIDEO_CODING_QUALITY_PRESET_QUALITY_BIT_KHR defines quality focus case.

// Provided by VK_KHR_video_queue
typedef VkFlags VkVideoCodingQualityPresetFlagsKHR;

VkVideoCodingQualityPresetFlagsKHR is a bitmask type for setting a mask of zero or more VkVideoCodingQualityPresetFlagBitsKHR.

The VkVideoReferenceSlotKHR structure is defined as:

// Provided by VK_KHR_video_queue
typedef struct VkVideoReferenceSlotKHR {
    VkStructureType                     sType;
    const void*                         pNext;
    int8_t                              slotIndex;
    const VkVideoPictureResourceKHR*    pPictureResource;
} VkVideoReferenceSlotKHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • slotIndex is the unique reference slot index used for the encode or decode operation.

  • pPictureResource is a pointer to a VkVideoPictureResourceKHR structure describing the picture resource bound to this slot index.

Valid Usage (Implicit)
  • VUID-VkVideoReferenceSlotKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_REFERENCE_SLOT_KHR

  • VUID-VkVideoReferenceSlotKHR-pNext-pNext
    pNext must be NULL

  • VUID-VkVideoReferenceSlotKHR-pPictureResource-parameter
    pPictureResource must be a valid pointer to a valid VkVideoPictureResourceKHR structure

The VkVideoPictureResourceKHR structure is defined as:

// Provided by VK_KHR_video_queue
typedef struct VkVideoPictureResourceKHR {
    VkStructureType    sType;
    const void*        pNext;
    VkOffset2D         codedOffset;
    VkExtent2D         codedExtent;
    uint32_t           baseArrayLayer;
    VkImageView        imageViewBinding;
} VkVideoPictureResourceKHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • codedOffset is the offset to be used for the picture resource.

  • codedExtent is the extent to be used for the picture resource.

  • baseArrayLayer is the first array layer to be accessed for the Decode or Encode Operations.

  • imageViewBinding is a VkImageView image view representing this picture resource.

Valid Usage (Implicit)
  • VUID-VkVideoPictureResourceKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_PICTURE_RESOURCE_KHR

  • VUID-VkVideoPictureResourceKHR-pNext-pNext
    pNext must be NULL

  • VUID-VkVideoPictureResourceKHR-imageViewBinding-parameter
    imageViewBinding must be a valid VkImageView handle

35.4.9. End of the Video Session

To end video decode or encode operations, call:

// Provided by VK_KHR_video_queue
void vkCmdEndVideoCodingKHR(
    VkCommandBuffer                             commandBuffer,
    const VkVideoEndCodingInfoKHR*              pEndCodingInfo);
  • commandBuffer is the command buffer to be filled by this function.

  • pEndCodingInfo is a pointer to a VkVideoEndCodingInfoKHR structure.

Valid Usage (Implicit)
  • VUID-vkCmdEndVideoCodingKHR-commandBuffer-parameter
    commandBuffer must be a valid VkCommandBuffer handle

  • VUID-vkCmdEndVideoCodingKHR-pEndCodingInfo-parameter
    pEndCodingInfo must be a valid pointer to a valid VkVideoEndCodingInfoKHR structure

  • VUID-vkCmdEndVideoCodingKHR-commandBuffer-recording
    commandBuffer must be in the recording state

  • VUID-vkCmdEndVideoCodingKHR-commandBuffer-cmdpool
    The VkCommandPool that commandBuffer was allocated from must support decode, or encode operations

  • VUID-vkCmdEndVideoCodingKHR-renderpass
    This command must only be called outside of a render pass instance

  • VUID-vkCmdEndVideoCodingKHR-bufferlevel
    commandBuffer must be a primary VkCommandBuffer

Host Synchronization
  • Host access to the VkCommandPool that commandBuffer was allocated from must be externally synchronized

Command Properties
Command Buffer Levels Render Pass Scope Supported Queue Types

Primary

Outside

Decode
Encode

The VkVideoEndCodingInfoKHR structure is defined as:

// Provided by VK_KHR_video_queue
typedef struct VkVideoEndCodingInfoKHR {
    VkStructureType             sType;
    const void*                 pNext;
    VkVideoEndCodingFlagsKHR    flags;
} VkVideoEndCodingInfoKHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • flags is reserved for future use.

Valid Usage (Implicit)
  • VUID-VkVideoEndCodingInfoKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_END_CODING_INFO_KHR

  • VUID-VkVideoEndCodingInfoKHR-pNext-pNext
    pNext must be NULL

  • VUID-VkVideoEndCodingInfoKHR-flags-zerobitmask
    flags must be 0

35.4.10. Video Session Control Command

To apply dynamic controls to video decode or video operations, call:

// Provided by VK_KHR_video_queue
void vkCmdControlVideoCodingKHR(
    VkCommandBuffer                             commandBuffer,
    const VkVideoCodingControlInfoKHR*          pCodingControlInfo);
  • commandBuffer is the command buffer to be filled by this function for setting encode rate control parameters.

  • pCodingControlInfo is a pointer to a VkVideoCodingControlInfoKHR structure.

Valid Usage (Implicit)
  • VUID-vkCmdControlVideoCodingKHR-commandBuffer-parameter
    commandBuffer must be a valid VkCommandBuffer handle

  • VUID-vkCmdControlVideoCodingKHR-pCodingControlInfo-parameter
    pCodingControlInfo must be a valid pointer to a valid VkVideoCodingControlInfoKHR structure

  • VUID-vkCmdControlVideoCodingKHR-commandBuffer-recording
    commandBuffer must be in the recording state

  • VUID-vkCmdControlVideoCodingKHR-commandBuffer-cmdpool
    The VkCommandPool that commandBuffer was allocated from must support decode, or encode operations

  • VUID-vkCmdControlVideoCodingKHR-renderpass
    This command must only be called outside of a render pass instance

  • VUID-vkCmdControlVideoCodingKHR-bufferlevel
    commandBuffer must be a primary VkCommandBuffer

Host Synchronization
  • Host access to the VkCommandPool that commandBuffer was allocated from must be externally synchronized

Command Properties
Command Buffer Levels Render Pass Scope Supported Queue Types

Primary

Outside

Decode
Encode

The VkVideoCodingControlInfoKHR structure is defined as:

// Provided by VK_KHR_video_queue
typedef struct VkVideoCodingControlInfoKHR {
    VkStructureType                 sType;
    const void*                     pNext;
    VkVideoCodingControlFlagsKHR    flags;
} VkVideoCodingControlInfoKHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • flags is a bitmask of VkVideoCodingControlFlagsKHR specifying control flags.

Valid Usage (Implicit)
  • VUID-VkVideoCodingControlInfoKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_CODING_CONTROL_INFO_KHR

  • VUID-VkVideoCodingControlInfoKHR-pNext-pNext
    pNext must be NULL or a pointer to a valid instance of VkVideoEncodeRateControlInfoKHR

  • VUID-VkVideoCodingControlInfoKHR-sType-unique
    The sType value of each struct in the pNext chain must be unique

  • VUID-VkVideoCodingControlInfoKHR-flags-parameter
    flags must be a valid combination of VkVideoCodingControlFlagBitsKHR values

The vkCmdControlVideoCodingKHR flags are defined with the following enumeration:

// Provided by VK_KHR_video_queue
typedef enum VkVideoCodingControlFlagBitsKHR {
    VK_VIDEO_CODING_CONTROL_DEFAULT_KHR = 0,
    VK_VIDEO_CODING_CONTROL_RESET_BIT_KHR = 0x00000001,
} VkVideoCodingControlFlagBitsKHR;
  • VK_VIDEO_CODING_CONTROL_RESET_BIT_KHR indicates a request for the bound video session context to be reset to init state.

// Provided by VK_KHR_video_queue
typedef VkFlags VkVideoCodingControlFlagsKHR;

VkVideoCodingControlFlagsKHR is a bitmask type for setting a mask of zero or more VkVideoCodingControlFlagBitsKHR.

35.5. Video Decode Operations

Before the application can start recording Vulkan command buffers for the Video Decode Operations, it must do the following, beforehand:

  1. Ensure that the implementation can decode the Video Content by querying the supported codec operations and profiles using vkGetPhysicalDeviceQueueFamilyProperties2.

  2. By using vkGetPhysicalDeviceVideoFormatPropertiesKHR and providing one or more video profiles, choose the Vulkan formats supported by the implementation. The formats for output and reference pictures must be queried and chosen separately. Refer to the section on enumeration of supported video formats.

  3. Before creating an image to be used as a video picture resource, obtain the supported image creation parameters by querying with vkGetPhysicalDeviceFormatProperties2 and vkGetPhysicalDeviceImageFormatProperties2 using one of the reported formats and adding VkVideoProfilesKHR to the pNext chain of VkFormatProperties2. When querying the parameters with vkGetPhysicalDeviceImageFormatProperties2 for images targeting decoded output and reference (DPB) pictures, the VkPhysicalDeviceImageFormatInfo2::usage field should contain VK_IMAGE_USAGE_VIDEO_DECODE_DST_BIT_KHR and VK_IMAGE_USAGE_VIDEO_DECODE_DPB_BIT_KHR, respectively.

  4. Create none, some, or all of the required images for the decoded output and reference pictures. More Video Picture Resources can be created at some later point if needed while processing the decoded content. Also, if the decoded picture size is expected to change, the images can be created based on the maximum decoded content size required.

  5. Create the video session to be used for video decode operations. Before creating the Decode Video Session, the decode capabilities should be queried with vkGetPhysicalDeviceVideoCapabilitiesKHR to obtain the limits of the parameters allowed by the implementation for a particular codec profile.

  6. Bind memory resources with the decode video session by calling vkBindVideoSessionMemoryKHR. The video session cannot be used until memory resources are allocated and bound to it. In order to determine the required memory sizes and heap types of the device memory allocations, vkGetVideoSessionMemoryRequirementsKHR should be called.

  7. Create one or more Session Parameter objects for use across command buffer recording operations, if required by the codec extension in use. These objects must be created against a video session with the parameters required by the codec. Each Session Parameter object created is a child object of the associated Session object and cannot be bound in the command buffer with any other Session Object.

The recording of Video Decode Commands against a Vulkan command buffer consists of the following sequence:

  1. vkCmdBeginVideoCodingKHR starts the recording of one or more Video Decode operations in the command buffer. For each Video Decode Command operation, a Video Session must be bound to the command buffer within this command. This command establishes a Vulkan Video Decode Context that consists of the bound Video Session Object, Session Parameters Object, and the required Video Picture Resources. The established Video Decode Context is in effect until the vkCmdEndVideoCodingKHR command is recorded. If more Video Decode operations are to be required after the vkCmdEndVideoCodingKHR command, another Video Decode Context can be started with the vkCmdBeginVideoCodingKHR command.

  2. vkCmdDecodeVideoKHR specifies one or more compressed data buffers to be decoded. The VkVideoDecodeInfoKHR parameters, and the codec extension structures chained to this, specify the details of the decode operation.

  3. vkCmdControlVideoCodingKHR records operations against the decoded data, decoding device, or the Video Session state.

  4. vkCmdEndVideoCodingKHR signals the end of the recording of the Vulkan Video Decode Context, as established by vkCmdBeginVideoCodingKHR.

In addition to the above, the following commands can be recorded between vkCmdBeginVideoCodingKHR and vkCmdEndVideoCodingKHR:

  • Query operations

  • Global Memory Barriers

  • Buffer Memory Barriers

  • Image Memory Barriers (these must be used to transition the Video Picture Resources to the proper VK_IMAGE_LAYOUT_VIDEO_DECODE_DPB_KHR and VK_IMAGE_LAYOUT_VIDEO_DECODE_DST_KHR layouts).

  • Pipeline Barriers

  • Events

  • Timestamps

  • Device Groups (device mask)

The following Video Decode related commands must be recorded outside the Vulkan Video Decode Context established with the vkCmdBeginVideoCodingKHR and vkCmdEndVideoCodingKHR commands:

  • Sparse Memory Binding

  • Copy Commands

  • Clear Commands

35.5.1. Video Picture Decode Modes

There are a few ways that the vkCmdDecodeVideoKHR can be configured for the Video Picture Decode Operations, based on:

The most basic Video Picture Decode operation with the vkCmdDecodeVideoKHR command is to output the decoded pixel data without using any DPB Reference Pictures and without updating any DPB Slot’s indexes.

In this case, the following VkVideoDecodeInfoKHR parameters must be set:

Video Picture Decode operations with the vkCmdDecodeVideoKHR command, requiring one or more Reference Pictures for the predictions of the values of samples for the decoded output picture would require DPB Slots with Valid Picture Reference.

In this case, the following VkVideoDecodeInfoKHR parameters must be set:

After the vkCmdDecodeVideoKHR operation is completed successfully, the VkVideoDecodeInfoKHR::pSetupReferenceSlot->pPictureResource->imageViewBinding pixel data will be updated with the decoded content. The operation will not update any DPB Slot with Reference Pictures data. However, any DPB Slot activation, invalidation, or deactivation operations requested via VkVideoDecodeInfoKHR::pReferenceSlots are still going to be performed.

image/svg+xml 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Dpb Decode dstImageView Bitstream Reference Pictures pReferenceEntries
Figure 26. Decoding a Frame to VkImageView without a slot update for a Reference Picture
Video Picture Decode with a Reference Picture slot update and using optional Reference Pictures

When it is known that the picture to be decoded will be used as a reference picture for subsequent decode operations, one of the available DPB Slots needs to be selected for activation and update operations as part of the vkCmdDecodeVideoKHR command.

Based on whether a decode operation with reference pictures or without reference pictures is required, the vkCmdDecodeVideoKHR should be configured with parameters as described in the previous sections. In addition, one of the available DPB Slots must be selected by the application, activated with resources and then set-up for an update with the decode operation.

In this case, the following VkVideoDecodeInfoKHR parameters must be set:

After the vkCmdDecodeVideoKHR operation has completed successfully, the decoded content will be available in the resource provided for VkVideoDecodeInfoKHR::pSetupReferenceSlot->pPictureResource->imageViewBinding. In addition, this operation will update the selected DPB Slot with Reference Pictures data. Any other DPB Slot activation,invalidation, or deactivation operation requested via the VkVideoDecodeInfoKHR::pReferenceSlots are going to be performed as well.

image/svg+xml 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Dpb Decode dstImageView Bitstream Reference Pictures pReferenceEntries Target Dpb Picture Slot dpbSlotIndex
Figure 27. Decoding a Frame to VkImageView with an update to a Reference Pictures DPB Slot

35.5.2. Video Decode Command Buffer Commands

To decode a frame, call:

// Provided by VK_KHR_video_decode_queue
void vkCmdDecodeVideoKHR(
    VkCommandBuffer                             commandBuffer,
    const VkVideoDecodeInfoKHR*                 pFrameInfo);
  • commandBuffer is the command buffer to be filled with this function for decode frame command.

  • pFrameInfo is a pointer to a VkVideoDecodeInfoKHR structure.

Valid Usage (Implicit)
  • VUID-vkCmdDecodeVideoKHR-commandBuffer-parameter
    commandBuffer must be a valid VkCommandBuffer handle

  • VUID-vkCmdDecodeVideoKHR-pFrameInfo-parameter
    pFrameInfo must be a valid pointer to a valid VkVideoDecodeInfoKHR structure

  • VUID-vkCmdDecodeVideoKHR-commandBuffer-recording
    commandBuffer must be in the recording state

  • VUID-vkCmdDecodeVideoKHR-commandBuffer-cmdpool
    The VkCommandPool that commandBuffer was allocated from must support decode operations

  • VUID-vkCmdDecodeVideoKHR-renderpass
    This command must only be called outside of a render pass instance

  • VUID-vkCmdDecodeVideoKHR-bufferlevel
    commandBuffer must be a primary VkCommandBuffer

Host Synchronization
  • Host access to the VkCommandPool that commandBuffer was allocated from must be externally synchronized

Command Properties
Command Buffer Levels Render Pass Scope Supported Queue Types

Primary

Outside

Decode

The VkVideoDecodeInfoKHR structure is defined as:

// Provided by VK_KHR_video_decode_queue
typedef struct VkVideoDecodeInfoKHR {
    VkStructureType                   sType;
    const void*                       pNext;
    VkVideoDecodeFlagsKHR             flags;
    VkOffset2D                        codedOffset;
    VkExtent2D                        codedExtent;
    VkBuffer                          srcBuffer;
    VkDeviceSize                      srcBufferOffset;
    VkDeviceSize                      srcBufferRange;
    VkVideoPictureResourceKHR         dstPictureResource;
    const VkVideoReferenceSlotKHR*    pSetupReferenceSlot;
    uint32_t                          referenceSlotCount;
    const VkVideoReferenceSlotKHR*    pReferenceSlots;
} VkVideoDecodeInfoKHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure. All the codec specific structures related to each frame(picture parameters, quantization matrix, etc.) must be chained here and pass to decode session with the function call vkCmdDecodeVideoKHR.

  • flags is a bitmask of VkVideoDecodeFlagBitsKHR specifying decode flags, reserved for future versions of this specification.

  • codedOffset is the coded offset of the decode operations. The purpose of this field is interpreted based on the codec extension.

  • codedExtent is the coded size of the decode operations.

  • srcBuffer is the source buffer that holds the encoded bitstream.

  • srcBufferOffset is the buffer offset where the valid encoded bitstream starts in srcBuffer. It must meet the alignment requirement minBitstreamBufferOffsetAlignment within VkVideoCapabilitiesKHR queried with the vkGetPhysicalDeviceVideoCapabilitiesKHR function.

  • srcBufferRange is the size of the srcBuffer with valid encoded bitstream, starting from srcBufferOffset. It must meet the alignment requirement minBitstreamBufferSizeAlignment within VkVideoCapabilitiesKHR queried with the vkGetPhysicalDeviceVideoCapabilitiesKHR function.

  • dstPictureResource is the destination Decoded Output Picture Resource.

  • pSetupReferenceSlot is NULL or a pointer to a VkVideoReferenceSlotKHR structure used for generating a DPB reference slot and Picture Resource. pSetupReferenceSlot->slotIndex specifies the slot index number to use as a target for producing the DPB data. slotIndex must reference a valid entry as specified in VkVideoBeginCodingInfoKHR via the pReferenceSlots within the vkCmdBeginVideoCodingKHR command that established the Vulkan Video Decode Context for this command.

  • referenceSlotCount is the number of the DPB Reference Pictures that will be used when this decoding operation is executing.

  • pReferenceSlots is a pointer to an array of VkVideoReferenceSlotKHR structures specifying the DPB Reference pictures that will be used when this decoding operation is executing.

Valid Usage (Implicit)
  • VUID-VkVideoDecodeInfoKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_DECODE_INFO_KHR

  • VUID-VkVideoDecodeInfoKHR-pNext-pNext
    pNext must be NULL

  • VUID-VkVideoDecodeInfoKHR-flags-parameter
    flags must be a valid combination of VkVideoDecodeFlagBitsKHR values

  • VUID-VkVideoDecodeInfoKHR-srcBuffer-parameter
    srcBuffer must be a valid VkBuffer handle

  • VUID-VkVideoDecodeInfoKHR-dstPictureResource-parameter
    dstPictureResource must be a valid VkVideoPictureResourceKHR structure

  • VUID-VkVideoDecodeInfoKHR-pSetupReferenceSlot-parameter
    pSetupReferenceSlot must be a valid pointer to a valid VkVideoReferenceSlotKHR structure

  • VUID-VkVideoDecodeInfoKHR-pReferenceSlots-parameter
    pReferenceSlots must be a valid pointer to an array of referenceSlotCount valid VkVideoReferenceSlotKHR structures

  • VUID-VkVideoDecodeInfoKHR-referenceSlotCount-arraylength
    referenceSlotCount must be greater than 0

The vkCmdDecodeVideoKHR flags are defined with the following enumeration:

// Provided by VK_KHR_video_decode_queue
typedef enum VkVideoDecodeFlagBitsKHR {
    VK_VIDEO_DECODE_DEFAULT_KHR = 0,
    VK_VIDEO_DECODE_RESERVED_0_BIT_KHR = 0x00000001,
} VkVideoDecodeFlagBitsKHR;
  • VK_VIDEO_DECODE_RESERVED_0_BIT_KHR The current version of the specification has reserved this value for future use.

// Provided by VK_KHR_video_decode_queue
typedef VkFlags VkVideoDecodeFlagsKHR;

VkVideoDecodeFlagsKHR is a bitmask type for setting a mask of zero or more VkVideoDecodeFlagBitsKHR.

35.6. Video Encode Operations

Before the application can start recording Vulkan command buffers for the Video Encode Operations, it must do the following, beforehand:

  1. Ensure that the implementation can encode the Video Content by querying the supported codec operations and profiles using vkGetPhysicalDeviceQueueFamilyProperties2.

  2. By using vkGetPhysicalDeviceVideoFormatPropertiesKHR and providing one or more video profiles, choose the Vulkan formats supported by the implementation. The formats for input and reference pictures must be queried and chosen separately. Refer to the section on enumeration of supported video formats.

  3. Before creating an image to be used as a video picture resource, obtain the supported image creation parameters by querying with vkGetPhysicalDeviceFormatProperties2 and vkGetPhysicalDeviceImageFormatProperties2 using one of the reported formats and adding VkVideoProfilesKHR to the pNext chain of VkFormatProperties2. When querying the parameters with vkGetPhysicalDeviceImageFormatProperties2 for images targeting input and reference (DPB) pictures, the VkPhysicalDeviceImageFormatInfo2::usage field should contain VK_IMAGE_USAGE_VIDEO_ENCODE_SRC_BIT_KHR and VK_IMAGE_USAGE_VIDEO_ENCODE_DPB_BIT_KHR, respectively.

  4. Create none, some, or all of the required images for the input and reference pictures. More Video Picture Resources can be created at some later point if needed while processing the content to be encoded. Also, if the size of the picture to be encoded is expected to change, the images can be created based on the maximum expected content size.

  5. Create the video session to be used for video encode operations. Before creating the Encode Video Session, the encode capabilities should be queried with vkGetPhysicalDeviceVideoCapabilitiesKHR to obtain the limits of the parameters allowed by the implementation for a particular codec profile.

  6. Bind memory resources with the encode video session by calling vkBindVideoSessionMemoryKHR. The video session cannot be used until memory resources are allocated and bound to it. In order to determine the required memory sizes and heap types of the device memory allocations, vkGetVideoSessionMemoryRequirementsKHR should be called.

  7. Create one or more Session Parameter objects for use across command buffer recording operations, if required by the codec extension in use. These objects must be created against a video session with the parameters required by the codec. Each Session Parameter object created is a child object of the associated Session object and cannot be bound in the command buffer with any other Session Object.

The recording of Video Encode Commands against a Vulkan Command Buffer consists of the following sequence:

  1. vkCmdBeginVideoCodingKHR starts the recording of one or more Video Encode operations in the command buffer. For each Video Encode Command operation, a Video Session must be bound to the command buffer within this command. This command establishes a Vulkan Video Encode Context that consists of the bound Video Session Object, Session Parameters Object, and the required Video Picture Resources. The established Video Encode Context is in effect until the vkCmdEndVideoCodingKHR command is recorded. If more Video Encode operations are to be required after the vkCmdEndVideoCodingKHR command, another Video Encode Context can be started with the vkCmdBeginVideoCodingKHR command.

  2. vkCmdEncodeVideoKHR specifies one or more frames to be encoded. The VkVideoEncodeInfoKHR parameters, and the codec extension structures chained to this, specify the details of the encode operation.

  3. vkCmdControlVideoCodingKHR records operations against the encoded data, encoding device, or the Video Session state.

  4. vkCmdEndVideoCodingKHR signals the end of the recording of the Vulkan Video Encode Context, as established by vkCmdBeginVideoCodingKHR.

In addition to the above, the following commands can be recorded between vkCmdBeginVideoCodingKHR and vkCmdEndVideoCodingKHR:

  • Query operations

  • Global Memory Barriers

  • Buffer Memory Barriers

  • Image Memory Barriers (these must be used to transition the Video Picture Resources to the proper VK_IMAGE_LAYOUT_VIDEO_ENCODE_SRC_KHR and VK_IMAGE_LAYOUT_VIDEO_ENCODE_DPB_KHR layouts).

  • Pipeline Barriers

  • Events

  • Timestamps

  • Device Groups (device mask)

The following Video Encode related commands must be recorded outside the Vulkan Video Encode Context established with the vkCmdBeginVideoCodingKHR and vkCmdEndVideoCodingKHR commands:

  • Sparse Memory Binding

  • Copy Commands

  • Clear Commands

35.6.1. Video Encode Vulkan Command Buffer Commands

To launch an encode operation that results in bitstream generation, call:

// Provided by VK_KHR_video_encode_queue
void vkCmdEncodeVideoKHR(
    VkCommandBuffer                             commandBuffer,
    const VkVideoEncodeInfoKHR*                 pEncodeInfo);
  • commandBuffer is the command buffer to be filled with this function for encoding to generate a bitstream.

  • pEncodeInfo is a pointer to a VkVideoEncodeInfoKHR structure.

Valid Usage (Implicit)
  • VUID-vkCmdEncodeVideoKHR-commandBuffer-parameter
    commandBuffer must be a valid VkCommandBuffer handle

  • VUID-vkCmdEncodeVideoKHR-pEncodeInfo-parameter
    pEncodeInfo must be a valid pointer to a valid VkVideoEncodeInfoKHR structure

  • VUID-vkCmdEncodeVideoKHR-commandBuffer-recording
    commandBuffer must be in the recording state

  • VUID-vkCmdEncodeVideoKHR-commandBuffer-cmdpool
    The VkCommandPool that commandBuffer was allocated from must support encode operations

  • VUID-vkCmdEncodeVideoKHR-renderpass
    This command must only be called outside of a render pass instance

  • VUID-vkCmdEncodeVideoKHR-bufferlevel
    commandBuffer must be a primary VkCommandBuffer

Host Synchronization
  • Host access to the VkCommandPool that commandBuffer was allocated from must be externally synchronized

Command Properties
Command Buffer Levels Render Pass Scope Supported Queue Types

Primary

Outside

Encode

The VkVideoEncodeInfoKHR structure is defined as:

// Provided by VK_KHR_video_encode_queue
typedef struct VkVideoEncodeInfoKHR {
    VkStructureType                   sType;
    const void*                       pNext;
    VkVideoEncodeFlagsKHR             flags;
    uint32_t                          qualityLevel;
    VkExtent2D                        codedExtent;
    VkBuffer                          dstBitstreamBuffer;
    VkDeviceSize                      dstBitstreamBufferOffset;
    VkDeviceSize                      dstBitstreamBufferMaxRange;
    VkVideoPictureResourceKHR         srcPictureResource;
    const VkVideoReferenceSlotKHR*    pSetupReferenceSlot;
    uint32_t                          referenceSlotCount;
    const VkVideoReferenceSlotKHR*    pReferenceSlots;
} VkVideoEncodeInfoKHR;
  • sType is the type of this structure.

  • pNext is a pointer to a structure extending this structure. A codec-specific extension structure must be chained to specify what bitstream unit to generate with this encode operation.

  • flags is a bitmask of VkVideoEncodeFlagBitsKHR specifying encode flags, and is reserved for future versions of this specification.

  • qualityLevel is the coding quality level of the encoding. It is defined by the codec-specific extensions.

  • codedExtent is the coded size of the encode operations.

  • dstBitstreamBuffer is the buffer where the encoded bitstream output will be produced.

  • dstBitstreamBufferOffset is the offset in the dstBitstreamBuffer where the encoded bitstream output will start. dstBitstreamBufferOffset’s value must be aligned to VkVideoCapabilitiesKHR::minBitstreamBufferOffsetAlignment, as reported by the implementation.

  • dstBitstreamBufferMaxRange is the maximum size of the dstBitstreamBuffer that can be used while the encoded bitstream output is produced. dstBitstreamBufferMaxRange’s value must be aligned to VkVideoCapabilitiesKHR::minBitstreamBufferSizeAlignment, as reported by the implementation.

  • srcPictureResource is the Picture Resource of the Input Picture to be encoded by the operation.

  • pSetupReferenceSlot is a pointer to a VkVideoReferenceSlotKHR structure used for generating a reconstructed reference slot and Picture Resource. pSetupReferenceSlot->slotIndex specifies the slot index number to use as a target for producing the Reconstructed (DPB) data. pSetupReferenceSlot must be one of the entries provided in VkVideoBeginCodingInfoKHR via the pReferenceSlots within the vkCmdBeginVideoCodingKHR command that established the Vulkan Video Encode Context for this command.

  • referenceSlotCount is the number of Reconstructed Reference Pictures that will be used when this encoding operation is executing.

  • pReferenceSlots is NULL or a pointer to an array of VkVideoReferenceSlotKHR structures that will be used when this encoding operation is executing. Each entry in pReferenceSlots must be one of the entries provided in VkVideoBeginCodingInfoKHR via the pReferenceSlots within the vkCmdBeginVideoCodingKHR command that established the Vulkan Video Encode Context for this command.

Multiple vkCmdEncodeVideoKHR commands may be recorded within a Vulkan Video Encode Context. The execution of each vkCmdEncodeVideoKHR command will result in generating codec-specific bitstream units. These bitstream units are generated consecutively into the bitstream buffer specified in dstBitstreamBuffer of VkVideoEncodeInfoKHR within the vkCmdBeginVideoCodingKHR command. The produced bitstream is the sum of all these bitstream units, including any padding between the bitstream units. Any bitstream padding must be filled with data compliant to the codec standard so as not to cause any syntax errors during decoding of the bitstream units with the padding included. The range of the bitstream buffer written can be queried via video encode bitstream buffer range queries.

Valid Usage (Implicit)
  • VUID-VkVideoEncodeInfoKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_ENCODE_INFO_KHR

  • VUID-VkVideoEncodeInfoKHR-pNext-pNext
    pNext must be NULL

  • VUID-VkVideoEncodeInfoKHR-flags-parameter
    flags must be a valid combination of VkVideoEncodeFlagBitsKHR values

  • VUID-VkVideoEncodeInfoKHR-dstBitstreamBuffer-parameter
    dstBitstreamBuffer must be a valid VkBuffer handle

  • VUID-VkVideoEncodeInfoKHR-srcPictureResource-parameter
    srcPictureResource must be a valid VkVideoPictureResourceKHR structure

  • VUID-VkVideoEncodeInfoKHR-pSetupReferenceSlot-parameter
    pSetupReferenceSlot must be a valid pointer to a valid VkVideoReferenceSlotKHR structure

  • VUID-VkVideoEncodeInfoKHR-pReferenceSlots-parameter
    pReferenceSlots must be a valid pointer to an array of referenceSlotCount valid VkVideoReferenceSlotKHR structures

  • VUID-VkVideoEncodeInfoKHR-referenceSlotCount-arraylength
    referenceSlotCount must be greater than 0

The vkCmdEncodeVideoKHR flags are defined with the following enumeration:

// Provided by VK_KHR_video_encode_queue
typedef enum VkVideoEncodeFlagBitsKHR {
    VK_VIDEO_ENCODE_DEFAULT_KHR = 0,
    VK_VIDEO_ENCODE_RESERVED_0_BIT_KHR = 0x00000001,
} VkVideoEncodeFlagBitsKHR;
  • VK_VIDEO_ENCODE_RESERVED_0_BIT_KHR The current version of the specification has reserved this value for future use.

// Provided by VK_KHR_video_encode_queue
typedef VkFlags VkVideoEncodeFlagsKHR;

VkVideoEncodeFlagsKHR is a bitmask type for setting a mask of zero or more VkVideoEncodeFlagBitsKHR.

The VkVideoEncodeRateControlInfoKHR structure is defined as:

// Provided by VK_KHR_video_encode_queue
typedef struct VkVideoEncodeRateControlInfoKHR {
    VkStructureType                            sType;
    const void*                                pNext;
    VkVideoEncodeRateControlFlagsKHR           flags;
    VkVideoEncodeRateControlModeFlagBitsKHR    rateControlMode;
    uint32_t                                   averageBitrate;
    uint16_t                                   peakToAverageBitrateRatio;
    uint16_t                                   frameRateNumerator;
    uint16_t                                   frameRateDenominator;
    uint32_t                                   virtualBufferSizeInMs;
} VkVideoEncodeRateControlInfoKHR;
  • sType is the type of this structure.

  • pNext is NULL or a pointer to a structure extending this structure.

  • flags is a bitmask of VkVideoEncodeRateControlModeFlagBitsKHR specifying encode rate control flags.

  • rateControlMode is a VkVideoEncodeRateControlModeFlagBitsKHR value specifying the encode rate control mode.

  • averageBitrate is the average bitrate in bits/second. Valid when rate control is not VK_VIDEO_ENCODE_RATE_CONTROL_MODE_NONE_BIT_KHR.

  • peakToAverageBitrateRatio is the peak bitrate to average bitrate in percentage. Valid when rate control is VK_VIDEO_ENCODE_RATE_CONTROL_MODE_VBR_BIT_KHR.

  • frameRateNumerator is the numerator of the frame rate. Valid when rate control is not VK_VIDEO_ENCODE_RATE_CONTROL_MODE_NONE_BIT_KHR.

  • frameRateDenominator is the denominator of the frame rate. Valid when rate control is not VK_VIDEO_ENCODE_RATE_CONTROL_MODE_NONE_BIT_KHR.

  • virtualBufferSizeInMs is the leaky bucket model virtual buffer size in milliseconds, with respect to peak bitrate. Valid when rate control is not VK_VIDEO_ENCODE_RATE_CONTROL_MODE_NONE_BIT_KHR. For example, virtual buffer size is (virtualBufferSizeInMs * peakToAverageBitrateRatio * averageBitrate / 100000).

A codec-specific extension structure for further rate control parameter settings may be chained to VkVideoEncodeRateControlInfoKHR.

Valid Usage (Implicit)
  • VUID-VkVideoEncodeRateControlInfoKHR-sType-sType
    sType must be VK_STRUCTURE_TYPE_VIDEO_ENCODE_RATE_CONTROL_INFO_KHR

  • VUID-VkVideoEncodeRateControlInfoKHR-flags-parameter
    flags must be a valid combination of VkVideoEncodeRateControlFlagBitsKHR values

  • VUID-VkVideoEncodeRateControlInfoKHR-flags-requiredbitmask
    flags must not be 0

  • VUID-VkVideoEncodeRateControlInfoKHR-rateControlMode-parameter
    rateControlMode must be a valid VkVideoEncodeRateControlModeFlagBitsKHR value

The rate control modes are defined with the following enums:

// Provided by VK_KHR_video_encode_queue
typedef enum VkVideoEncodeRateControlModeFlagBitsKHR {
    VK_VIDEO_ENCODE_RATE_CONTROL_MODE_NONE_BIT_KHR = 0,
    VK_VIDEO_ENCODE_RATE_CONTROL_MODE_CBR_BIT_KHR = 1,
    VK_VIDEO_ENCODE_RATE_CONTROL_MODE_VBR_BIT_KHR = 2,
} VkVideoEncodeRateControlModeFlagBitsKHR;
  • VK_VIDEO_ENCODE_RATE_CONTROL_MODE_NONE_BIT_KHR for disabling rate control.

  • VK_VIDEO_ENCODE_RATE_CONTROL_MODE_CBR_BIT_KHR for constant bitrate rate control mode.

  • VK_VIDEO_ENCODE_RATE_CONTROL_MODE_VBR_BIT_KHR for variable bitrate rate control mode.

// Provided by VK_KHR_video_encode_queue
typedef VkFlags VkVideoEncodeRateControlModeFlagsKHR;

VkVideoEncodeRateControlModeFlagsKHR is a bitmask type for setting a mask of zero or more VkVideoEncodeRateControlModeFlagBitsKHR.

The vkCmdControlVideoCodingKHR flags are defined with the following enumeration:

// Provided by VK_KHR_video_encode_queue
typedef enum VkVideoEncodeRateControlFlagBitsKHR {
    VK_VIDEO_ENCODE_RATE_CONTROL_DEFAULT_KHR = 0,
    VK_VIDEO_ENCODE_RATE_CONTROL_RESET_BIT_KHR = 0x00000001,
} VkVideoEncodeRateControlFlagBitsKHR;
  • VK_VIDEO_ENCODE_RESERVED_0_BIT_KHR The current version of the specification has reserved this value for future use.

// Provided by VK_KHR_video_encode_queue
typedef VkFlags VkVideoEncodeRateControlFlagsKHR;

VkVideoEncodeRateControlFlagsKHR is a bitmask type for setting a mask of zero or more VkVideoEncodeRateControlFlagBitsKHR.