Class VideoEncoder

java.lang.Object
org.javagi.base.ProxyInstance
org.gnome.gobject.TypeInstance
org.gnome.gobject.GObject
org.gnome.gobject.InitiallyUnowned
org.freedesktop.gstreamer.gst.GstObject
org.freedesktop.gstreamer.gst.Element
org.freedesktop.gstreamer.video.VideoEncoder
All Implemented Interfaces:
Preset, Proxy
Direct Known Subclasses:
VideoEncoder.VideoEncoder$Impl
@Generated("org.javagi.JavaGI") public abstract class VideoEncoder extends Element implements Preset

This base class is for video encoders turning raw video into encoded video data.

GstVideoEncoder and subclass should cooperate as follows.

Configuration

  • Initially, GstVideoEncoder calls start when the encoder element is activated, which allows subclass to perform any global setup.
  • GstVideoEncoder calls setFormat to inform subclass of the format of input video data that it is about to receive. Subclass should setup for encoding and configure base class as appropriate (e.g. latency). While unlikely, it might be called more than once, if changing input parameters require reconfiguration. Baseclass will ensure that processing of current configuration is finished.
  • GstVideoEncoder calls stop at end of all processing.

Data processing

* Base class collects input data and metadata into a frame and hands
  this to subclass' `handleFrame.`

* If codec processing results in encoded data, subclass should call
  `gstVideoEncoderFinishFrame` to have encoded data pushed
  downstream.

* If implemented, baseclass calls subclass `prePush` just prior to
  pushing to allow subclasses to modify some metadata on the buffer.
  If it returns GST_FLOW_OK, the buffer is pushed downstream.

* GstVideoEncoderClass will handle both srcpad and sinkpad events.
  Sink events will be passed to subclass if `event` callback has been
  provided.

Shutdown phase

  • GstVideoEncoder class calls stop to inform the subclass that data parsing will be stopped.

Subclass is responsible for providing pad template caps for source and sink pads. The pads need to be named "sink" and "src". It should also be able to provide fixed src pad caps in getcaps by the time it calls gstVideoEncoderFinishFrame.

Things that subclass need to take care of:

  • Provide pad templates
  • Provide source pad caps before pushing the first buffer
  • Accept data in handleFrame and provide encoded results to gstVideoEncoderFinishFrame.

The GstVideoEncoder:qos property will enable the Quality-of-Service features of the encoder which gather statistics about the real-time performance of the downstream elements. If enabled, subclasses can use gst_video_encoder_get_max_encode_time() to check if input frames are already late and drop them right away to give a chance to the pipeline to catch up.

  • Constructor Details

    • VideoEncoder

      public VideoEncoder(MemorySegment address)
      Create a VideoEncoder instance for the provided memory address.
      Parameters:
      address - the memory address of the native object

    • VideoEncoder

      public VideoEncoder()
      Create a new VideoEncoder.

  • Method Details

    • getType

      public static @Nullable Type getType()
      Get the GType of the VideoEncoder class.
      Returns:
      the GType

    • getMemoryLayout

      public static MemoryLayout getMemoryLayout()
      The memory layout of the native struct.
      Returns:
      the memory layout

    • asParent

      protected VideoEncoder asParent()
      Return this instance as if it were its parent type. Comparable to the Java super keyword, but ensures the parent typeclass is also used in native code.
      Overrides:
      asParent in class Element
      Returns:
      the instance as if it were its parent type

    • allocateOutputBuffer

      public Buffer allocateOutputBuffer(long size)
      Helper function that allocates a buffer to hold an encoded video frame for encoder's current GstVideoCodecState.
      Parameters:
      size - size of the buffer
      Returns:
      allocated buffer

    • allocateOutputFrame

      public FlowReturn allocateOutputFrame(VideoCodecFrame frame, long size)

      Helper function that allocates a buffer to hold an encoded video frame for encoder's current GstVideoCodecState. Subclass should already have configured video state and set src pad caps.

      The buffer allocated here is owned by the frame and you should only keep references to the frame, not the buffer.

      Parameters:
      frame - a GstVideoCodecFrame
      size - size of the buffer
      Returns:
      FlowReturn.OK if an output buffer could be allocated

    • dropFrame

      public void dropFrame(VideoCodecFrame frame)
      Removes frame from the list of pending frames, releases it and posts a QoS message with the frame's details on the bus. Similar to calling gst_video_encoder_finish_frame() without a buffer attached to frame, but this function additionally stores events from frame as pending, to be pushed out alongside the next frame submitted via gst_video_encoder_finish_frame().
      Parameters:
      frame - a GstVideoCodecFrame
      Since:
      1.26

    • finishFrame

      public FlowReturn finishFrame(VideoCodecFrame frame)

      frame must have a valid encoded data buffer, whose metadata fields are then appropriately set according to frame data or no buffer at all if the frame should be dropped. It is subsequently pushed downstream or provided to prePush. In any case, the frame is considered finished and released.

      If frame does not have a buffer attached, it will be dropped, and a QoS message will be posted on the bus. Events from frame will be pushed out immediately.

      After calling this function the output buffer of the frame is to be considered read-only. This function will also change the metadata of the buffer.

      Parameters:
      frame - an encoded GstVideoCodecFrame
      Returns:
      a GstFlowReturn resulting from sending data downstream

    • finishSubframe

      public FlowReturn finishSubframe(VideoCodecFrame frame)

      If multiple subframes are produced for one input frame then use this method for each subframe, except for the last one. Before calling this function, you need to fill frame->output_buffer with the encoded buffer to push.

      You must call gst_video_encoder_finish_frame() for the last sub-frame to tell the encoder that the frame has been fully encoded.

      This function will change the metadata of frame and frame->output_buffer will be pushed downstream.

      Parameters:
      frame - a GstVideoCodecFrame being encoded
      Returns:
      a GstFlowReturn resulting from pushing the buffer downstream.
      Since:
      1.18

    • getAllocator

      public void getAllocator(@Nullable Out<Allocator> allocator, @Nullable AllocationParams params)

      Lets GstVideoEncoder sub-classes to know the memory allocator used by the base class and its params.

      Unref the allocator after use it.

      Parameters:
      allocator - the GstAllocator used
      params - the GstAllocationParams of allocator

    • getFrame

      public @Nullable VideoCodecFrame getFrame(int frameNumber)
      Get a pending unfinished GstVideoCodecFrame
      Parameters:
      frameNumber - system_frame_number of a frame
      Returns:
      pending unfinished GstVideoCodecFrame identified by frameNumber.

    • getFrames

      public List<VideoCodecFrame> getFrames()
      Get all pending unfinished GstVideoCodecFrame
      Returns:
      pending unfinished GstVideoCodecFrame.

    • getLatency

      public void getLatency(@Nullable ClockTime minLatency, @Nullable ClockTime maxLatency)
      Query the configured encoding latency. Results will be returned via minLatency and maxLatency.
      Parameters:
      minLatency - address of variable in which to store the configured minimum latency, or null
      maxLatency - address of variable in which to store the configured maximum latency, or null

    • getMaxEncodeTime

      public ClockTimeDiff getMaxEncodeTime(VideoCodecFrame frame)

      Determines maximum possible encoding time for frame that will allow it to encode and arrive in time (as determined by QoS events). In particular, a negative result means encoding in time is no longer possible and should therefore occur as soon/skippy as possible.

      If no QoS events have been received from downstream, or if GstVideoEncoder:qos is disabled this function returns G_MAXINT64.

      Parameters:
      frame - a GstVideoCodecFrame
      Returns:
      max decoding time.
      Since:
      1.14

    • getMinForceKeyUnitInterval

      public ClockTime getMinForceKeyUnitInterval()
      Returns the minimum force-keyunit interval, see gst_video_encoder_set_min_force_key_unit_interval() for more details.
      Returns:
      the minimum force-keyunit interval
      Since:
      1.18

    • getOldestFrame

      public @Nullable VideoCodecFrame getOldestFrame()
      Get the oldest unfinished pending GstVideoCodecFrame
      Returns:
      oldest unfinished pending GstVideoCodecFrame

    • getOutputState

      public @Nullable VideoCodecState getOutputState()
      Get the current GstVideoCodecState
      Returns:
      GstVideoCodecState describing format of video data.

    • isQosEnabled

      public boolean isQosEnabled()
      Checks if this VideoEncoder is currently configured to handle Quality-of-Service events from downstream.
      Returns:
      true if the encoder is configured to perform Quality-of-Service.
      Since:
      1.14

    • mergeTags

      public void mergeTags(@Nullable TagList tags, TagMergeMode mode)

      Sets the video encoder tags and how they should be merged with any upstream stream tags. This will override any tags previously-set with gst_video_encoder_merge_tags().

      Note that this is provided for convenience, and the subclass is not required to use this and can still do tag handling on its own.

      MT safe.

      Parameters:
      tags - a GstTagList to merge, or NULL to unset previously-set tags
      mode - the GstTagMergeMode to use, usually GST_TAG_MERGE_REPLACE

    • negotiate

      public boolean negotiate()
      Negotiate with downstream elements to currently configured GstVideoCodecState. Unmark GST_PAD_FLAG_NEED_RECONFIGURE in any case. But mark it again if negotiate fails.
      Returns:
      true if the negotiation succeeded, else false.

    • proxyGetcaps

      public Caps proxyGetcaps(@Nullable Caps caps, @Nullable Caps filter)
      Returns caps that express caps (or sink template caps if caps == NULL) restricted to resolution/format/... combinations supported by downstream elements (e.g. muxers).
      Parameters:
      caps - initial caps
      filter - filter caps
      Returns:
      a GstCaps owned by caller

    • releaseFrame

      public void releaseFrame(VideoCodecFrame frame)
      Removes frame from list of pending frames and releases it, similar to calling gst_video_encoder_finish_frame() without a buffer attached to the frame, but does not post a QoS message or do any additional processing. Events from frame are moved to the pending events list.
      Parameters:
      frame - a GstVideoCodecFrame
      Since:
      1.26

    • setHeaders

      public void setHeaders(List<Buffer> headers)
      Set the codec headers to be sent downstream whenever requested.
      Parameters:
      headers - a list of GstBuffer containing the codec header

    • setLatency

      public void setLatency(ClockTime minLatency, ClockTime maxLatency)
      Informs baseclass of encoding latency. If the provided values changed from previously provided ones, this will also post a LATENCY message on the bus so the pipeline can reconfigure its global latency.
      Parameters:
      minLatency - minimum latency
      maxLatency - maximum latency

    • setMinForceKeyUnitInterval

      public void setMinForceKeyUnitInterval(ClockTime interval)
      Sets the minimum interval for requesting keyframes based on force-keyunit events. Setting this to 0 will allow to handle every event, setting this to GST_CLOCK_TIME_NONE causes force-keyunit events to be ignored.
      Parameters:
      interval - minimum interval
      Since:
      1.18

    • setMinPts

      public void setMinPts(ClockTime minPts)

      Request minimal value for PTS passed to handle_frame.

      For streams with reordered frames this can be used to ensure that there is enough time to accommodate first DTS, which may be less than first PTS

      Parameters:
      minPts - minimal PTS that will be passed to handle_frame
      Since:
      1.6

    • setOutputState

      public @Nullable VideoCodecState setOutputState(Caps caps, @Nullable VideoCodecState reference)

      Creates a new GstVideoCodecState with the specified caps as the output state for the encoder. Any previously set output state on this VideoEncoder will be replaced by the newly created one.

      The specified caps should not contain any resolution, pixel-aspect-ratio, framerate, codec-data, .... Those should be specified instead in the returned GstVideoCodecState.

      If the subclass wishes to copy over existing fields (like pixel aspect ratio, or framerate) from an existing GstVideoCodecState, it can be provided as a reference.

      If the subclass wishes to override some fields from the output state (like pixel-aspect-ratio or framerate) it can do so on the returned GstVideoCodecState.

      The new output state will only take effect (set on pads and buffers) starting from the next call to gst_video_encoder_finish_frame().

      Parameters:
      caps - the GstCaps to use for the output
      reference - An optional reference GstVideoCodecState
      Returns:
      the newly configured output state.

    • setQosEnabled

      public void setQosEnabled(boolean enabled)
      Configures this VideoEncoder to handle Quality-of-Service events from downstream.
      Parameters:
      enabled - the new qos value.
      Since:
      1.14

    • close

      protected boolean close()
      Optional. Called when the element changes to GST_STATE_NULL. Allows closing external resources.

    • decideAllocation

      protected boolean decideAllocation(Query query)
      Optional. Setup the allocation parameters for allocating output buffers. The passed in query contains the result of the downstream allocation query. Subclasses should chain up to the parent implementation to invoke the default handler.

    • finish

      protected FlowReturn finish()
      Optional. Called to request subclass to dispatch any pending remaining data (e.g. at EOS).

    • flush

      protected boolean flush()
      Optional. Flush all remaining data from the encoder without pushing it downstream. Since: 1.2

    • getcaps

      protected Caps getcaps(Caps filter)
      Optional. Allows for a custom sink getcaps implementation (e.g. for multichannel input specification). If not implemented, default returns gst_video_encoder_proxy_getcaps applied to sink template caps.

    • handleFrame

      protected FlowReturn handleFrame(VideoCodecFrame frame)
      Provides input frame to subclass.

    • open

      protected boolean open()
      Optional. Called when the element changes to GST_STATE_READY. Allows opening external resources.

    • prePush

      protected FlowReturn prePush(VideoCodecFrame frame)
      Optional. Allows subclass to push frame downstream in whatever shape or form it deems appropriate. If not provided, provided encoded frame data is simply pushed downstream.

    • proposeAllocation

      protected boolean proposeAllocation(Query query)
      Optional. Propose buffer allocation parameters for upstream elements. Subclasses should chain up to the parent implementation to invoke the default handler.

    • reset

      protected boolean reset(boolean hard)
      Optional. Allows subclass (encoder) to perform post-seek semantics reset. Deprecated.

    • setFormat

      protected boolean setFormat(VideoCodecState state)
      Optional. Notifies subclass of incoming data format. GstVideoCodecState fields have already been set according to provided caps.

    • sinkEvent

      protected boolean sinkEvent(Event event)
      Optional. Event handler on the sink pad. This function should return TRUE if the event was handled and should be discarded (i.e. not unref'ed). Subclasses should chain up to the parent implementation to invoke the default handler.

    • sinkQuery

      protected boolean sinkQuery(Query query)
      Optional. Query handler on the sink pad. This function should return TRUE if the query could be performed. Subclasses should chain up to the parent implementation to invoke the default handler. Since: 1.4

    • srcEvent

      protected boolean srcEvent(Event event)
      Optional. Event handler on the source pad. This function should return TRUE if the event was handled and should be discarded (i.e. not unref'ed). Subclasses should chain up to the parent implementation to invoke the default handler.

    • srcQuery

      protected boolean srcQuery(Query query)
      Optional. Query handler on the source pad. This function should return TRUE if the query could be performed. Subclasses should chain up to the parent implementation to invoke the default handler. Since: 1.4

    • start

      protected boolean start()
      Optional. Called when the element starts processing. Allows opening external resources.

    • stop

      protected boolean stop()
      Optional. Called when the element stops processing. Allows closing external resources.

    • transformMeta

      protected boolean transformMeta(VideoCodecFrame frame, Meta meta)
      Optional. Transform the metadata on the input buffer to the output buffer. By default this method is copies all meta without tags and meta with only the "video" tag. subclasses can implement this method and return true if the metadata is to be copied. Since: 1.6