gstaudiodecoder

## gstaudiodecoder gstaudiodecoder — Base class for audio decoders |

## Synopsis

    #include <gst/audio/gstaudiodecoder.h>

    struct              GstAudioDecoder;
    struct              GstAudioDecoderClass;
    #define             GST_AUDIO_DECODER_ERROR             (el,
                                                             weight,
                                                             domain,
                                                             code,
                                                             text,
                                                             debug,
                                                             ret)
    #define             GST_AUDIO_DECODER_SINK_NAME
    #define             GST_AUDIO_DECODER_SINK_PAD          (obj)
    #define             GST_AUDIO_DECODER_SRC_NAME
    #define             GST_AUDIO_DECODER_SRC_PAD           (obj)
    GstFlowReturn       gst_audio_decoder_finish_frame      (GstAudioDecoder *dec,
                                                             GstBuffer *buf,
                                                             gint frames);
    GstAudioInfo *      gst_audio_decoder_get_audio_info    (GstAudioDecoder *dec);
    gint                gst_audio_decoder_get_byte_time     (GstAudioDecoder *dec);
    gint                gst_audio_decoder_get_delay         (GstAudioDecoder *dec);
    gboolean            gst_audio_decoder_get_drainable     (GstAudioDecoder *dec);
    void                gst_audio_decoder_get_latency       (GstAudioDecoder *dec,
                                                             GstClockTime *min,
                                                             GstClockTime *max);
    gint                gst_audio_decoder_get_max_errors    (GstAudioDecoder *dec);
    gint64              gst_audio_decoder_get_min_latency   (GstAudioDecoder *dec);
    gboolean            gst_audio_decoder_get_needs_format  (GstAudioDecoder *dec);
    void                gst_audio_decoder_get_parse_state   (GstAudioDecoder *dec,
                                                             gboolean *sync,
                                                             gboolean *eos);
    gboolean            gst_audio_decoder_get_plc           (GstAudioDecoder *dec);
    gint                gst_audio_decoder_get_plc_aware     (GstAudioDecoder *dec);
    gint64              gst_audio_decoder_get_tolerance     (GstAudioDecoder *dec);
    void                gst_audio_decoder_set_byte_time     (GstAudioDecoder *dec,
                                                             gboolean enabled);
    void                gst_audio_decoder_set_drainable     (GstAudioDecoder *dec,
                                                             gboolean enabled);
    void                gst_audio_decoder_set_latency       (GstAudioDecoder *dec,
                                                             GstClockTime min,
                                                             GstClockTime max);
    void                gst_audio_decoder_set_max_errors    (GstAudioDecoder *dec,
                                                             gint num);
    void                gst_audio_decoder_set_min_latency   (GstAudioDecoder *dec,
                                                             gint64 num);
    void                gst_audio_decoder_set_needs_format  (GstAudioDecoder *dec,
                                                             gboolean enabled);
    void                gst_audio_decoder_set_plc           (GstAudioDecoder *dec,
                                                             gboolean enabled);
    void                gst_audio_decoder_set_plc_aware     (GstAudioDecoder *dec,
                                                             gboolean plc);
    void                gst_audio_decoder_set_tolerance     (GstAudioDecoder *dec,
                                                             gint64 tolerance);

## Object Hierarchy

      GObject
       +----GstObject
             +----GstElement
                   +----GstAudioDecoder

## Properties

      "min-latency"              gint64                : Read / Write
      "plc"                      gboolean              : Read / Write
      "tolerance"                gint64                : Read / Write

## Description

This base class is for audio decoders turning encoded data into raw audio samples.

GstAudioDecoder and subclass should cooperate as follows.

1. Configuration

   - Initially, GstAudioDecoder calls `start` when the decoder element is activated, which allows subclass to perform any global setup. Base class (context) parameters can already be set according to subclass capabilities (or possibly upon receive more information in subsequent `set_format`).

   - GstAudioDecoder calls `set_format` to inform subclass of the format of input audio data that it is about to receive. While unlikely, it might be called more than once, if changing input parameters require reconfiguration.

   - GstAudioDecoder calls `stop` at end of all processing.

2. Data processing

   - Base class gathers input data, and optionally allows subclass to parse this into subsequently manageable (as defined by subclass) chunks. Such chunks are subsequently referred to as 'frames', though they may or may not correspond to 1 (or more) audio format frame.

   - Input frame is provided to subclass' `handle_frame`.

   - If codec processing results in decoded data, subclass should call `gst_audio_decoder_finish_frame` to have decoded data pushed downstream.

   - Just prior to actually pushing a buffer downstream, it is passed to `pre_push`. Subclass should either use this callback to arrange for additional downstream pushing or otherwise ensure such custom pushing occurs after at least a method call has finished since setting src pad caps.

   - During the parsing process GstAudioDecoderClass will handle both srcpad and sinkpad events. Sink events will be passed to subclass if `event` callback has been provided.

3. Shutdown phase

   - GstAudioDecoder class calls `stop` to inform the subclass that data parsing will be stopped.

Subclass is responsible for providing pad template caps for source and sink pads. The pads need to be named "sink" and "src". It also needs to set the fixed caps on srcpad, when the format is ensured. This is typically when base class calls subclass' `set_format` function, though it might be delayed until calling `gst_audio_decoder_finish_frame`.

In summary, above process should have subclass concentrating on codec data processing while leaving other matters to base class, such as most notably timestamp handling. While it may exert more control in this area (see e.g. `pre_push`), it is very much not recommended.

In particular, base class will try to arrange for perfect output timestamps as much as possible while tracking upstream timestamps. To this end, if deviation between the next ideal expected perfect timestamp and upstream exceeds "tolerance", then resync to upstream occurs (which would happen always if the tolerance mechanism is disabled).

In non-live pipelines, baseclass can also (configurably) arrange for output buffer aggregation which may help to redue large(r) numbers of small(er) buffers being pushed and processed downstream.

On the other hand, it should be noted that baseclass only provides limited seeking support (upon explicit subclass request), as full-fledged support should rather be left to upstream demuxer, parser or alike. This simple approach caters for seeking and duration reporting using estimated input bitrates.

Things that subclass need to take care of:

- Provide pad templates

- Set source pad caps when appropriate

- Set user-configurable properties to sane defaults for format and implementing codec at hand, and convey some subclass capabilities and expectations in context.

- Accept data in `handle_frame` and provide encoded results to `gst_audio_decoder_finish_frame`. If it is prepared to perform PLC, it should also accept NULL data in `handle_frame` and provide for data for indicated duration.

## Details

### struct GstAudioDecoder

    struct GstAudioDecoder;

The opaque GstAudioDecoder data structure.

Since 0.10.36

### struct GstAudioDecoderClass

    struct GstAudioDecoderClass {
      GstElementClass element_class;

      /* virtual methods for subclasses */

      gboolean      (*start)              (GstAudioDecoder *dec);

      gboolean      (*stop)               (GstAudioDecoder *dec);

      gboolean      (*set_format)         (GstAudioDecoder *dec,
                                           GstCaps *caps);

      GstFlowReturn (*parse)              (GstAudioDecoder *dec,
                                           GstAdapter *adapter,
                                           gint *offset, gint *length);

      GstFlowReturn (*handle_frame)       (GstAudioDecoder *dec,
                                           GstBuffer *buffer);

      void          (*flush)              (GstAudioDecoder *dec, gboolean hard);

      GstFlowReturn (*pre_push)           (GstAudioDecoder *dec,
                                           GstBuffer **buffer);

      gboolean      (*event)              (GstAudioDecoder *dec,
                                           GstEvent *event);
    };

Subclasses can override any of the available virtual methods or not, as needed. At minimum `handle_frame` (and likely `set_format`) needs to be overridden.

GstElementClass `element_class`; | The parent class structure
`start` () | Optional. Called when the element starts processing. Allows opening external resources.
`stop` () | Optional. Called when the element stops processing. Allows closing external resources.
`set_format` () | Notifies subclass of incoming data format (caps).
`parse` () | Optional. Allows chopping incoming data into manageable units (frames) for subsequent decoding. This division is at subclass discretion and may or may not correspond to 1 (or more) frames as defined by audio format.
`handle_frame` () | Provides input data (or NULL to clear any remaining data) to subclass. Input data ref management is performed by base class, subclass should not care or intervene, and input data is only valid until next call to base class, most notably a call to `gst_audio_decoder_finish_frame()`.
`flush` () | Optional. Instructs subclass to clear any codec caches and discard any pending samples and not yet returned decoded data. `hard` indicates whether a FLUSH is being processed, or otherwise a DISCONT (or conceptually similar).
`pre_push` () | Optional. Called just prior to pushing (encoded data) buffer downstream. Subclass has full discretionary access to buffer, and a not OK flow return will abort downstream pushing.
`event` () | Optional. Event handler on the sink pad. This function should return TRUE if the event was handled and should be discarded (i.e. not unref'ed).

Since 0.10.36

### GST_AUDIO_DECODER_ERROR()

    #define             GST_AUDIO_DECODER_ERROR(el, weight, domain, code, text, debug, ret)

Utility function that audio decoder elements can use in case they encountered a data processing error that may be fatal for the current "data unit" but need not prevent subsequent decoding. Such errors are counted and if there are too many, as configured in the context's max_errors, the pipeline will post an error message and the application will be requested to stop further media processing. Otherwise, it is considered a "glitch" and only a warning is logged. In either case, `ret` is set to the proper value to return to upstream/caller (indicating either GST_FLOW_ERROR or GST_FLOW_OK).

`el` : | the base audio decoder element that generates the error
`weight` : | element defined weight of the error, added to error count
`domain` : | like CORE, LIBRARY, RESOURCE or STREAM (see gstreamer-GstGError)
`code` : | error code defined for that domain (see gstreamer-GstGError)
`text` : | the message to display (format string and args enclosed in parentheses)
`debug` : | debugging information for the message (format string and args enclosed in parentheses)
`ret` : | variable to receive return value

Since 0.10.36

### GST_AUDIO_DECODER_SINK_NAME

    #define GST_AUDIO_DECODER_SINK_NAME    "sink"

The name of the templates for the sink pad.

Since 0.10.36

### GST_AUDIO_DECODER_SINK_PAD()

    #define GST_AUDIO_DECODER_SINK_PAD(obj)        (((GstAudioDecoder *) (obj))->sinkpad)

Gives the pointer to the sink GstPad object of the element.

`obj` : | base audio codec instance

Since 0.10.36

### GST_AUDIO_DECODER_SRC_NAME

    #define GST_AUDIO_DECODER_SRC_NAME     "src"

The name of the templates for the source pad.

Since 0.10.36

### GST_AUDIO_DECODER_SRC_PAD()

    #define GST_AUDIO_DECODER_SRC_PAD(obj)         (((GstAudioDecoder *) (obj))->srcpad)

Gives the pointer to the source GstPad object of the element.

`obj` : | base audio codec instance

Since 0.10.36

### gst_audio_decoder_finish_frame ()

    GstFlowReturn       gst_audio_decoder_finish_frame      (GstAudioDecoder *dec,
                                                             GstBuffer *buf,
                                                             gint frames);

Collects decoded data and pushes it downstream.

`buf` may be NULL in which case the indicated number of frames are discarded and considered to have produced no output (e.g. lead-in or setup frames). Otherwise, source pad caps must be set when it is called with valid data in `buf`.

Note that a frame received in `gst_audio_decoder_handle_frame()` may be invalidated by a call to this function.

`dec` : | a GstAudioDecoder
`buf` : | decoded data
`frames` : | number of decoded frames represented by decoded data
Returns : | a GstFlowReturn that should be escalated to caller (of caller)

Since 0.10.36

### gst_audio_decoder_get_audio_info ()

    GstAudioInfo *      gst_audio_decoder_get_audio_info    (GstAudioDecoder *dec);

`dec` : | a GstAudioDecoder
Returns : | a GstAudioInfo describing the input audio format

Since 0.10.36

### gst_audio_decoder_get_byte_time ()

    gint                gst_audio_decoder_get_byte_time     (GstAudioDecoder *dec);

`dec` : | a GstAudioDecoder
Returns : | currently configured byte to time conversion setting

Since 0.10.36

### gst_audio_decoder_get_delay ()

    gint                gst_audio_decoder_get_delay         (GstAudioDecoder *dec);

`dec` : | a GstAudioDecoder
Returns : | currently configured decoder delay

Since 0.10.36

### gst_audio_decoder_get_drainable ()

    gboolean            gst_audio_decoder_get_drainable     (GstAudioDecoder *dec);

Queries decoder drain handling.

`dec` : | a GstAudioDecoder
Returns : | TRUE if drainable handling is enabled. MT safe.

Since 0.10.36

### gst_audio_decoder_get_latency ()

    void                gst_audio_decoder_get_latency       (GstAudioDecoder *dec,
                                                             GstClockTime *min,
                                                             GstClockTime *max);

Sets the variables pointed to by `min` and `max` to the currently configured latency.

`dec` : | a GstAudioDecoder
`min` : | a pointer to storage to hold minimum latency. [out][allow-none]
`max` : | a pointer to storage to hold maximum latency. [out][allow-none]

Since 0.10.36

### gst_audio_decoder_get_max_errors ()

    gint                gst_audio_decoder_get_max_errors    (GstAudioDecoder *dec);

`dec` : | a GstAudioDecoder
Returns : | currently configured decoder tolerated error count.

Since 0.10.36

### gst_audio_decoder_get_min_latency ()

    gint64              gst_audio_decoder_get_min_latency   (GstAudioDecoder *dec);

Queries decoder's latency aggregation.

`dec` : | a GstAudioDecoder
Returns : | aggregation latency. MT safe.

Since 0.10.36

### gst_audio_decoder_get_needs_format ()

    gboolean            gst_audio_decoder_get_needs_format  (GstAudioDecoder *dec);

Queries decoder required format handling.

`dec` : | a GstAudioDecoder
Returns : | TRUE if required format handling is enabled. MT safe.

Since 0.10.36

### gst_audio_decoder_get_parse_state ()

    void                gst_audio_decoder_get_parse_state   (GstAudioDecoder *dec,
                                                             gboolean *sync,
                                                             gboolean *eos);

Return current parsing (sync and eos) state.

`dec` : | a GstAudioDecoder
`sync` : | a pointer to a variable to hold the current sync state
`eos` : | a pointer to a variable to hold the current eos state

Since 0.10.36

### gst_audio_decoder_get_plc ()

    gboolean            gst_audio_decoder_get_plc           (GstAudioDecoder *dec);

Queries decoder packet loss concealment handling.

`dec` : | a GstAudioDecoder
Returns : | TRUE if packet loss concealment is enabled. MT safe.

Since 0.10.36

### gst_audio_decoder_get_plc_aware ()

    gint                gst_audio_decoder_get_plc_aware     (GstAudioDecoder *dec);

`dec` : | a GstAudioDecoder
Returns : | currently configured plc handling

Since 0.10.36

### gst_audio_decoder_get_tolerance ()

    gint64              gst_audio_decoder_get_tolerance     (GstAudioDecoder *dec);

Queries current audio jitter tolerance threshold.

`dec` : | a GstAudioDecoder
Returns : | decoder audio jitter tolerance threshold. MT safe.

Since 0.10.36

### gst_audio_decoder_set_byte_time ()

    void                gst_audio_decoder_set_byte_time     (GstAudioDecoder *dec,
                                                             gboolean enabled);

Allows baseclass to perform byte to time estimated conversion.

`dec` : | a GstAudioDecoder
`enabled` : | whether to enable byte to time conversion

Since 0.10.36

### gst_audio_decoder_set_drainable ()

    void                gst_audio_decoder_set_drainable     (GstAudioDecoder *dec,
                                                             gboolean enabled);

Configures decoder drain handling. If drainable, subclass might be handed a NULL buffer to have it return any leftover decoded data. Otherwise, it is not considered so capable and will only ever be passed real data.

MT safe.

`dec` : | a GstAudioDecoder
`enabled` : | new state

Since 0.10.36

### gst_audio_decoder_set_latency ()

    void                gst_audio_decoder_set_latency       (GstAudioDecoder *dec,
                                                             GstClockTime min,
                                                             GstClockTime max);

Sets decoder latency.

`dec` : | a GstAudioDecoder
`min` : | minimum latency
`max` : | maximum latency

Since 0.10.36

### gst_audio_decoder_set_max_errors ()

    void                gst_audio_decoder_set_max_errors    (GstAudioDecoder *dec,
                                                             gint num);

Sets numbers of tolerated decoder errors, where a tolerated one is then only warned about, but more than tolerated will lead to fatal error. Default is set to GST_AUDIO_DECODER_MAX_ERRORS.

`dec` : | a GstAudioDecoder
`num` : | max tolerated errors

Since 0.10.36

### gst_audio_decoder_set_min_latency ()

    void                gst_audio_decoder_set_min_latency   (GstAudioDecoder *dec,
                                                             gint64 num);

Sets decoder minimum aggregation latency.

MT safe.

`dec` : | a GstAudioDecoder
`num` : | new minimum latency

Since 0.10.36

### gst_audio_decoder_set_needs_format ()

    void                gst_audio_decoder_set_needs_format  (GstAudioDecoder *dec,
                                                             gboolean enabled);

Configures decoder format needs. If enabled, subclass needs to be negotiated with format caps before it can process any data. It will then never be handed any data before it has been configured. Otherwise, it might be handed data without having been configured and is then expected being able to do so either by default or based on the input data.

MT safe.

`dec` : | a GstAudioDecoder
`enabled` : | new state

Since 0.10.36

### gst_audio_decoder_set_plc ()

    void                gst_audio_decoder_set_plc           (GstAudioDecoder *dec,
                                                             gboolean enabled);

Enable or disable decoder packet loss concealment, provided subclass and codec are capable and allow handling plc.

MT safe.

`dec` : | a GstAudioDecoder
`enabled` : | new state

Since 0.10.36

### gst_audio_decoder_set_plc_aware ()

    void                gst_audio_decoder_set_plc_aware     (GstAudioDecoder *dec,
                                                             gboolean plc);

Indicates whether or not subclass handles packet loss concealment (plc).

`dec` : | a GstAudioDecoder
`plc` : | new plc state

Since 0.10.36

### gst_audio_decoder_set_tolerance ()

    void                gst_audio_decoder_set_tolerance     (GstAudioDecoder *dec,
                                                             gint64 tolerance);

Configures decoder audio jitter tolerance threshold.

MT safe.

`dec` : | a GstAudioDecoder
`tolerance` : | new tolerance

Since 0.10.36

## Property Details

### The `"min-latency"` property

      "min-latency"              gint64                : Read / Write

Aggregate output data to a minimum of latency time (ns).

Allowed values: >= 0

Default value: 0

### The `"plc"` property

      "plc"                      gboolean              : Read / Write

Perform packet loss concealment (if supported).

Default value: FALSE

### The `"tolerance"` property

      "tolerance"                gint64                : Read / Write

Perfect ts while timestamp jitter/imperfection within tolerance (ns).

Allowed values: >= 0

Default value: 0

## See Also

GstBaseTransform