draft-ietf-payload-rtp-jpegxs-00.txt   draft-ietf-payload-rtp-jpegxs-01.txt 
Payload Working Group S. Lugan Payload Working Group S. Lugan
Internet-Draft G. Rouvroy Internet-Draft G. Rouvroy
Intended status: Standards Track A. Descampe Intended status: Standards Track A. Descampe
Expires: August 29, 2019 intoPIX Expires: October 12, 2019 intoPIX
T. Richter T. Richter
IIS IIS
A. Willeme A. Willeme
UCL/ICTEAM UCL/ICTEAM
February 25, 2019 April 10, 2019
RTP Payload Format for ISO/IEC 21122 (JPEG XS) RTP Payload Format for ISO/IEC 21122 (JPEG XS)
draft-ietf-payload-rtp-jpegxs-00 draft-ietf-payload-rtp-jpegxs-01
Abstract Abstract
This document specifies a Real-Time Transport Protocol (RTP) payload This document specifies a Real-Time Transport Protocol (RTP) payload
format to be used for transporting JPEG XS (ISO/IEC 21122) encoded format to be used for transporting JPEG XS (ISO/IEC 21122) encoded
video. JPEG XS is a low-latency, lightweight image coding system video. JPEG XS is a low-latency, lightweight image coding system.
allowing for an increased resolution and frame rate, while offering Compared to an uncompressed video use case, it allows higher
visually lossless quality with reduced amount of resources such as resolutions and frame rates, while offering visually lossless
power and bandwidth. quality, reduced power consumption, and end-to-end latency confined
to a fraction of a frame.
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/. Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on August 29, 2019. This Internet-Draft will expire on October 12, 2019.
Copyright Notice Copyright Notice
Copyright (c) 2019 IETF Trust and the persons identified as the Copyright (c) 2019 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of (https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 18 skipping to change at page 2, line 19
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Conventions, Definitions, and Abbreviations . . . . . . . . . 3 2. Conventions, Definitions, and Abbreviations . . . . . . . . . 3
3. Media Format Description . . . . . . . . . . . . . . . . . . 4 3. Media Format Description . . . . . . . . . . . . . . . . . . 4
3.1. Image Data Structures . . . . . . . . . . . . . . . . . . 4 3.1. Image Data Structures . . . . . . . . . . . . . . . . . . 4
3.2. Codestream . . . . . . . . . . . . . . . . . . . . . . . 5 3.2. Codestream . . . . . . . . . . . . . . . . . . . . . . . 5
3.3. Video Support Box . . . . . . . . . . . . . . . . . . . . 6 3.3. Video support box and colour specification box . . . . . 5
4. Payload Format . . . . . . . . . . . . . . . . . . . . . . . 6 4. Payload Format . . . . . . . . . . . . . . . . . . . . . . . 5
4.1. Payload Header . . . . . . . . . . . . . . . . . . . . . 7 4.1. Payload Header . . . . . . . . . . . . . . . . . . . . . 6
4.2. Payload Data . . . . . . . . . . . . . . . . . . . . . . 9 4.2. Payload Data . . . . . . . . . . . . . . . . . . . . . . 8
4.3. Traffic Shaping and Delivery Timing . . . . . . . . . . . 10 4.3. Traffic Shaping and Delivery Timing . . . . . . . . . . . 10
5. Congestion Control Considerations . . . . . . . . . . . . . . 10 5. Congestion Control Considerations . . . . . . . . . . . . . . 10
6. Payload Format Parameters . . . . . . . . . . . . . . . . . . 11 6. Payload Format Parameters . . . . . . . . . . . . . . . . . . 10
6.1. Media Type Definition . . . . . . . . . . . . . . . . . . 11 6.1. Media Type Definition . . . . . . . . . . . . . . . . . . 10
6.2. Mapping to SDP . . . . . . . . . . . . . . . . . . . . . 14 6.2. Mapping to SDP . . . . . . . . . . . . . . . . . . . . . 13
6.2.1. General . . . . . . . . . . . . . . . . . . . . . . . 14 6.2.1. General . . . . . . . . . . . . . . . . . . . . . . . 13
6.2.2. Media type and subtype . . . . . . . . . . . . . . . 14 6.2.2. Media type and subtype . . . . . . . . . . . . . . . 14
6.2.3. Traffic shaping . . . . . . . . . . . . . . . . . . . 15 6.2.3. Traffic shaping . . . . . . . . . . . . . . . . . . . 14
6.2.4. Offer/Answer Considerations . . . . . . . . . . . . . 15 6.2.4. Offer/Answer Considerations . . . . . . . . . . . . . 14
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 15 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 15
8. Security Considerations . . . . . . . . . . . . . . . . . . . 15 8. Security Considerations . . . . . . . . . . . . . . . . . . . 15
9. RFC Editor Considerations . . . . . . . . . . . . . . . . . . 16 9. RFC Editor Considerations . . . . . . . . . . . . . . . . . . 16
10. References . . . . . . . . . . . . . . . . . . . . . . . . . 16 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 16
10.1. Normative References . . . . . . . . . . . . . . . . . . 16 10.1. Normative References . . . . . . . . . . . . . . . . . . 16
10.2. Informative References . . . . . . . . . . . . . . . . . 18 10.2. Informative References . . . . . . . . . . . . . . . . . 18
10.3. URIs . . . . . . . . . . . . . . . . . . . . . . . . . . 19 10.3. URIs . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 19 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 18
1. Introduction 1. Introduction
This document specifies a payload format for packetization of JPEG XS This document specifies a payload format for packetization of JPEG XS
encoded video signals into the Real-time Transport Protocol (RTP) encoded video signals into the Real-time Transport Protocol (RTP)
[RFC3550]. [RFC3550].
JPEG XS is a low-latency, lightweight image coding system allowing The JPEG XS coding system offers compression and recompression of
for an increased resolution and frame rate, while offering visually image sequences with very moderate computational resources while
lossless quality with reduced amount of resources such as power and remaining robust under multiple compression and decompression cycles
bandwidth. and mixing of content sources, e.g. embedding of subtitles, overlays
or logos. Typical target compression ratios ensuring visually
lossless quality are in the range of 2:1 to 10:1, depending on the
nature of the source material. The end-to-end latency can be
confined to a fraction of a frame, typically between a small number
of lines down to below a single line.
2. Conventions, Definitions, and Abbreviations 2. Conventions, Definitions, and Abbreviations
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119]. document are to be interpreted as described in RFC 2119 [RFC2119].
Application Data Unit: Application Data Unit (ADU)
The unit of source data provided as payload to the transport The unit of source data provided as payload to the transport
layer, and corresponding, in this RTP payload definition, to a layer, and corresponding, in this RTP payload definition, to a
JPEG XS frame. JPEG XS frame.
EOC Marker Colour specification box
A marker that consists of the two bytex 0xff 0x10 indicating the A ISO colour specification box defined in ISO/IEC 21122-3
start of a JPEG XS codestream. [ISO21122-3] that includes colour-related metadata required to
correctly display JPEG XS frames, such as colour primaries,
transfer characteristics and matrix coefficients.
JPEG XS codestream: EOC marker
A marker that consists of the two bytes 0xff11 indicating the end
of a JPEG XS codestream.
JPEG XS codestream
A sequence of bytes representing a compressed image formatted A sequence of bytes representing a compressed image formatted
according to JPEG XS Part 1 [ISO21122-1]. according to JPEG XS Part 1 [ISO21122-1].
JPEG XS frame: JPEG XS codestream header
The concatenation of a Video Support Box, as defined in JPEG XS A sequence of bytes at the beginning of each JPEG XS codestream
Part 3 [ISO21122-3], and a JPEG XS codestream. encoded in multiple markers and marker segments that does not
carry entropy coded data, but metadata such as the frame dimension
and component precision.
JPEG XS stream: JPEG XS frame
A JPEG XS stream is a sequence of frames, where each frame is The concatenation of a video support box, as defined in JPEG XS
coded independently of each other. For the purpose of RTP Part 3 [ISO21122-3], a colour specification box, as defined as
transport, each frame forms an Application Data Unit (ADU). well in JPEG XS Part 3 [ISO21122-3] and a JPEG XS codestream.
Marker: JPEG XS header segment
The concatenation of a video support box, as defined in JPEG XS
Part 3 [ISO21122-3], a colour specification box, as defined as
well in JPEG XS Part 3 [ISO21122-3] and a JPEG XS codestream
header.
JPEG XS stream
A sequence of JPEG XS frames
Marker
A two-byte functional sequence that is part of a JPEG XS A two-byte functional sequence that is part of a JPEG XS
codestream starting with a 0xff byte and a subsequent byte codestream starting with a 0xff byte and a subsequent byte
defining its function. defining its function.
Marker Segment: Marker segment
A marker along with a 16-bit marker size and payload data A marker along with a 16-bit marker size and payload data
following the size. following the size.
JPEG XS Header: SOC marker
A sequence of bytes at the beginning of each JPEG XS codestream A marker that consists of the two bytes 0xff10 indicating the
encoded in multiple markers and marker segments that does not start of a JPEG XS codestream.
carry entropy coded data, but metadata such as the frame dimension
and component precision.
SOC Marker
A marker that consists of the two bytex 0xff 0x11 indicating the
end of a JPEG XS codestream.
Video Support Box:
A ISO video support box in the sense of ISO/IEC 15444-1
[ISO15444-1] defined in ISO/IEC 21122-3 [ISO21122-3] that includes
metadata required to play back a JPEG XS video stream, such as its
color space, its maximum bitrate, its subsampling structure, its
buffer model and its frame rate.
JPEG XS Header Segment: Video support box
The concatenation of a Video Support Box and JPEG XS header. A ISO video support box defined in ISO/IEC 21122-3 [ISO21122-3]
that includes metadata required to play back a JPEG XS stream,
such as its maximum bitrate, its subsampling structure, its buffer
model and its frame rate.
Slice: Slice
The smallest independently decodable unit of a JPEG XS codestream, The smallest independently decodable unit of a JPEG XS codestream,
bearing in mind that it decodes to wavelet coefficients which bearing in mind that it decodes to wavelet coefficients which
still require inverse wavelet filtering to give an image. still require inverse wavelet filtering to give an image.
Slice group: Metadata
A contiguous sequence of slices. Data that consists either of the JPEG XS header segment or the EOC
marker.
Fragment:
A fragment consists of one slice group, possibly preceded by a
JPEG XS header segment (if the slice group is the first one of a
JPEG XS frame), and possibly followed by the EOC marker (if the
slice group is the last one of a JPEG XS frame).
3. Media Format Description 3. Media Format Description
3.1. Image Data Structures 3.1. Image Data Structures
JPEG XS is a low-latency lightweight image coding system for coding JPEG XS is a low-latency lightweight image coding system for coding
continuous-tone grayscale or continuous-tone color digital images. continuous-tone grayscale or continuous-tone colour digital images.
This coding system provides an efficient representation of image This coding system provides an efficient representation of image
signals through the mathematical tool of wavelet analysis. The signals through the mathematical tool of wavelet analysis. The
wavelet filter process separates each component into multiple bands, wavelet filter process separates each component into multiple bands,
where each band consists of multiple coefficients describing the where each band consists of multiple coefficients describing the
image signal of a given component within a frequency domain specific image signal of a given component within a frequency domain specific
to the wavelet filter type, i.e. the particular filter corresponding to the wavelet filter type, i.e. the particular filter corresponding
to the band. to the band.
Wavelet coefficients are grouped into precincts, where each precinct Wavelet coefficients are grouped into precincts, where each precinct
skipping to change at page 5, line 7 skipping to change at page 5, line 13
region of the image. region of the image.
One or multiple precincts are furthermore combined into slices One or multiple precincts are furthermore combined into slices
consisting of an integral number of precincts. Precincts do not consisting of an integral number of precincts. Precincts do not
cross slice boundaries, and wavelet coefficients in precincts that cross slice boundaries, and wavelet coefficients in precincts that
are part of different slices can be decoded independently from each are part of different slices can be decoded independently from each
other. Note, however, that the wavelet transformation runs across other. Note, however, that the wavelet transformation runs across
slice boundaries. A slice always extends over the full width of the slice boundaries. A slice always extends over the full width of the
image, but may only cover parts of its height. image, but may only cover parts of its height.
Multiple contiguous slices are combined into slice groups. Slice Data that is not part of any slice is metadata. It consists of
groups along with preceding and/or following metadata form fragments. either the JPEG XS header segment preceeding any slice data, or the
A fragment, and by that the corresponding slice group, is sized such EOC marker which follows the last slice.
that it is spread over at least two distinct RTP packets, except for
the last fragment of an Application Data Unit.
Slice groups within a frame are enumerated from top to bottom by the
slice group counter. That is, the first slice group of a frame is
slice group #0, and the slice group counter increments by 1 from top
to bottom for each slice group, and by that for each fragment.
Figure 1 shows an example of packets, slices, slice groups and
fragments. In this Figure, MDT indicates metadata preceding or
following slice groups, SlcGrp the slice groups and Slc the slices.
As seen there, a fragment may contain more than one slice if the
slices are too short to fill up an entire packet, and fragment and
packet boundaries need only to align at the start and the end of the
ADU. Fragments may extend over more than two packets, depending on
their size, but a packet never contains two entire fragments or more.
Slice group and fragment boundaries coincide, except for the first
and the last fragment, which include additional metadata. Unlike
regularly sized packets, the fragment and the slice group size may
vary.
<------------------- Application Data Unit (ADU) ------------------->
+-----------+-----------+-----------+-----------+-/ /-+-------------+
| Packet #0 | Packet #1 | Packet #2 | Packet #3 | | Packet #n-1 |
+-----------+---+-------+-----------+---+-------+-/ /-+-------------+
| Fragment #0 | Fragment #1 | Fragment #m-1 |
+---+-----------+-----------------------+---------/ /-----------+---+
|MDT| SlcGrp #0 | SlcGrp #1 | SlcGrp #m-1 | M |
+---+-----------+-----------------------+---------/ /-----------+---+
|MDT|Slc#0 Slc#1| Slc #2 | Slc #k-1 | M |
+---+-----------+-----------------------+---------/ /-----------+---+
Figure 1: Slice Groups and Fragments
3.2. Codestream 3.2. Codestream
The overall codestream format, including the definition of all The overall codestream format, including the definition of all
markers, is further defined in ISO/IEC 21122-1 [ISO21122-1]. It markers, is further defined in ISO/IEC 21122-1 [ISO21122-1]. It
represents sample values of a single frame, bare any interpretation represents sample values of a single frame, bare any interpretation
relative to a colorspace. relative to a colour space.
3.3. Video Support Box 3.3. Video support box and colour specification box
While the information defined in the codestream is sufficient to While the information defined in the codestream is sufficient to
reconstruct the sample values of one video frame, the interpretation reconstruct the sample values of one video frame, the interpretation
of the samples remains undefined by the codestream itself. This of the samples remains undefined by the codestream itself. This
interpretation, including the color space, frame rate and other interpretation is given by the video support box and the colour
information significant to play a JPEG XS stream are contained in the specification box which contain significant information to correctly
Video Support Box, which precedes each JPEG XS codestream. The play the JPEG XS stream. The layout and syntax of these boxes,
syntax of the Video Support Box follows ISO/IEC 15444-1 [ISO15444-1]; together with their content, are defined in ISO/IEC 21122-3
it consists of multiple subboxes, each with a particular meaning. [ISO21122-3]. The video support box provides information on the
Its contents, in particular its subboxes are defined in ISO/IEC maximum bitrate, the frame rate, the subsampling image format, the
21122-3 [ISO21122-3]. timecode of the current JPEG XS frame, the profile, level and
sublevel used (as defined in ISO/IEC 21122-2 [ISO21122-2]), and
optionally on the buffer model and the mastering display metadata.
The colour specification box indicates the colour primaries, transfer
characteristics, matrix coefficients and video full range flag needed
to specify the colour space of the video stream.
4. Payload Format 4. Payload Format
This section specifies the payload format for JPEG XS video streams This section specifies the payload format for JPEG XS streams over
over the Real-time Transport Protocol (RTP) [RFC3550]. the Real-time Transport Protocol (RTP) [RFC3550].
In order to be transported over RTP, each JPEG XS stream is In order to be transported over RTP, each JPEG XS stream is
transported in a distinct RTP stream, identified by a distinct SSRC. transported in a distinct RTP stream, identified by a distinct SSRC.
Each of those RTP streams is divided into Application Data Units A JPEG XS stream is divided into Application Data Units (ADUs), each
(ADUs). ADU corresponding to a single JPEG XS frame.
Each ADU is split into packets, depending e.g. on the Maximum An ADU is split into multiple RTP packet payloads. Figure 1 shows an
Transmission unit (MTU) of the network. Every packet shall have the example of how slices and metadata fit into the payload of RTP
same size, except the last packet of every ADU which could be packets ("Hdr" denotes a RTP packet header). As seen there, each
shorter. Packet boundaries shall coincide with ADU boundaries, i.e. packet contains metadata or data from a single slice, but a slice or
the first (resp. last) byte of an ADU shall be the first (resp. last) metadata may extend over multiple packets. The payload of every
byte of an RTP packet payload data. packet shall have the same size (based e.g. on the Maximum Transfer
Unit of the network), except (possibly) the last packet of a slice or
metadata. The boundaries of a slice or metadata shall coincide with
the boundaries of the payload of a packet, i.e. the first (resp.
last) byte of a slice or metadata shall be the first (resp. last)
byte of the payload. In particular, this implies that the EOC marker
is sent in a packet of its own.
The following figure illustrates the RTP payload header used in order RTP +-----+------------------------+
to transport a JPEG XS stream. Packet #1 | Hdr | JPEG XS header segment |
+-----+------------------------+
RTP +-----+---------------------------+
Packet #2 | Hdr | Slice 0 |
+-----+---------------------------+
RTP +-----+---------------------------------------------+
Packet #3 | Hdr | Slice 1 (part 1/3) |
+-----+---------------------------------------------+
RTP +-----+---------------------------------------------+
Packet #4 | Hdr | Slice 1 (part 2/3) |
+-----+---------------------------------------------+
RTP +-----+---------------------+
Packet #5 | Hdr | Slice 1 (part 3/3) |
+-----+---------------------+
...
RTP +-----+-----+
Packet #N | Hdr | EOC |
+-----+-----+
Figure 1: Example of slices and metadata of an ADU
4.1. Payload Header
Figure 2 illustrates the RTP payload header used in order to
transport a JPEG XS stream.
0 1 2 3 0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+---+-+-+-------+-+-------------+-------------------------------+ +---+-+-+-------+-+-------------+-------------------------------+
| V |P|X| CC |M| PT | Sequence number | | V |P|X| CC |M| PT | Sequence number |
+---+-+-+-------+-+-------------+-------------------------------+ +---+-+-+-------+-+-------------+-------------------------------+
| Timestamp | | Timestamp |
+---------------------------------------------------------------+ +---------------------------------------------------------------+
| Synchronization source (SSRC) identifier | | Synchronization source (SSRC) identifier |
+===============================================================+ +===============================================================+
| Contributing source (CSRC) identifiers | | Contributing source (CSRC) identifiers |
| .... | | .... |
+-----+-+-+---------+---------------------+---------------------+ +-+-------------+-----------------------+-----------------------+
| Ver |f|c| SlcGrp | SlcGrpOffset | Frame counter | |L|Frame counter| Slice counter | Packet counter |
+-----+-+-+---------+---------------------+---------------------+ +-+-------------+-----------------------+-----------------------+
| Data | | Data |
+---------------------------------------------------------------+ +---------------------------------------------------------------+
Figure 2: RTP and payload headers Figure 2: RTP and payload headers
4.1. Payload Header
The version (V), padding (P), extension (X), CSRC count (CC), The version (V), padding (P), extension (X), CSRC count (CC),
sequence number, synchronization source (SSRC) and contributing sequence number, synchronization source (SSRC) and contributing
source (CSRC) fields follow their respective definitions in RFC 3550 source (CSRC) fields follow their respective definitions in RFC 3550
[RFC3550]. [RFC3550].
The timestamp should be based on a globally synchronized 90 kHz clock The timestamp SHOULD be based on a 90 kHz clock reference.
reference, and should correspond to the number of cycles since the
SMPTE Epoch (as per defined in SMPTE ST 2059-1:2015 [SMPTE-ST2059])
modulo 2^32:
timestamp = floor(time_since_epoch*90000) % 2^32
where time_since_epoch is the time elapsed since the SMPTE Epoch,
expressed in seconds as a real number, and floor indicates rounding
to the next lower integer.
As per specified in RFC 3550 [RFC3550] and RFC 4175 [RFC4175], the As per specified in RFC 3550 [RFC3550] and RFC 4175 [RFC4175], the
RTP timestamp designates the sampling instant of the first octet of RTP timestamp designates the sampling instant of the first octet of
the frame to which the RTP packet belongs. Packets shall not include the frame to which the RTP packet belongs. Packets shall not include
data from multiple frames, and all packets belonging to the same data from multiple frames, and all packets belonging to the same
frame shall have the same timestamp. Several successive RTP packets frame shall have the same timestamp. Several successive RTP packets
will consequently have equal timestamps if they belong to the same will consequently have equal timestamps if they belong to the same
frame (that is until the marker bit is set to 1, marking the last frame (that is until the marker bit is set to 1, marking the last
packet of the frame), and the timestamp is only increased when a new packet of the frame), and the timestamp is only increased when a new
frame begins. frame begins.
If the sampling instant does not correspond to an integer value of If the sampling instant does not correspond to an integer value of
the clock, the value shall be truncated to the next lowest integer, the clock, the value shall be truncated to the next lowest integer,
with no ambiguity. with no ambiguity.
The remaining fields are defined as follows: The remaining fields are defined as follows:
Marker (M) [1 bit]: Marker (M) [1 bit]:
The marker bit is used to indicate the last packet of a frame. The M bit is used to indicate the last packet of a frame. This
This enables a decoder to finish decoding the frame, where it enables a decoder to finish decoding the frame, where it otherwise
otherwise may need to wait for the next packet to explicitly know may need to wait for the next packet to explicitly know that the
that the frame is finished. frame is finished.
Payload Type (PT) [7 bits]: Payload Type (PT) [7 bits]:
A dynamically allocated payload type field that designates the A dynamically allocated payload type field that designates the
payload as JPEG XS video. payload as JPEG XS video.
Ver [3 bits]: Last (L) [1 bit]:
This field indicates the version number of the payload header. The L bit is set to indicate the last packet of a slice or
The value of this field shall be 0 for the purpose of this edition metadata. It enables the decoder to already start decoding a
of the RFC. slice without having to wait for the full frame to finish, and
thus allows low-latency decoding. As the end of the frame also
f [1 bit]: ends the metadata packet containing the EOC marker, the L bit is
The f field shall be set if a new fragment is started within this set whenever the M bit is set.
packet, i.e. if this packet contains the first byte of a fragment.
NOTE: The JPEG XS header segment and the first slice group form a
fragment. For that reason, the f-bit remains unset in the packet
that contains the first byte of slice group 0 but does not also
contains the first byte of the Video Support box. All other slice
groups form fragments of their own. The f bit allows a quick
identification of packets that start a fragment. The SlcGrpOffset
field (see below) can be used to identify the start of a slice
group.
c [1 bit]:
The c field is a one-bit field that is set if the fragment to
which the first byte of the packet belongs extends througout a
subsequent packet.
SlcGrp [5 bits]:
The SlcGrp (Slice Group) field contains the slice group index
modulo 32 that is contained in the fragment that is started in
this packet. If no fragment starts in this packet, it contains
the slice group index modulo 64 of the slice group that is
contained in the fragment to which the first byte of the payload
data of this packet belongs.
SlcGrpOffset [11 bits]:
This field indicates the byte offset of the slice header marker Frame counter [7 bits]:
(SLH, hex 0xff20, see ISO/IEC 21122-1 [ISO21122-1]) of the slice This field identifies the frame number modulo 128 to which a
group that starts in this packet, relative to the Ver field. If packet belongs. Frame numbers increment by 1 for each frame
no slice group starts in this packet, this field shall be 0. transmitted. The frame number, in addition to the time stamp, may
help the decoder to manage its input buffer and to bring packets
back into their natural order.
NOTE: Since the payload data has a non-zero offset within a Slice counter [12 bits]:
packet, this field can also be used to identify whether a slice This field identifies the slice modulo 4096 to which the packet
group starts in a packet. If 0, no slice group starts in this contributes. If the data does not belong to a particular slice,
packet. Consequently, for all slice groups in a frame except the i.e. is metadata, this field shall have its maximal value, namely
first one, this field will be non-zero if and only if the f-field 4095 = 0x0fff. Otherwise, it is the slice index modulo 4096.
is set. Slice indices count from 0 at the top of the frame to their
maximum number.
Frame counter [11 bits]: Packet counter [12 bits]:
Counter indicating the current frame number modulo 2^11. The This field identifies the packet number modulo 4096 within a
frame number is incremented by one at the beginning of each frame, slice. The packet counter is reset to 0 at the start of a slice,
and stays constant throughout all packets that contribute to to and incremented by 1 for every packet that contributes to the same
the same frame. slice. For metadata, the packet counter starts from 0 for the
video support box and picture header, and increments throughout
all metadata.
4.2. Payload Data 4.2. Payload Data
The payload data of a JPEG XS transport stream consists of a The payload data of a JPEG XS RTP stream consists of a concatenation
concatenation of multiple JPEG XS Frames. of multiple JPEG XS frames.
Each JPEG XS frame is the concatenation of multiple fragments where Each JPEG XS frame is the concatenation of multiple slices and
each fragment contains one and only one slice group. The first metadata. The first metadata of a frame contains the JPEG XS header
fragment of a frame also contains the Video Support box and the JPEG segment and the last metadata contains the EOC marker. Figure 3
XS header, the last fragment also contains the EOC marker. Figure 3
depicts this layout. depicts this layout.
^ +-------------------------------------------+ ^ +--------[ JPEG XS header segment ]---------+
| | Video Support Box | | | Video support box |
| | +-------------------------------------+ | | | +-------------------------------------+ | Slice counter = 0x0fff
| | | Sub boxes of the Video Support Box | | | | | Sub boxes of the video support box | |
Frag- | +-------------------------------------+ | JPEG | +-------------------------------------+ |
ment | : additional sub-boxes of the VE-Box : | XS | : additional sub-boxes of the vs-box : |
#0 | +-------------------------------------+ | Header | +-------------------------------------+ |
| | | Seg- | |
| +-------------------------------------------+ ment +-------------------------------------------+
| | JPEG XS Header | | | Colour specification box |
| | +-------------------------------------+ | | | +-------------------------------------+ |
| | | SOC Marker | | | | | Specification method (METH = 5) | |
| | +-------------------------------------+ | | | +-------------------------------------+ |
| | : Additional Marker Segments : | | | : additional fields of the cs-box : |
| | +-------------------------------------+ | | | +-------------------------------------+ |
| | | | | |
| +-------------------------------------------+ v +-------------------------------------------+
| | Slice Group #0 | | JPEG XS codestream header |
| | +-------------------------------------+ | | +-------------------------------------+ |
| | | Slice #0 of Slice Group #0 | | | | SOC marker | |
| | | +-------------------------------+ | | | +-------------------------------------+ |
| | | | SLH Marker | | | | : Additional Marker segments : |
| | | +-------------------------------+ | | | +-------------------------------------+ |
| | | : Entropy Coded Data : | | | | M = 0, L = 1
| | | +-------------------------------+ | | +-------------------------------------------+
| | +-------------------------------------+ | +----------------[ Slices ]-----------------+
| | | Slice #1 of Slice Group #0 | | | Slice #0 |
| | : : | | +-------------------------------------+ |
| | +-------------------------------------+ | | | SLH Marker | |
| | | Slice #n-1 of Slice Group #0 | | | +-------------------------------------+ |
| | : : | | : Entropy Coded Data : |
v | +-------------------------------------+ | | | | |
^ +-------------------------------------------+ | +-------------------------------------+ |
| | Slice Group #1 | | | M = 0, L = 1
Frag- : : +-------------------------------------------+
ment : : | Slice #1 |
#1 : : : : M = 0, L = 1
| : : +-------------------------------------------+
v +-------------------------------------------+ : :
: : +-------------------------------------------+
^ +-------------------------------------------+ | Slice #n-1 |
| | Slice Group #n-1 | : :
Frag- : : +-------------------------------------------+
ment : : +----------[ End-of-codestream ]------------+
#n-1 +-------------------------------------------+ | EOC marker | Slice counter = 0x0fff
| | EOC Marker | +-------------------------------------------+ M = 1, L = 1
v +-------------------------------------------+
Figure 3: JPEG XS Payload Data Figure 3: JPEG XS Payload Data
4.3. Traffic Shaping and Delivery Timing 4.3. Traffic Shaping and Delivery Timing
The traffic shaping and delivery timing shall be in accordance with The traffic shaping and delivery timing shall be in accordance with
the Network Compatibility Model compliance definitions specified in the Network Compatibility Model compliance definitions specified in
SMPTE ST 2110-21 [SMPTE-ST2110-21] for either Narrow Linear Senders SMPTE ST 2110-21 [SMPTE-ST2110-21] for either Narrow Linear Senders
(Type NL) or Wide Senders (Type W). (Type NL) or Wide Senders (Type W). The session description shall
include a format-specific parameter of either TP=2110TPNL or
TP=2110TPW to indicate compliance with Type NL or Type W
respectively.
NOTE: The Virtual Receiver Buffer Model compliance definitions of ST NOTE: The Virtual Receiver Buffer Model compliance definitions of ST
2110-21 do not apply. 2110-21 do not apply.
5. Congestion Control Considerations 5. Congestion Control Considerations
Congestion control for RTP SHALL be used in accordance with RFC 3550 Congestion control for RTP SHALL be used in accordance with RFC 3550
[RFC3550], and with any applicable RTP profile: e.g., RFC 3551 [RFC3550], and with any applicable RTP profile: e.g., RFC 3551
[RFC3551]. An additional requirement if best-effort service is being [RFC3551]. An additional requirement if best-effort service is being
used is users of this payload format MUST monitor packet loss to used is users of this payload format MUST monitor packet loss to
skipping to change at page 11, line 4 skipping to change at page 10, line 25
NOTE: The Virtual Receiver Buffer Model compliance definitions of ST NOTE: The Virtual Receiver Buffer Model compliance definitions of ST
2110-21 do not apply. 2110-21 do not apply.
5. Congestion Control Considerations 5. Congestion Control Considerations
Congestion control for RTP SHALL be used in accordance with RFC 3550 Congestion control for RTP SHALL be used in accordance with RFC 3550
[RFC3550], and with any applicable RTP profile: e.g., RFC 3551 [RFC3550], and with any applicable RTP profile: e.g., RFC 3551
[RFC3551]. An additional requirement if best-effort service is being [RFC3551]. An additional requirement if best-effort service is being
used is users of this payload format MUST monitor packet loss to used is users of this payload format MUST monitor packet loss to
ensure that the packet loss rate is within acceptable parameters. ensure that the packet loss rate is within acceptable parameters.
Circuit Breakers [RFC8083] is an update to RTP [RFC3550] that defines Circuit Breakers [RFC8083] is an update to RTP [RFC3550] that defines
criteria for when one is required to stop sending RTP Packet Streams criteria for when one is required to stop sending RTP Packet Streams
and applications implementing this standard MUST comply with it. RFC and applications implementing this standard MUST comply with it. RFC
8085 [RFC8083] provides additional information on the best practices 8085 [RFC8085] provides additional information on the best practices
for applying congestion control to UDP streams. for applying congestion control to UDP streams.
6. Payload Format Parameters 6. Payload Format Parameters
6.1. Media Type Definition 6.1. Media Type Definition
Type name: video Type name: video
Subtype name: jpeg-xs Subtype name: jxsv
Required parameters: Required parameters:
rate: The RTP timestamp clock rate. Applications using this rate: The RTP timestamp clock rate. Applications using this
payload format SHOULD use a value of 90000. payload format SHOULD use a value of 90000.
Optional parameters: Optional parameters:
profile: The JPEG XS profile in use, as defined in JPEG XS Part 2 profile: The JPEG XS profile in use, as defined in ISO/IEC 21122-2
[ISO21122-2]. (JPEG XS Part 2) [ISO21122-2].
level: The JPEG XS level in use, as defined in JPEG XS Part 2 level: The JPEG XS level in use, as defined in ISO/IEC 21122-2
[ISO21122-2]. (JPEG XS Part 2) [ISO21122-2].
sublevel: The JPEG XS sublevel in use, as defined in JPEG XS Part 2 sublevel: The JPEG XS sublevel in use, as defined in ISO/IEC
[ISO21122-2]. 21122-2 (JPEG XS Part 2) [ISO21122-2].
sampling: Signals the color difference signal sub-sampling sampling: Signals the colour difference signal sub-sampling
structure. structure.
Signals utilizing the non-constant luminance Y'C'B C'R signal Signals utilizing the non-constant luminance Y'C'B C'R signal
format of Recommendation ITU-R BT.601-7, Recommendation ITU-R format of Recommendation ITU-R BT.601-7, Recommendation ITU-R
BT.709-6, Recommendation ITU-R BT.2020-2, or Recommendation ITU-R BT.709-6, Recommendation ITU-R BT.2020-2, or Recommendation ITU-R
BT.2100 shall use the appropriate one of the following values for BT.2100 shall use the appropriate one of the following values for
the Media Type Parameter "sampling": the Media Type Parameter "sampling":
YCbCr-4:4:4 (4:4:4 sampling) YCbCr-4:4:4 (4:4:4 sampling)
YCbCr-4:2:2 (4:2:2 sampling) YCbCr-4:2:2 (4:2:2 sampling)
skipping to change at page 14, line 30 skipping to change at page 14, line 9
ST 2110-10 [SMPTE-ST2110-10]. ST 2110-10 [SMPTE-ST2110-10].
The information carried in the media type specification has a The information carried in the media type specification has a
specific mapping to fields in the Session Description Protocol (SDP), specific mapping to fields in the Session Description Protocol (SDP),
which is commonly used to describe RTP sessions. which is commonly used to describe RTP sessions.
6.2.2. Media type and subtype 6.2.2. Media type and subtype
The media type ("video") goes in SDP "m=" as the media name. The media type ("video") goes in SDP "m=" as the media name.
The media subtype ("jpeg-xs") goes in SDP "a=rtpmap" as the encoding The media subtype ("jxsv") goes in SDP "a=rtpmap" as the encoding
name. The RTP clock rate in "a=rtpmap" MUST be 90000, which for the name, followed by a slash ("/") and the required parameter "rate"
payload format defined in this document is a 90 kHz clock. The corresponding to the RTP timestamp clock rate (which for the payload
remaining parameters go in the SDP "a=fmtp" attribute by copying them format defined in this document MUST be 90000). The optional
directly from the MIME media type string as a semicolon-separated parameters go in the SDP "a=fmtp" attribute by copying them directly
list of parameter=value pairs. from the MIME media type string as a semicolon-separated list of
parameter=value pairs.
A sample SDP mapping for JPEG XS video is as follows: A sample SDP mapping for JPEG XS video is as follows:
m=video 30000 RTP/AVP 112 m=video 30000 RTP/AVP 112
a=rtpmap:112 jpeg-xs/90000 a=rtpmap:112 jxsv/90000
a=fmtp:112 sampling=YCbCr-4:2:2; width=1920; height=1080; a=fmtp:112 sampling=YCbCr-4:2:2; width=1920; height=1080;
depth=10; colorimetry=BT709; TCS=SDR; RANGE=FULL depth=10; colorimetry=BT709; TCS=SDR;
RANGE=FULL; TP=2110TPNL
In this example, a dynamic payload type 112 is used for JPEG XS In this example, a JPEG XS RTP stream is being sent to UDP
video. The RTP sampling clock is 90 kHz. Note that the "a=fmtp:" destination port 30000, with an RTP dynamic payload type of 112 and a
line has been wrapped to fit this page, and will be a single long media clock rate of 90000 Hz. Note that the "a=fmtp:" line has been
line in the SDP file. wrapped to fit this page, and will be a single long line in the SDP
file.
6.2.3. Traffic shaping 6.2.3. Traffic shaping
The SDP object shall include the TP parameter and may include the The SDP object shall include the TP parameter (either 2110TPNL or
CMAX parameter as specified in SMPTE ST 2110-21 [SMPTE-ST2110-21]. 2110TPW as specified in Section 4.3) and may include the CMAX
parameter as specified in SMPTE ST 2110-21 [SMPTE-ST2110-21].
6.2.4. Offer/Answer Considerations 6.2.4. Offer/Answer Considerations
The following considerations apply when using SDP offer/answer The following considerations apply when using SDP offer/answer
procedures [RFC3264] to negotiate the use of the JPEG XS payload in procedures [RFC3264] to negotiate the use of the JPEG XS payload in
RTP: RTP:
o The "encode" parameter can be used for sendrecv, sendonly, and o The "encode" parameter can be used for sendrecv, sendonly, and
recvonly streams. Each encode type MUST use a separate payload recvonly streams. Each encode type MUST use a separate payload
type number. type number.
o Any unknown parameter in an offer MUST be ignored by the receiver o Any unknown parameter in an offer MUST be ignored by the receiver
and MUST NOT be included in the answer. and MUST NOT be included in the answer.
7. IANA Considerations 7. IANA Considerations
This memo requests that IANA registers video/jpeg-xs as specified in This memo requests that IANA registers video/jxsv as specified in
Section 6.1. The media type is also requested to be added to the Section 6.1. The media type is also requested to be added to the
IANA registry for "RTP Payload Format MIME types" [1]. IANA registry for "RTP Payload Format MIME types" [1].
8. Security Considerations 8. Security Considerations
RTP packets using the payload format defined in this specification RTP packets using the payload format defined in this specification
are subject to the security considerations discussed in the RTP are subject to the security considerations discussed in the RTP
specification [RFC3550] and in any applicable RTP profile such as specification [RFC3550] and in any applicable RTP profile such as
RTP/AVP [RFC3551], RTP/AVPF [RFC4585], RTP/SAVP [RFC3711], or RTP/ RTP/AVP [RFC3551], RTP/AVPF [RFC4585], RTP/SAVP [RFC3711], or RTP/
SAVPF [RFC5124]. This implies that confidentiality of the media SAVPF [RFC5124]. This implies that confidentiality of the media
skipping to change at page 16, line 44 skipping to change at page 16, line 26
Note to RFC Editor: This section may be removed after carrying out Note to RFC Editor: This section may be removed after carrying out
all the instructions of this section. all the instructions of this section.
RFC XXXX is to be replaced by the RFC number this specification RFC XXXX is to be replaced by the RFC number this specification
receives when published. receives when published.
10. References 10. References
10.1. Normative References 10.1. Normative References
[ISO15444-1]
International Organization for Standardization (ISO) -
International Electrotechnical Commission (IEC),
"Information technology - JPEG 2000 image coding system:
Core coding system", ISO/IEC IS 15444-1, 2016,
<https://www.iso.org/standard/70018.html>.
[ISO21122-1] [ISO21122-1]
International Organization for Standardization (ISO) - International Organization for Standardization (ISO) -
International Electrotechnical Commission (IEC), International Electrotechnical Commission (IEC),
"Information technology - Low-latency lightweight image "Information technology - JPEG XS low-latency lightweight
coding system - Part 1: Core coding system", ISO/IEC DIS image coding system - Part 1: Core coding system", ISO/
21122-1, under development, IEC PRF 21122-1, under development,
<https://www.iso.org/standard/74535.html>. <https://www.iso.org/standard/74535.html>.
[ISO21122-2] [ISO21122-2]
International Organization for Standardization (ISO) - International Organization for Standardization (ISO) -
International Electrotechnical Commission (IEC), International Electrotechnical Commission (IEC),
"Information technology - Low-latency lightweight image "Information technology - JPEG XS low-latency lightweight
coding system - Part 2: Profiles and buffer models", ISO/ image coding system - Part 2: Profiles and buffer models",
IEC DIS 21122-2, under development, ISO/IEC PRF 21122-2, under development,
<https://www.iso.org/standard/74535.html>. <https://www.iso.org/standard/74536.html>.
[ISO21122-3] [ISO21122-3]
International Organization for Standardization (ISO) - International Organization for Standardization (ISO) -
International Electrotechnical Commission (IEC), International Electrotechnical Commission (IEC),
"Information technology - Low-latency lightweight image "Information technology - JPEG XS low-latency lightweight
coding system - Part 3: Transport and container formats", image coding system - Part 3: Transport and container
ISO/IEC NP 21122-3, under development, formats", ISO/IEC FDIS 21122-3, under development,
<https://www.iso.org/standard/74537.html>. <https://www.iso.org/standard/74537.html>.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997, DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>. <https://www.rfc-editor.org/info/rfc2119>.
[RFC3264] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model [RFC3264] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model
with Session Description Protocol (SDP)", RFC 3264, with Session Description Protocol (SDP)", RFC 3264,
DOI 10.17487/RFC3264, June 2002, DOI 10.17487/RFC3264, June 2002,
skipping to change at page 19, line 10 skipping to change at page 18, line 38
[RFC7201] Westerlund, M. and C. Perkins, "Options for Securing RTP [RFC7201] Westerlund, M. and C. Perkins, "Options for Securing RTP
Sessions", RFC 7201, DOI 10.17487/RFC7201, April 2014, Sessions", RFC 7201, DOI 10.17487/RFC7201, April 2014,
<https://www.rfc-editor.org/info/rfc7201>. <https://www.rfc-editor.org/info/rfc7201>.
[RFC7202] Perkins, C. and M. Westerlund, "Securing the RTP [RFC7202] Perkins, C. and M. Westerlund, "Securing the RTP
Framework: Why RTP Does Not Mandate a Single Media Framework: Why RTP Does Not Mandate a Single Media
Security Solution", RFC 7202, DOI 10.17487/RFC7202, April Security Solution", RFC 7202, DOI 10.17487/RFC7202, April
2014, <https://www.rfc-editor.org/info/rfc7202>. 2014, <https://www.rfc-editor.org/info/rfc7202>.
[SMPTE-ST2059]
Society of Motion Picture and Television Engineers, "SMPTE
Standard - Generation and Alignment of Interface Signals
to the SMPTE Epoch", SMPTE ST 2059-1:2015, 2015,
<https://doi.org/10.5594/SMPTE.ST2059-1.2015>.
10.3. URIs 10.3. URIs
[1] http://www.iana.org/assignments/rtp-parameters [1] http://www.iana.org/assignments/rtp-parameters
Authors' Addresses Authors' Addresses
Sebastien Lugan Sebastien Lugan
intoPIX S.A. intoPIX S.A.
Rue Emile Francqui, 9 Rue Emile Francqui, 9
1435 Mont-Saint-Guibert 1435 Mont-Saint-Guibert
Belgium Belgium
Phone: +32 10 23 84 70 Phone: +32 10 23 84 70
Email: s.lugan@sine.sd2.net Email: D313B41E@dynmail.crt1.net
URI: http://www.intopix.com URI: http://www.intopix.com
Gael Rouvroy Gael Rouvroy
intoPIX S.A. intoPIX S.A.
Rue Emile Francqui, 9 Rue Emile Francqui, 9
1435 Mont-Saint-Guibert 1435 Mont-Saint-Guibert
Belgium Belgium
Phone: +32 10 23 84 70 Phone: +32 10 23 84 70
Email: g.rouvroy@intopix.com Email: g.rouvroy@intopix.com
URI: http://www.intopix.com URI: http://www.intopix.com
 End of changes. 62 change blocks. 
295 lines changed or deleted 262 lines changed or added

This html diff was produced by rfcdiff 1.47. The latest version is available from http://tools.ietf.org/tools/rfcdiff/