The more I-frames the video stream has, the more editable it is. However, having more I-frames increases the stream size. In order to save bandwidth and disk space, videos prepared for internet broadcast often have only one I-frame per GOP.
The GOP structure is often referred by two numbers, for example M=3, N=12. The first one tells the distance between two anchor frames (I or P). The second one tells the distance between two full images (I-frames): it is the GOP length. For the example M=3 N=12, the GOP structure is IBBPBBPBBPBBI. Instead of the M parameter one can use the maximal count of B-frames between two consecutive anchor frames.
7. H.264 coding/Layer structure
7.1 H.264 General Layer
7.2 H.264 detailed coding structure/Layer
In order to handle all kinds of application, H.264 provide a middle adaptor layer, NAL, which can convert video data into two kinds of format, byte-stream or packet-based, the former is then encapsulated by MPEG-2 TS or MPEG-4 container, the latter is then encapsulated by RTP/IP or other IP-based protocol.
7.3 H.264 Stream Structure
What you call \– it‘s probably \byte stream\It‘s important to use the proper terminology to discuss stream validity or multiplexing matters
7.3.1 SODB
Let‘s start from the SODB (String Of Data Bits). This is the real raw H.264 stream. The syntax is specified ISO/IEC 14496-10 in a form of bit string syntax.
7.3.2 RBSP
SODB is not very convenient because it‘s a bit stream, not a byte stream. It means the number of bits for one syntax may not be byte-aligned and therefore difficult to process. This is why the standard introduces RBSP (Raw Byte Sequence Payload). RBSP stores SODB in a byte stream so that the first bit of each syntax is always aligned at the first bit of a byte.
RBSP type is specified in NAL unit head, following will give a detailed description.
7.3.3 NAL Unit
The problem of RBSP is that RBSP may contain any byte pattern. Since it doesn‘t have special synchronization byte sequence, to find the synchronization point in RBSP (For example, to find the first byte of IDR picture), you may have to parse every single bit syntax from the beginning of the file.
NAL Unit is another wrapper layer to prevent certain byte pattern from occurring in the stream. When RBSP has any of 0×000000, 0×000001, 0×000002, and 0×000003, they are converted to 0×00000300, 0×00000301, 0×00000302, and 0×00000303 respectively in
NAL unit. Therefore, we can use any of 0×000000, 0×000001, 0×000002, or 0×000003 as a special synchronization byte sequence.
NAL Unit also adds one byte header to indicate the type of the NAL Unit.
Following we will give detailed NAL head format description.
SEI: SEI unit可以存放影片簡介、版權宣告、使用者自行定義的資料…等
IDR: IDR unit的內容全是採用intra-prediction編碼，所以自己本身即可完全解碼，不用參考其他access unit的資料, 每個壓縮視訊序列的第一個access unit必須為IDR access unit SPS: see following PPS: see following
7.3.4 NAL sample: SPS (Sequence parameter set) trace Sample
Which apply to a series of consecutive coded video pictures called a coded video sequence.
-------------------- SEQUENCE PARAMETER SET --------------------
@0 NAL unit header: forbidden_zero_bit u(1) 0 [0] @1 NAL unit header: nal_ref_idc u(2) 3 [11]
@3 NAL unit header: nal_unit_type u(5) 7 [00111] @8 SPS: profile_idc u(8) 77 [01001101] @16 SPS: constraint_set0_flag u(1) 0 [0] @17 SPS: constraint_set1_flag u(1) 1 [1] @18 SPS: constraint_set2_flag u(1) 0 [0] @19 SPS: constraint_set3_flag u(1) 1 [1] @20 SPS: reserved_zero_4bits u(4) 0 [0000] @24 SPS: level_idc u(8) 11 [00001011] @32 SPS: seq_parameter_set_id ue(v) 0 [1]
@33 SPS: log2_max_frame_num_minus_4 ue(v) 5 [00110] @38 SPS: pic_order_cnt_type ue(v) 0 [1]
@39 SPS: log2_max_pic_order_cnt_lsb_minus4 ue(v) 2 [011] @42 SPS: num_ref_frames ue(v) 0 [1]
@43 SPS: gaps_in_frame_num_value_allowed_flag u(1) 1 [1] @44 SPS: pic_width_in_mbs_minus1 ue(v) 10 [0001011] @51 SPS: pic_height_in_map_units_minus1 ue(v) 8 [0001001] @58 SPS: frame_mbs_only_flag u(1) 1 [1] @59 SPS: direct_8x8_inference_flag u(1) 1 [1] @60 SPS: frame_cropping_flag u(1) 0 [0] @61 SPS: vui_parameters_present_flag u(1) 0 [0] -------------------- SUBSET SEQUENCE PARAMETER SET -------------------- @0 NAL unit header: forbidden_zero_bit u(1) 0 [0] @1 NAL unit header: nal_ref_idc u(2) 3 [11]
@3 NAL unit header: nal_unit_type u(5) 15 [01111] @8 SPS: profile_idc u(8) 86 [01010110] @16 SPS: constraint_set0_flag u(1) 0 [0] @17 SPS: constraint_set1_flag u(1) 1 [1] @18 SPS: constraint_set2_flag u(1) 0 [0] @19 SPS: constraint_set3_flag u(1) 1 [1] @20 SPS: reserved_zero_4bits u(4) 0 [0000] @24 SPS: level_idc u(8) 12 [00001100] @32 SPS: seq_parameter_set_id ue(v) 0 [1] @33 SPS: chroma_format_idc ue(v) 1 [010] @36 SPS: bit_depth_luma_minus8 ue(v) 0 [1]