noggin wrote: ↑
Fri Dec 14, 2018 9:57 am
I can't help with specific set-ups but I an offer some codec advice.
Sorry, a fair amount of that is only partly true.
An H264 I frame is more efficiently coded than MJPEG. MJPEG really should be avoided unless you have a very good reason to use it (needing more than 1080P would be the only reason I can think of on the Pi).
H264 supports I(ntra-coded), P(redicted) and B(idirectional) frames.
With P frames the decoder will already have the reference frames required to decode the frame - it will have minimal effect on latency.
Yes, B-frames require all reference frames, which will normally include later frames in the stream and therefore increase latency.
The Pi encoder doesn't support B-frames, therefore it's a non-issue.
It is up to the decoder as to whether it fills the full DPB (Decoded Picture Buffer) that is allowed by the specified by the H264 level and profile, or does it only wait for the reference frames actually used to encode the frame.
On the Pi my recollection is that if you have provided the codec with framed data and correctly framed it, then it will check whether it has the necessary references and decode it as soon as it has the required frames.
Putting lots of B frames in a GOP (eg IBBBBBP) would increase latency as the decoder needs the P frame before it can decode any of the B's. Irrelevant on the Pi as it doesn't support B frames. The default GOP on the Pi is an I and 59 P's, which is an I frame every 2 seconds at the default 30fps. You can change that via raspivid's -g parameter.
To jjl64 (the OP), the main issue with raspivid, netcat, and hello_video is the buffering in the Linux kernel when you pipe data around. If you send directly to the network from raspivid (eg raspivid -o udp://188.8.131.52.4:1234), then adding the -fl flag means it calls fflush after every write to send the data over the network.
Piping the output of netcat to feed it into hello_video will have the same issue. There is a flag you can pass to switch to unbuffered operation (setvbuf with mode _IONBF), but IIRC there is an issue the setting will apply to the terminal after your application exits, therefore you want to put a signal handler in to restore the correct behaviour on quit. You still won't have framed data there, so there is likely to still be buffering within the codec unless you add additional bitstream parsing.
Worth also adding that multicast over Wifi can be rather hit and miss. If they're on the same Wifi network then generally it gets treated as broadcast (and that's fine). If you have a mix of Wifi and wired devices then the behaviour can depend on exactly how the AP and network switches behave.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.