Video Encoding on iOS 7

Last year, I published a sample that showed how to use hardware accelerated video encoding on the iPhone for uses such as network streaming. Changes made to video encoding in iOS 7, and on the iPhone 5s, broke this mechanism, and I’ve today updated the source here to fix this.

One issue was pretty trivial: my code to divide up NALUs into frames was not quite correct. Apple added an SEI NALU to the stream, and this broke my scheme. I’ve fixed the scheme so this works correctly now.

The second problem is somewhat more difficult. On the iPhone 5s, the encoder now supports frame re-ordering: every other frame uses bi-directional prediction, and so needs to be decoded before the other frame of the pair. So the (decoding) order of the frames in the file is not the same as the timestamps that came from the incoming frames.

There is a way to resolve this, by decoding the ‘picture-order-count’ values from the H264 elementary stream. This is tediously difficult to do, but I have added code to do this for the subset of POC types that Apple uses. However, it appears that there is a bug.

Update 22nd April 2014

I’ve looked at this again, and found the bug (which was in the frame separation logic in fact). With this fix, frame re-ordering now works correctly, using the picture order count to select the correct timestamp. This version is now published here.