SDL2 and FFmpeg on Apple Silicon

Recently I wrote some code to validate a bitstream.  My co-worker told me he had various problems playing it using FFmpeg, and asked me if I want to have a go at it.  That sounds like a fun little side project, so I decided to do the exercise on the laptop while I am waiting in the parking lot for my kids to be done with their activities.  My goals were simple, using libavformat and libavcodec to demux and decode the MP4 file, which contains H.264 video originally from a camera.  Then I take the decoded frame and blit it into a window using libsdl2.  Here are my notes.

SDL2 Headers and Libraries

Under OSX, you should get SDL2 several ways.  From official web site you can download a disk image with the framework, which you need to copy to your system's /Library/Frameworks directory.  This means when you compile or link, you need to specify -F /Library/Frameworks directory to access the framework, and you must link with -f SDL2.  Or, you can use pre-built libraries from projects like HomeBrew, MacPorts, or Fink and do everything Unix-style with sdl2-config utility.  Both works and I did this with Makefile, so the Unix-style library was simpler.

Suppressing FFmpeg Debug Output

By default, FFmpeg libraries will print debugging prints to console.  Some of them could be very annoying, like the warning from libswscale that there is no accelerated function to perform colorspace conversation from YUV to RGB.  you'd never see it on x86 hardware, but Apple Silicon I was using tripped over this on every frame.  Solution was to include libavutil/logs.h so you can call av_log_set_level(AV_LOG_FATAL).

Weird Color

I used libswscale to perform colorspace conversion and the initial results all had a blue/purple tint.  This is addressed easily by getting a scale context with AV_PIX_FMT_BGR24 instead of AV_PIX_FMT_RGB24.  Or I suppose it is just as easy to create SDL surfaces from memory using different masks for the color components.

SDL Crashing and Window Not Shown

I had initially used a different thread for graphics operations.  But that was not working under OS X.  It turns out you want to do all the initialization from the main thread.  Once I got over that, no warnings or errors, but no window either.  This was because I forgot to set up the event loop.  You need to start pumping events for SDL window to show up.

OpenGL Deprecated

Apple really wants you to use Metal instead.  Depending on your situation, maybe it is easier to call SDL_BlitStretch() instead of plumbing your own shaders.

End-of-stream

If you try to decode past end of stream/file, you'll get back a frame with odd color format.  This will cause an assert in libswscale.  Detecting such condition is easy, but you have to pay attention to error code AVERROR_EOF.  Generally FFmpeg APIs return a negative number when there is a problem, you must check those negative values.