I solved this error (with help from Nicolas George on ffmpeg-users) by rebuilding libass with fontconfig enabled. It seems that the libass package in yum did not have it enabled which makes it pretty useless. You may also have to compile your own fontconfig depending on which version you build of libass, as the repo version is 2.4.1. My steps:
# remove libass and fontconfig
yum erase libass fontconfig
# get fontconfig
wget https://cgit.freedesktop.org/fontconfig/snapshot/fontconfig-2.4.2.tar.gz
tar -xzf fontconfig-2.4.2.tar.gz
cd fontconfig-2.4.2
./autogen.sh
make && make install
# get libass
wget https://libass.googlecode.com/files/libass-0.10.1.tar.gz
tar libass-0.10.1.tar.gz
cd libass-0.10.1
./configure --enable-fontconfig
make && make install
# Then you'll need to rebuild and configure ffmpeg with --enable-libass --enable-fontconfig