That was it. I am using the Unite theme. They listed it as supporting gallery format, but it was missing in the functions.php. The following change fixed it.
// Enable support for Post Formats.
add_theme_support( 'post-formats', array( 'aside', 'image', 'video', 'quote', 'link', 'gallery' ) );
