We did have a plugin installed that was providing those things. It was disabled however. I just deleted it and cleared cache.
I think however I identified the problem – We built the pages initially through WPBakery Page Builder and it looks like some pages were switched to Elementor. When I tried editing a page in Elementor, the changes were published.
How do we switch back to WPBakery?