So the issue is from plugin version 2.3.0 onwards. Kept on trying downgrading the plugin to a version lower. No luck till 2.3.0, but when the plugin was downgraded to version 2.2.0 everything on the website looked how it was supposed to look. The moment plugin is upgraded to version 2.3.0 or higher, the website looks funny :).
Hope the developers will look into it and resolve it quickly.