Figured it my self:
There are two plugins for WPFly cart plugin:
1. WPC AJAX Add to Cart for WooCommerce
2. WPC Fly Cart for WooCommerce
The first one is only for specifying on which products you want flycart enabled like, All, variable, simple etc.
After removing this plugin, the products which had the limit set to 1 item per order, always opened the fly cart without incrementing the quantity.
