One post, two categories: need category-specific permalink and template

I have two blog categories on my site, and I have category-specific sidebars for when the single post is viewed. On rare occasions both categories are assigned to a post, and the permalink (and therefore the template) utilizes the category with the lower ID number.

Is there a way I can override this, so that if you’re on the cat1 archives page and click the link, the permalink is site.com/cat1/post and it uses the cat1 template, but if you’re on the cat2 archives page and click the link the permalink is site.com/cat2/post and it uses the cat2 template?

I recognize the canonical url issues this might cause, but this is for rare occasions. The category-specific sidebar functionality trumps the potential SEO issues.