Associating a Custom Post Type with the User Object

I’m am trying to get my head around how you would create a custom post type that links to the user object.

I want to create a custom post type called Contact which stores a contacts first, last name, job title etc

However, I would like this “Add Contact” to appear in the User menu and link to a user when you edit that user. It would be a one (user) to many (contacts) relationship.

Basically I want to be able to assign many contacts which are not wordpress users to a user object.

Any ideas or plugins out there that will do this??

Cheers