Templates in child theme?

Yes, you can!

For the base of the widget you can create a file in your theme like this

View when logged in
ultimate-member/templates/login-widget/login-view.php

View when logged out
ultimate-member/templates/login-widget/login-form.php

However, if you chose an ultimate member form as an option then you may want to edit the Ultimate Member template instead using this method https://docs.ultimatemember.com/article/119-overriding-default-ultimate-member-profile-templates