How to customize the addresses output in My Account

If you mean the look of it, it has classes that you can style with CSS.

If you mean the actual address data and layout, then the woocommerce_my_account_my_address_formatted_address filter can be used to change the data, and woocommerce_localisation_address_formats is how you change address format for WooCommerce.

See here for an example of the last one: https://stackoverflow.com/questions/66860888/woocommerce-custom-localization-address-format-for-specific-country