The real answer is, it's a total pain. Most of the emails are in the language files, so you have to hack html into those and pray it works. The better way to do it IS with a plugin, and there are several options:
Depends what are your needs, but language overrides can help with most of the problems. On this way you can add HTML code into the language override and that means you can include most of the things you want.
Changing CORE files sometimes could be difficult and on next Joomla update that will be overridden again (in most cases).
Try with language override. If you are using Joomla 3 you can go to Extensions > Languages > Overrides > New.
Then on new window search for the "Value" - your existing text, and then just update that text to what you want. You can add more text than you have today.