You need to create a template override for the article view -
default.php template file.
So there you can create a "
modal button" for the register link, to open the registration view inside a modal window.
This blog post explains how to create modals in Joomla 3. It covers the bootstrap way for Joomla 3 with many examples, but also has a reference to an older article with examples using the mootools way - which still works on Joomla 3 by the way.
Regarding the loading of the registration screen, I would suggest to add the
tmpl=component in the link, so it will only load the component output and not the whole template.
So the link to the registration page will become like:
<?php $link = new JUri(JRoute::_('index.php?option=com_users&view=login&Itemid=' . $itemId .'&tmpl=component', false)); ?>
You will also might want to deal with the an auto-closing of the modal window after successful login and refresh of the current window.
An alternative way to achieve that is to try and load a popup Ajax login module instead of linking to the registration page.
Check this module: SC Login.
See this answer here that describes how to load a module from inside a component: Displaying more articles in the article view
- Again I haven't tried this solution with the popup login module, but there is big chance it will work.