The controller that deals with redirecting the form upon registration is at /components/com_users/controllers/registration.php
Looking at this code, it seems to redirect back to the form view if:
- There are validation errors. The validation is checked using JForm's validate function.
- The user registration process fails. This function exists at /components/com_users/models/registration.php and fails if it is unable to access the database to add the user or if it is unable to send an email.
In all these cases, an error message should be sent to the screen. To see these, your template needs the code
<jdoc:include type="message" />. The users component also uses this to show a successful registration message to the user, so it is required for this component to work.
My guess would be that your server is failing to send emails (it sounds like it can access the database), and that you don't have a message area in your template to see the error message.
However, I'd also try it without the captcha, and make sure a completely vanilla version also has the problem.