You can use JSON format in the controller directly by adding .json suffix and changing format in your URL to json. In this case the controller would be placed in /administrator/components/com_example/controllers/ajax.json.php. It's contents:
defined('_JEXEC') or die;
class ExampleControllerAjax ...
This component is developed using Akeeba FOF, so you need to read the documentation to know how to complete the task.
Almost everything is configured using XML, this is to some extent of course.
This question (problem) was successfully and quite quickly solved yesterday, working effectively together with @Mick on the solution. And this question is very well detailed, well described and it included the exact error message from Joomla, so it is a good example of a well-built question on SE.
The solution was relatively simple: we had to change the ...