Joomla 4 API Question

Question

How to convert a custom Joomla 3 to Joomla 4, and use the Joomla 4 web services API?

I have an extension I wrote for J3 and with a bit of tweaking it works just fine in J4. I want to add web services to the extension since I am building a custom Android app for the site.

Do I need to implement additional code in my extension to have it properly work with the Joomla 4 web services API?

I will expand a bit on the component that I have written. It is a simple administrator component with 2 views, clients and records. The clients table is a simple table with id, client name, notes, and a published field. The records table is also a simple table with an id, a technician id, a client id, notes, time_in, time_out, and published.

This component is going to store a list of time in and time out records for a technician going on site for a given client for reporting purposes.

I have created an api folder for the component with a src folder similar to com_tags. Very simple, single display view and controller. I also have created web services plugin, again mimicking com_tags and simple changing the names of the extension and the controller name in the apiparseroute.

The error I am receiving in /administrator/logs/everything.php:

2022-06-15T16:20:44+00:00   CRITICAL xxx.xxx.xxx.xxx    error   Uncaught Throwable of type Joomla\CMS\Router\Exception\RouteNotFoundException thrown with message "Unable to handle request for route `v1/timetracker/clients`.". Stack trace: 
#0 [ROOT]/libraries/src/Application/ApiApplication.php(241): Joomla\CMS\Router\ApiRouter->parseApiRoute('GET')
#1 [ROOT]/libraries/src/Application/ApiApplication.php(104): Joomla\CMS\Application\ApiApplication->route()
#2 [ROOT]/libraries/src/Application/CMSApplication.php(278): Joomla\CMS\Application\ApiApplication->doExecute()
#3 [ROOT]/api/includes/app.php(54): Joomla\CMS\Application\CMSApplication->execute()
#4 [ROOT]/api/index.php(31): require_once('/home/customer/...')
#5 {main}

I am willing to share a zipped version of the component upon request if it helps since there is nothing in it that is proprietary. We are only using it for admins on the backend to have a visual reference. The majority of the work will be done through an Anroid app I am currently developing.

<?php
//WEB SERVICES PLUGIN
defined('_JEXEC') or die;
use Joomla\CMS\Plugin\CMSPlugin;
use Joomla\CMS\Router\ApiRouter;

class PlgWebservicesTimetracker extends CMSPlugin
{
    protected $autoloadLanguage = true;

    public function onBeforeApiRoute(&$router)
    {
        $router->createCRUDRoutes(
            'v1/timetracker',
            'clients',
            ['component' => 'com_timetracker']
        );
    }
}

//ClientsController.php
<?php
namespace Joomla\Component\Timetracker\Api\Controller;

\defined('_JEXEC') or die;

use Joomla\CMS\MVC\Controller\ApiController;

class ClientsController extends ApiController
{
    protected $contentType = 'clients';
    protected $default_view = 'clients';
}

//JsonapiView.php
<?php
namespace Joomla\Component\Timetracker\Api\View\Clients;

\defined('_JEXEC') or die;

use Joomla\CMS\MVC\View\JsonApiView as BaseApiView;

class JsonapiView extends BaseApiView
{
    protected $fieldsToRenderItem = [
        'id',
        'business_name',
    ];
    protected $fieldsToRenderList = [
        'id',
        'business_name',
    ];
}

Stack Trace

2022-06-20T19:30:18+00:00
Uncaught Throwable of type Joomla\CMS\Router\Exception\RouteNotFoundException thrown with message "Unable to handle request for route `v1/timetracker/clients`.". 
Stack trace: #0 [ROOT]/libraries/src/Application/ApiApplication.php(241): Joomla\CMS\Router\ApiRouter->parseApiRoute('GET')
#1 [ROOT]/libraries/src/Application/ApiApplication.php(104): Joomla\CMS\Application\ApiApplication->route()
#2 [ROOT]/libraries/src/Application/CMSApplication.php(278): Joomla\CMS\Application\ApiApplication->doExecute()
#3 [ROOT]/api/includes/app.php(54): Joomla\CMS\Application\CMSApplication->execute()
#4 [ROOT]/api/index.php(31): require_once('/home/customer/...')
#5 {main}

Answer 1

Lets say your component name is test

At plugins/webservices/test/test.php

class PlgWebservicesTest extends CMSPlugin{
private $router;

public function onBeforeApiRoute(&$router)
{
    $this->router = $router;

    $router->createCRUDRoutes('v1/test/orders', 'orders', ['component' => 'com_test']);
    // Custom route for fetching orders assigned to a specific driver
    $this->createDriverOrdersRoute('v1/test/orders/driver','orders',['component' => 'com_test']);
    
}


/**
 * Create route for fetching orders assigned to a specific driver
 *
 * @param   ApiRouter  &$router  The API Routing object
 *
 * @return  void
 *
 * @since   1.0.1
 */
private function createDriverOrdersRoute($baseName,$controller,$defaults)
{

    $publicGets = false;

    $getDefaults = array_merge(['public' => $publicGets], $defaults);

    $routes = [
        // new Route(['GET'], $baseName, $controller . '.displayList', [], $getDefaults),
        new Route(['GET'], $baseName . '/:id', $controller . '.getDriverOrders', ['id' => '(\d+)'], $getDefaults),//create custom response
        // new Route(['GET'], $baseName . '/:id', $controller . '.getDriverOrders', ['id' => '(\d+)'], $getDefaults),//create custom response
        // new Route(['POST'], $baseName, $controller . '.add', [], $defaults),
        // new Route(['PATCH'], $baseName . '/:id', $controller . '.edit', ['id' => '(\d+)'], $defaults),
        // new Route(['DELETE'], $baseName . '/:id', $controller . '.delete', ['id' => '(\d+)'], $defaults),
    ];

    // $router->addRoute(new Route(['GET'], $baseName . '/:id', $controller . '.getDriverOrders', ['id' => '(\d+)'], $getDefaults));
    $this->router->addRoutes($routes);
}

}

at this line of code

new Route(['GET'], $baseName . '/:id', $controller . '.getDriverOrders', ['id' => '(\d+)'], $getDefaults),//create custom response

getDriverOrders is the function we call from the controller located at api/components/com_test/Controller/OrdersController.php

The default function for displaying a list is the displayList so if we use

new Route(['GET'], $baseName . '/:id', $controller . '.displayList', ['id' => '(\d+)'], $getDefaults),//create custom response

it will load what the default com_test/src/Model/OrdersModel.php getListQuery loads.

So in controller we will create the getDriverOrders which will handle the difference. We passed in the id of the driver so in api/components/com_test/Controller/OrdersController.php

namespace Test\Component\Test\Api\Controller;

\defined('_JEXEC') or die;

use Joomla\CMS\MVC\Controller\ApiController; // use Joomla\CMS\Factory; use Joomla\CMS\Response\JsonResponse; use Joomla\CMS\Filter\InputFilter; use Joomla\CMS\Uri\Uri; use Tobscure\JsonApi\Exception\InvalidParameterException; use Joomla\CMS\Language\Text;

class OrdersController extends ApiController {

protected $contentType = 'orders';
protected $default_view = 'orders';

 /**
 * Orders list view with filtering of data based on driver
 *
 * @return  static  A BaseController object to support filtering drivers.
 *
 * @since   4.0.0
 * @throws  InvalidParameterException
 */
public function getDriverOrders()
{   
    $filter        = InputFilter::getInstance();

    // set filter for the model
    $driverId = $this->input->getInt('id', 0);

    if ($driverId > 0) {
        $this->modelState->set('filter.driver', $filter->clean($driverId, 'INT'));
    }else{
        $error = Text::sprintf('JLIB_FORM_VALIDATE_FIELD_INVALID', 'id');
        throw new InvalidParameterException($error, 400, null, 'registrationDateStart');
    }

    // call the orders model list
    return parent::displayList();

}

}

this will set a filter to the getListQuery in the model.The way to use the filter to filter your data in you component model at the path /components/com_test/src/models/OrdersModel.php at getListQuery is like this:

    protected function getListQuery()
{
    // Create a new query object.
    $db    = $this->getDbo();
    $query = $db->getQuery(true);

    // Select the required fields from the table.
    $query->select(
        $this->getState(
            'list.select', 'DISTINCT a.*'
        )
    );

    $query->from($db->quoteName('#__test_orders','a'));
    
    

    // Filter by search in title
    $search = $this->getState('filter.search');

    if (!empty($search))
    {
        if (stripos($search, 'id:') === 0)
        {
            $query->where('a.id = ' . (int) substr($search, 3));
        }
        else
        {
            $search = $db->Quote('%' . $db->escape($search, true) . '%');
            $query->where('( a.name LIKE '.$search.' )');
        }
    }
    
    //Filtering driver
    $filter_driver = $this->state->get("filter.driver");
    if ($filter_driver) {
        $query->where("a.driver = '".$db->escape($filter_driver)."'");
    }

    // $user = Factory::getUser();

    // only new or proccessing orders
    $query->where('a.status!=2 AND a.status!=5');

    //only when searching for driver or when assigned to the current driver user and its not completed/delivered
    // $query->where('a.driver = 0 OR (a.driver='.$db->quote($user->id).' AND a.status<>5)');

    // Add the list ordering clause.
    $orderCol  = $this->state->get('list.ordering', 'id');
    $orderDirn = $this->state->get('list.direction', 'DESC');

    if ($orderCol && $orderDirn)
    {
        $query->order($db->escape($orderCol . ' ' . $orderDirn));
    }

    return $query;
}

more specific this lines of code

//Filtering driver
$filter_driver = $this->state->get("filter.driver");
if ($filter_driver) {
    $query->where("a.driver = '".$db->escape($filter_driver)."'");
}

If you dont want to use your joomla component model you could handle right in the api/components/com_test/Controller/OrdersController.php and the function getDriverOrders will become like this Alternative solution using full custom response:

namespace Test\Component\Test\Api\Controller; \defined('_JEXEC') or die;

use Joomla\CMS\MVC\Controller\ApiController; use Joomla\CMS\Factory; use Joomla\CMS\Response\JsonResponse; use Joomla\CMS\Filter\InputFilter; use Joomla\CMS\Uri\Uri; use Tobscure\JsonApi\Exception\InvalidParameterException; use Joomla\CMS\Language\Text;

class OrdersController extends ApiController {

protected $contentType = 'orders';

protected $default_view = 'orders';

public function getDriverOrders()
{   

    $this->app->getSession()->set('application.queue', $this->app->getMessageQueue());

    header('Content-Type: application/json');
    
    // uncomment those 3 line for test purposes to check the response
    // $response_data = null;
    // echo new JsonResponse($response_data, 'Check', true);
    // $this->app->close();

    // Get the input
    $input = $this->input;
    $driverId = $input->getInt('id', 0);
    
    // Get the database object
    $db = Factory::getDbo();

    // Build the query to get orders assigned to the specified driver_id
    $query = $db->getQuery(true)
        ->select('*')
        ->from($db->quoteName('#__test_orders'))
        ->where($db->quoteName('driver') . ' = ' . $db->quote($driverId));

    // Execute the query
    $db->setQuery($query);
    $orders = $db->loadObjectList();

    // Return the result as JSON response
    echo new JsonResponse($orders, 'Check', false);
    $this->app->close();
}

}

Answer 2

Academic explanation of J! 4 webservices route mapping

This is an explanation of how Joomla 4 (hereafter referred to as J! 4) webservices route resolution and method finding works.

Disclaimer What follows is an explanation only. I'm attempting to educate you as to what J! 4 is trying to make you do. It is not an endorsement. I do NOT recommend jumping down "The Joomla 4 Way" (hereafter referred to as The J! 4 way) rabbit hole, because as you will shortly find out, the abstraction tree is both deep, wide & you'll end up being forced to write a very specific J! 4 dialect of PHP as well as jump between multiple files in order to persuade the architecture to get things done. In my original answer on this question I attempt to liberate developers coming from other paradigms from getting caught in the J! 4 class hierarchy trap. Neither am I against OOP, I'm a fan of classes, when used in moderation. J! 4 is not an architecture of OOP moderation, it is OOP taken to some rather extensive lengths. That's great for the ivory tower or if you have years to invest in the architecture, most daily programmers do not have such luxurious time outlay to get entangled in this architecture.
Let's do this! Get yourself a powerful caffeinated beverage, buckle your seatbelts and here we go...

Where the webservices route lifecycle starts

In your webservices plugin (where routing and our request lifecycle starts) The innocent looking code which registers a route "The J! 4 Way" is as follows:

    /**
     * Registers com_users's API's routes in the application
     *
     * @param   ApiRouter  &$router  The API Routing object
     *
     * @return  void
     *
     * @since   4.0.0
     */
    public function onBeforeApiRoute(&$router)
    {
        $router->createCRUDRoutes(
            'v1/users',
            'users',
            ['component' => 'com_users']
        );

In order to understand what's happening here we are forced to go read:

[siteroot]/libraries/src/Router/ApiRouter.php

In that file (the ApiRouter class) you'll find a method called createCRUDRoutes which is responsible for the actual mapping of your arbitrary route string (the v1/users string in the plugin's route declaration above) to a controller, and its methods.

First the code in the ApiRouter class:

    public function createCRUDRoutes($baseName, $controller, $defaults = [], $publicGets = false)
    {
        $getDefaults = array_merge(['public' => $publicGets], $defaults);

        $routes = [
            new Route(['GET'], $baseName, $controller . '.displayList', [], $getDefaults),
            new Route(['GET'], $baseName . '/:id', $controller . '.displayItem', ['id' => '(\d+)'], $getDefaults),
            new Route(['POST'], $baseName, $controller . '.add', [], $defaults),
            new Route(['PATCH'], $baseName . '/:id', $controller . '.edit', ['id' => '(\d+)'], $defaults),
            new Route(['DELETE'], $baseName . '/:id', $controller . '.delete', ['id' => '(\d+)'], $defaults),
        ];

        $this->addRoutes($routes);
    }

Here $baseName is the arbitrary route string you defined in the webservices plugin i.e. v1/users in our example.
$controller is the class name of the controller that J! 4 will look for in the following folder:

[siteroot]/api/components/com_users/Controllers/UsersController.php

Finally, there is no useful difference between $getDefaults and $defaults, essentially this should boil down (array_merge) to a single associative array that looks like:
['public' => false, 'component' => 'com_<yourComponentName>']
In fact, you could just structure a single associative array as shown and you'd be perfectly fine (in the granular un-joomla case shown in my other answer, else for the J! 4 way, just set ['component' => 'com_<yourComponentName>'] like the handful of J! 4 examples show.

Notice also that in createCRUDRoutes we see concatenation of the controller string with suffixes like:
.displayList, .displayItem, .add, .edit, and .delete this naturally leads one to think that the controller thus mapped might contain those methods (because they're essentially forcing you to conduct business using those precise names) which is indeed what J! 4 expects. With a twist of OOP abstraction the core both satisfies this expectation of J! 4 AND hides that clarity from YOU, the developer.

Let's take a look at the core [siteroot]/api/components/com_users/Controllers/UsersController.php as shipped in the api folder i.e. A webservice controller...

/**
 * The users controller
 * @since  4.0.0
 */
class UsersController extends ApiController
{
    /**
     * The content type of the item.
     * @var    string
     * @since  4.0.0
     */
    protected $contentType = 'users';

    /**
     * The default view for the display method.
     * @var    string
     * @since  4.0.0
     */
    protected $default_view = 'users';

    /**
     * Method to allow extended classes to manipulate the data to be saved for an extension.
     * @param   array  $data  An array of input data.
     * @return  array
     * @since   4.0.0
     */
    protected function preprocessSaveData(array $data): array
    {
        ...
    }

    /**
     * User list view with filtering of data
     * @return  static  A BaseController object to support chaining.
     * @since   4.0.0
     * @throws  InvalidParameterException
     */
    public function displayList()
    {
        $apiFilterInfo = $this->input->get('filter', [], 'array');
        $filter        = InputFilter::getInstance();

        if (\array_key_exists('state', $apiFilterInfo))
        {
            $this->modelState->set('filter.state', $filter->clean($apiFilterInfo['state'], 'INT'));
        }

        if (\array_key_exists('active', $apiFilterInfo))
        {
            $this->modelState->set('filter.active', $filter->clean($apiFilterInfo['active'], 'INT'));
        }

        if (\array_key_exists('groupid', $apiFilterInfo))
        {
            $this->modelState->set('filter.group_id', $filter->clean($apiFilterInfo['groupid'], 'INT'));
        }

        if (\array_key_exists('search', $apiFilterInfo))
        {
            $this->modelState->set('filter.search', $filter->clean($apiFilterInfo['search'], 'STRING'));
        }

        /*
        * What's happenning here is that the above code is populating a bunch of data on the instance property called
        * $modelState (defined in ApiController.php which is the base class of UsersController.php in the api folder).
        * Then at the end (the line below) a static call to the base class's (the parent:: call) displayList() method happens
        * This basically executes that method using the updated $modelState object details to populate the model in one go.
        * What it's doing is looking at your REST request and seeing if you specified any of the filter keys `state`, `active` etc
        * then it's setting those filtration config options.
        * 
        */
        return parent::displayList();
    }
}

I commented on the indirection happenning in the displayList() method above.
You might naturally be wondering "Where are the rest of the method names we saw hard-coded in createCRUDRoutes"??
Ah, those are all inherited from ApiController.php!

So we read [siteroot]/libraries/src/MVC/Controller/ApiController.php
and indeed there they are, we also see references to the model like the snippet below (interspersed all over ApiController.php)

        /** @var \Joomla\CMS\MVC\Model\AdminModel $model */
        $model = $this->getModel(Inflector::singularize($this->contentType));

So that's telling us that the model is dynamically picked from the singularization (plural to singular) of the value set in $contentType in our api controller UsersController.php, nowehere does the code comment on this.

So you might wonder "Ok, well which model (site-side or admin-side)?" which is a perfectly natural question to ask.
The only clue is from the @var annotation which indicates: \Joomla\CMS\MVC\Model\AdminModel $model
The model is always taken from the admin side.

Next steps:

1. You now need to (if you're not a seasoned J! 4 developer) determine which method names you need to implement in your admin model in order for ApiController.php or J! 4 not to throw exceptions at you, silently fail, or otherwise meltdown.
2. You do this by highlighting the $model variable throughout all the methods in the ApiController class and observing the calls to specific methods that are made, YOU need to make sure your admin model's methods are named the same names AND return the same shapes of data.
3. You'll also need to update the fields you wish to see displayed by editing the JsonapiView.php file under the View/<ControllerName> folder. These are all the relationships you'll need to stay mindful of (under production time constraints mind you) as you strive to bring a single endpoint online.
4. Files you need to touch in order to bring one REST endpoint online:

• [siteroot]/plugins/webservices/<yourWebservicesPluginName>/<yourWebservicesPluginName>.php
• [siteroot]/api/components/com_<ComponentName>/src/Controllers/<ControllerName>Controller.php
• [siteroot]/api/components/com_<ComponentName>/src/View/<ControllerName>/JsonapiView.php
• [siteroot]/administrator/components/com_<ComponentName>/src/Models/<ModelName>Model.php

Summary:
As you can see from our little dive into the request and class inheritance lifecycle (specifically the way it has been done in J! 4), there is no documented (to say nothing of official let alone simple) way to implement 'webservices' in J! 4 which are not completely marching in lockstep with all the conventions and expected object shapes of Joomla (which if you're not a seasoned J! developer, you'll need to log out to everything.php via print_r($objectInQuestion, true) all those shapes in order to get a grasp of how all the J! 4 methods expect to have responses.

For every arbitrarily defined endpoint you wish to bring online, (even though J! 4 tries to mulitply that by 5x for all of its CRRUD routes (read list & read single item for 2x R's)) you now have 4 files to intricately contend with across a relatively disparate folder structure within the 'same' component (but only because J! 4 is trying to glue all these parts together behind the scenes)

This is "The Joomla 4 way" of webservices.

I hope this sheds light upon your quandary and I also hope this makes it easier to understand why a professional developer might not wish to engage in this song and dance just to reap the benefits of J! 4's ecosystem, user management, token generation and other extant capabilities. You don't have to give up on Joomla to use it efficiently (in terms of time invested times complexity and technical debt accrued for capability output delivered). I mean you can subject yourself to it if you want to, but then you'll have to be subject to all of the aforementioned constraints and mutliple failure points when a particular object isn't "just so" from the expectation of some other architectural piece.

In my other answer I take the Un-Joomla path which only asks that you add your granular routes (like most other frameworks you're probably already used to) in the webservices plugin file, and then simply write your request handling and response logic in a public method which will live in your controller (as specified in the webservices route plugin php file) residing at [siteroot]/api/components/com_<ComponentName>/src/Controllers/<ControllerName>Controller.php

That's two files!, One for the routes list, easily granularly accomplished one route per line and the other is the actual file that does the work. Done. Period. End of sentence.

At the end of the day one needs to assess the effects of technology choice and prescribed architectural adherence on developer velocity. Most of the time we're getting paid to do this and "The Joomla way" makes for a much slower pace of progress i.e. developer velocity than the pace of progress afforded by something like express. The beautiful insight to be gleaned here is that we can "Have our Joomla cake and eat it too", and that is what my "joomla-tools" repo listed on my github in my other answer is all about.

I hope you learned something.

Answer 3

Here is an easy way to setup your routes and get on with getting work done (vs fighting with archiac and inefficient Joomla-isms)

<?php
defined('_JEXEC') or die;
use Joomla\CMS\Plugin\CMSPlugin;
use Joomla\Router\Route;
use Joomla\CMS\Log\Log;

class PlgWebservicesAirport extends CMSPlugin
{
  protected $autoloadLanguage = true;

  public function onBeforeApiRoute(&$router)
  {
    // A nice granular way to do it.
    // new Route(['HTTP_METHOD'],  'arbitrary/pattern/string',                     '<CONTROLLER_NAME>.PUBLIC_METHOD_NAME',               [], $defaults)
    // Obviously substitute the COMPONENTNAME (lowercase no spaces), <CONTROLLER_NAME> as lowercase, & PUBLIC_METHOD_NAME as camelcase.
    // controllers are to be places in [site_root]/api/components/src/Controllers/<CONTROLLER_NAME>Controller.php

    // So the 'hangars' controller below would in fact be located at:  [site_root]/api/components/com_airport/src/Controllers/HangarsController.php
    // inside of it would be a public method called getHangarsByAirline() etc

    // An obvious example for ease of comprehension
    $defaults    = array_merge(['public' => false], ['component' => 'com_airport']);
    $routes = [
      /* My cool routes */
      new Route(['GET'],   'v1/airport/hangars/by/airline/:airLineName',     'hangars.getHangarsByAirline',                ['airLineName'    => '(filter.+validation.+regex)'], $defaults),
      /* No filtration regex allows ALL patterns to pass through into Jinput on the controller side. */
      new Route(['GET'],  'v1/airport/hangar/by/id/:id',                          'hangars.getHangarById',                 ['id' => '(\d{1,9})'], $defaults),
      /* No filtration regex allows ALL patterns to pass through into Jinput on the controller side. */
      new Route(['GET'],  'v1/airport/lounges/by/airline/:airLineName',    'lounges.getLoungesByAirline',           [], $defaults),
      /* No url parameter, no checking necessary, so you need to grab the POST body via $req = json_decode( $this->input->json->getRaw() ); on the controller side) */
      new Route(['POST'],  'v1/airport/purchase/ticket',                     'tickets.purchaseTicket',               [], $defaults)
    ];
    $router->addRoutes($routes);
  }
}

Now if you'd like to have a clean set of component + webservices plugin, that will take days of research for J! 4 (which I spent a few months learning).

So to save future me (and you now too) the pain of component and plugin authoring, I wrote a python script that makes working J! 4 scaffolds of the component and webservices plugin. Check it out on my github and have fun developing with J! 4!

Node0's extension generator joomla-tools on github

Answer 4

The URL you're trying to enter or the defined route is wrong. You're trying to access v1/timetracker/clients but the route is defined as v1/timetracker. You should update it accordingly:

$router->createCRUDRoutes(
    'v1/timetracker/clients',
    'clients',
    ['component' => 'com_timetracker']
);