4

How to convert a custom Joomla 3 to Joomla 4, and use the Joomla 4 web services API?

I have an extension I wrote for J3 and with a bit of tweaking it works just fine in J4. I want to add web services to the extension since I am building a custom Android app for the site.

Do I need to implement additional code in my extension to have it properly work with the Joomla 4 web services API?

I will expand a bit on the component that I have written. It is a simple administrator component with 2 views, clients and records. The clients table is a simple table with id, client name, notes, and a published field. The records table is also a simple table with an id, a technician id, a client id, notes, time_in, time_out, and published.

This component is going to store a list of time in and time out records for a technician going on site for a given client for reporting purposes.

I have created an api folder for the component with a src folder similar to com_tags. Very simple, single display view and controller. I also have created web services plugin, again mimicking com_tags and simple changing the names of the extension and the controller name in the apiparseroute.

The error I am receiving in /administrator/logs/everything.php:

2022-06-15T16:20:44+00:00   CRITICAL xxx.xxx.xxx.xxx    error   Uncaught Throwable of type Joomla\CMS\Router\Exception\RouteNotFoundException thrown with message "Unable to handle request for route `v1/timetracker/clients`.". Stack trace: 
#0 [ROOT]/libraries/src/Application/ApiApplication.php(241): Joomla\CMS\Router\ApiRouter->parseApiRoute('GET')
#1 [ROOT]/libraries/src/Application/ApiApplication.php(104): Joomla\CMS\Application\ApiApplication->route()
#2 [ROOT]/libraries/src/Application/CMSApplication.php(278): Joomla\CMS\Application\ApiApplication->doExecute()
#3 [ROOT]/api/includes/app.php(54): Joomla\CMS\Application\CMSApplication->execute()
#4 [ROOT]/api/index.php(31): require_once('/home/customer/...')
#5 {main}

I am willing to share a zipped version of the component upon request if it helps since there is nothing in it that is proprietary. We are only using it for admins on the backend to have a visual reference. The majority of the work will be done through an Anroid app I am currently developing.

<?php
//WEB SERVICES PLUGIN
defined('_JEXEC') or die;
use Joomla\CMS\Plugin\CMSPlugin;
use Joomla\CMS\Router\ApiRouter;

class PlgWebservicesTimetracker extends CMSPlugin
{
    protected $autoloadLanguage = true;

    public function onBeforeApiRoute(&$router)
    {
        $router->createCRUDRoutes(
            'v1/timetracker',
            'clients',
            ['component' => 'com_timetracker']
        );
    }
}

//ClientsController.php
<?php
namespace Joomla\Component\Timetracker\Api\Controller;

\defined('_JEXEC') or die;

use Joomla\CMS\MVC\Controller\ApiController;

class ClientsController extends ApiController
{
    protected $contentType = 'clients';
    protected $default_view = 'clients';
}

//JsonapiView.php
<?php
namespace Joomla\Component\Timetracker\Api\View\Clients;

\defined('_JEXEC') or die;

use Joomla\CMS\MVC\View\JsonApiView as BaseApiView;

class JsonapiView extends BaseApiView
{
    protected $fieldsToRenderItem = [
        'id',
        'business_name',
    ];
    protected $fieldsToRenderList = [
        'id',
        'business_name',
    ];
}

Stack Trace

2022-06-20T19:30:18+00:00
Uncaught Throwable of type Joomla\CMS\Router\Exception\RouteNotFoundException thrown with message "Unable to handle request for route `v1/timetracker/clients`.". 
Stack trace: #0 [ROOT]/libraries/src/Application/ApiApplication.php(241): Joomla\CMS\Router\ApiRouter->parseApiRoute('GET')
#1 [ROOT]/libraries/src/Application/ApiApplication.php(104): Joomla\CMS\Application\ApiApplication->route()
#2 [ROOT]/libraries/src/Application/CMSApplication.php(278): Joomla\CMS\Application\ApiApplication->doExecute()
#3 [ROOT]/api/includes/app.php(54): Joomla\CMS\Application\CMSApplication->execute()
#4 [ROOT]/api/index.php(31): require_once('/home/customer/...')
#5 {main}
2
  • 1
    Do post the relevant code. The error you're getting means the page is not found. Double check the URL you're trying to enter, the routes in your plugin and make sure that plugin is enabled.
    – Sharky
    Jun 16 at 5:53
  • @Sharky I added code examples for reference to my API files. Jun 16 at 15:50

3 Answers 3

3

Academic explanation of J! 4 webservices route mapping

This is an explanation of how Joomla 4 (hereafter referred to as J! 4) webservices route resolution and method finding works.

Disclaimer What follows is an explanation only. I'm attempting to educate you as to what J! 4 is trying to make you do. It is not an endorsement. I do NOT recommend jumping down "The Joomla 4 Way" (hereafter referred to as The J! 4 way) rabbit hole, because as you will shortly find out, the abstraction tree is both deep, wide & you'll end up being forced to write a very specific J! 4 dialect of PHP as well as jump between multiple files in order to persuade the architecture to get things done. In my original answer on this question I attempt to liberate developers coming from other paradigms from getting caught in the J! 4 class hierarchy trap. Neither am I against OOP, I'm a fan of classes, when used in moderation. J! 4 is not an architecture of OOP moderation, it is OOP taken to some rather extensive lengths. That's great for the ivory tower or if you have years to invest in the architecture, most daily programmers do not have such luxurious time outlay to get entangled in this architecture.
Let's do this! Get yourself a powerful caffeinated beverage, buckle your seatbelts and here we go...

Where the webservices route lifecycle starts

In your webservices plugin (where routing and our request lifecycle starts) The innocent looking code which registers a route "The J! 4 Way" is as follows:

    /**
     * Registers com_users's API's routes in the application
     *
     * @param   ApiRouter  &$router  The API Routing object
     *
     * @return  void
     *
     * @since   4.0.0
     */
    public function onBeforeApiRoute(&$router)
    {
        $router->createCRUDRoutes(
            'v1/users',
            'users',
            ['component' => 'com_users']
        );

In order to understand what's happening here we are forced to go read:

[siteroot]/libraries/src/Router/ApiRouter.php

In that file (the ApiRouter class) you'll find a method called createCRUDRoutes which is responsible for the actual mapping of your arbitrary route string (the v1/users string in the plugin's route declaration above) to a controller, and its methods.

First the code in the ApiRouter class:

    public function createCRUDRoutes($baseName, $controller, $defaults = [], $publicGets = false)
    {
        $getDefaults = array_merge(['public' => $publicGets], $defaults);

        $routes = [
            new Route(['GET'], $baseName, $controller . '.displayList', [], $getDefaults),
            new Route(['GET'], $baseName . '/:id', $controller . '.displayItem', ['id' => '(\d+)'], $getDefaults),
            new Route(['POST'], $baseName, $controller . '.add', [], $defaults),
            new Route(['PATCH'], $baseName . '/:id', $controller . '.edit', ['id' => '(\d+)'], $defaults),
            new Route(['DELETE'], $baseName . '/:id', $controller . '.delete', ['id' => '(\d+)'], $defaults),
        ];

        $this->addRoutes($routes);
    }

Here $baseName is the arbitrary route string you defined in the webservices plugin i.e. v1/users in our example.
$controller is the class name of the controller that J! 4 will look for in the following folder:

[siteroot]/api/components/com_users/Controllers/UsersController.php

Finally, there is no useful difference between $getDefaults and $defaults, essentially this should boil down (array_merge) to a single associative array that looks like:
['public' => false, 'component' => 'com_<yourComponentName>']
In fact, you could just structure a single associative array as shown and you'd be perfectly fine (in the granular un-joomla case shown in my other answer, else for the J! 4 way, just set ['component' => 'com_<yourComponentName>'] like the handful of J! 4 examples show.

Notice also that in createCRUDRoutes we see concatenation of the controller string with suffixes like:
.displayList, .displayItem, .add, .edit, and .delete this naturally leads one to think that the controller thus mapped might contain those methods (because they're essentially forcing you to conduct business using those precise names) which is indeed what J! 4 expects. With a twist of OOP abstraction the core both satisfies this expectation of J! 4 AND hides that clarity from YOU, the developer.

Let's take a look at the core [siteroot]/api/components/com_users/Controllers/UsersController.php as shipped in the api folder i.e. A webservice controller...


/**
 * The users controller
 * @since  4.0.0
 */
class UsersController extends ApiController
{
    /**
     * The content type of the item.
     * @var    string
     * @since  4.0.0
     */
    protected $contentType = 'users';

    /**
     * The default view for the display method.
     * @var    string
     * @since  4.0.0
     */
    protected $default_view = 'users';

    /**
     * Method to allow extended classes to manipulate the data to be saved for an extension.
     * @param   array  $data  An array of input data.
     * @return  array
     * @since   4.0.0
     */
    protected function preprocessSaveData(array $data): array
    {
        ...
    }

    /**
     * User list view with filtering of data
     * @return  static  A BaseController object to support chaining.
     * @since   4.0.0
     * @throws  InvalidParameterException
     */
    public function displayList()
    {
        $apiFilterInfo = $this->input->get('filter', [], 'array');
        $filter        = InputFilter::getInstance();

        if (\array_key_exists('state', $apiFilterInfo))
        {
            $this->modelState->set('filter.state', $filter->clean($apiFilterInfo['state'], 'INT'));
        }

        if (\array_key_exists('active', $apiFilterInfo))
        {
            $this->modelState->set('filter.active', $filter->clean($apiFilterInfo['active'], 'INT'));
        }

        if (\array_key_exists('groupid', $apiFilterInfo))
        {
            $this->modelState->set('filter.group_id', $filter->clean($apiFilterInfo['groupid'], 'INT'));
        }

        if (\array_key_exists('search', $apiFilterInfo))
        {
            $this->modelState->set('filter.search', $filter->clean($apiFilterInfo['search'], 'STRING'));
        }

        /*
        * What's happenning here is that the above code is populating a bunch of data on the instance property called
        * $modelState (defined in ApiController.php which is the base class of UsersController.php in the api folder).
        * Then at the end (the line below) a static call to the base class's (the parent:: call) displayList() method happens
        * This basically executes that method using the updated $modelState object details to populate the model in one go.
        * What it's doing is looking at your REST request and seeing if you specified any of the filter keys `state`, `active` etc
        * then it's setting those filtration config options.
        * 
        */
        return parent::displayList();
    }
}

I commented on the indirection happenning in the displayList() method above.
You might naturally be wondering "Where are the rest of the method names we saw hard-coded in createCRUDRoutes"??
Ah, those are all inherited from ApiController.php!

So we read [siteroot]/libraries/src/MVC/Controller/ApiController.php
and indeed there they are, we also see references to the model like the snippet below (interspersed all over ApiController.php)

        /** @var \Joomla\CMS\MVC\Model\AdminModel $model */
        $model = $this->getModel(Inflector::singularize($this->contentType));

So that's telling us that the model is dynamically picked from the singularization (plural to singular) of the value set in $contentType in our api controller UsersController.php, nowehere does the code comment on this.

So you might wonder "Ok, well which model (site-side or admin-side)?" which is a perfectly natural question to ask.
The only clue is from the @var annotation which indicates: \Joomla\CMS\MVC\Model\AdminModel $model
The model is always taken from the admin side.

Next steps:

  1. You now need to (if you're not a seasoned J! 4 developer) determine which method names you need to implement in your admin model in order for ApiController.php or J! 4 not to throw exceptions at you, silently fail, or otherwise meltdown.
  2. You do this by highlighting the $model variable throughout all the methods in the ApiController class and observing the calls to specific methods that are made, YOU need to make sure your admin model's methods are named the same names AND return the same shapes of data.
  3. You'll also need to update the fields you wish to see displayed by editing the JsonapiView.php file under the View/<ControllerName> folder. These are all the relationships you'll need to stay mindful of (under production time constraints mind you) as you strive to bring a single endpoint online.
  4. Files you need to touch in order to bring one REST endpoint online:
  • [siteroot]/plugins/webservices/<yourWebservicesPluginName>/<yourWebservicesPluginName>.php
  • [siteroot]/api/components/com_<ComponentName>/src/Controllers/<ControllerName>Controller.php
  • [siteroot]/api/components/com_<ComponentName>/src/View/<ControllerName>/JsonapiView.php
  • [siteroot]/administrator/components/com_<ComponentName>/src/Models/<ModelName>Model.php

Summary:
As you can see from our little dive into the request and class inheritance lifecycle (specifically the way it has been done in J! 4), there is no documented (to say nothing of official let alone simple) way to implement 'webservices' in J! 4 which are not completely marching in lockstep with all the conventions and expected object shapes of Joomla (which if you're not a seasoned J! developer, you'll need to log out to everything.php via print_r($objectInQuestion, true) all those shapes in order to get a grasp of how all the J! 4 methods expect to have responses.

For every arbitrarily defined endpoint you wish to bring online, (even though J! 4 tries to mulitply that by 5x for all of its CRRUD routes (read list & read single item for 2x R's)) you now have 4 files to intricately contend with across a relatively disparate folder structure within the 'same' component (but only because J! 4 is trying to glue all these parts together behind the scenes)

This is "The Joomla 4 way" of webservices.

I hope this sheds light upon your quandary and I also hope this makes it easier to understand why a professional developer might not wish to engage in this song and dance just to reap the benefits of J! 4's ecosystem, user management, token generation and other extant capabilities. You don't have to give up on Joomla to use it efficiently (in terms of time invested times complexity and technical debt accrued for capability output delivered). I mean you can subject yourself to it if you want to, but then you'll have to be subject to all of the aforementioned constraints and mutliple failure points when a particular object isn't "just so" from the expectation of some other architectural piece.

In my other answer I take the Un-Joomla path which only asks that you add your granular routes (like most other frameworks you're probably already used to) in the webservices plugin file, and then simply write your request handling and response logic in a public method which will live in your controller (as specified in the webservices route plugin php file) residing at [siteroot]/api/components/com_<ComponentName>/src/Controllers/<ControllerName>Controller.php

That's two files!, One for the routes list, easily granularly accomplished one route per line and the other is the actual file that does the work. Done. Period. End of sentence.

At the end of the day one needs to assess the effects of technology choice and prescribed architectural adherence on developer velocity. Most of the time we're getting paid to do this and "The Joomla way" makes for a much slower pace of progress i.e. developer velocity than the pace of progress afforded by something like express. The beautiful insight to be gleaned here is that we can "Have our Joomla cake and eat it too", and that is what my "joomla-tools" repo listed on my github in my other answer is all about.

I hope you learned something.

1
  • 1
    Beautifully explained. Thanks for the deep dive. I am still figuring out my own implementation to see if I can get this going properly. At the moment I have built my own REST API that shares a database with J!4 and implements the Joomla framework for interaction with the database. So far it is working really well. Jul 31 at 11:50
1

Here is an easy way to setup your routes and get on with getting work done (vs fighting with archiac and inefficient Joomla-isms)

<?php
defined('_JEXEC') or die;
use Joomla\CMS\Plugin\CMSPlugin;
use Joomla\Router\Route;
use Joomla\CMS\Log\Log;

class PlgWebservicesAirport extends CMSPlugin
{
  protected $autoloadLanguage = true;

  public function onBeforeApiRoute(&$router)
  {
    // A nice granular way to do it.
    // new Route(['HTTP_METHOD'],  'arbitrary/pattern/string',                     '<CONTROLLER_NAME>.PUBLIC_METHOD_NAME',               [], $defaults)
    // Obviously substitute the COMPONENTNAME (lowercase no spaces), <CONTROLLER_NAME> as lowercase, & PUBLIC_METHOD_NAME as camelcase.
    // controllers are to be places in [site_root]/api/components/src/Controllers/<CONTROLLER_NAME>Controller.php

    // So the 'hangars' controller below would in fact be located at:  [site_root]/api/components/com_airport/src/Controllers/HangarsController.php
    // inside of it would be a public method called getHangarsByAirline() etc

    // An obvious example for ease of comprehension
    $defaults    = array_merge(['public' => false], ['component' => 'com_airport']);
    $routes = [
      /* My cool routes */
      new Route(['GET'],   'v1/airport/hangars/by/airline/:airLineName',     'hangars.getHangarsByAirline',                ['airLineName'    => '(filter.+validation.+regex)'], $defaults),
      /* No filtration regex allows ALL patterns to pass through into Jinput on the controller side. */
      new Route(['GET'],  'v1/airport/hangar/by/id/:id',                          'hangars.getHangarById',                 ['id' => '(\d{1,9})'], $defaults),
      /* No filtration regex allows ALL patterns to pass through into Jinput on the controller side. */
      new Route(['GET'],  'v1/airport/lounges/by/airline/:airLineName',    'lounges.getLoungesByAirline',           [], $defaults),
      /* No url parameter, no checking necessary, so you need to grab the POST body via $req = json_decode( $this->input->json->getRaw() ); on the controller side) */
      new Route(['POST'],  'v1/airport/purchase/ticket',                     'tickets.purchaseTicket',               [], $defaults)
    ];
    $router->addRoutes($routes);
  }
}

Now if you'd like to have a clean set of component + webservices plugin, that will take days of research for J! 4 (which I spent a few months learning).

So to save future me (and you now too) the pain of component and plugin authoring, I wrote a python script that makes working J! 4 scaffolds of the component and webservices plugin. Check it out on my github and have fun developing with J! 4!

Node0's extension generator joomla-tools on github

5
  • I've gone a ways down the rabbit hole of API development in J! 4 and will be updating componentMaker.py and pluginMaker.py to output code that is designed for speed, clarity (in both reading and writing) and generally getting things done. This means the controller "just does the work, responds and then exits immediately" no model, no view, just a method that does the work. Want abstraction? Install a Joomla library package and pull it in with a namespace. It's time to get efficient with J! 4, streamlined, like express, rails, and other fast frameworks. Jul 22 at 4:40
  • I played around with some of this code and it didn't make a lot of sense to me honestly. Is there better documentation on your Github page for using these routes? Jul 27 at 17:34
  • @TerryCarter Hi Terry, thanks for your question. Yes! There is a better experience. If you clone the joomla-tools repo on my github (make sure you have Python 3.6+ and pip3 install sh) then run componentMaker.py and pluginMaker.py with the --help flag, you'll be able to generate a component whose name you can feed to the pluginMaker.py input param and obtain a set of component + plugin extensions that you can install and then modify the plugin-routes to the controller methods (also the controller methods themselves) to get actual responses. I'll work on a turnkey demo code-gen default. Jul 27 at 20:17
  • I have the same API issue telling ` Component not found.. The Legacy Dispatcher is beeing used looking here for the api controller: htdocs/project1/api/components/com_foo/foo.php. Is it possible to use the J4 API with an J3 MVC structure or is the /src structure needed in the frontend component directory?
    – maidan
    Sep 22 at 18:45
  • @maidan technically speaking you could specify any containing folder for the MVC folders to live in under the /api/components/com_foo folder, this detail is specified in the manifest xml file in the following section: <api> <files folder="api"> <folder>src</folder> </files> </api> Now the core J4! components all use the src folder. If you want an example of a structure that J4! will accept (installable) try my repo out and generate a custom com and websvc plg pair to try out. If you run into any trouble please open a github issue on my joomla-tools repo. Sep 29 at 20:05
0

The URL you're trying to enter or the defined route is wrong. You're trying to access v1/timetracker/clients but the route is defined as v1/timetracker. You should update it accordingly:

$router->createCRUDRoutes(
    'v1/timetracker/clients',
    'clients',
    ['component' => 'com_timetracker']
);
6
  • Uncaught Throwable of type Exception thrown with message "Component not found.". Stack trace: #0 Jun 20 at 3:48
  • Post the stack trace.
    – Sharky
    Jun 20 at 5:18
  • I added the stack trace. Jun 21 at 11:18
  • Did you post the right trace? It looks the same as the one before the change.
    – Sharky
    Jun 22 at 5:26
  • Yes, that was the trace immediately after updating the script. Jun 23 at 11:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.