4

I'm just about to migrate my site from Joomla 3 to Joomla 4. It's a pretty big website with thousands of articles that leads to problems using the new Joomla smart search.

The indexing of the articles for the smart search lets the database grow up to almost 2 GB. That's way too much. Just wondering whether anyone with the same problem figured out how to save the data for the component into an extra database what might be a solution for this mess.

As the old Joomla search won't be supported with Joomla 5 anymore, I'd like to avoid this alternative.

Improve this question
4
  • 2 GB database is scary. But how large is your website (in words or characters of all texts)? And which languages?
    – Sven
    Oct 18, 2022 at 11:19
  • It is indeed. I've decided to use the old Joomla search as long as possible. Some alternatives I've tested led to other problems and bugs. There are over 4000 articles in German.
    – mib1980
    Oct 19, 2022 at 12:12
  • Are you allowed to share a link just to get an impression of the type of articles that led to such a large database? Or is this an internal site?
    – Sven
    Oct 20, 2022 at 13:19
  • 1
    Hey Sven, you can have a look at hitchecker.de ...
    – mib1980
    Oct 21, 2022 at 14:46

3 Answers 3

Reset to default
4

Some big sites use external indexing software / services to index and search their site. Some are available as Software as a Service, while others can be installed on your own resources.

For Joomla 4 and 3 there are commercial extensions available to connect your Joomla to such search services. A plugin will register every save/trash function in your website, and communicate it accordingly to the external service. Your Joomla search will use the external service to retrieve results for specific keywords entered by the visitor. This all happens under the hood. The visitor stays on your website and does not notice that the search action was done externally outside your website.

Elastic Search

Algolia

Improve this answer
5
  • 1
    Joomla Geek's elasticsearch is great. Provided you can set up elasticsearch yourself, you don't need to use an external service. You could run it on the same server as your website(assuming you have the resources) or an external one. Not sure how the size will compare to Joomla's smart search, but it's definitely worth a look.
    – TryHarder
    Jul 7, 2022 at 18:07
  • @Sven I have approved your suggested edit, but let me just say that if you are in any way affiliated with that product or company, it is required by Stack Exchange that you announce your relationship.
    – mickmackusa ♦
    Oct 21, 2022 at 1:44
  • @mickmackusa I changed the "About" block of my profile. Is this enough?
    – Sven
    Oct 22, 2022 at 7:20
  • Editing your profile is good, but we actually need attribution to be declared with the content. I think maybe the best play is for you to 1. copy your content to your clipboard, 2. click on the blue "edited yesterday" link under the answer, 3. Click the Rollback option to revert the version to before your initial edit, 4. Post a new answer by pasting your content then state that you are the great Dr. Sven. This way it is clear to everyone that you are the author, you may be biased in your recommendation, and you can potentially gain rep/privileges from contributing. @Sven
    – mickmackusa ♦
    Oct 22, 2022 at 7:25
  • Sorry @Sven I didn't realise the Rollback was something that needed to be unlocked.
    – mickmackusa ♦
    Oct 22, 2022 at 22:15
2

If the site is that big then I would rewrite it if I couldn't find a nimble solution. Just to note - remember that every site that has 10-20k records a month needs archive tables or table partitioning. 500k up to 1mil records kills single database table.

I work on Joomla and Laravel mostly. Apache Solr is the way to go with heavy search (yeaaa Algolia has made a driver for Laravel). On the other side, I've never tried Joomla with pgsql, but I've tested big data with a Laravel postgre driver.

Postgre with correct indices give speed and boost in search functionality. Mysql is simply not as optimised in comparison.

You are asking big things which require big decision.

At the very end, you can try extensions, but ey, extensions do many things before they reach the end. Some advanced topics are easier to implement in pure frameworks, so I would rather switch if you feel able.

Improve this answer
2

If your documents are in German, this might be a niche solution (SaaS) for you:

SEMPRIA-Search

a cognitive search engine based on natural language understanding, currently only for German

Caveat: I am working for this small company.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.