I have the following PHP code working to bulk add array items from JSON to com_content. The code is executed by directly accessing the file in the web browser. For example, http://examples.com/json-articles.php.
<?php
if (!defined( '_JEXEC' )) {
define( '_JEXEC', 1 );
define( 'JPATH_BASE', realpath(dirname(__FILE__)));
require_once ( JPATH_BASE .'/includes/defines.php' );
require_once ( JPATH_BASE .'/includes/framework.php' );
defined( 'DS' ) or define( 'DS', DIRECTORY_SEPARATOR );
}
$app = JFactory::getApplication( 'site' );
$curl = curl_init();
curl_setopt_array($curl, array(
CURLOPT_URL => 'https://api.dtn.com/publishing/news/articles?categoryId=16%2C17%2C18&limit=3&maxAge=30&apikey=placeholder',
CURLOPT_RETURNTRANSFER => true,
CURLOPT_ENCODING => '',
CURLOPT_MAXREDIRS => 10,
CURLOPT_TIMEOUT => 0,
CURLOPT_FOLLOWLOCATION => true,
CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
CURLOPT_CUSTOMREQUEST => 'GET',
));
$json = curl_exec($curl);
curl_close($curl);
$data = json_decode($json, true);
foreach ($data as $value) {
$articleData = array(
'id' => 0,
'catid' => 8,
'title' => ucwords(strtolower($value['title'])),
'alias' => '',
'introtext' => strip_tags($value['storySummary']),
'fulltext' => strip_tags($value['content']),
'state' => 1,
'access' => 2,
'created_by' => 332,
'language' => '*'
);
$articleId = createArticle($articleData);
if(!$articleId) {
echo JText::_('Article(s) could not be created.');
}
}
function createArticle($data) {
$data['rules'] = array(
'core.edit.delete' => array(),
'core.edit.edit' => array(),
'core.edit.state' => array(),
);
$basePath = JPATH_ADMINISTRATOR . '/components/com_content';
require_once $basePath.'/models/article.php';
$config = array();
$articleModel = new ContentModelArticle($config);
if(!$articleModel->save($data)) {
$errMsg = $articleModel->getError();
return false;
} else {
$id = $articleModel->getItem()->id;
return $id;
}
}
Now I am needing to convert it to a CLI app that can be executed at a set time everyday using a CRON job. I have reviewed existing CLI scripts in the CLI directory and have coded the following so far. I used this Joomla StackExchange post and started with the garbagecron.php
CLI script as references.
<?php
/**
* @package Joomla.Cli
*
* @copyright (C) 2012 Open Source Matters, Inc. <https://www.joomla.org>
* @license GNU General Public License version 2 or later; see LICENSE.txt
*/
/**
* A command line cron job to bulk import JSON data into Joomla articles
*/
// Initialize Joomla framework
const _JEXEC = 1;
// Load system defines
if (file_exists( dirname(__DIR__) . '/defines.php' ) ) {
require_once dirname(__DIR__) . '/defines.php';
}
if (!defined( '_JDEFINES' )) {
define( 'JPATH_BASE', dirname(__DIR__) );
require_once JPATH_BASE . '/includes/defines.php';
}
if (!defined( '_JEXEC' )) {
define( '_JEXEC', 1 );
define( 'JPATH_BASE', realpath(dirname(__FILE__)));
require_once ( JPATH_BASE .'/includes/defines.php' );
require_once ( JPATH_BASE .'/includes/framework.php' );
}
// Get the framework.
require_once JPATH_LIBRARIES . '/import.legacy.php';
// Bootstrap the CMS libraries.
require_once JPATH_LIBRARIES . '/cms.php';
/**
* CLI script to bulk import JSON data into Joomla articles.
*
* @since 2.5
*/
class BulkArticles extends JApplicationCli {
public function doExecute() {
$this->out(JText::_('Start'));
// Fetch the JSON data
$jsonData = $this->fetchData();
// Run the insertArticle bulk data import to articles function
$this->insertArticle($jsonData);
$this->out(JText::_('Complete'));
}
private function fetchData() {
$curl = curl_init();
curl_setopt_array($curl, array(
CURLOPT_URL => 'https://api.dtn.com/publishing/news/articles?categoryId=16%2C17%2C18&limit=10&maxAge=30&apikey=placeholder',
CURLOPT_RETURNTRANSFER => true,
CURLOPT_ENCODING => '',
CURLOPT_MAXREDIRS => 10,
CURLOPT_TIMEOUT => 0,
CURLOPT_FOLLOWLOCATION => true,
CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
CURLOPT_CUSTOMREQUEST => 'GET',
));
$json = curl_exec($curl);
curl_close($curl);
$data = json_decode($json, true);
return $data;
}
private function insertArticle($data) {
foreach ($data as $value) {
// Outputs expected results from JSON data
// echo($value['title']);
// echo("<br/>");
// echo($value['storySummary']);
// echo("<br/>");
// echo($value['content']);
// echo("<br/>");
$articleData = array(
'id' => 0,
'catid' => 8,
'title' => ucwords(strtolower($value['title'])),
'introtext' => strip_tags($value['storySummary']),
'fulltext' => strip_tags($value['content']),
'state' => 1,
'access' => 2,
'created_by' => 332,
'language' => '*'
);
$data['rules'] = array(
'core.edit.delete' => array(),
'core.edit.edit' => array(),
'core.edit.state' => array(),
);
$basePath = JPATH_ADMINISTRATOR . '/components/com_content';
require_once $basePath.'/models/article.php';
$config = array();
$articleModel = new ContentModelArticle($config);
if(!$articleModel->save($data)) {
$errMsg = $articleModel->getError();
return false;
} else {
$id = $articleModel->getItem()->id;
return $id;
}
echo 'Article created';
}
}
}
JApplicationCli::getInstance('BulkArticles')->execute();
Update - Sample JSON data
[
{
"id": "fbb0427e-7529-4a74-b390-87bc21330068__2556796",
"links": {
"article": {
"href": "/publishing/news/articles/fbb0427e-7529-4a74-b390-87bc21330068__2556796"
}
},
"title": "DTN Closing Livestock Comments",
"subTitle": "Lower Trend Blankets Livestock Futures Ahead of Close",
"storySummary": "<span class=\"news_summary_content\">As the markets head into the last trading day of the week, they're hopeful that more support will push the contracts higher ahead of the weekend.</span>",
"publishDate": "2022-01-27T22:21:00Z",
"product": null,
"author": {
"name": "ShayLe Stewart"
},
"thumbNailImage": {
"name": "2020closinglivestock.jpg",
"imageUrl": "https://www.dtnpf.com/mydtn-public-core-portlet/servlet/GetStoredImage?category=CMS&symbolicName=2020closinglivestock.jpg",
"caption": "(DTN file photo)"
},
"cmsCategoryId": null,
"dtnpfUrl": "https://www.dtnpf.com/agriculture/web/ag/news/article/2022/01/27/lower-trend-blankets-livestock-ahead",
"language": "en",
"topic": "DTN/Ag/Livestock",
"content": "<span class=\"news_summary_content\">As the markets head into the last trading day of the week, they're hopeful that more support will push the contracts higher ahead of the weekend.</span>",
"status": null,
"audioLinks": null,
"mediaLinks": null
},
{
"id": "07027E30",
"links": {
"article": {
"href": "/publishing/news/articles/07027E30"
}
},
"title": "Canadian Agriculture PM News",
"subTitle": "Canadian Agriculture PM News",
"storySummary": "<p class=\"news_content\">Canadian Financial Close: Loonie hits three-week low\nBy MarketsFarm</p>",
"publishDate": "2022-01-27T21:35:42Z",
"product": null,
"author": null,
"thumbNailImage": null,
"cmsCategoryId": null,
"dtnpfUrl": "",
"language": "en",
"topic": "DTN/Ag",
"content": "\n<div style=\"background:#FFFFFF\">\n Canadian Financial Close: Loonie hits three-week low\n By MarketsFarm\n WINNIPEG, Jan. 27 (MarketsFarm) - The Canadian dollar fell \nto its lowest level in three weeks the day after the Bank of \nCanada (BoC) announced it was holding its key interest rate at \n0.25 per cent.\n The loonie was at US$0.7867 or US$1=C$1.2712 on Thursday, \ndown from Wednesday's close of US$0.7933 or US$1=C$1.2606. Both \nthe BoC and the United States Federal Reserve announced on \nWednesday they were keeping their interest rates steady for the \ntime being, but both signaled increases to counter inflation in \nthe coming months.\n A strengthening greenback allowed the U.S. Dollar Index to \njump 1.30 of a point to 97.25, its highest level since June \n2020.\n Benchmark crude oil prices were down on Thursday. Brent \ncrude oil slipped US$0.18 per barrel to US$89.78. West Texas \nIntermediate (WTI) crude oil dropped US$0.30 to US$87.05/barrel. \nWestern Canadian Select (WCS) crude oil increased US$0.64 at \nUS$74.27/barrel. \n The TSX/S&P Composite Index declined 51.78 points to \n20,544.11.\n Gold nosedived US$35.50 per ounce to US$1,796.50, \ncontinuing its drop after the Fed's announcement.\n Canada's agricultural sector fared as follows:\n\nFarmer's Edge Inc. dn $ 0.01 at $ 2.76\nLinamar Corp. up $ 1.15 at $ 69.07\nMaple Leaf Foods up $ 0.09 at $ 29.55\nNutrien Ltd. dn $ 0.44 at $ 88.31\nRitchie Bros Auctioneers Inc. dn $ 1.52 at $ 73.74\n \n (All figures are in Canadian dollars.)\n END\n\n\n</div>\n",
"status": null,
"audioLinks": null,
"mediaLinks": null
}
]
Any help or tips is greatly appreciated.
fetchData
function is working, but I am having issues with thecreateArticle
function. I am getting the following warnings in Terminal when executing the CLI script:PHP Warning: Undefined variable $articleData
,PHP Warning: Undefined variable $value
,PHP Warning: Undefined variable $data
, andPHP Warning: foreach() argument must be of type array|object, null given
. I updated my question with the current code.$articleExists
asfalse
, then immediately checking if the value is falsey in thecreateArticle()
method? Can you offer a sample payload that might be returned fromhttps://api.dtn.com/publishing/news/articles
? It looks like you are expecting upto 10 articles. This suggests after json_decoding, there will be an array of arrays(rows). This means thatJFilterOutput::stringURLSafe($value['title'])
outside of a loop will not correctly access thetitle
element because$value
needs to refer to a "row" andtitle
is a key in a row.$articleExists
comes from. Just want to avoid duplicate articles or risk of failure of imports if duplicates exist. Thanks for your help.