Errors de-crypting libsodium encrypted values from database

Question

I've been implementing some encryption at rest in one of our Joomla components, but keep getting a couple of errors when de-crypting the values when they're written and read from a database.

So I wrote a basic wrapper class for crypt...

/**
 * This class provides functions related to encryption
 *
 * @since  2.5
 */
class crypt
{

    public $cipher = FALSE;

    public function __construct()
    {

        # test private key
        $private_key_string = "51111110024135111761721919212413";
        # test public key
        $public_key_string = "76172191921241351111110024135111";

        # get the cipher object
        $cipher = new JCryptCipherSodium;
        # generate a key
        $key = $cipher->generateKey();
        # set the nonce
        $cipher->setNonce(\Sodium\randombytes_buf(\Sodium\CRYPTO_BOX_NONCEBYTES));

        # set the key values
        $key->private = $private_key_string;
        $key->public  = $public_key_string;

        $this->key = $key;
        $this->cipher = $cipher;
        return TRUE;
    }

    public function en($string){
        #echo  "\n\nencrypt\n";
        $key = $this->key;
        $encrypted = $this->cipher->encrypt($string, $key);
        #echo  "encrypted: " . $encrypted . "\n";
        return $encrypted;
    }

    public function de($string){
        #echo  "\n\ndecrypt\n";
        $key = $this->key;
        $decrypted = $this->cipher->decrypt($string, $key);
        #echo  "decrypted: " . $decrypted . "\n";
        return $decrypted;
    }
}

I know this is not best practice, with keys, but I was just starting off as a proof of concept.

This works fine with a simple encrypt then decrypt, like...

JLoader::register("crypt","cli/crypt_helper.php");      

$crypt = new crypt();
$enc = $crypt->en("hello world, how you doing");
$de = $crypt->de($enc);
echo "de: " . $de . "\n";

...but when I write to a database and read back, I get the errors mentioned above. So with code like...

# database test
$db = JFactory::getDbo();
$query = $db->getQuery(true);
# Insert columns.
$columns = array('f1',);
# Insert values.
$values = array($db->quote( $crypt->en( $test_value ) ) );
# Prepare the insert query.
$query
    ->insert($db->quoteName('#__piota_results'))
    ->columns($db->quoteName($columns))
    ->values(implode(',', $values));
# Set the query using our newly populated query object and execute it.
$db->setQuery($query);
$db->execute();

# Create a new query object.
$query = $db->getQuery(true);
# Order it by the ordering field.
$query->select($db->quoteName(array('f1')));
$query->from($db->quoteName('#__piota_results'));
$query->order('id DESC');
$query->setLimit('1');
# Reset the query using our newly populated query object.
$db->setQuery($query);
# Load the results as a list of stdClass objects (see later for more options on retrieving data).
$result = $db->loadResult();

echo "result: " . $result . "\n";
echo "result: " . $crypt->de( $result ) . "\n";

I've tried on two different systems, with slightly different errors.

System 1

• PHP 7.2.6
• Joomla! 3.8.8
• MariaDB 5.5.5-10.1.34

System 1 error message: Invalid MAC this is, I think, coming from https://github.com/joomla/joomla-cms/blob/9ae0742f28faa1d4b5d4ae4a84619f115ed3f9ea/libraries/vendor/paragonie/sodium_compat/src/Crypto.php#L1004

System 2

• PHP 7.2.6
• Joomla! 3.8.6
• MySQL 5.7.22

System 2 error message: Argument 1 must be at least CRYPTO_BOX_MACBYTES long which is, I think, coming from https://github.com/joomla/joomla-cms/blob/9ae0742f28faa1d4b5d4ae4a84619f115ed3f9ea/libraries/vendor/paragonie/sodium_compat/src/Compat.php#L1069

The database field is of the type "longtext" with collation "utf8mb4_bin".

I did some reading, but since libsodium is quite new in Joomla, there is not so much docs or info on it.

I tried using base64_encode and base64_decode in-case MySQL was making a mess of the data, but it was all the same.

I looked at...

• https://issues.joomla.org/tracker/joomla-cms/16754
• https://github.com/paragonie/sodium_compat
• http://php.net/manual/en/book.sodium.php

...but with no luck.

Any ideas, thoughts or pointers would be greatly appreciated. Thanks ❤️

Once I get this all working I'll try and write a wiki page on libsodium on https://docs.joomla.org

Answer 1

This works fine with a simple encrypt then decrypt, like...

...but when I write to a database and read back, I get the errors mentioned above.

This is actually expected behavior, because the nonce isn't being stored with the ciphertext. You can verify this by how SodiumCipher is defined.

Try storing the nonce alongside the ciphertext and using it when decrypting. (Don't reuse nonces for multiple rows.)

Also, you will want to encode before storing and decode after retrieving (as your question alluded to), using base64 or hex.

Answer 2

The error messages Invalid MAC and Argument 1 must be at least CRYPTO_BOX_MACBYTES long are definitely misleading.

I had to try it for myself in order to find that both errors appear randomly based on the content of the string.

Despite what I expected, $db->quote() does NOT encode correctly the string returned by crypt. base64_encode/decode are sufficient: see my whole code (I changed to the #__banners table, for ease of testing)

$helperPath = dirname(__FILE__) ."/crypt_helper.php";
JLoader::register("crypt",$helperPath);      
$test_value = 'the <b>quick brown fox jumps</b> over the lazy dog<br>';
$crypt = new crypt();

# database test
$db = JFactory::getDbo();
$query = $db->getQuery(true);
$columns = array('name','params');
$values = array($db->quote('test'), $db->quote( 
    base64_encode( $crypt->en( $test_value ) )
    ));

# Prepare the insert query.
$query
    ->insert($db->quoteName('#__banners'))
    ->columns($db->quoteName($columns))
    ->values(implode(',',$values));
# Set the query using our newly populated query object and execute it.
echo $query .'<br>'; 
$db->setQuery($query);
$db->execute();

# Create a new query object.
$query = $db->getQuery(true);
# Order it by the ordering field.
$query->select($db->quoteName('params'));
$query->from($db->quoteName('#__banners'));
$query->order('id DESC');
$query->setLimit('1');
# Reset the query using our newly populated query object.
$db->setQuery($query);
# Load the results as a list of stdClass objects (see later for more options on retrieving data).
$result = $db->loadResult();

echo "result: " . $result . "\n";
echo "result: " . $crypt->de( base64_decode($result )) . "\n";

die();

These is the video output of the query, inserted and retrieved data:

INSERT INTO `#__banners`
(`name`,`params`) VALUES 
('test','pKihow6a5Dfx/7p+dlGc5qOnBgOvoJ3+TcZu82sCQU/EWYxnsdxRiAJ6cjB2l433LlSOgTWT0i1Sn/uAWdy/hzwRlRUKaQ==')

result: pKihow6a5Dfx/7p+dlGc5qOnBgOvoJ3+TcZu82sCQU/EWYxnsdxRiAJ6cjB2l433LlSOgTWT0i1Sn/uAWdy/hzwRlRUKaQ==

result: the <b>quick brown fox jumps</b> over the lazy dog<br>

Note 1: any attempt without base64_encode fails. I tried it with 10 bytes or 1K of text and it works just fine.

Note 2: $db->insertObject() is much easier to use than your approach, it handles the $db->quote() calls for you, but still requires base64_encode:

$db = JFactory::getDbo();
$object = new stdClass();
$object->name = 'test';
$object->params = base64_encode($crypt->en( $test_value ));
$db->insertObject('#__banners', $object);