php - Implementing goMongoDB-like Query expression object evaluation

Question

I've been looking for a MongoDb-like ( http://docs.mongodb.org/manual/applications/read/#find, docs.mongodb.org/manual/reference/operators/ ) query expression object evaluation function implementation or a class. It may cover not all the advanced features, and should have extensible architecture.

MongoDB-like query expression objects are easy for understanding and usage, providing ability to write clean, self-explaining code, because both query and objects to search in, are associative arrays.

Basically talking its a convenient function to extract information from php arrays. Knowing the array structure(the arrayPath), it will allow to perform operations on multidimensional arrays data, without the need for multiple nested loops.

If you are not familiar with MongoDb, take a look at a given expression object and array to search in.

I wrote it as JSON string for simplicity. The object contents makes no sense, just showng the MongoDb query syntax.

MongoDb-like query expression object

{
    "name": "Mongo",
    "type": "db",
    "arch": {
        "$in": [
            "x86",
            "x64"
        ]
    },
    "version": {
        "$gte": 22
    },
    "released": {
        "$or": {
            "$lt": 2013,
            "$gt": 2012
        }
    }
}

The array to search in

[
    {
        "name": "Mongo",
        "type": "db",
        "release": {
            "arch": "x86",
            "version": 22,
            "year": 2012
        }
    },
    {
        "name": "Mongo",
        "type": "db",
        "release": {
            "arch": "x64",
            "version": 21,
            "year": 2012
        }
    },
    {
        "name": "Mongo",
        "type": "db",
        "release": {
            "arch": "x86",
            "version": 23,
            "year": 2013
        }
    }
]

Find using Mongo-like query expressions

So, with the help of the function, we should be able to issue the following query to the target array.

$found=findLikeMongo($array, $queryExpr); //resulting in a $array[0] value;
//@return found array

Get array path using Mongo-like query expressions

$arrayPath=getPathFromMongo($array, $queryExpr);// resulting in array("0")
//@return array path, represented as an array where entries are consecutive keys.

Homework

• I found that goessner.net/articles/JsonPath/ could possibly cover my needs(not being an exact match because it uses Xpath-like expressions), the caveat is, that it heavily relies on regular expressions and string parsing, what will definitely slow it down compared to array only(JSON like) implementation.

• Also I've found a similar question here, @stackoverflow Evaluating MongoDB-like JSON Queries in PHP. The resulting answer was to use some SPL functions, which I am used to avoid most of the time.
  Wonder if the author had came up with function, he had been trying to develop.

• The possible arrayPath implementation was found on thereisamoduleforthat.com/content/dealing-deep-arrays-php, thus the lack of this implementation, is that it relies on pointers.

I know its not a trivial question with a oneliner answer, that's why I'm asking it before starting the actual development of my own class.

I appreciate architecture tips, related or similar code, which may be a good practice example for building php "if..else" expressions on the fly.emphasized text

How to write a non-SPL version?

@Baba provided an excellent class, which is written with the use of SPL. I wonder how to rewrite this code without SPL.

There are two reasons for this

• calling the class multiple times will give function overhead, that can be avoided rewriting it in raw PHP.
• it would be easily portable to raw Javascript where SPL is not available, leading to easier code maintenance on both platforms.

Results

The created ArrayQuery class is published on Github, consider checking-out the repository for updates.

SPL, raw PHP version and Chequer2 FORP profiler output

In brief-

1. the raw PHP version performs 10x faster than the SPL one, consuming 20% less memory.
2. Chequer2 class performs 40% slower than PHP SPL class, and almost 20x slower than raw PHP version.
3. MongoDb is the fastest(10x faster than raw PHP implementation and consumes 5x less memory), do not use these classes unless you are sure you want to avoid interaction with MongoDb.

MongoDb version

SPL version

Raw PHP(latest ArrayQuery class) version

Chequer2 version

MongoDb reference test profiling code

$m = new MongoClient(); // connect
$db = $m->testmongo; // select a database
$collection = $db->data;
$loops=100;
for ($i=0; $i<$loops; $i++) {
    $d = $collection->find(array("release.year" => 2013));
}
print_r( iterator_to_array($d) );

PHP with SPL class profiling code

include('data.php');
include('phpmongo-spl.php');
$s = new ArrayCollection($array, array("release.year" => 2013),false);
$loops=100;
for ($i=0; $i<$loops; $i++) {
    $d = $s->parse();
}
print_r( $d );

The SPL class parse() function has been slightly modified to return the value after execution, it could be also be modified to accept expression, but it's not essential for profiling purposes as the expression is being reevaluated every time.

raw PHP(latest ArrayQuery class) profiling code

include('data.php');
include('phpmongo-raw.php');
$s = new ArrayStandard($array);
$loops=100;
for ($i=0; $i<$loops; $i++) {
    $d = $s->find(array("release.year" => 2013));
}
print_r( $d );

chequer2 PHP profiling code

<?php
include('data.php');
include('../chequer2/Chequer.php');
$query=array("release.year" => 2013);

$loops=100;
for ($i=0; $i<$loops; $i++) {
    $result=Chequer::shorthand('(.release.year > 2012) ? (.) : NULL')
        ->walk($array);

}
print_r($result);
?>

data used(same as @baba provided in his answer)

$json = '[{
    "name":"Mongo",
    "type":"db",
    "release":{
        "arch":"x86",
        "version":22,
        "year":2012
    }
},
{
    "name":"Mongo",
    "type":"db",
    "release":{
        "arch":"x64",
        "version":21,
        "year":2012
    }
},
{
    "name":"Mongo",
    "type":"db",
    "release":{
        "arch":"x86",
        "version":23,
        "year":2013
    }
},      
{
    "key":"Diffrent",
    "value":"cool",
    "children":{
        "tech":"json",
        "lang":"php",
        "year":2013
    }
}
]';

$array = json_decode($json, true);

the forp-ui slightly modified sample ui loader(to be called with ?profile=FILE_TO_PROFILE)

<!doctype html>
<html>
    <head>
        <style>
            body {margin : 0px}
        </style>
    </head>
    <body>
        <div class="forp"></div>
<?php
register_shutdown_function(
    function() {
        // next code can be append to PHP scripts in dev mode
        ?>
        <script src="../forp-ui/js/forp.min.js"></script>
        <script>
        (function(f) {
            f.find(".forp")
             .each(
                function(el) {
                    el.css('margin:50px;height:300px;border:1px solid #333');
                }
             )
             .forp({
                stack : <?php echo json_encode(forp_dump()); ?>,
                //mode : "fixed"
             })
        })(forp);
        </script>
        <?php
    }
);

// start forp
forp_start();

// our PHP script to profile
include($_GET['profile']);

// stop forp
forp_end();
?>
</body>
</html>

Answer 1

Introduction

I think Evaluating MongoDB-like JSON Queries in PHP has given all the Information you need. all you need is to be creative with the solution and you achieve what you want

The Array

Lets assume we have the follow json converted to array

$json = '[{
    "name":"Mongo",
    "type":"db",
    "release":{
        "arch":"x86",
        "version":22,
        "year":2012
    }
},
{
    "name":"Mongo",
    "type":"db",
    "release":{
        "arch":"x64",
        "version":21,
        "year":2012
    }
},
{
    "name":"Mongo",
    "type":"db",
    "release":{
        "arch":"x86",
        "version":23,
        "year":2013
    }
},      
{
    "key":"Diffrent",
    "value":"cool",
    "children":{
        "tech":"json",
        "lang":"php",
        "year":2013
    }
}
]';

$array = json_decode($json, true);

Example 1

check if key - Different would be as simple as

echo new ArrayCollection($array, array("key" => "Diffrent"));

Output

{"3":{"key":"Diffrent","value":"cool","children":{"tech":"json","lang":"php","year":2013}}}

Example 2 Check if release year is 2013

echo new ArrayCollection($array, array("release.year" => 2013));

Output

{"2":{"name":"Mongo","type":"db","release":{"arch":"x86","version":23,"year":2013}}}

Example 3

Count where Year is 2012

$c = new ArrayCollection($array, array("release.year" => 2012));
echo count($c); // output 2 

Example 4

Lets take from your example where you want to check version is grater than 22

$c = new ArrayCollection($array, array("release.version" => array('$gt'=>22)));
echo $c;

Output

{"2":{"name":"Mongo","type":"db","release":{"arch":"x86","version":23,"year":2013}}}

Example 5

Check if release.arch value is IN a set such as [x86,x100] (Example)

$c = new ArrayCollection($array, array("release.arch" => array('$in'=>array("x86","x100"))));
foreach($c as $var)
{
    print_r($var);
}

Output

Array
(
    [name] => Mongo
    [type] => db
    [release] => Array
        (
            [arch] => x86
            [version] => 22
            [year] => 2012
        )

)
Array
(
    [name] => Mongo
    [type] => db
    [release] => Array
        (
            [arch] => x86
            [version] => 23
            [year] => 2013
        )

)

Example 6

Using Callable

$year = 2013;
$expression = array("release.year" => array('$func' => function ($value) use($year) {
    return $value === 2013;
}));

$c = new ArrayCollection($array, $expression);

foreach ( $c as $var ) {
    print_r($var);
}

Output

Array
(
    [name] => Mongo
    [type] => db
    [release] => Array
        (
            [arch] => x86
            [version] => 23
            [year] => 2013
        )

)

Example 7

Register your own expression name

$c = new ArrayCollection($array, array("release.year" => array('$baba' => 3)), false);
$c->register('$baba', function ($a, $b) {
    return substr($a, - 1) == $b;
});
$c->parse();
echo $c;

Output

{"2":{"name":"Mongo","type":"db","release":{"arch":"x86","version":23,"year":2013}}}

Class Used

class ArrayCollection implements IteratorAggregate, Countable, JsonSerializable {
    private $array;
    private $found = array();
    private $log;
    private $expression;
    private $register;

    function __construct(array $array, array $expression, $parse = true) {
        $this->array = $array;
        $this->expression = $expression;
        $this->registerDefault();
        $parse === true and $this->parse();
    }

    public function __toString() {
        return $this->jsonSerialize();
    }

    public function jsonSerialize() {
        return json_encode($this->found);
    }

    public function getIterator() {
        return new ArrayIterator($this->found);
    }

    public function count() {
        return count($this->found);
    }

    public function getLog() {
        return $this->log;
    }

    public function register($offset, $value) {
        if (strpos($offset, '$') !== 0)
            throw new InvalidArgumentException('Expresiion name must always start with "$" sign');

        if (isset($this->register[$offset]))
            throw new InvalidArgumentException(sprintf('Expression %s already registred .. Please unregister It first'));

        if (! is_callable($value)) {
            throw new InvalidArgumentException(sprintf('Only callable value can be registred'));
        }

        $this->register[$offset] = $value;
    }

    public function unRegister($offset) {
        unset($this->register[$offset]);
    }

    public function parse() {
        $it = new RecursiveIteratorIterator(new RecursiveArrayIterator($this->array));
        foreach ( $it as $k => $items ) {
            if ($this->evaluate($this->getPath($it), $items)) {
                $this->found[$it->getSubIterator(0)->key()] = $this->array[$it->getSubIterator(0)->key()];
            }
        }
    }

    private function registerDefault() {
        $this->register['$eq'] = array($this,"evaluateEqal");
        $this->register['$not'] = array($this,"evaluateNotEqual");

        $this->register['$gte'] = array($this,"evaluateGreater");
        $this->register['$gt'] = array($this,"evaluateGreater");

        $this->register['$lte'] = array($this,"evaluateLess");
        $this->register['$lt'] = array($this,"evaluateLess");

        $this->register['$in'] = array($this,"evalueateInset");

        $this->register['$func'] = array($this,"evalueateFunction");
        $this->register['$fn'] = array($this,"evalueateFunction");
        $this->register['$f'] = array($this,"evalueateFunction");
    }

    private function log($log) {
        $this->log[] = $log;
    }

    private function getPath(RecursiveIteratorIterator $it) {
        $keyPath = array();
        foreach ( range(1, $it->getDepth()) as $depth ) {
            $keyPath[] = $it->getSubIterator($depth)->key();
        }
        return implode(".", $keyPath);
    }

    private function checkType($a, $b) {
        if (gettype($a) != gettype($b)) {
            $this->log(sprintf("%s - %s  is not same type of %s - %s", json_encode($a), gettype($a), json_encode($b), gettype($b)));
            return false;
        }
        return true;
    }

    private function evaluate($key, $value) {
        $o = $r = 0; // Obigation & Requirement
        foreach ( $this->expression as $k => $options ) {
            if ($k !== $key)
                continue;

            if (is_array($options)) {
                foreach ( $options as $eK => $eValue ) {
                    if (strpos($eK, '$') === 0) {
                        $r ++;
                        $callable = $this->register[$eK];
                        $callable($value, $eValue) and $o ++;
                    } else {
                        throw new InvalidArgumentException('Missing "$" in expession key');
                    }
                }
            } else {

                $r ++;
                $this->evaluateEqal($value, $options) and $o ++;
            }
        }
        return $r > 0 && $o === $r;
    }

    private function evaluateEqal($a, $b) {
        return $a == $b;
    }

    private function evaluateNotEqual($a, $b) {
        return $a != $b;
    }

    private function evaluateLess($a, $b) {
        return $this->checkType($a, $b) and $a < $b;
    }

    private function evaluateGreater($a, $b) {
        return $this->checkType($a, $b) and $a > $b;
    }

    private function evalueateInset($a, array $b) {
        return in_array($a, $b);
    }

    private function evalueateFunction($a, callable $b) {
        return $b($a);
    }
}

Summary

It may cover not all the advanced features, and should have extensible architecture

The above class shows a typical example of what you want .. you can easy decouple it , extend it to support compound expressions like $and and $or

MongoDB-like query expression objects are easy for understanding and usage, providing ability to write clean, self-explaining code, because both query and objects to search in, are associative arrays.

Why not just write the array to a MongoDB database rather than working it arrays ?? It more efficient and it would save you a lot of troubles

I must also mention that use the best tool for the best job ... What you want is basically a function of a Database

Basically talking its a convenient function to extract information from php arrays. Knowing the array structure(the arrayPath), it will allow to perform operations on multidimensional arrays data, without the need for multiple nested loops.

The example shows how using a path to search for value but you are still dependent on loading the array to memory and your class performing multiple recursion ans loops which is not as efficient as a database .

I appreciate architecture tips, related or similar code, which may be a good practice example for building php "if..else" expressions on the fly.

Do you really mean you want all those just in here ???