Setup of Mongo ODM in PHP 7 – performance considered

In a previous topic I have described our journey in optimising a symfony app ( the-journey-of-a-symfony-api-from-150ms-to-20ms ). Some of you have asked about specific configs, so here we go:

Install mongo-odm and the adapter:

php composer.phar require alcaeus/mongo-php-adapter
php composer.phar require doctrine/mongodb-odm-bundle

Now let’s configure it:

            server: mongodb://%mongo_server%
                connect: true
                connectTimeoutMS: 300 # as stated in the original post: timeouts are very important
    default_database: %database_name%
            # this is also important to reduce disk usage, you could also use other backends for caching this
            metadata_cache_driver: apc   
            retry_connect:              1
            retry_query:                1
            auto_mapping: true

To have the best control I recommend to allow slave readonly only from code. For this you can simply use:

$this->mongo->getManager()->getClassMetadata(MyDocumentType::class)->slaveOkay = true

Install doctrine mongo bundle on php7 enviroments

I have recently made a new symfony project on php7 and I had a lot of problems installing mongo doctrine.

What I have tried:

  • Directly mention packages in composer.json and run composer update
  • Putting all new packages “@stable” version

None of the above worked, but I have found a very specific way that it worked and the order of the steps is very important:

php composer.phar require alcaeus/mongo-php-adapter
php composer.phar require doctrine/mongodb-odm-bundle

The journey of a Symfony API from 150ms to 20ms

I have been involved in the development of an app based on symfony 2.8 which also used solr, Mongo and redis.

The problem at hand: response times were slow (see table below). With some bold objectives we went down the road.



Best practices says to enable caching, but before going there let’s try to see what can we optimize, as caching may make some things harder to discover.

At this point you should make sure that you have the proper measuring tools (APM and load testing). For this we have used jmeter and free new relic. Get some numbers before starting the process and have patience (some results will start be seen after days).

Use your DB engine properly

What i mean is for mysql/mongo for example put in place proper indexes and analyse the queries your run.
Think about that a query that you runs every few hours could lock your tables/collections for a few seconds and this could increase your response time. Our particular case was in regards to missing indexes in mongo.

Another bottleneck were the Solr query. We have made some tweaks to both schema and config of the solr it self and boom…. from calls that took 2-3 seconds… we don’t have any above 500ms (the 99% at 3000 was mostly generated by solr calls)

Use the latest versions

Or at least somewhere around :). For example our upgrade from

php5.6 to php70 has helped us with a boost of 25% (and some headaches dues to required upgrade of Mongo driver and doctrine missing native support for the new driver).

Update 2016.12.1: Someone asked about the upgrade of doctrine+mongo to PHP 7. As many may have observed already, the old driver is deprecated and doctrine-mongo-odm is not compatible with the new driver. But someone made a cool transition package ( that worked great for us.
Another problem encountered during the upgrade of ODM was related to the field annotation, that we had to migrate from @MongoDB\String @MongoDB\Int to @MongoDB\Field(type=”string”)

upgrade Mongo from 2.4 to 3.2 – well… someone used a very old version of Mongo when the cluster was made. This was a bit tricky as we had to take the whole app down for about 1 hour as we couldn’t migrate from 2.4 to 3.2 without taking a full backup and restoring it on a new cluster.


If you don’t need it, remove it! If you need it in development, load it just in dev/test env.

Some examples of bundles disabled on our app were:

  • SecurityBundle – our API is read-only and it’s nothing that needs to be protected (or at least you could declare a dedicated firewall for the public API sections)
  • SwiftMailerBundle
  • SensioFrameworkExtraBundle
  • TwigBundle
  • AsseticBundle


Make sure all configs are in production mode. Some examples would be:

  • make as little as possible IO (also consider logs in the process)
  • Doctrine makes a lot of cache files for metadata. Put it in APC/APCU.
                server: mongodb://%mongo_servers%
                    connect: true
                    connectTimeoutMS: 300
        default_database: %database_name%
                metadata_cache_driver: apc
                retry_connect:              1
                retry_query:                1
                auto_mapping: true
  • persistent connections where possible (we had some problems with redis after doing this, because we were using 2 databases for different purposes and switching dbs… was a nightmare)
  • Make sure you read from slaves. (we were killing the Mongo master because of this)
    • Here we need to control when to read from slave… so we haven’t allowed it by default, but we enabled it on demand:
      $container->get('doctrine_mongodb')->getManager()->getClassMetadata('<Entity Name>')->slaveOkay = true
  • Try to consider what the timeout of the clients are and use them in your app. If you know your client is ignoring your response, why bother completing the entire request (example: timeout -300ms, but a internal http call takes 3000ms. This would keep you busy for nothing )

Queue it!

If you don’t need something now, just put in a queue and continue with serving requests. Write operations (to db or disk) are usually heavier than putting it on a queue and handling it on a separate server.

We use RabbitMQ with this bundle:


It’s an API and we chose to have twig enabled only in dev environment (so that we can have the profiler work). 🙂

Try to use php directly, but if you can’t live without it, at least install php twig extension (brings some boost in performance)

Container / DI

Here we also did a little comparison with other bare frameworks like slilex. We choose to stay with symfony as the performance penalty was minor compared with the advantages:

  • yml configs were actually cached as php code at deploy
  • yml errors were caught during deploy phase (at cache warmup) and faulty containers can not be released live
  • Personally I started to use a lot ContainerPass’es and also “tags”
  • We already knew how to define and use services in symfony and brining a new framework make lead to some beginner mistakes.


This was one of the final steps we added. A few tricks to be considered here:

  • Make sure your connection to the cache server is persistent
  • Consider with what TTL you should cache (depending on change freq). If you have a large data set and freq updates on various events, you can set a high TTL and make the update process to also flush the cache
  • you can try multi level caches if you need it (apc + memcache), but it will also generate some headaches
  • To avoid complicated logic inside your methods you could use a cool caching bundle:
  • You might have a lot of cache evictions due to large cache.
    • Try to reduce the size of caches (is it really required that often?)
    • Add additional capacity to the caching server.

Nginx / FPM config

Here we did some changes after we discovered that under high load we had increased response time, but our servers were free (CPU and memory). We have increase the number of worker/child processes for both nginx and php-fpm and this allowed us higher load.


I have recently upgraded one of work project to PHP 7. What a bliss… Just by upgrading we have managed to reduce response time for Symfony based API from 100ms to 75ms (25% boost). (Note: responsibility of the decreased timing might also be from some updated libraries and drivers)
I have also managed to upgrade the present blog to the new PHP version and a similar improvement was observed.

Other major improvements from my perspective:

  • Strong type return methods (this can help increase the code quality and will help resolve some bugs in some early stages). I was a bit disappointed that overloading is still not available.
  • Syntax errors are now catchable

Cache Bundle for Symfony

Have you ever felt that your classes are burdened with caching logic instead of proper logic? I know you did, and so did I. In an effort to improve readability and quality of the code, me and my colleagues at eMAG have developed internally a library that will remove all the extra logic for caching from method body.

And after some time we have decided to open source it, so that others can also use it and help develop it.

The library is available on github: . Also there is available also a small use guide.


wakup tool v2

Today i’ve worked hard to make some changes to “wakeup tool” and so it came v2.

Changes that worth mentioning:

  • New layout based on bootstrap. It’s responsive now 🙂
  • Grouped hosts by type (Linux, Windows, Webserver). Each with it’s own verification procedure
  • Changed WOL mechanism in order to remove dependencies on some installed binaries.
  • Changed autodiscovery mechanism. Now it will check if host alive trough checking ports 3389,22,80. Again removed dependencies on external binaries.
  • Changed log in mechanism to sfDoctrineGuard in order to allow future features that require permissions. If you require integration with Active Directory have a look here.

You can have a look yourself at:

Why Scala is not so cool

A friend of mine is currently attending a Scala course on courseraand she showed me one of her issues searching for an advice. Unfortunately for her after 5 minutes i was leaving the desk as i was unable to come up with any idea.

Based on this experience i tried to have a look at Scala docummentation, so i’ve started with introduction and first thing i’ve seen was that is Java based. Considering my previous experience with a programming language writen over Java, Coldfusion(a huge resource consumer), this wasn’t the best “nice to meet you” conversation. Basically Scala is compiling your code into Java binary compatible file.

Scala programs run on the Java VM, are byte code compatible with Java

A plus was the fact that major companies were already using Scala.

For example, at Twitter, the social networking service, Robey Pointer moved their core message queue from Ruby to Scala. This change was driven by the company’s need to reliably scale their operation to meet fast growing Tweet rates, already reaching 5000 per minute during the Obama Inauguration. Robeys thinking behind the Twitter Kestrel project is explained in the developers live journal. His concise 1500 lines of Scala code can be seen as he has generously made them available as an open source project.

So far introduction hasn’t brought very much confidence in continuing, so i asked my friend to give me some key points that would represent the an advantage of using it in a real case. First and only point(due to lack of time) mentioned was a better stack management while using recursive functions. At this point i said that all recursive methods can be implemented also iterative in order to avoid stack problems while using large volumes of data. On the other hand this may raise complexity of the code.

This conversation being over i’ve came back to Scala site searching for some code examples. Look what i found:

Any method which takes a single parameter can be used as an infix operator in Scala. Here is the definition of classMyBool which defines three methods andor, and negate.

class MyBool(x: Boolean) {
  def and(that: MyBool): MyBool = if (x) that else this
  def or(that: MyBool): MyBool = if (x) this else that
  def negate: MyBool = new MyBool(!x)
def not(x: MyBool) = x negate; // semicolon required here
def xor(x: MyBool, y: MyBool) = (x or y) and not(x and y)

Why a programming language requires such a feature? Why don’t they like obj.method(param) ?

Adobe WebUp #9

This time, it was quite short (19:45 – 21:10) with only 2 presentation, both directly related to some cool js libraries.

First presentation was kept by a student(Andrei Oprea), that is an intern at Adobe RO. He presented an online version of tool that resembles with PowerPoint –
This tool is based on a node.js server with front-end developed using less, jQuery, bootstrap and require.js. About jQuery, less and bootstrap i heard also before, but require.js was the new stuff.

What’s require.js? It’s a dependency management library, that can help you a lot in terms of concurrency issues at load(we all had this problem at least once).

Continue reading Adobe WebUp #9

HTTPS has problems

It seems like nothing is secure those days. I found a post about possible security issues with HTTPS on Firefox and Chrome. The issues is realted to some features enabled by default in those browsers(TLS compression or SPDY) that would allow access to cookies from https sites. In the latest versions of Firefox and Chrome those features were disabled, so update as soon as possible.

Surprising for me it’s that this happends to Firefox and Chrome and not to Internet Explorer, but it seems like the fact that they don’t implement all the new cool stuff saved them.

Also as webmaster you can protect your users by disabling those features in your webserver.

wakeup tool

Today was the first expose to public of  the wakeup tool. It’s available on github here

The tool is designed for companies that want to reduce the amount of power consumer by the computers while employees are not using them, but also offer possibility during VPN connections to access the machines.

It requires wake-on-lan to be enabled from bios and, for the auto-discovery functionality, it requires that computer would have Windows Shares enabled( it uses netbios scans ). Also computer may be added manually to the list.

Currently documentation and interface are still in work, but the core functionality is working.