Memory leaks #1150

crystoy · 2019-12-17T03:27:49Z

Q	A
Bug report?	yes
Feature request?	no
BC Break report?	no
RFC?	no

Steps required to reproduce the problem

Run project which uses this serializer in swoole http server(or other CLI mode that can reproduce this case), memory_limit is set to 128M
The simplified code looks like this:

<?php

...

$serializer = $this->getSerializer();
$server->handle('/', function($request, $response) use ($serializer) {
    $data = ['OK'];
    $response->send($serializer->serialize($data, 'json'));
});

...

public function getSerializer(): Serializer
{
    $builder = SerializerBuilder::create();
    $builder->setCacheDir(ROOT_DIR . '/temp/runtime/serializer');
    $builder->setSerializationContextFactory(function () {
        return SerializationContext::create()
            ->setSerializeNull(true);
    });
    $builder->configureHandlers(function (HandlerRegistry $registry) {
        $registry->registerSubscribingHandler(new DateTimeHandler());
    });

    return $builder->build();
}

Run apache ab test: ab -n 100000 http://example.test/, "n" should be a number bigger enough

Expected Result

Memory usage should always at a low-level, 10~20M, etc.

Actual Result

Memory usage increases to 128M until the project crashes

I'm not sure if it is caused by this serializer, but when I remove it, everything goes well again.
Does anyone else encoutered this problem?
Any help would be appreciated, thanks.

goetas · 2019-12-17T09:04:12Z

if you change:

$server->handle('/', function($request, $response) use ($serializer) {
    $data = ['OK'];
    $response->send($serializer->serialize($data, 'json'));
});

to

for ($i = 0, $i<1000000; $i++) {
    $data = ['OK'];
    $serializer->serialize($data, 'json');
};

and just run it as a script?

crystoy · 2019-12-17T09:22:20Z

Thanks for your help.
I tried this script and it seems no memory leaks at all...
I will check other points to find the problem.

goetas · 2019-12-17T09:27:42Z

Then is related on how your application is built.

goetas · 2019-12-17T09:28:26Z

Is this a reactphp application?

crystoy · 2019-12-17T09:35:49Z

Then is related on how your application is built.

I think it is.

Is this a reactphp application?

No, it's an application based on swoole, an extreme fast asynchronous engine which is widely used in china.

goetas · 2019-12-17T09:38:07Z

Consider that if you are using symfony, their compiled DI container leaks memory when executed in a loop. But I think here you will have to figure it out. (related to https://bugs.php.net/bug.php?id=75914)

crystoy · 2019-12-17T09:45:24Z

Wow...I'm using symfony DI indeed!
I'll do some test to check it

goetas · 2019-12-17T09:54:46Z

if you disable the feature that splits the DI container into multiple files, you should have no leaks.

crystoy · 2019-12-17T10:09:24Z

Sorry I have no idea how to disable this feature...
And what makes me confused is that when I use a json_encode() instead of serializer, it also solve the problem...

crystoy · 2019-12-18T07:44:50Z

After many times of try, I believe it's not symfony DI's fault, because I even rewrite the DI container but without any effects...
And last I find maybe it's related on symfony console component.
Here are some test cases without using symfony DI, it needs to run in CLI mode with swoole installed

<?php

use JMS\Serializer\SerializerBuilder;
use JMS\Serializer\SerializerInterface;
use Swoole\Coroutine\Http\Server;
use Swoole\Http\Request;
use Swoole\Http\Response;
use Symfony\Component\Console\Application;
use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;

require 'vendor/autoload.php';

$serializer = SerializerBuilder::create()->build();

class StartCommand extends Command
{

    /**
     * @var SerializerInterface
     */
    private $serializer = null;

    public function setSerializer(SerializerInterface $serializer)
    {
        $this->serializer= $serializer;
    }

    protected function execute(InputInterface $input, OutputInterface $output)
    {
        go(function () {
            $server = new Server('127.0.0.1', 3000);
            $server->handle('/', function (Request $request, Response $response) {
                $data = ['OK'];
                if ($this->serializer) {
                    $response->end($this->serializer->serialize($data, 'json'));
                } else {
                    $response->end(json_encode($data));
                }
            });
            $server->start();
        });

        return 0;
    }

}

// serializer and symfony console
$test1 = function () use ($serializer) {
    $app = new Application();
    $command = new StartCommand('start');
    $command->setSerializer($serializer);
    $app->add($command);
    $app->run();
};

// json_encode and symfony console
$test2 = function () {
    $app = new Application();
    $app->add(new StartCommand('start'));
    $app->run();
};

// serializer and no symfony console
$test3 = function () use ($serializer) {
    go(function () use ($serializer) {
        $server = new Server('127.0.0.1', 3000);
        $server->handle('/', function (Request $request, Response $response) use ($serializer) {
            $data = ['OK'];
            $response->end($serializer->serialize($data, 'json'));
        });
        $server->start();
    });
};

//$test1(); // About 88M memory usage(in monitor) after 50,000 requests
//$test2(); // About 8.5M memory usage(in monitor) after 50,000 requests
//$test3(); // About 14.5M memory usage(in monitor) after 50,000 requests

@goetas

goetas · 2019-12-18T08:25:20Z

What is the memory after a single iteration?

crystoy · 2019-12-18T08:36:08Z

What is the memory after a single iteration?

I didn't test it.

BUT I find a usable solution ...
Just place the go() function at the top level, not in execute() method will solve the problem...
I don't know why, but it just works!
There maybe some conflicts between swoole and symfony console, I can't figure it out, hope someone can do..
Anyway, thanks a lot for your careful help

<?php

use JMS\Serializer\SerializerBuilder;
use JMS\Serializer\SerializerInterface;
use Swoole\Coroutine\Http\Server;
use Swoole\Http\Request;
use Swoole\Http\Response;
use Symfony\Component\Console\Application;
use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;

require 'vendor/autoload.php';

$serializer = SerializerBuilder::create()->build();

class StartCommand extends Command
{

    /**
     * @var SerializerInterface
     */
    private $serializer = null;

    public function setSerializer(SerializerInterface $serializer)
    {
        $this->serializer= $serializer;
    }

    protected function execute(InputInterface $input, OutputInterface $output)
    {
        $server = new Server('127.0.0.1', 3000);
        $server->handle('/', function (Request $request, Response $response) {
            $data = ['OK'];
            if ($this->serializer) {
                $response->end($this->serializer->serialize($data, 'json'));
            } else {
                $response->end(json_encode($data));
            }
        });
        $server->start();

        return 0;
    }

}

// This is correct
go(function () use ($serializer) {
    $app = new Application();
    $command = new StartCommand('start');
    $command->setSerializer($serializer);
    $app->add($command);
    $app->run();
});

goetas · 2019-12-18T10:13:15Z

When having memory leaks, I use https://github.com/BitOne/php-meminfo to hunt them down

…

On Wed, 18 Dec 2019, 09:36 crystoy, ***@***.***> wrote: What is the memory after a single iteration? I didn't test it. BUT I find a usable solution ... Just place the go() function at the top level, not in execute() method will solve the problem... I don't know why, but it just works! There maybe some conflicts between swoole and symfony console, I can't figure it out, hope someone can do.. Anyway, thanks a lot for your careful help <?phpuse JMS\Serializer\SerializerBuilder;use JMS\Serializer\SerializerInterface;use Swoole\Coroutine\Http\Server;use Swoole\Http\Request;use Swoole\Http\Response;use Symfony\Component\Console\Application;use Symfony\Component\Console\Command\Command;use Symfony\Component\Console\Input\InputInterface;use Symfony\Component\Console\Output\OutputInterface;require 'vendor/autoload.php';$serializer = SerializerBuilder::create()->build();class StartCommand extends Command{ /** * @var SerializerInterface */ private $serializer = null; public function setSerializer(SerializerInterface $serializer) { $this->serializer= $serializer; } protected function execute(InputInterface $input, OutputInterface $output) { $server = new Server('127.0.0.1', 3000); $server->handle('/', function (Request $request, Response $response) { $data = ['OK']; if ($this->serializer) { $response->end($this->serializer->serialize($data, 'json')); } else { $response->end(json_encode($data)); } }); $server->start(); return 0; }}// This is correctgo(function () use ($serializer) { $app = new Application(); $command = new StartCommand('start'); $command->setSerializer($serializer); $app->add($command); $app->run();}); — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1150?email_source=notifications&email_token=AAF5UJY7F22SEBXYE6FOYNLQZHOHRA5CNFSM4J3VFXKKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHFLE2A#issuecomment-566932072>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAF5UJY2NAHGHSBRNJUITH3QZHOHRANCNFSM4J3VFXKA> .

crystoy · 2019-12-18T10:22:07Z

OK, nice tool, I'll have a try later, thanks

goetas closed this as completed Dec 17, 2019

crystoy mentioned this issue Dec 26, 2019

使用symfony/console+jms/serializer中遇到的内存泄漏问题 swoole/swoole-src#3040

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Memory leaks #1150

Memory leaks #1150

crystoy commented Dec 17, 2019 •

edited

Loading

goetas commented Dec 17, 2019

crystoy commented Dec 17, 2019

goetas commented Dec 17, 2019

goetas commented Dec 17, 2019

crystoy commented Dec 17, 2019

goetas commented Dec 17, 2019

crystoy commented Dec 17, 2019

goetas commented Dec 17, 2019

crystoy commented Dec 17, 2019

crystoy commented Dec 18, 2019 •

edited

Loading

goetas commented Dec 18, 2019

crystoy commented Dec 18, 2019

goetas commented Dec 18, 2019 via email

crystoy commented Dec 18, 2019

Memory leaks #1150

Memory leaks #1150

Comments

crystoy commented Dec 17, 2019 • edited Loading

Steps required to reproduce the problem

Expected Result

Actual Result

goetas commented Dec 17, 2019

crystoy commented Dec 17, 2019

goetas commented Dec 17, 2019

goetas commented Dec 17, 2019

crystoy commented Dec 17, 2019

goetas commented Dec 17, 2019

crystoy commented Dec 17, 2019

goetas commented Dec 17, 2019

crystoy commented Dec 17, 2019

crystoy commented Dec 18, 2019 • edited Loading

goetas commented Dec 18, 2019

crystoy commented Dec 18, 2019

goetas commented Dec 18, 2019 via email

crystoy commented Dec 18, 2019

crystoy commented Dec 17, 2019 •

edited

Loading

crystoy commented Dec 18, 2019 •

edited

Loading