Application Plugin Interface

[Yaliang]

Introducing an application plugin interface can provide flexible context reference for components(for example, UI) which are built on top of Presto main.

[Yaliang]

+cc @billonahill @dabaitu

[electrum]

Passing Injector here won't work, since plugins can't see Presto's Guice from inside their class loader, by design. We intentionally do not expose the internal details of Presto to plugins. Anything needed by a plugin service should be explicitly part of the SPI.

[Yaliang]

How about setting couple context groups as different ApplicationContexts in SPI just like ConnectorContext and initialize them in ApplicationManager?

[electrum]

What is a "context group"? I feel like I'm missing a lot of context around the purpose of this pull request. Can you explain more about the use case?

[Yaliang]

Initially, I suppose to bind UI on the port other than the API port here: #7106. In that PR, I just add a new module inside the PrestoServer and rebind the instance in the initialized injector to another Bootstrap, which looks hacky. I feel it would be reasonable adding an application plugin which runs on top of presto-main, and potentially passing some context instances from the initialized injector in some way. I agree that it shouldn't pass the whole injector and it shouldn't expose all internal instance to plugins.

[electrum]

At Facebook, we run an extended version of Presto. Create a module that depends on presto-main and write an extension of PrestoServer with additional modules:

public class ExtendedPresto
        extends PrestoServer
{
    public static void main(String[] args)
    {
        new ExtendedPresto().run();
    }

    @Override
    protected Iterable<? extends Module> getAdditionalModules()
    {
        return ImmutableList.of(...);
    }
}

Package it up by creating a module that is basically a copy of presto-server. You'll need the provisio-maven-plugin declaration from presto-root with <extensions>true</extensions> in order to use the <packaging>provisio</packaging> to build the server tarball.

(To be clear, we don't have a fork of Presto internally. We depend on presto-main and all other released artifacts that are published to Maven Central. All our extensions use integration points like the one described above.)

[Yaliang]

The idea of the application plugin is that the additional modules can be built within an isolated Bootstrap and with an isolated config file while are still able to refer some instances in the PrestoServer. In the example of #7106, if the config file is isolated, it would be much easier to config the port for the UI server; if the Bootstrap is isolated, it would be much easier to install the HttpServerModule. Do you think we can achieve the same goal with ExtendedPresto?

[electrum]

You can do this with ExtendedPresto. For example, one thing we do is bind a Thrift server to integrate with internal monitoring tools (using https://github.com/facebook/swift), which binds its own config classes.

return ImmutableList.of(
        new ThriftCodecModule(),
        new ThriftClientModule(),
        new ThriftServerModule(),
        new Fb303StatusModule(),
        ...);

[Yaliang]

I think the scenarios here are a little bit different. In your scenario, the additional modules are brand new, the config keys are different to those from other modules. However, I wish to add the HttpServerModule whose usage is purely for hosting UI(and potentially some URL forward for the resources related to UI) as the additional module. While the main Bootstrap

      modules.add(
             new NodeModule(),
             new DiscoveryModule(),
             new HttpServerModule(),          <-----here
             new JsonModule(),
             new JaxrsModule(true),
             new MBeanModule(),
             new JmxModule(),
             new JmxHttpModule(),
             ...);

already has HttpServerModule module serving UI, API and more. Since the module is same, they would have same property keys, which makes the things slightly difficult.

[electrum]

Are you saying you want to replace the Airlift HttpServerModule with your own version? If that's the case, we could change getAdditionalModules() to be just getModules(), letting to change/replace the entire list.

Alternatively, and/or as an interim solution, you can just fork the PrestoServer class and modify it as necessary.

[Yaliang]

I could say cleaner. The goal is letting two Airlift HttpServerModules to be installed. One is serving UI(and forwarding partial API) and the other one is serving API and the rest if any.