WebProtégé Next Generation Overview
The newest version of WebProtege is built using a microservices architecture. We are gradually transitioning WebProtege away from being a monolithic application to one that is composed as a bunch of small services. This will make WebProtege more extensible going forward and will allow it to scale better as we receive more and more projects on webprotege.stanford.edu and as ontologies and knowledge graphs get larger and larger. Third parties will also be able to create their own isolated backend-extensions, which are akin to plugins for Desktop Protégé.
This document gives a brief overview of the set of WebProtégé microservices and the external services that WebProtégé relies upon.
The table below outlines the backend micro-services along with the default ports that they run on. Any WebProtégé installation will need to set up and run these sevices along with any external dependencies. This is currently a manual process, but going forward we will be automating the build and deployment process.
The core services, that WebProtégé absolutely requrires, are: webprotege-backend-service, webprotege-authorization service, and the webprotege-gwt-gateway-api.
Executable JARs are Spring Boot Applications. The default port can be overriden using the server.port configuration property.
External dependencies should be available before services are started but thereafter services may be started in any order. Services are implemented as executable JAR files and may be started in the usual way (java -jar <JAR FileName>) with configuration properties specified as command line arguments in the usual way (using the -D flag).
The WebProtégé front-end is currently implemented as a Google Web Toolkit (GWT) application. This is a servlet-container based application and should be run using a servlet container, such as Tomcat. It also must be run using Java 11.
| Service Name | Brief Description and Relevant Configuration Properties | Default Port | Form | External Dependencies |
|---|---|---|---|---|
| webprotege-backend-service | Represents the current backend monolith You must use webprotege.directories.datato specify the WebProtégé data directory.Use spring.data.mongodb.urito configure the connection to MongoDB, if not using defaults. Releases are available here. Example run command: java -Xmx10G -Dwebprotege.directories.data=/data -jar webprotege-backend-service-0.1.0.jar Note: We are gradually transitioning away from this monolithic service using the strangler vine pattern [Fowler, Richardson]. |
7770 | Spring Boot Application (Executable JAR) | Apache Pulsar, MongoDB, MinIO |
| webprotege-authorization-service | Manages user roles and capabilities. Use spring.data.mongodb.urito configure the connection to MongoDB, if not using defaults.Releases are available here. Example run command: java -jar webprotege-authorization-service-0.9.0.jar Note: This service requires a running instance of Apache Pulsar for testing. |
7771 | Spring Boot Application (Executable JAR) | Apache Pulsar, MongoDB |
| webprotege-ontology-processing-service | Processes uploaded ontologies by parsing them, resolving owl:imports and converting them to a format that can be quickly parsed and loaded into a project and revision history. The following configuration properties must be specified when running this service: webprotege.directories.data, minio.clientKey, minio.clientSecret, minio.endPointReleases are available here. |
7772 | Spring Boot Application (Executable JAR) | Apache Pulsar, MinIO |
| webprotege-snapshot-generator-service | Generates snapshots of a particular project revision for download or downstream processing. The following configuration properties must be specified when running this service: webprotege.directories.data, minio.clientKey, minio.clientSecret, minio.endPointReleases are available here. |
7773 | Spring Boot Application (Executable JAR) | Apache Pulsar, MinIO |
| webprotege-gwt-api-gateway | Provides a unified API for the webprotege-gwt-frontend. The following configuration properties should be specified: security.oauth2.resourceserver.jwt.issuer-uri, which should specify the URL to the keycloak server (e.g. http://localhost:8180/auth/realms/webprotege)Releases are available here. Example run command: java -jar webprotege-gwt-api-gateway-0.1.0.jar
|
7777 | Spring Boot Application (Executable JAR) | Apache Pulsar, Keycloak |
| webprotege-gwt-ui | Serves the user interface and translates requests from GWT, sent via GWT-RPC, to backend requests routed via the API Gate Way service. Note that this must be run under Java 11 (it will not work with later versions of Java due to illegal reflective access errors) | 8080 | WAR | Keycloak |
The following diagram gives a flavor of the over all design of WebProtégé
WebProtégé requires the following third party services:
| Service | Description | Default Port |
|---|---|---|
| Apache Pulsar | WebProtege uses asynchronous messaging for inter-service communication on the backend. This is facilitated by Apache Pulsar | 9092 |
| Apache Zoo Keeper | Used in combination with Apache Pulsar | 2181 |
| MongoDB | MongoDB is used to store a variety of json documents that represent structured information that WebProtege uses for things like forms, tags, user-roles and capabilities, user-activity, | 27019 |
| MinIO | MinIO is used for blob storage of project snapshots (for download) and uploaded and processed ontologies. MinIO can be run locally or, more commonly, it is used as an interface to something like Amazon S3. | 9000 |
| Apache Tomcat | The main WebProtege user interface is written using The Google Web Tooklit (GWT) and this runs as a container based web-app. | 8080 (Note this clashes with Keycloak) |
| Keycloak | Provides authentication services. Note that the default port is 8080, but this can be overridden when starting keycloak using the jboss.socket.binding.port-offsetproperty with a value of 100.WebProtégé will attemt to connect to Keycloak with a client id of webprotege and it expects there to be a webprotege realm as well, however this can be overridden when starting the API gateway service. |
8180 |
WebProtégé also requires access to the local file system for storing ontology change logs. Change logs are currently stored using the BinaryOWL log format. WebProtégé needs to be able to create and append changes to files in order for this functionality to be available.
WebProtégé uses an asynchornous style of communication between backend services. You can read more about it on the API documentation page.