Software module for answering questions on processes

ℹ️ About ℹ️

This is a software module developed to answer questions about linear business processes using sophisticated machine learning techniques.

This module represents a final thesis written in the form of a program project for the Faculty of Informatics in Pula under the mentorship of Nikola Tanković. The topic is based on the program engineering course.

It was developed using Google Colaboratory running Python 3.7 notebook. It uses different HuggingFace models, frameworks like TensorFlow and PyTorch and datasets like

📜 Features 📜

For a given JSON object that represents a set of business processes (all being linear in the nature) and training data that is oriented around these processes, the module is capable to answer the following question (for a specific business process):

• P1: What are all the tasks of the process
• P2: What is the duration of the process?
• P3: What is the description of an individial task?
• P4: What is the duration of a certain task?
• P5: What task comes after the specific task?
• P6: Out of domain (none of the above)

🤖 How does it work 🤖

For a certain question, the module goes through few logic units.

First, the user intent recognition module. This module was designed to categorize a question into one of the 6 predetermined question domains (6th being an "empty domain" for questions that do not belong to any other domain). The module was developed by Aldo Ferlatti (source).

There exist 2 types of domains: simple and complex. After the domain of the question is figured out, main module checks whether the domain is a simple or a complex one. If the domain is a simple domain, then all the data is already available via JSON model. After extracting the data, the main module prints out a message with the data. On the other hand, if the domain is a complex domain, further logic is required. Complex domains require some additional extraction from the question itself, so the next logic module is data extraction module.

The data extraction module is composed of few different submodules, one being named entity recognition module (NER module). NER module is finetuned to recognise the names of the tasks and mark them as an "Organisation" entity. NER module was built by Deepak moonat (source). By doing this, the module "zooms into" the original question to find the most matching part of the question to one of the task names. Once "zoomed in" (or extracted), next submodule transforms the result of the previous submodule into multidimenstional vector space and performs a cosine similarity among all the task names to find which of the task names the extracted data matches the most. For this task, all-MiniLM-L6-v2 sentence transformer was used. This module yields a number, indicating what is the list index of the task being referenced in the original question (assuming that a complex domain always contains a task name in the question).

Finally, after the additional data extraction (specifically the task number), the rest of the data is available via JSON model, which is then printed out by the main module.

This process is repeated until a user inputs a letter "q" indicating the termination of the interaction.

Example of the JSON model:

json = [
    {
        "name": "Foo Bar",
        "phases": [
            {
                "name": "Foo",
                "alias": ["<other>", "<names>", "<for>", "<foo>"],
                "description": "<your description goes here>",
                "duration": "<Foo duration goes here>",
            },
            {
                "name": "Bar",
                "alias": ["<alias are optional>"],
                "description": "...",
                "duration": "...",
            },
        ],
        "duration": "<duration of the Foo Bar>",
    },
    {
        "name": "Fry the eggs",
        "phases": [
            {
                "name": "Set up the oven",
                "alias": ["Start", "First step", "Oven"],
                "description": "Turn on your oven...",
                "duration": "few seconds",
            },
            ...
            {
                "name": "Bon appetite!",
                "alias": ["Final step", "ready meal"],
                "description": "Your meal is now ready...",
                "duration": "None",
            },
        ],
        "duration": "5-10 minutes",
    },
]

Demo sample (translated):

To end the conversation, enter 'q'

>>> how long does the practice last
The process 'practice' takes 2 month

>>> I need to do the practice
The process stages for the 'Practice' process are: selecting preferences, filling out the application form and submitting the practice log

>>> how long does it take to select preferences
The 'preference selection' process takes 1 month

>>> Who is Joe?
Unfortunately, I do not understand your question

>>> q
Good bye! ( ^_^)/

💻 Running it on local machine 💻

You can run the code on Google Colaboratory platform (click me). Initial execution of the code should last for ~30 min on CPU and ~5 min on GPU mode.

If you, however, want to download the notebook and run it localy, there is a quick 4 minute video tutorial on YouTube to help you set up Jupyter Notebooks and all the necessary environment. Clone the repository (or download ZIP), and run the notebook. You may need to pip install some libraries since the working notebook was developed in Google Colaboratory, which has some common libraries preinstalled.