tutorials/3. Validating models and simulations/2. Validating simulation experiments (SED-ML files).ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "38d2cc02-4bf8-4244-aa4c-5112edcb3fd6",
   "metadata": {},
   "source": [
    "# Validating simulation experiments (SED-ML files)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "54dd0772-fca4-4819-9094-807fb90c7caf",
   "metadata": {},
   "source": [
    "This tutorial illustrates how to validate simulation experiments encoded in the [Simulation Experiment Description Markup Language (SED-ML)](http://sed-ml.org/). This format uses [URNs](https://sed-ml.org/urns.html) to describe the language of each model. This format uses the [Kinetic Simulation Algorithm Ontology (KiSAO)](https://github.com/SED-ML/KiSAO) to describe algorithms and algorithm parameters.\n",
    "\n",
    "<div class=\"alert alert-block alert-info\">\n",
    "    The methods described here do not validate the sources of the models (e.g., CellML, SBML files) of SED-ML files or targets for changes to models or for outputs of simulations of models. More comprehensive validation is available through validation of entire simulation projects (COMBINE/OMEX archives). This encompasses validation of the sources of models in SED-ML files and targets for models for model changes and simulation observables. Please see the <a href=\"4.%20Validating%20simulation%20projects%20(COMBINE-OMEX%20archives).ipynb\">simulation project validation tutorial</a> for more information.\n",
    "</div>\n",
    "\n",
    "<div class=\"alert alert-block alert-info\">\n",
    "    SED-ML is under active development. The SED-ML community is working toward a single interpretation of SED-ML. BioSimulators supports the interpretation of SED-ML described at <a href=\"https://docs.biosimulations.org/concepts/conventions/simulation-experiments/\">https://docs.biosimulations.org</a>.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e3ff26f-f6dd-4001-8369-0bc4227c4477",
   "metadata": {},
   "source": [
    "## 1. Validate a SED-ML file online"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a3f43f57-3d3e-4009-a7fa-363ba2b050aa",
   "metadata": {},
   "source": [
    "The easiest way to validate a SED-ML file is to use the web interface at https://run.biosimulations.org. An HTTP API for validating SED-ML files is also available at [https://combine.api.biosimulations.org](https://combine.api.biosimulations.org/)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "35937c08-11c7-4efb-bf50-ef621729e721",
   "metadata": {
    "tags": []
   },
   "source": [
    "## 2. Validate a SED-ML file with the BioSimulators command-line application"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a494796c-3473-4c69-94c1-ba89b4c53384",
   "metadata": {},
   "source": [
    "First, install [BioSimulators-utils](https://github.com/biosimulators/Biosimulators_utils). Installation instructions are available at [https://docs.biosimulators.org](https://docs.biosimulators.org/Biosimulators_utils). Note, BioSimulators-utils must be installed with the installation options for the model languages that you wish to validate. A Docker image with BioSimulators utils and all dependencies is also available ([`ghcr.io/biosimulators/biosimulators`](https://github.com/biosimulators/Biosimulators/pkgs/container/biosimulators))."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad43eced-2ee5-48b4-868b-e9e9d8bc7e3d",
   "metadata": {},
   "source": [
    "Inline help for the `biosimulators-utils` command-line program is available by running the program with the `--help` option."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "5d10d4de-fda2-4d8b-b54c-c767146eca3e",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Invalid -W option ignored: invalid module name: 'biosimulators_utils.warnings'\n",
      "usage: biosimulators-utils [-h] [-d] [-q] [-v]\n",
      "                           {convert,exec,validate-project,validate-metadata,validate-simulation,validate-model,build-project}\n",
      "                           ...\n",
      "\n",
      "Utilities for working with containerized biosimulation tools\n",
      "\n",
      "optional arguments:\n",
      "  -h, --help            show this help message and exit\n",
      "  -d, --debug           full application debug mode\n",
      "  -q, --quiet           suppress all console output\n",
      "  -v, --version         show program's version number and exit\n",
      "\n",
      "sub-commands:\n",
      "  {convert,exec,validate-project,validate-metadata,validate-simulation,validate-model,build-project}\n",
      "    convert             Convert files among formats\n",
      "    exec                Execute a model project (COMBINE/OMEX archive)\n",
      "    validate-project    Validate a model project (COMBINE/OMEX archive)\n",
      "    validate-metadata   Validate metadata (OMEX Metadata file)\n",
      "    validate-simulation\n",
      "                        Validate a simulation experiment (SED-ML file)\n",
      "    validate-model      Validate a model (e.g., CellML, SBML file)\n",
      "    build-project       Build a modeling project\n"
     ]
    }
   ],
   "source": [
    "!biosimulators-utils --help"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "73c28ef7-6d41-4b33-8b3b-e9629d7a48ca",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Invalid -W option ignored: invalid module name: 'biosimulators_utils.warnings'\n",
      "usage: biosimulators-utils validate-simulation [-h] filename\n",
      "\n",
      "Validate a simulation experiment (SED-ML file)\n",
      "\n",
      "positional arguments:\n",
      "  filename    Path to a SED-ML file\n",
      "\n",
      "optional arguments:\n",
      "  -h, --help  show this help message and exit\n"
     ]
    }
   ],
   "source": [
    "!biosimulators-utils validate-simulation --help"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91a88f05-0522-4894-80ec-43bc89386312",
   "metadata": {},
   "source": [
    "Next, use the command-line program to validate the [simulation experiment](../_data/Ciliberto-J-Cell-Biol-2003-morphogenesis-checkpoint-continuous.sedml)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "910d0b3d-9059-4c83-8e20-e28b8220687e",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Invalid -W option ignored: invalid module name: 'biosimulators_utils.warnings'\n"
     ]
    }
   ],
   "source": [
    "!biosimulators-utils validate-simulation ../_data/Ciliberto-J-Cell-Biol-2003-morphogenesis-checkpoint-continuous.sedml"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e04e8798-906c-4bbc-93d8-ca4458752c2d",
   "metadata": {},
   "source": [
    "If the simulation is invalid, a list of errors will be printed to your console."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c929c603-6676-4a2e-a6a5-6ac703d4d63d",
   "metadata": {},
   "source": [
    "## 3. Validate a SED-ML file programmatically with Python"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "675310b7-f67b-487d-a01c-edc42a0e15a0",
   "metadata": {},
   "source": [
    "First, install [BioSimulators-utils](https://github.com/biosimulators/Biosimulators_utils). Installation instructions are available at [https://docs.biosimulators.org](https://docs.biosimulators.org/Biosimulators_utils). Note, BioSimulators-utils must be installed with the installation options for the model languages that you wish to validate. A Docker image with BioSimulators utils and all dependencies is also available ([`ghcr.io/biosimulators/biosimulators`](https://github.com/biosimulators/Biosimulators/pkgs/container/biosimulators))."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f8ddc08-537f-4945-9328-5cf3fd661c85",
   "metadata": {},
   "source": [
    "Next, import BioSimulators-utils' SED-ML reader."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "4708120d-da66-4355-8011-130f2d67c187",
   "metadata": {},
   "outputs": [],
   "source": [
    "from biosimulators_utils.sedml.io import SedmlSimulationReader"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e8de44c8-ff53-4d92-a3b7-9052653a57cc",
   "metadata": {},
   "source": [
    "Next, use this class to read the SED-ML file. To skip validating the sources of models and targets to them, set the `validate_models_with_language` argument to `False`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "6fe99a0d-8e91-45d0-b3d4-b42ce2fc2088",
   "metadata": {},
   "outputs": [],
   "source": [
    "reader = SedmlSimulationReader()\n",
    "simulation_filename = '../_data/Ciliberto-J-Cell-Biol-2003-morphogenesis-checkpoint-continuous.sedml'\n",
    "sed_doc = reader.run(simulation_filename, validate_models_with_languages=False, validate_targets_with_model_sources=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db298a08-9a46-4d12-bd0b-b65491bf4c14",
   "metadata": {},
   "source": [
    "The `run` method raises exceptions on invalid SED-ML files. The `run` method also stores lists of errors and warnings in the instance of `SedmlSimulationReader`. This information can be accessed by catching the exceptions raised by the `run` method. Human-readable messages can then be printed out using the `flatten_nested_list_of_strings` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "86ab3916-6d22-42b4-a568-e85f0f051962",
   "metadata": {},
   "outputs": [],
   "source": [
    "try:\n",
    "    sed_doc = reader.run(simulation_filename, validate_models_with_languages=False, validate_targets_with_model_sources=False)\n",
    "except (ValueError, NotImplementedError):\n",
    "    pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "b692d258-c800-43cb-8b96-5abaa24203ac",
   "metadata": {},
   "outputs": [],
   "source": [
    "from biosimulators_utils.utils.core import flatten_nested_list_of_strings\n",
    "from warnings import warn\n",
    "\n",
    "if reader.warnings:\n",
    "    warn(flatten_nested_list_of_strings(reader.warnings), UserWarning)\n",
    "\n",
    "if reader.errors:\n",
    "    raise ValueError(flatten_nested_list_of_strings(reader.errors))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91540815-6a21-49a6-bd14-f5c0ae31d69e",
   "metadata": {},
   "source": [
    "The output of the `run` method (`sed_doc`) is a data structure that represents all of the information contained in the SED-ML file, including the models, simulations, tasks, reports, and plots. This data structure is an instance of `biosimulators_utils.sedml.data_model.SedDocument`. The `vars` method and `inspect` module can be used to inspect the attributes and methods of this data structure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "9bf59ac0-f003-48ae-a9f1-3da66611e552",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'level': 1,\n",
       " 'version': 3,\n",
       " 'models': [<biosimulators_utils.sedml.data_model.Model at 0x7f9db42d1a60>],\n",
       " 'simulations': [<biosimulators_utils.sedml.data_model.UniformTimeCourseSimulation at 0x7f9db42d1d90>],\n",
       " 'tasks': [<biosimulators_utils.sedml.data_model.Task at 0x7f9db42d1940>],\n",
       " 'data_generators': [<biosimulators_utils.sedml.data_model.DataGenerator at 0x7f9db42d1700>,\n",
       "  <biosimulators_utils.sedml.data_model.DataGenerator at 0x7f9df8173880>,\n",
       "  <biosimulators_utils.sedml.data_model.DataGenerator at 0x7f9db431b0d0>,\n",
       "  <biosimulators_utils.sedml.data_model.DataGenerator at 0x7f9db42d1b50>,\n",
       "  <biosimulators_utils.sedml.data_model.DataGenerator at 0x7f9db431b370>,\n",
       "  <biosimulators_utils.sedml.data_model.DataGenerator at 0x7f9db42b6340>,\n",
       "  <biosimulators_utils.sedml.data_model.DataGenerator at 0x7f9db42b6250>,\n",
       "  <biosimulators_utils.sedml.data_model.DataGenerator at 0x7f9db42b6bb0>,\n",
       "  <biosimulators_utils.sedml.data_model.DataGenerator at 0x7f9db42b6ca0>,\n",
       "  <biosimulators_utils.sedml.data_model.DataGenerator at 0x7f9db42b6520>,\n",
       "  <biosimulators_utils.sedml.data_model.DataGenerator at 0x7f9db42b6d90>,\n",
       "  <biosimulators_utils.sedml.data_model.DataGenerator at 0x7f9db42b6610>,\n",
       "  <biosimulators_utils.sedml.data_model.DataGenerator at 0x7f9db42b6ac0>,\n",
       "  <biosimulators_utils.sedml.data_model.DataGenerator at 0x7f9db42b6f70>,\n",
       "  <biosimulators_utils.sedml.data_model.DataGenerator at 0x7f9db42b68b0>,\n",
       "  <biosimulators_utils.sedml.data_model.DataGenerator at 0x7f9db42b69a0>,\n",
       "  <biosimulators_utils.sedml.data_model.DataGenerator at 0x7f9db42b69d0>,\n",
       "  <biosimulators_utils.sedml.data_model.DataGenerator at 0x7f9db42b6700>,\n",
       "  <biosimulators_utils.sedml.data_model.DataGenerator at 0x7f9db42b6e80>,\n",
       "  <biosimulators_utils.sedml.data_model.DataGenerator at 0x7f9db42b60d0>,\n",
       "  <biosimulators_utils.sedml.data_model.DataGenerator at 0x7f9db42b6dc0>],\n",
       " 'outputs': [<biosimulators_utils.sedml.data_model.Report at 0x7f9db42d1610>,\n",
       "  <biosimulators_utils.sedml.data_model.Plot2D at 0x7f9db42454c0>,\n",
       "  <biosimulators_utils.sedml.data_model.Plot2D at 0x7f9db42ae520>,\n",
       "  <biosimulators_utils.sedml.data_model.Plot2D at 0x7f9db3b82fd0>,\n",
       "  <biosimulators_utils.sedml.data_model.Plot2D at 0x7f9db42db0a0>],\n",
       " 'metadata': None}"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vars(sed_doc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "3b9b0ad0-4a11-4e46-9d21-5a63636fb2fd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'__init__': <bound method SedDocument.__init__ of <biosimulators_utils.sedml.data_model.SedDocument object at 0x7f9db42d1eb0>>,\n",
       " 'is_equal': <bound method SedDocument.is_equal of <biosimulators_utils.sedml.data_model.SedDocument object at 0x7f9db42d1eb0>>,\n",
       " 'to_tuple': <bound method SedDocument.to_tuple of <biosimulators_utils.sedml.data_model.SedDocument object at 0x7f9db42d1eb0>>}"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import inspect\n",
    "dict(inspect.getmembers(sed_doc, predicate=inspect.ismethod))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}