examples/Tutorial_1-carrots_demo/carrots_demo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "pZgQLLVe1p47"
   },
   "source": [
    "\n",
    "## 1.1 Introduction to PyDP\n",
    "The PyDP package provides a Python API into [Google's Differential Privacy library](https://github.com/google/differential-privacy). This example uses the alpha 0.4 version of the package that has the following limitations:\n",
    "\n",
    "\n",
    "*   Supports Linux (Windows coming soon)\n",
    "*  Currently implements an algorithm to support computing private mean using a Laplace noise generation technique.\n",
    "* Supports only integer and floating point values\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Example using Differential Privacy library\n",
    "\n",
    "In this directory, we give a simple example of how to use the Python Differential\n",
    "Privacy library.\n",
    "\n",
    "## Zoo Animals\n",
    "\n",
    "There are around 182 animals at Farmer Alex's zoo. Every day, Alex feeds\n",
    "the animals as many carrots as they desire. The animals record how many carrots\n",
    "they have eaten per day. For this particular day, the number of carrots eaten\n",
    "can be seen in `animals_and_carrots.csv`.\n",
    "\n",
    "At the end of each day, Alex often asks aggregate questions about how many\n",
    "carrots everyone ate. For example, he wants to know how many carrots are eaten\n",
    "each day, so he knows how many to order the next day. The animals are fearful\n",
    "that Alex will use the data against their best interest. For example, Alex could\n",
    "get rid of the animals who eat the most carrots!\n",
    "\n",
    "To protect themselves, the animals decide to use the Python Differential Privacy\n",
    "library to aggregate their data before reporting it to Alex. This way, the\n",
    "animals can control the risk that Alex will identify individuals' data while\n",
    "maintaining an adequate level of accuracy so that Alex can continue to run the\n",
    "zoo effectively.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 102
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 8271,
     "status": "ok",
     "timestamp": 1587961311947,
     "user": {
      "displayName": "Ben Szymkow",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj8JCwpOTvlaeImB7WktdTucJJTZ_kDAprKpf_sOA=s64",
      "userId": "08454492702843508366"
     },
     "user_tz": -600
    },
    "id": "h67VRxjSSQ9b",
    "outputId": "6da55f3e-cfb8-4503-a179-ec2613a7f8bd"
   },
   "outputs": [],
   "source": [
    "# Install the PyDP package\n",
    "! pip install python-dp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "ipGLypbRTDSc"
   },
   "outputs": [],
   "source": [
    "import pydp as dp  # by convention our package is to be imported as dp (for Differential Privacy!)\n",
    "from pydp.algorithms.laplacian import BoundedSum, BoundedMean, Count, Max\n",
    "import pandas as pd\n",
    "import statistics  # for calculating mean without applying differential privacy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data\n",
    "\n",
    "Each row in `animals_and_carrots.csv` is composed of the name of an animal, and\n",
    "the number of carrots it has eaten, comma-separated.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 204
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 1171,
     "status": "ok",
     "timestamp": 1587961449410,
     "user": {
      "displayName": "Ben Szymkow",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj8JCwpOTvlaeImB7WktdTucJJTZ_kDAprKpf_sOA=s64",
      "userId": "08454492702843508366"
     },
     "user_tz": -600
    },
    "id": "jM3luaaKSQ9h",
    "outputId": "de008de4-f261-47ef-de5a-d33d23792c9d"
   },
   "outputs": [],
   "source": [
    "# get carrots data from our public github repo\n",
    "url = \"https://raw.githubusercontent.com/OpenMined/PyDP/dev/examples/Tutorial_1-carrots_demo/animals_and_carrots.csv\"\n",
    "df = pd.read_csv(url, sep=\",\", names=[\"animal\", \"carrots_eaten\"])\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Taking the mean of all the entries in a normal fashion without applying the DP library. This is the actual mean of all the records."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Per-animal Privacy\n",
    "\n",
    "Notice that each animal owns at most one row in the data. This means that we\n",
    "provide per-animal privacy. Suppose that some animal appears multiple times in\n",
    "the csv file. That animal would own more than one row in the data. In this case,\n",
    "using this DP library would not guarantee per-animal privacy! The animals would\n",
    "first have to pre-process their data in a way such that each animal doesn't own\n",
    "more than one row.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "5_pUyyKzW1YC"
   },
   "outputs": [],
   "source": [
    "# calculates mean without applying differential privacy\n",
    "def mean_carrots() -> float:\n",
    "    return statistics.mean(list(df[\"carrots_eaten\"]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Private Mean uses Differential Privacy Library by Google to calculate the Mean. To preserve privacy, Laplacian mechanism is used.\n",
    "\n",
    "The function takes the argument privacy_budget as input.\n",
    "\n",
    "It is a number between 0 and 1, denoting privacy threshold\n",
    "\n",
    "It measures the acceptable loss of privacy (with 0 meaning no loss is acceptable).\n",
    "\n",
    "`BoundedMean.quick_result()` takes a List of integer/ float as an input and returns the mean of the list values.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "0AMReuBXTaLV"
   },
   "outputs": [],
   "source": [
    "# calculates mean applying differential privacy\n",
    "def private_mean(privacy_budget: float) -> float:\n",
    "    x = BoundedMean(privacy_budget, 0, 1, 100)\n",
    "    return x.quick_result(list(df[\"carrots_eaten\"]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, the value of the private mean varies compared to the mean calculated using non-private statistical methods.\n",
    "\n",
    "This difference in value corresponds to the privacy that is actually preserved for individual records in it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 51
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 749,
     "status": "ok",
     "timestamp": 1587961454996,
     "user": {
      "displayName": "Ben Szymkow",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj8JCwpOTvlaeImB7WktdTucJJTZ_kDAprKpf_sOA=s64",
      "userId": "08454492702843508366"
     },
     "user_tz": -600
    },
    "id": "vgkt7BuZW3Vw",
    "outputId": "f5708cb5-1477-4f05-851b-b40ae7d3125d"
   },
   "outputs": [],
   "source": [
    "print(\"Mean: \", mean_carrots())\n",
    "print(\"Private Mean: \", private_mean(0.8))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Counts number of animals who ate more than 'limit' carrots without applying the DP library. This is the actual number of such animals."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Calculates number of animals who ate more than \"limit\" carrots without applying differential privacy.\n",
    "def count_above(limit: int) -> int:\n",
    "    return df[df.carrots_eaten > limit].count()[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Private Count Above uses Differential Privacy Library by Google to calculate the number of rows with value above limit. To preserve privacy, Laplacian mechanism is used.\n",
    "\n",
    "The function takes the argument privacy_budget as input.\n",
    "\n",
    "It is a number between 0 and 1, denoting privacy threshold\n",
    "\n",
    "It measures the acceptable loss of privacy (with 0 meaning no loss is acceptable).\n",
    "\n",
    "The function also takes the limit as an argument.\n",
    "\n",
    "`Count.quick_result()` takes a List of integer/ float as an input and returns the count of elements in the list. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Calculates number of animals who ate more than \"limit\" carrots applying differential privacy.\n",
    "def private_count_above(privacy_budget: float, limit: int) -> int:\n",
    "    x = Count(privacy_budget, dtype=\"int\")\n",
    "    return x.quick_result(list(df[df.carrots_eaten > limit][\"carrots_eaten\"]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, the value of Private Count Above varies compared to the Count calculated using normal Statistical methods.\n",
    "\n",
    "This difference in values refers to that privacy is actually preserved for individual records in it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Above 70:\\t\" + str(count_above(70)))\n",
    "print(\"private count above:\\t\" + str(private_count_above(1, 70)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Taking Max of all the entries in a normal fashion without Applying the DP library. This is the actual maximum of carrots eaten of all the records."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Function to return the maximum of the number of carrots eaten by any one animal without appyling differential privacy.\n",
    "def max() -> int:\n",
    "    return df.max()[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Private Max uses Differential Privacy Library by Google to calculate the maximum out of all the values. To preserve privacy, Laplacian mechanism is used.\n",
    "\n",
    "The function takes the argument privacy_budget as input.\n",
    "\n",
    "It is a number between 0 and 1, denoting privacy threshold\n",
    "\n",
    "It measures the acceptable loss of privacy (with 0 meaning no loss is acceptable).\n",
    "\n",
    "`Max.quick_result()` takes a List of integer/ float as an input and returns the list max value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Function to return the maximum of the number of carrots eaten by any one animal appyling differential privacy.\n",
    "def private_max(privacy_budget: float) -> int:\n",
    "    # 0 and 150 are the upper and lower limits for the search bound.\n",
    "    x = Max(epsilon = privacy_budget, lower_bound = 0, upper_bound = 100, dtype=\"int\")\n",
    "    return x.quick_result(list(df[\"carrots_eaten\"]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, the value of Private Max varies compared to the Max calculated using normal Statistical methods.\n",
    "\n",
    "This difference in values refers to that privacy is actually preserved for individual records in it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Max:\\t\" + str(max()))\n",
    "print(\"private max:\\t\" + str(private_max(1)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Taking Sum of all the entries in a normal fashion without Applying the DP library. This is the actual sum of carrots eaten by all the animals."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Function to calculate sum of carrots eaten without applying differential privacy.\n",
    "def sum_carrots() -> int:\n",
    "    return df.sum()[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Private Sum uses Differential Privacy Library by Google to calculate the sum of all the values. To preserve privacy, Laplacian mechanism is used.\n",
    "\n",
    "The function takes the argument privacy_budget as input.\n",
    "\n",
    "It is a number between 0 and 1, denoting privacy threshold\n",
    "\n",
    "It measures the acceptable loss of privacy (with 0 meaning no loss is acceptable).\n",
    "\n",
    "`BoundedSum.quick_result()` takes a List of integer/ float as an input and returns the list sum."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Function to calculate sum of carrots eaten applying differential privacy.\n",
    "def private_sum(privacy_budget: float) -> int:\n",
    "    x = BoundedSum(epsilon = privacy_budget, delta = 0, lower_bound= 1, upper_bound = 100, dtype=\"float\")\n",
    "    return x.quick_result(list(df[\"carrots_eaten\"]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Sum:\\t\" + str(sum_carrots()))\n",
    "print(\"Private Sum:\\t\" + str(private_sum(1)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## How to Run\n",
    "\n",
    "```python PyDP/example/carrots.py```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "1.1 - Introductions to PyDP.ipynb",
   "provenance": [
    {
     "file_id": "https://github.com/OpenMined/PyDP/blob/collab/examples/notebooks/001%20-%20Private%20Mean%20with%20Medical%20Data.ipynb",
     "timestamp": 1587868929244
    }
   ]
  },
  "interpreter": {
   "hash": "1806825b8165262bd0a77e4a07198af7f39d6caf9d567dfbb2cdda81fa2124ef"
  },
  "kernelspec": {
   "display_name": "Python 3.8.10 ('pydp-bl_tS4CM-py3.8')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}