Colab-Exercises.ipynb

{
  "metadata": {
    "colab": {
      "name": "pydata-llama-index-tutorial-colab.ipynb",
      "private_outputs": false,
      "provenance": [],
      "cell_execution_strategy": "setup"
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU",
    "gpuClass": "standard"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "id": "12976e25-8fc1-48d4-9186-9d2bcea0de92",
      "metadata": {},
      "source": [
        "# Running this notebook in Google Colab\n",
        "This notebook is intended to run in Google Colab [here](https://colab.research.google.com/github/datakami/pydata-llama-index-tutorial/blob/main/Colab-Exercises.ipynb).\n",
        "\n",
        "Simply execute the code below to install the neccessary dependencies and download the data."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "9da5883c-d4ec-43e0-bc95-835f875b6c77",
      "metadata": {},
      "outputs": [],
      "source": [
        "%pip install llama-index==0.8.5.post2 sentence-transformers==2.2.2 loguru==0.7.0\n",
        "# download the data\n",
        "from urllib.request import urlretrieve\n",
        "path, _ = urlretrieve(\"https://github.com/datakami/pydata-llama-index-tutorial/archive/refs/heads/main.zip\")\n",
        "import zipfile\n",
        "with zipfile.ZipFile(path, 'r') as zip_ref:\n",
        "    zip_ref.extractall(\".\")\n",
        "!mv pydata-llama-index-tutorial-main/data pydata-llama-index-tutorial-main/indices ."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "c913d2a2-f87f-4505-abb4-49e6b2291e52",
      "metadata": {},
      "source": [
        "# Part 1: Querying"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "0e084dce-bd60-43c1-a13d-cd9636ea2856",
      "metadata": {},
      "source": [
        "If you want to use the OpenAPI API during this tutorial, make a file `secret.py` with `openai_api_key = \"YOURKEYHERE\"`.\n",
        "\n",
        "How to use this notebook: \n",
        "1. Execute all cells under \"Setup\"\n",
        "2. Fill in the exercises below"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "8feb43be-6be1-4cd1-933f-cd3a77052e33",
      "metadata": {},
      "source": [
        "## Setup"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "cdb21f09-827a-412c-8b19-c40fd14825a1",
      "metadata": {},
      "source": [
        "### Imports"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "effd39ab-b6ba-4d69-8e8f-39f382d7712f",
      "metadata": {},
      "outputs": [],
      "source": [
        "import pprint\n",
        "import os\n",
        "import sys\n",
        "from pathlib import Path\n",
        "\n",
        "# logging for lazy people :)\n",
        "from loguru import logger\n",
        "\n",
        "# we're not importing specific methods or classes so it's clear when we actually call llama_index!\n",
        "import llama_index"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "4b056809-10bc-4f4d-90bf-a9b8e8acd34e",
      "metadata": {},
      "source": [
        "### Logging"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "5f44daa1-a3e6-45bf-bc6e-8c84fad646b2",
      "metadata": {},
      "outputs": [],
      "source": [
        "logger.remove()\n",
        "logger.add(sys.stdout, format=\"{time} - {level} - {message}\", level=\"DEBUG\")\n",
        "logger.add(\"tutorial_part_1.log\", level=\"DEBUG\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "ee221890-7faa-4b1a-b593-ffe15d5976dc",
      "metadata": {},
      "source": [
        "### Tell the notebook where files are stored"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "84e2e8b1-5b94-4dcf-b109-9e8351b087b4",
      "metadata": {},
      "outputs": [],
      "source": [
        "DATA_PATH = Path(\"data/pydata/schedule.json\")\n",
        "INDEX_PATH = Path(\"indices/pydata_schedule_index/\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "6b4f10b1-759c-459b-bece-1495c099cf62",
      "metadata": {},
      "source": [
        "### Tell `llama-index` to use a local embeddings model for retrieval"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "3420c723-b6f0-4c1f-8a5f-9da6b875747b",
      "metadata": {},
      "source": [
        "More information about this embeddings model: https://www.sbert.net/docs/pretrained_models.html#model-overview\n",
        "\n",
        "Note that all-minilm-l6-v2 has a maximum size of 256 tokens."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "0c4b220f-e86c-48e6-a069-ec4d84a2b119",
      "metadata": {},
      "outputs": [],
      "source": [
        "embed_model = \"local:sentence-transformers/all-minilm-l6-v2\"\n",
        "llm = None"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "85531cae-9ec3-47f3-b4c1-e9df4cea39f4",
      "metadata": {},
      "outputs": [],
      "source": [
        "service_context = llama_index.ServiceContext.from_defaults(\n",
        "  embed_model=embed_model, chunk_size=256, llm=llm\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "7b317df6-0c8c-4ec8-beb9-ce932bc34a50",
      "metadata": {},
      "source": [
        "### Load a vector index with the PyData Amsterdam 2023 schedule"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "81545f8f-f540-4b5a-bc39-eac70da0771a",
      "metadata": {},
      "source": [
        "Load the index from file."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "3b8bc3e5-6bc7-4970-9a1f-c722ba931956",
      "metadata": {},
      "outputs": [],
      "source": [
        "if not os.path.exists(INDEX_PATH):\n",
        "    logger.error(\"Index file for part 1 does not exist on disk. :(\")\n",
        "else:\n",
        "    try:                                                                             \n",
        "        storage_context = llama_index.StorageContext.from_defaults(persist_dir=INDEX_PATH)\n",
        "        index = llama_index.load_index_from_storage(storage_context, service_context=service_context)\n",
        "        logger.info(\"Loaded index from local storage\")                               \n",
        "    except Exception as e:                                                           \n",
        "        logger.error(e) "
      ]
    },
    {
      "cell_type": "markdown",
      "id": "cede9a40-f6e1-42f3-92bf-4519bfad87ef",
      "metadata": {},
      "source": [
        "## Exercises"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "4b4d0cad-f3bb-4467-b3ed-ec87e5fa966a-q",
      "metadata": {},
      "source": [
        "### Create a retriever from the vector index `index`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "4b4d0cad-f3bb-4467-b3ed-ec87e5fa966a-a1",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "4b4d0cad-f3bb-4467-b3ed-ec87e5fa966a-a2",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "id": "f343d465-8b22-4778-91e7-b33f3c63e826-q",
      "metadata": {},
      "source": [
        "### Retrieve chunks that mention llama-index"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "f343d465-8b22-4778-91e7-b33f3c63e826-a1",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "f343d465-8b22-4778-91e7-b33f3c63e826-a2",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "id": "01462a67-ebcc-4282-a7c0-5fadb0cc75a9-q",
      "metadata": {},
      "source": [
        "### Retrieve chunks related to causal machine learning"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "01462a67-ebcc-4282-a7c0-5fadb0cc75a9-a1",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "01462a67-ebcc-4282-a7c0-5fadb0cc75a9-a2",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "id": "fc675732-2371-4a41-8a11-4c00577bd4e8-q",
      "metadata": {},
      "source": [
        "### Print the talk title and speaker for results related to causal machine learning"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "fc675732-2371-4a41-8a11-4c00577bd4e8-a1",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "fc675732-2371-4a41-8a11-4c00577bd4e8-a2",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "id": "11b98123-b560-4bc9-b611-3f2db8705262-q",
      "metadata": {},
      "source": [
        "### Create a new retriever that retrieves more than 2 results. Hint: [llama-index Retriever documentation](https://gpt-index.readthedocs.io/en/v0.8.5.post2/api_reference/query/retrievers/vector_store.html)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "11b98123-b560-4bc9-b611-3f2db8705262-a1",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "11b98123-b560-4bc9-b611-3f2db8705262-a2",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "id": "84f418a4-19d4-4ec0-b102-714de283c7fc-q",
      "metadata": {},
      "source": [
        "### Find all talks about causal inference at PyData"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "84f418a4-19d4-4ec0-b102-714de283c7fc-a1",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "84f418a4-19d4-4ec0-b102-714de283c7fc-a2",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "id": "f81ee400-d0c5-4ad2-b68e-09c84ce4044c",
      "metadata": {},
      "source": [
        "## Querying the vector index with an external LLM"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "601ced37-8de0-4d5e-8abb-6fae18bf1803",
      "metadata": {},
      "source": [
        "### Set OpenAI API key (optional)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "62f5b922-be03-435f-b581-053dfb47e5d7",
      "metadata": {},
      "outputs": [],
      "source": [
        "from secret import openai_api_key"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "883fc70c-f18f-412c-bae5-3ac06a090a92",
      "metadata": {},
      "source": [
        "### Use OpenAI's gpt-3.5-turbo for querying"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "d4d6ce8a-85d1-4d29-a182-a749782d14c3",
      "metadata": {},
      "outputs": [],
      "source": [
        "llm = llama_index.llms.OpenAI(model=\"gpt-3.5-turbo\", api_key=openai_api_key)\n",
        "service_context = llama_index.ServiceContext.from_defaults(\n",
        "  embed_model=embed_model, chunk_size=256, llm=llm\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "54c91e61-7b94-4a1c-a68b-b22553e8b78d",
      "metadata": {},
      "outputs": [],
      "source": [
        "if not os.path.exists(INDEX_PATH):\n",
        "    logger.error(\"Index file for part 1 does not exist on disk. :(\")\n",
        "else:\n",
        "    try:                                                                             \n",
        "        storage_context = llama_index.StorageContext.from_defaults(persist_dir=INDEX_PATH)\n",
        "        index = llama_index.load_index_from_storage(storage_context, service_context=service_context)\n",
        "        logger.info(\"Loaded index from local storage\")                               \n",
        "    except Exception as e:                                                           \n",
        "        logger.error(e) "
      ]
    },
    {
      "cell_type": "markdown",
      "id": "de9d00b1-c133-4af2-b9f7-0e82113797f3-q",
      "metadata": {},
      "source": [
        "### Create a query engine"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "de9d00b1-c133-4af2-b9f7-0e82113797f3-a1",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "de9d00b1-c133-4af2-b9f7-0e82113797f3-a2",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "id": "64136237-b60d-48b5-89d7-e749b71e061d-q",
      "metadata": {},
      "source": [
        "### Find talks that might be interesting for startup founders"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "64136237-b60d-48b5-89d7-e749b71e061d-a1",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "64136237-b60d-48b5-89d7-e749b71e061d-a2",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "id": "c913d2a2-f87f-4505-abb4-49e6b2291e52",
      "metadata": {},
      "source": [
        "# Part 2: Building a custom vector index"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "47bccc4d-52f3-4610-afae-1b08278dad37",
      "metadata": {},
      "source": [
        "How to use this notebook: \n",
        "1. Execute all cells under \"Setup\"\n",
        "2. Fill in the exercises below"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "e03ae8f4-83b1-49a8-bf1e-19180366a102",
      "metadata": {},
      "source": [
        "## Setup"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "038506f9-dcca-400c-b10b-5e7e612b1792",
      "metadata": {},
      "source": [
        "### Imports"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "effd39ab-b6ba-4d69-8e8f-39f382d7712f",
      "metadata": {},
      "outputs": [],
      "source": [
        "import pprint\n",
        "import json\n",
        "import os\n",
        "import sys\n",
        "from pathlib import Path\n",
        "from loguru import logger\n",
        "import llama_index"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "a4bd8c85-03d8-4894-8dfe-b419d4f6b6f2",
      "metadata": {},
      "source": [
        "### Logging"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "5f44daa1-a3e6-45bf-bc6e-8c84fad646b2",
      "metadata": {},
      "outputs": [],
      "source": [
        "logger.remove()\n",
        "logger.add(sys.stdout, format=\"{time} - {level} - {message}\", level=\"DEBUG\")\n",
        "logger.add(\"tutorial_part_1.log\", level=\"DEBUG\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "84e2e8b1-5b94-4dcf-b109-9e8351b087b4",
      "metadata": {},
      "outputs": [],
      "source": [
        "DATA_PATH = Path(\"data/julia_evans/blogposts.json\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "da94c113-deb9-49fc-a072-3f82f82afadd",
      "metadata": {},
      "source": [
        "### Load Julia Evans blogpost data from file"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "a5876b6c-0373-43d4-8f28-26b40dbf5f2a",
      "metadata": {},
      "outputs": [],
      "source": [
        "with open(DATA_PATH, 'r') as infile:\n",
        "    blogposts = json.loads(infile.read())"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "c02640e6-afcf-4eb6-a5e7-e89cf7914cd3",
      "metadata": {},
      "outputs": [],
      "source": [
        "logger.info(f\"Loaded {len(blogposts)} blogposts from file.\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "db2b0d29-ce87-454f-b7b8-e6ddd3a52c87",
      "metadata": {},
      "outputs": [],
      "source": [
        "for idx, post in enumerate(blogposts):\n",
        "    print(f\"{idx+1}: {post['title']}\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "867fe918-339e-4de3-a614-f73a44f4b949",
      "metadata": {},
      "outputs": [],
      "source": [
        "print(\"Example blogpost:\")\n",
        "pprint.pprint(blogposts[0])"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "6b4f10b1-759c-459b-bece-1495c099cf62",
      "metadata": {},
      "source": [
        "### Create a service context\n",
        "- No OpenAI API calls\n",
        "- No large local LLM\n",
        "- Just use the smallest sentence-transformers embeddings model"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "c0bfd191-3038-4ad7-a073-58b93e94f8d1",
      "metadata": {},
      "source": [
        "- all-minilm-l6-v2 has a maximum size of 256 tokens\n",
        "- source: https://www.sbert.net/docs/pretrained_models.html#model-overview"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "85531cae-9ec3-47f3-b4c1-e9df4cea39f4",
      "metadata": {},
      "outputs": [],
      "source": [
        "service_context = llama_index.ServiceContext.from_defaults(\n",
        "  embed_model=\"local:sentence-transformers/all-minilm-l6-v2\", chunk_size=256, llm=None\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "0fb5492d-e290-461a-a3f4-55f23818fee1",
      "metadata": {},
      "source": [
        "### Create documents from the blogposts"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "a03e2bf3-5d46-40d1-9ba5-c6d8dda9d0c0",
      "metadata": {},
      "outputs": [],
      "source": [
        "documents = [llama_index.Document(text=blogpost['text']) for blogpost in blogposts]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "e829de61-44ef-4701-b01d-e24bb34b6d78",
      "metadata": {},
      "outputs": [],
      "source": [
        "len(documents)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "91772358-47dc-4e6a-861a-aa9e295805ca",
      "metadata": {},
      "outputs": [],
      "source": [
        "print(\"Example document:\")\n",
        "documents[0]"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "9e8bd061-8ff4-4dd2-922c-117365fd8bdf",
      "metadata": {},
      "source": [
        "# Exercises"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "6282009a-3ccd-43fb-92dd-d04383bf4740-q",
      "metadata": {},
      "source": [
        "### Create a vector index from `documents` (this could take a minute because we're processing lots of text)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "6282009a-3ccd-43fb-92dd-d04383bf4740-a1",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "6282009a-3ccd-43fb-92dd-d04383bf4740-a2",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "id": "50650136-6ea7-41ed-9b9b-816da3bded7c-q",
      "metadata": {},
      "source": [
        "### Retrieve blogposts about DNS"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "50650136-6ea7-41ed-9b9b-816da3bded7c-a1",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "50650136-6ea7-41ed-9b9b-816da3bded7c-a2",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "id": "e3780a5b-4b23-4c1a-a733-4c0c58998ab7-q",
      "metadata": {},
      "source": [
        "### It's really hard to figure out from which blogposts these results are! Create a set of Documents that has the blogpost title as the id."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "e3780a5b-4b23-4c1a-a733-4c0c58998ab7-a1",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "e3780a5b-4b23-4c1a-a733-4c0c58998ab7-a2",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "id": "47688e4b-56bb-40a5-83dd-6e98002ad2ac-q",
      "metadata": {},
      "source": [
        "### Build a vector index from these new documents"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "47688e4b-56bb-40a5-83dd-6e98002ad2ac-a1",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "47688e4b-56bb-40a5-83dd-6e98002ad2ac-a2",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "id": "e4f577c5-98af-4a3d-8fc4-324a2ab0bc1a-q",
      "metadata": {},
      "source": [
        "### Retrieve blogposts about DNS using the new index. This time, use the id of the source node to check from which blogpost they originate."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "e4f577c5-98af-4a3d-8fc4-324a2ab0bc1a-a1",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "e4f577c5-98af-4a3d-8fc4-324a2ab0bc1a-a2",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "id": "f4b91d79-a54f-4f1a-aaf3-8ce1c01dbc8d-q",
      "metadata": {},
      "source": [
        "### Store your index to disk"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "f4b91d79-a54f-4f1a-aaf3-8ce1c01dbc8d-a1",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "f4b91d79-a54f-4f1a-aaf3-8ce1c01dbc8d-a2",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "id": "dbd7f948-cb5d-49be-a505-86e27e915ea9-q",
      "metadata": {},
      "source": [
        "### Load your index from disk in a new index variable. Hint: you need two ingredients: a storage context and a service context."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "dbd7f948-cb5d-49be-a505-86e27e915ea9-a1",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "dbd7f948-cb5d-49be-a505-86e27e915ea9-a2",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "id": "dd76ae61-63bb-4ae3-af27-765209ddc4a1-q",
      "metadata": {},
      "source": [
        "### Expand the documents in your index with metadata such as title, original URL, author, and update time."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "dd76ae61-63bb-4ae3-af27-765209ddc4a1-a1",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "dd76ae61-63bb-4ae3-af27-765209ddc4a1-a2",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "id": "22637948-dd35-401d-b30e-c7a13825cc07-q",
      "metadata": {},
      "source": [
        "### Create an index from these new documents. Retrieve 20 search results about Nix (a package manager)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "22637948-dd35-401d-b30e-c7a13825cc07-a1",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "22637948-dd35-401d-b30e-c7a13825cc07-a2",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "id": "bc456b4b-1818-4bfe-b68e-78adb9796ca4-q",
      "metadata": {},
      "source": [
        "### For all 20 chunks, print their score, url and the date the blogpost was published."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "bc456b4b-1818-4bfe-b68e-78adb9796ca4-a1",
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "bc456b4b-1818-4bfe-b68e-78adb9796ca4-a2",
      "metadata": {},
      "outputs": [],
      "source": []
    }
  ],
  "nbformat": 4,
  "nbformat_minor": 5
}