[DOC] Organizing the tutorial, splitting it across low-level and high-level (#39)

Author: Yoric

.github/workflows/test.yml

@@ -43,7 +43,7 @@ jobs:
         if-no-files-found: ignore
 
   test_notebook:
-    name: Run the Jupyter notebook tutorial (ubuntu)
+    name: Run the Jupyter notebook tutorials (ubuntu)
     runs-on: ubuntu-latest
     strategy:
       matrix:
@@ -64,8 +64,9 @@ jobs:
       - name: Run notebook
         run: |
           hatch run pip install jupyter
-          hatch run jupyter execute examples/tutorial.ipynb
-          hatch run jupyter execute examples/ml_tutorial.ipynb
+          for tutorial in examples/tutorial*.ipynb; do
+            hatch run jupyter execute "$tutorial";
+          done
 
   publish:
     name: Publish to PyPI

README.md

@@ -44,8 +44,10 @@ $ pipx install quantum-evolution-kernel
 
 ## Usage
 
-For the time being, the easiest way to learn how to use this package is to look
-at the [example notebook](examples/tutorial.ipynb).
+We have a two parts tutorial:
+
+1. [Using a Quantum Device to extract machine-learning features](examples/tutorial%201%20-%20Using%20a%20Quantum%20Device%20to%20Extract%20Machine-Learning%20Features.ipynb);
+2. [Machine Learning with the Quantum Evolution Kernel](examples/tutorial%202%20-%20Machine-Learning%20with%20the%20Quantum%20EvolutionKernel.ipynb)
 
 See also the [full API documentation](https://pqs.pages.pasqal.com/quantum-evolution-kernel/).
 

examples/tutorial.ipynb … Extract Machine-Learning Features.ipynbexamples/tutorial.ipynb renamed to examples/tutorial 1 - Using a Quantum Device to Extract Machine-Learning Features.ipynb examples/tutorial.ipynb renamed to examples/tutorial 1 - Using a Quantum Device to Extract Machine-Learning Features.ipynb

@@ -4,27 +4,19 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# QEK from A to Z\n",
+    "# Tutorial 1: using a Quantum Device to extract machine-learning features\n",
     "\n",
-    "This notebook reproduces the results of the [QEK paper](https://journals.aps.org/pra/abstract/10.1103/PhysRevA.107.042615).\n",
+    "This notebook reproduces the first part of the [QEK paper](https://journals.aps.org/pra/abstract/10.1103/PhysRevA.107.042615) using the library's high-level API, dedicated to features extraction using a Quantum Device.\n",
     "\n",
-    "At the end, you will be able to:\n",
-    "1. Extract the embeddings of a molecular dataset\n",
-    "2. Compile these embeddings into Pulse sequences for use on a Quantum Device or an amulator.\n",
-    "3. Run the Pulse sequences on a Quantum Device or an emulator.\n",
+    "By the end of this notebook, you will know how to:\n",
     "\n",
+    "1. Setup import for a molecular dataset (the library supports other type of graphs, of course).\n",
+    "2. Setup compilation and execution of these graphs for execution on a Quantum Device (either an emulator or a physical QPU).\n",
+    "3. Launch the execution and extract the relevant machine-learning features.\n",
     "\n",
-    "We also provide a another [machine learning tutorial](ml_tutorial.ipynb) to train an SVM with the QEK (Quantum Evolution Kernel) kernel and benchmark the performance reported in the paper."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
+    "A [companion notebook](./tutorial%202%20-%20Machine-Learning%20with%20the%20Quantum%20EvolutionKernel.ipynb) will guide you through machine-learning with QEK.\n",
     "\n",
-    "import torch_geometric.datasets as pyg_dataset"
+    "If, instead of using the library's high-level API, you prefer digging a bit closer to the qubits, you may prefer the companion [low-level notebook](./tutorial%201a%20-%20Using%20a%20Quantum%20Device%20to%20Extract%20Machine-Learning%20Features%20-%20low-level.ipynb) that mirrors this notebook, but using a lower-level API that will let you experiment with different quantum pulses."
    ]
   },
   {
@@ -42,8 +34,10 @@
    "metadata": {},
    "outputs": [],
    "source": [
+    "import torch_geometric.datasets as pyg_dataset\n",
+    "\n",
     "# Load the original PTC-FM dataset\n",
-    "og_ptcfm = [data for data in pyg_dataset.TUDataset(root=\"dataset\", name=\"PTC_FM\")]\n",
+    "og_ptcfm = pyg_dataset.TUDataset(root=\"dataset\", name=\"PTC_FM\")\n",
     "\n",
     "display(\"Loaded %s samples\" % (len(og_ptcfm), ))"
    ]
@@ -54,7 +48,7 @@
    "source": [
     "To extract machine-learning features from our dataset, we will need to configure a feature extractor. This library provides several feature extractors to either make use of a physical quantum device (QPU), or a variety of emulators.\n",
     "\n",
-    "To configure a feature extractor, we will need to give it a _compiler_, whose task is to take a list of graphs, extract embeddings and compile these embeddings to _sequences of pulses_, the format that can be executed  by either a QPU or an emulator. For this tutorial, our dataset is composed of molecule graphs encoded with the PTC-FM conventions, so we will use the `PTCFMGraphCompiler`:"
+    "To configure a feature extractor, we will need to provide a _compiler_, whose task is to take a list of graphs, extract embeddings and compile these embeddings to _sequences of pulses_, the format that can be executed  by either a QPU or an emulator. For this tutorial, our dataset is composed of molecule graphs following the PTCFM conventions, so we will use the `PTCFMCompiler`:"
    ]
   },
   {
@@ -72,7 +66,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "This library provides other compilers from other formats of graphs, including the `MoleculeGraphCompiler` and general-purpose graph compilers for pytorch_geometric or networkx graphs."
+    "This library also provides other compilers from a variety of graph formats."
    ]
   },
   {
@@ -111,7 +105,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "As you can see, the number of sequences compiled is lower than the number of samples loaded. Some of this is due to limitations within the algorithm (not all graphs can be efficiently laid out for execution on a Quantum Device), while others are due to the limitations of the emulator we target (which at the time of this writing is limited to 50 qubits).\n",
+    "As you can see, the number of sequences compiled is lower than the number of samples loaded. Some of this is due to limitations within the algorithm (not all graphs can be efficiently laid out for execution on a Quantum Device), while others are due to the limitations of the emulator we target (which at the time of this writing is limited to ~50 qubits).\n",
     "\n",
     "We may now run the extraction on the emulator:"
    ]
@@ -125,6 +119,8 @@
     "# Limit the number of qubits for this run, for performance reasons.\n",
     "# You can increase this value to higher number of qubits, but this\n",
     "# notebook will take longer to execute and may run out of memory.\n",
+    "#\n",
+    "# On our test computer, the practical limit is around 10 qubits.\n",
     "max_qubits = 5\n",
     "processed_dataset = extractor.run(max_qubits=max_qubits)\n",
     "display(\"Extracted features from %s samples\"% (len(processed_dataset.states), ))"
@@ -134,7 +130,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "If you wish to extract features from more samples, feel free to increase the value of `max_qubits` above. However, you will soon run into limitations of a quantum emulator, and possibly crash this notebook. At this point, you have other options, such as using `EmuMPSExtractor` instead of `QutipExtractor`, a more recent emulator that features much better performance in most cases, or you can run the extraction on a physical QPU."
+    "If you wish to extract features from more samples, feel free to increase the value of `MAX_QUBITS` above. However, you will soon run into limitations of a quantum emulator, and possibly crash this notebook. At this point, you have other options, such as using `EmuMPSExtractor` instead of `QutipExtractor`, a more recent emulator that features much better performance in most cases, or you can run the extraction on a physical QPU."
    ]
   },
   {
@@ -178,11 +174,11 @@
     "\n",
     "    # Launch the execution.\n",
     "    processed_dataset = extractor.run()\n",
-    "    display(\"Work enqueued with ids %s\" % (processed_dataset.batch_ids, ))\n",
+    "    display(\"Work enqueued with ids %s\" % (extractor.batch_ids, ))\n",
     "\n",
     "    # ...and wait for the results.\n",
     "    await processed_dataset\n",
-    "    display(\"Extracted states from %s samples\"% (len(processed_dataset.states), ))"
+    "    display(\"Extracted features from %s samples\"% (len(processed_dataset.states), ))"
    ]
   },
   {
@@ -219,8 +215,7 @@
    "outputs": [],
    "source": [
     "import qek.data.dataset as qek_dataset\n",
-    "from qek.data.dataset import ProcessedData\n",
-    "processed_dataset: list[ProcessedData] = qek_dataset.load_dataset(file_path=\"ptcfm_processed_dataset.json\")\n",
+    "processed_dataset = qek_dataset.load_dataset(file_path=\"ptcfm_processed_dataset.json\")\n",
     "print(f\"Size of the quantum compatible dataset = {len(processed_dataset)}\")"
    ]
   },
@@ -283,7 +278,38 @@
     "- each of the keys represents one possible state for the register (which represents the graph), with each qubit (which represents a single node) being in state `0` or `1`;\n",
     "- the corresponding value is the number of samples observed with this specific state of the register.\n",
     "\n",
-    "In this example, for instance, we can see that the state observed most frequently is `10000001010`, with 43/1000 samples.\n"
+    "In this example, for instance, we can see that the state observed most frequently is `10000001010`, with 43/1000 samples.\n",
+    "\n",
+    "Note: Since Quantum Devices are inherently non-deterministic, you will probably obtained different samples if you run this on a Quantum Device instead of loading the dataset.\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Machine learning-features\n",
+    "\n",
+    "From the state dictionary, we derive as machine-learning feature the _distribution of excitation_. We'll use this in a second to define our kernel."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "dataset_example.draw_excitation()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# What now?\n",
+    "\n",
+    "What we have seen so far covers the use of a Quantum Device to extract machine-learning features.\n",
+    "\n",
+    "For the next step, we'll see [how to use these features for machine learning](./tutorial%202%20-%20Machine-Learning%20with%20the%20Quantum%20EvolutionKernel.ipynb)."
    ]
   }
  ],
@@ -303,7 +329,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.15"
+   "version": "3.12.3"
   }
  },
  "nbformat": 4,