{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "3b6524c2-5754-4a45-a303-505426e03d50",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from dotenv import load_dotenv, find_dotenv\n",
    "_ = load_dotenv(find_dotenv())\n",
    "openai_api_key = os.environ[\"OPENAI_API_KEY\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8111186a-4244-4c68-9389-24ffd8c523e9",
   "metadata": {},
   "source": [
    "## Previous name: GPT Index"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "35e5f3ba-5209-4706-8040-511a9b959b4f",
   "metadata": {},
   "source": [
    "## Home Page: Pitch\n",
    "* Unleash the power of LLMs over your data\n",
    "    * Data Ingestion\n",
    "        * Unstructured data: PDF, Text, Video, Images, etc.\n",
    "        * Strucured data: Excel, SQL, etc.\n",
    "        * Semi-strucured data: API's Slack, Salesforce, Notion, etc. \n",
    "    * Data Indexing\n",
    "        * Store (save)\n",
    "        * Index (find)\n",
    "        * Integrate with vector stores and databases \n",
    "    * Query Interface\n",
    "        * Accepts any input prompt over your data\n",
    "        * Returns a knowledge-augmented response"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b2c4da6-602c-4cec-8e19-3d0c4d2aa8ac",
   "metadata": {},
   "source": [
    "## Home Page: Use Cases\n",
    "* Document QA\n",
    "* Data Augmented Chatbots\n",
    "* Knowledge Agents\n",
    "* Structured Analytics"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9bbecb3a-dd36-49de-ac3a-f554073267b8",
   "metadata": {},
   "source": [
    "## Home Page: Products\n",
    "* LlamaIndex (Python)\n",
    "* LlamaIndex.TS (Typescript version)\n",
    "* LlamaHub\n",
    "    * Llama Packs\n",
    "    * Data Loaders\n",
    "    * Agent tools \n",
    "* SEC Insights: end to end app\n",
    "* create-llama: CLI tool to install llamaindex from terminal"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b808ecc-23e7-4e21-9343-2235ce983e93",
   "metadata": {},
   "source": [
    "## Last features\n",
    "* [RAGs](https://github.com/run-llama/rags):\n",
    "    * Build, customize, and use multiple ChatGPTs over your data, all with natural language.\n",
    "    * RAGs is a Streamlit app that lets you create a RAG pipeline from a data source using natural language.\n",
    "* [LLama Packs](https://llamahub.ai/). Interesting llama packs:\n",
    "    * Resume screener\n",
    "    * Gmail OpenAI agent\n",
    "    * Deeplake multimodal retrieval\n",
    "    * Sub_question Webiate"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad9cfbc1-e3b0-4700-9fc2-354431e2ae6c",
   "metadata": {},
   "source": [
    "## Documentation: structure\n",
    "* Getting started\n",
    "* Use cases\n",
    "* Understanding LLamaIndex\n",
    "    * Tutorial series \n",
    "* Optimizing\n",
    "    * When you already have LlamaIndex app working and are looking to further refine it.\n",
    "    * List of first things you should try: embedding model, chunk size, customizations, etc.\n",
    "    * Fine tuning your model.\n",
    "* Module guides\n",
    "    * Guides to the individual components of LlamaIndex"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3fbca7f5-4787-4e01-8604-17d25aae919c",
   "metadata": {},
   "source": [
    "## Documentation: Starter Tutorial"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "4f79d62f-b4da-4f8c-9f32-8ff7bdd5f6a9",
   "metadata": {},
   "outputs": [],
   "source": [
    "#pip install llama-index"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "298937ea-5594-4834-8df9-9bb41e37ceec",
   "metadata": {},
   "source": [
    "#### Load Private Document"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "aef6580d-698c-4fdc-bb3d-f3db18f3c85b",
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index import SimpleDirectoryReader\n",
    "\n",
    "documents = SimpleDirectoryReader(\"data\").load_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b1b7147-5e15-4093-86b7-355fcc4a56b6",
   "metadata": {},
   "source": [
    "#### Create Vector Database (LlamaIndex call them \"indexes\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "f5108a82-0d47-4ea1-b146-0596bee6d561",
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index import VectorStoreIndex\n",
    "\n",
    "index = VectorStoreIndex.from_documents(documents)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e0303edf-ddd3-4219-b5b8-2e93cdbd90be",
   "metadata": {},
   "source": [
    "#### QA over private document"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "151655aa-87a8-4eb4-92c6-9077ad7b41e5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The article discusses the idea that startups can benefit from behaving like nonprofits. It suggests that focusing on making something people want and helping others can improve morale, attract support, and lead to success. Examples of successful startups that have followed this approach, such as Craigslist and Google, are mentioned. The article also explores the potential of combining benevolence with entrepreneurship, and how it can create a sense of mission and resilience. Overall, the article highlights the advantages of being good and how it can contribute to the success of startups.\n"
     ]
    }
   ],
   "source": [
    "query_engine = index.as_query_engine()\n",
    "\n",
    "response = query_engine.query(\"Summarize the article in less than 100 words.\")\n",
    "\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "684447bd-69be-45a2-b159-12626f795092",
   "metadata": {},
   "source": [
    "#### Save the vector database in your computer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "7c4c3251-2b19-4a1c-b7ba-38118528ef80",
   "metadata": {},
   "outputs": [],
   "source": [
    "index.storage_context.persist()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f412dd7-72a1-4cc0-b216-80cab6e027d7",
   "metadata": {},
   "source": [
    "By default, this will save the data to the directory storage, but you can change that by passing a `persist_dir` parameter."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a92c3c57-4401-4efb-b90b-6297ded5cd57",
   "metadata": {},
   "source": [
    "## Documentation: High-Level Concepts"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0610bbbe-b468-4a58-a61b-13c1ca733137",
   "metadata": {},
   "source": [
    "#### RAG\n",
    "* Your data is loaded\n",
    "* Your data is indexed: prepared for queries\n",
    "* When you ask a question, LlamaIndex gets the most relevant data from the vector database and passes your question and this most relevant data (called \"the context\") to the LLM so the LLM can redact a conversational answer."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1d72c4d-89fc-40bd-a0a8-61974c742250",
   "metadata": {},
   "source": [
    "#### Stages within RAG\n",
    "1. Loading\n",
    "2. Indexing: convert data into embeddings and metadata\n",
    "3. Storing: store your embeddings and metadata\n",
    "4. Querying\n",
    "    * sub-queries\n",
    "    * multi-step queries\n",
    "    * hybrid strategies\n",
    "5. Evaluation: checking how your accurate, faithful and fast responses to queries are"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "913e0a81-5652-462c-8fb4-687f939d445f",
   "metadata": {},
   "source": [
    "#### Important concepts within some of the previous stages\n",
    "1. Loading\n",
    "    * Document: data format (PDF, API, etc).\n",
    "    * Node: data chunk with metadata.\n",
    "    * Connector or Reader: connects with data sources.\n",
    "2. Indexing\n",
    "    * Indexing: transformation and storage of data into embeddings with metadata in vector databases.\n",
    "    * Embeddings: numerical representation of data.\n",
    "4. Querying\n",
    "    * Retrievers: how to retrieve relevant context from an index when given a query. The retrieval strategy is key to the performance of the app.\n",
    "    * Routers: determines which retriever will be used based on the reriever's metadata and the query.\n",
    "    * Node postprocessors: applies transformations, filtering and re-ranking logic to nodes.\n",
    "    * Response synthesizers: given a query and a set of retrieved text chunks, it generates the conversational response from an LLM."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "37c9cee4-0280-4100-b71a-de373b962896",
   "metadata": {},
   "source": [
    "#### Naming of the 3 main use cases\n",
    "* Query Engines: ask questions about your data.\n",
    "* Chat Engines: have a conversation with your data.\n",
    "* Agents: automated decision maker."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4a90d6fd-14ab-43aa-ad0d-be89ab7d0e35",
   "metadata": {},
   "source": [
    "## Documentation: Customization Tutorial"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "973baba1-3ab6-4363-b382-e36e4398119b",
   "metadata": {},
   "source": [
    "#### Starting point: basic RAG"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "07c98835-84ba-4f93-a135-6c0d09b5ee0e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The author does not consider themselves to be a particularly good person. They grew up suspicious of the concept of being good, associating it with being quiet. The author believes that being good is not about being sanctimonious, but rather about it being a practical and effective approach. They suggest that being good can improve morale, make others want to help, and help startups be decisive. The author also mentions that most explicitly benevolent projects lack accountability and that having good intentions alone is not enough to guarantee good effects.\n"
     ]
    }
   ],
   "source": [
    "from llama_index import VectorStoreIndex, SimpleDirectoryReader\n",
    "\n",
    "documents = SimpleDirectoryReader(\"data\").load_data()\n",
    "index = VectorStoreIndex.from_documents(documents)\n",
    "query_engine = index.as_query_engine()\n",
    "response = query_engine.query(\"In less than 100 words, what is the meaning of good for the author?\")\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "599b5eab-4ae7-4cae-b1f9-cf3ff2e470df",
   "metadata": {},
   "source": [
    "#### Parse the document into smaller chunks"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "dcff1d04-54c4-41b8-858d-21d1db0b35cc",
   "metadata": {},
   "outputs": [],
   "source": [
    "from llama_index import ServiceContext\n",
    "\n",
    "service_context = ServiceContext.from_defaults(chunk_size=1000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "befbda15-54aa-40ba-9329-1c6f0a6f683f",
   "metadata": {},
   "outputs": [],
   "source": [
    "index = VectorStoreIndex.from_documents(\n",
    "    documents, \n",
    "    service_context=service_context\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7544dbbb-365a-44ba-8ed3-1866331d7ef8",
   "metadata": {},
   "source": [
    "#### Use a different vector database"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "b005c4e7-2ab3-4ee6-a755-d241a7906038",
   "metadata": {},
   "outputs": [],
   "source": [
    "import chromadb\n",
    "from llama_index.vector_stores import ChromaVectorStore\n",
    "from llama_index import StorageContext\n",
    "\n",
    "chroma_client = chromadb.PersistentClient()\n",
    "chroma_collection = chroma_client.create_collection(\"quickstart\")\n",
    "vector_store = ChromaVectorStore(chroma_collection=chroma_collection)\n",
    "storage_context = StorageContext.from_defaults(\n",
    "    vector_store=vector_store\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "cab44594-1f08-4624-b98d-c888169bda1f",
   "metadata": {},
   "outputs": [],
   "source": [
    "index = VectorStoreIndex.from_documents(\n",
    "    documents, \n",
    "    storage_context=storage_context\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c674a5d-45c2-4861-8523-d7c5c1604e61",
   "metadata": {},
   "source": [
    "#### Retrieve more context when I query"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "731889a5-9064-4783-8b36-0b8b629ab400",
   "metadata": {},
   "outputs": [],
   "source": [
    "query_engine = index.as_query_engine(similarity_top_k=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd73b058-5f19-4b09-94ec-f2d12f3677d0",
   "metadata": {},
   "source": [
    "#### Use a different LLM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "2e624338-55c2-4a68-bbc9-250b13340490",
   "metadata": {},
   "outputs": [],
   "source": [
    "# from llama_index.llms import PaLM\n",
    "\n",
    "# service_context = ServiceContext.from_defaults(llm=PaLM())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4595d62a-8344-47d2-a8d8-497527e36550",
   "metadata": {},
   "source": [
    "#### Use a different response mode"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "9b06acdd-9226-4bf7-a6c8-ccfc5610b7ae",
   "metadata": {},
   "outputs": [],
   "source": [
    "query_engine = index.as_query_engine(response_mode=\"tree_summarize\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e364965-27ae-42bc-aae9-b36413b06d19",
   "metadata": {},
   "source": [
    "#### Stream the response back"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "152a965d-b30f-4659-b007-eb5837a035ef",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The author suggests that being \"good\" is not about claiming to be a morally superior person, but rather about making decisions and taking actions that are beneficial and helpful to others. It is not about being sanctimonious or self-righteous, but rather about being effective and successful in achieving positive outcomes. The author emphasizes that being good can serve as a compass for decision-making and can attract talented individuals who are motivated to make a positive impact on the world."
     ]
    }
   ],
   "source": [
    "query_engine = index.as_query_engine(streaming=True)\n",
    "response = query_engine.query(\"In less than 100 words, what is the meaning of good according to the author?\")\n",
    "response.print_response_stream()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0f76bf40-439f-4907-9a57-7521b45c6fad",
   "metadata": {},
   "source": [
    "#### Use a chatbot instead of a QA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "7988e811-1591-4032-a2cc-de6b0c5bba46",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The meaning of \"bad\" according to the author is subjective and can vary depending on the context. It could refer to something that is morally wrong, harmful, or undesirable. The author's interpretation of \"bad\" may also be influenced by personal beliefs, values, and cultural norms. Ultimately, the meaning of \"bad\" is open to interpretation and can differ from person to person.\n",
      "According to the author, their understanding of \"bad\" was influenced by their experiences as a child. They observed that adults often used the word \"good\" in relation to being quiet, which made them suspicious of it. This suggests that the author associated being \"bad\" with not conforming to societal expectations of being quiet or obedient. It implies that the author's interpretation of \"bad\" is connected to a sense of independence or non-conformity. However, it's important to note that the meaning of \"bad\" can still be subjective and may vary depending on individual perspectives and experiences.\n"
     ]
    }
   ],
   "source": [
    "query_engine = index.as_chat_engine()\n",
    "response = query_engine.chat(\"In less than 100 words, what is the meaning of bad according to the author?\")\n",
    "print(response)\n",
    "\n",
    "response = query_engine.chat(\"Oh interesting, tell me more.\")\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2bc467a4-0acd-42a6-92c6-fa7b242064db",
   "metadata": {},
   "source": [
    "## Documentation: The LlamaIndex Video Series\n",
    "* Build a document chatbot from scratch\n",
    "* Sub-questions\n",
    "* Manage documents from a source that is constantly updating like Discord\n",
    "* Combining SQL and Semantic Search"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6592ef2c-c39c-4963-9bfd-2a1be9be2c04",
   "metadata": {},
   "source": [
    "## Documentation: Use Cases\n",
    "* QA\n",
    "* Chatbots\n",
    "* Agents\n",
    "* Structured Data Extraction\n",
    "* Multi-modal"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2de60ff2-e07b-4f49-8f72-033171d6d511",
   "metadata": {},
   "source": [
    "## Documentation: Understanding (LI vs LC)\n",
    "* Using LLMs\n",
    "    * Different way of loading OpenAIEmbeddings than LC\n",
    "    * Similar approach to Prompt templates \n",
    "* Loading\n",
    "    * Very interesting: multi-purpose loader\n",
    "    * Splitter, chunk_size, chunk_overlap\n",
    "    * Creating chunks (nodes) manually\n",
    "    * Adding metadata to document (copied to nodes)\n",
    "    * Loading connectors from LLamaHub\n",
    "* Indexing\n",
    "    * Index types:\n",
    "        * Vector store index\n",
    "            * Nodes and embeddings\n",
    "            * Semantic search\n",
    "            * Top K Retrieval\n",
    "        * Summary index\n",
    "            * If you want to summarize the document \n",
    "        * Knowledge graph index\n",
    "            * If your data is a set of disconnected concepts (a \"graph\") \n",
    "* Storing\n",
    "    * by default, indexed data is stored only in memory\n",
    "    * creating embeddings is expensive\n",
    "    * store to avoid the time and cost of re-indexing\n",
    "    * save: .persist()\n",
    "    * load persisted index: load_index_from_storage()\n",
    "* Querying\n",
    "    * the most significant part of an LLM App\n",
    "    * stages: retrieval, postprocessing, response synthesis.\n",
    "    * customizing the stages of querying.\n",
    "* Putting it all together\n",
    "    * advanced techniques\n",
    "    * how to build a full-stack app\n",
    "        * React + Flask API\n",
    "* Observability: tracing and debugging.\n",
    "    * Logging\n",
    "    * Callbacks to help debug\n",
    "    * One-click observability with eval tools offered by partners (W&B, etc)\n",
    "* Evaluation.\n",
    "    * Response evaluation\n",
    "    * Retrieval evaluation\n",
    "    * Analizing the cost of your app\n",
    "        * MockLLM to predict token usage\n",
    "        * MockEmbedding"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "88432aad-0140-4075-897f-184bd7d86730",
   "metadata": {},
   "source": [
    "## Documentation: Optimizing\n",
    "* Advanced prompt techniques\n",
    "* Prompt engineering for RAG\n",
    "* Advanced retrieval strategies\n",
    "* Agentic strategies\n",
    "    * OpenAI Agent\n",
    "* Evaluation\n",
    "* Fine-tuning\n",
    "* Building performant RAG apps for production\n",
    "    * General techniques\n",
    "        * decoupling retrieval chunks vs syntesis chunks\n",
    "        * structured retrieval for large document sets\n",
    "        * dynamically retrieve chunks\n",
    "        * optimize context embeddings\n",
    "    * Long list of specific techniques\n",
    "* Building RAG from scratch (lower-level)  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "323f0185-ab10-4752-b9e0-aea066d1e354",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}