{ "cells": [ { "cell_type": "markdown", "id": "81c5fc85-e4e5-470c-b2bd-b523aeb0d870", "metadata": {}, "source": [ "## SEC Insights: Deep Dive" ] }, { "cell_type": "markdown", "id": "499993b1-de7c-451e-b6aa-78af6c1039d7", "metadata": {}, "source": [ "## Goals of the Deep Dive\n", "* Deploy to production.\n", "* Understand the role of LLM.\n", "* Understand the placement of the LlamaIndex code.\n", "* Check if there is testing framework in place.\n", "* Additional parts to pay attention to:\n", " * Github codespace\n", " * AWS S3\n", " * Cron job service\n", " * Ready to deploy to Vercel and Render\n", " * Local environment setup making use of\n", " * LocalStack" ] }, { "cell_type": "markdown", "id": "35a1f835-968c-4f9c-983a-0cd45203db55", "metadata": {}, "source": [ "## Tech Stack\n", "* Frontend\n", " * React / Next.js\n", " * Tailwind CSS\n", "* Backend\n", " * FastAPI\n", " * Docker\n", " * SQLAlchemy\n", " * OpenAI\n", " * gpt-3.5-turbo\n", " * text-embedding-ada-002\n", " * PGVector\n", " * LlamaIndex\n", "* Infrastructure\n", " * Render.com\n", " * Backend hosting\n", " * Postgres 15\n", " * Vercel\n", " * Frontend hosting\n", " * AWS\n", " * Cloudfront\n", " * S3 " ] }, { "cell_type": "markdown", "id": "59a33e95-0aba-4cdc-a260-86bd783cc545", "metadata": {}, "source": [ "## LLM Logic: role and placement\n", "* Role: advanced RAG application\n", "* Main placement: `/backend/app/chat/engine.py`\n", "* Secondary placements in same folder:\n", " * `messaging.py`\n", " * `pg_vector.py`\n", " * `a_response_synth.py`\n", "* Python version of LlamaIndex" ] }, { "cell_type": "markdown", "id": "d518c80f-0040-443d-b2a2-7ccd9cfde88e", "metadata": {}, "source": [ "## Testing functionality\n", "* Limited testing functionality in /backend/tests/app/chat/test_engine.py\n", " * TestGetChatHistory" ] }, { "cell_type": "markdown", "id": "f634a184-4b6a-452f-81b2-c018fbe31929", "metadata": {}, "source": [ "## What it is?\n", "* Chat application\n", "* RAG technique\n", "* Answers questions about SEC 10k and 10Q documents\n", "* Production-ready\n", "* Full-stack repo\n", "* Ready for you to fork" ] }, { "cell_type": "markdown", "id": "2c542e4b-79ff-4968-b4e0-2c68719be020", "metadata": {}, "source": [ "## Try it out\n", "[secinsights.ai](secinsights.ai)" ] }, { "cell_type": "markdown", "id": "8274cfea-e621-4186-9d20-d1988e7dc1e1", "metadata": {}, "source": [ "## Product Features\n", "* QA chat grounded in source-of-truth SEC documents\n", "* PDF viewer\n", "* Token-level streaming of chat responses\n", "* Streaming of reasoning steps (sub-questions)\n", "* Citation of source data\n", "* Use of API-based tools (in addition to semantic search)" ] }, { "cell_type": "markdown", "id": "7dcb8dfd-df87-4e01-8524-09c05234b6f0", "metadata": {}, "source": [ "![alt text](images/architecture-v1.png)" ] }, { "cell_type": "markdown", "id": "9e117747-ad8f-492e-b46c-f64579859ba1", "metadata": {}, "source": [ "## Architecture\n", "* Backend\n", " * Render.com: hosting most of the backend.\n", " * Similar to AWS but easier to use.\n", " * FastAPI Backend Service.\n", " * Integrates with Polygon.io to solve some of the quantitative questions about companies (revenue, etc). Good example of how to integrate API-based tools with the chat. \n", " * Postgres 15 database: PG vectorstore.\n", " * Cron job service: more on this on a next section below.\n", " * pulls the SEC documents from SEC's Edgar API.\n", " * stores the documents into the AWS S3's Public PDF Bucket.\n", " * calls OpenAI's embedding API to run the embeddings to the SEC files.\n", " * stores the embeddings in the PG (postgres) vectorstore. We use the PG Vectorstore Integration.\n", " * updates the Private StorageContext Bucket to update the metadata of the embeddings.\n", " * can run at whatever schedule you set in the render.yaml file.\n", " * Auto-scaling: dynamic automatic adjustment of the computational resources used based on the real demand.\n", " * All of the above prepared for us in the file render.yaml (see detailed explanation of the render.yaml file in the next section)" ] }, { "cell_type": "markdown", "id": "279dec2b-716e-4286-8939-eb2e70833774", "metadata": {}, "source": [ "* AWS S3: **you will have to setup this yourself**\n", " * Private StorageContext Bucket: metadata from the llamaindex library\n", " * Public PDF Bucket\n", " * See instructions on how to create an AWS S3 account in the next section." ] }, { "cell_type": "markdown", "id": "4a396a5b-c980-441f-9a22-cea47b722bec", "metadata": {}, "source": [ "* Frontend\n", " * NextJS\n", " * Interacts with the FastAPI chat endpoints. \n", " * Hosted in Vercel\n", " * Interacts with the FastAPI backend for some of the chat endpoints" ] }, { "cell_type": "markdown", "id": "3d51b9bb-616b-4ebd-b5cd-c4e7fdeb6431", "metadata": {}, "source": [ "* External services:\n", " * [Polygon.io](https://polygon.io) (Financial Data API, Numeric Data). Good example on how to integrate tools in your chat agent.\n", " * [Get free API key](https://polygon.io/dashboard/login?redirect=%2F) \n", " * [SEC's Edgar API](https://sec-api.io)\n", " * [Get free API Key](https://sec-api.io/signup/free) \n", " * OpenAI Service (LLM)\n", " * The cron service is the main worker that calls the OpenAI Embedding API to get the embeddings for the given SEC documents.\n", " * The cron service calls the Edgar API from the SEC, get the PDFs, store them in the AWS Public PDF Bucket, run the embeddings on them and store the embeddings in the Postgres database (we use the PG Vectorstore integration).\n", " * The cron job can run at whatever schedule you set in the render.yaml file.\n", " * Sentry.io (Production-level Monitoring Service. It will ping you whenever there is an error in the backend service, or threshold errors, etc. You can also do Performance Monitoring: what sections of the code are taking more time in your service).\n", " * More info on this in a next section below. " ] }, { "cell_type": "markdown", "id": "16c1bcef-d58c-4a6a-8fc2-81e3eaa8ac13", "metadata": {}, "source": [ "All the setup is open source and is easy to deploy on Vercel and Render.com" ] }, { "cell_type": "markdown", "id": "e9b30814-f53f-4315-ab3d-0fb6a3f6695e", "metadata": {}, "source": [ "## Contents of the render.yaml file" ] }, { "cell_type": "markdown", "id": "6809dd58-5c52-4c22-991c-03f72b9fff19", "metadata": {}, "source": [ "This code is a configuration for deploying and managing the app on a cloud hosting platform, Render.com, based on its structure and comments. Let's break it down by sections to better understand what it does:\n", "\n", "#### General Configurations:\n", "\n", "- `previewsEnabled: true`: Enables preview functionality for this project.\n", "\n", "#### Database Configuration:\n", "\n", "- `databases`: Defines a database for the application.\n", " - `name`: The internal name of the database.\n", " - `databaseName`: The actual name of the database.\n", " - `plan`: The service plan for the database (**in this case, `pro`**).\n", " - `previewPlan`: The plan for previews (in this case, `starter`).\n", "\n", "#### Services:\n", "\n", "Two main services are defined here, a web service and a cron service.\n", "\n", "1. **Web Service (`type: web`)**:\n", " - Deployed as a Docker service in a web environment.\n", " - Specifies the service name, runtime configuration, repository, and region.\n", " - Defines an auto-scaling range (between 2 and 10 instances) with specific targets for CPU and memory usage.\n", " - Configures a health check endpoint and a command for initial deployment.\n", " - Specific environment variables for the service, including the database URL and other settings.\n", "\n", "2. **Cron Service (`type: cron`)**:\n", " - A cron job executed in a Docker container.\n", " - Similar in configuration to the web service but designed to run on a schedule (though here it's configured to not run automatically).\n", " - Uses the same command as the web service for initial deployment.\n", "\n", "#### Environment Variable Groups:\n", "\n", "Groups of environment variables are defined for different purposes:\n", "\n", "- `general-settings`: Common variables for all environments, such as CORS configuration, S3 bucket names, CDN URL, etc.\n", "- `prod-web-secrets`: Environment-specific variables for the production environment, like API keys and AWS credentials.\n", "- `preview-web-secrets`: Similar variables for the preview environment.\n", "\n", "#### Summary:\n", "\n", "This code configures two main services (web and cron) on Render.com, with a database and a set of environment variables for different environments (production and preview). It includes details about scaling, service health, and specific deployment configurations, indicating a well-thought infrastructure for applications in production and development." ] }, { "cell_type": "markdown", "id": "9209ea58-6b3a-4880-823c-0cb947dfa372", "metadata": {}, "source": [ "## How to create an AWS S3 account" ] }, { "cell_type": "markdown", "id": "82e5788b-5935-40e0-9860-b36bbdb5fa45", "metadata": {}, "source": [ "Creating an account on Amazon Web Services (AWS) and starting to use the Amazon S3 (Simple Storage Service) involves several steps. Here is a guide through the process:\n", "\n", "1. **Sign Up for Amazon Web Services (AWS)**:\n", " - Visit the AWS website at [https://aws.amazon.com/](https://aws.amazon.com/).\n", " - Click on \"Create an AWS Account\".\n", " - Follow the instructions to create your account, which include providing your email address, creating a password, and choosing a name for your AWS account.\n", " - Then, you'll be asked to provide contact information and billing details (credit card information), as AWS operates on a pay-as-you-go basis.\n", "\n", "2. **Log in to the AWS Management Console**:\n", " - Once your account is active, log in to the AWS Management Console at [https://aws.amazon.com/console/](https://aws.amazon.com/console/).\n", " - Use the email and password you registered with to log in.\n", "\n", "3. **Access the Amazon S3 Service**:\n", " - In the AWS Management Console, search for \"S3\" in the search bar or find it under the \"Storage\" section.\n", " - Click on S3 to open the service management panel.\n", "\n", "4. **Create a Bucket in S3**:\n", " - Within the S3 console, you can start by creating a \"bucket\", which is a basic container where data is stored.\n", " - Click on \"Create bucket\".\n", " - Provide a unique name for the bucket and select the region where you want to store your data.\n", " - Configure additional options as needed, such as access control, versioning, etc.\n", " - Click on \"Create\" to finalize the creation of the bucket.\n", "\n", "5. **Upload Files to Your Bucket**:\n", " - Once the bucket is created, you can upload files to it. To do this, select your bucket and then use the \"Upload\" option to add files from your computer.\n", "\n", "6. **Configure Permissions and Policies**:\n", " - It's important to properly configure permissions and security policies to control who can access your files in S3.\n", "\n", "7. **Additional Uses**:\n", " - Besides storing files, you can use S3 for a variety of purposes, such as hosting static websites, as part of backup solutions, etc.\n", "\n", "Remember that AWS S3 is a pay-for-use service, so you will be charged for storage and data transfer according to AWS's pricing. Also, it's advisable to familiarize yourself with best practices for security and management to protect your data on AWS." ] }, { "cell_type": "markdown", "id": "781d18bb-7e8c-4e5e-acf7-c62b13b7a051", "metadata": {}, "source": [ "## Create a private bucket on AWS S3\n", "* name used for the bucket: private-storagecontext-bucket" ] }, { "cell_type": "markdown", "id": "fe1c7c18-51ef-45db-9403-2d19130a6ea2", "metadata": {}, "source": [ "Creating a private bucket in Amazon S3 (Simple Storage Service) is a process that involves two main parts: \n", "* creating the bucket\n", "* and setting its access policy to ensure it is private.\n", "\n", "Here's how you can do it:\n", "\n", "#### Step 1: Create the Bucket\n", "1. **Log in to AWS Management Console**:\n", " - Go to the [AWS Management Console](https://aws.amazon.com/console/) and log in with your account.\n", "\n", "2. **Open Amazon S3 Service**:\n", " - Once in the console, search for and select the S3 service.\n", "\n", "3. **Create a New Bucket**:\n", " - Click on \"Create bucket\".\n", " - Provide a name for your bucket following S3 naming rules.\n", " - Choose the region where you want to create the bucket.\n", " - Click \"Next\" to continue with the configuration.\n", "\n", "4. **Optional Configuration**:\n", " - In the options section, you can leave the default settings or adjust them as needed (versioning, logging, etc.).\n", " - Click \"Next\".\n", "\n", "5. **Set Permissions**:\n", " - This is where you will ensure the bucket is private.\n", " - Check “Block Public Access” to ensure there is no public access to the bucket.\n", " - Do not add any access control list (ACL) that allows public access.\n", "\n", "6. **Review and Create the Bucket**:\n", " - Review your settings and then click \"Create bucket\".\n", "\n", "#### Step 2 (Optional, recommended): Configure the Bucket Privacy Policy\n", "Once the bucket is created, you can add a bucket policy to further restrict access:\n", "\n", "1. **Select Your Bucket**:\n", " - In the S3 console, click on the name of your newly created bucket.\n", "\n", "2. **Go to the Permissions Section**:\n", " - Click on the “Permissions” tab.\n", "\n", "3. **Add a Bucket Policy**:\n", " - In the Permissions section, find and click on “Bucket Policy”.\n", " - Here you can add a JSON policy that defines who can access the bucket. To keep it private, you can use a policy that denies all access unless it comes from specific users or roles within your AWS account.\n", "\n", " Example of a basic policy for maintaining privacy:\n", " ```json\n", " {\n", " \"Version\": \"2012-10-17\",\n", " \"Statement\": [\n", " {\n", " \"Effect\": \"Deny\",\n", " \"Principal\": \"*\",\n", " \"Action\": \"s3:*\",\n", " \"Resource\": [\"arn:aws:s3:::your-bucket-name/*\", \"arn:aws:s3:::your-bucket-name\"]\n", " }\n", " ]\n", " }\n", " ```\n", " Replace `your-bucket-name` with the actual name of your bucket.\n", "\n", "4. **Save the Policy**:\n", " - Click “Save” to apply the policy.\n", "\n", "With these steps, you will have created a bucket in S3 that is private and whose access is restricted to only those whom you explicitly give permission through IAM policies or bucket policies." ] }, { "cell_type": "markdown", "id": "9d826895-a6e5-46ea-9d2f-1e2eadb64ae3", "metadata": {}, "source": [ "## How to create a public bucket on AWS S3\n", "* name used for the bucket: public-pdf-bucket" ] }, { "cell_type": "markdown", "id": "191ce726-9f6c-4679-b0db-6ead40498f6d", "metadata": {}, "source": [ "To create a public bucket in Amazon S3, you need to configure the bucket's access options to allow public access. However, it's important to consider the security implications of making a bucket public, as this could expose your data to anyone on the Internet. If you're sure that you need a public bucket (for example, to host assets for a static website), here's how you can do it:\n", "\n", "#### Step 1: Create the Bucket\n", "1. **Log in to AWS Management Console**:\n", " - Go to the [AWS Management Console](https://aws.amazon.com/console/) and log in.\n", "\n", "2. **Open the Amazon S3 Service**:\n", " - Once in the console, search for and select the S3 service.\n", "\n", "3. **Create a New Bucket**:\n", " - Click on \"Create bucket\".\n", " - Provide a name for your bucket and select the region.\n", " - Continue with the configuration until you reach the permissions section.\n", "\n", "#### Step 2: Configure Public Access Permissions\n", "In the permissions section during bucket creation:\n", "\n", "1. **Block Public Access Settings**:\n", " - Disable the \"Block all public access\" option. This will allow public access to the bucket.\n", " - AWS will warn you about the risks of doing this; ensure you understand the consequences.\n", "\n", "2. **Review and Create the Bucket**:\n", " - Review your configuration and then create the bucket.\n", "\n", "#### Step 3 (Optional, recommended): Set Up a Bucket Policy for Public Access\n", "Once the bucket is created, you need to define a bucket policy to explicitly allow public access:\n", "\n", "1. **Select Your Bucket**:\n", " - In the S3 console, click on the name of your bucket.\n", "\n", "2. **Go to the Permissions Section**:\n", " - Click on the \"Permissions\" tab.\n", "\n", "3. **Add a Bucket Policy**:\n", " - Click on “Bucket Policy”.\n", " - Add a policy that grants public read permissions. For example:\n", " ```json\n", " {\n", " \"Version\": \"2012-10-17\",\n", " \"Statement\": [\n", " {\n", " \"Effect\": \"Allow\",\n", " \"Principal\": \"*\",\n", " \"Action\": [\"s3:GetObject\"],\n", " \"Resource\": [\"arn:aws:s3:::your-bucket-name/*\"]\n", " }\n", " ]\n", " }\n", " ```\n", " - Replace `your-bucket-name` with the actual name of your bucket.\n", "\n", "4. **Save the Policy**:\n", " - Click on “Save” to apply the policy.\n", "\n", "#### Additional Considerations\n", "- **Security**: Keep in mind that making a bucket public can expose your data. Use this setting only when it's absolutely necessary and when the data is intended to be public.\n", "- **CORS Usage**: If your bucket will be used to serve content to websites on different domains, you might also need to configure CORS rules.\n", "\n", "By following these steps, you will have created a bucket in S3 that is publicly accessible, meaning anyone with the correct URL can access or download the files stored in it." ] }, { "cell_type": "markdown", "id": "b6a3a522-9478-4825-89f8-7c2ab6c2e532", "metadata": {}, "source": [ "## What is a Cron Job and how to set it up in Render.com" ] }, { "cell_type": "markdown", "id": "f03f71f9-0a07-4399-9a73-b3a23a6ced26", "metadata": {}, "source": [ "#### The Cron Job configuration on the render.yaml file\n", "\n", "The configuration of the cron job specified in the `render.yaml` file indicates that it will be deployed and executed on Render.com, but within a Docker environment. That is, although the execution of the cron job is managed through Render.com, the environment in which the cron job runs is based on Docker.\n", "\n", "To clarify:\n", "\n", "- **Deployment on Render.com**: The cron job is configured and managed through Render.com. This includes control over when and how the cron job starts, as well as the provisioning of the necessary resources for its execution.\n", "\n", "- **Execution in Docker**: The cron job executes inside a Docker container. This means that the cron job's code runs in an isolated environment provided by Docker. The use of Docker ensures that the cron job has a consistent execution environment, with the required dependencies and configuration already set up.\n", "\n", "So, in summary, the cron job is configured to be deployed and managed by Render.com, but it executes inside a Docker container as defined in the `render.yaml` file." ] }, { "cell_type": "markdown", "id": "7d7a4b79-dff6-4bd2-bc0c-6724043836d7", "metadata": {}, "source": [ "#### What is a Cron Job and how to set it up on Render.com\n", "A cron job is a scheduled task that runs automatically at a specified time. It is a common feature in Unix-type operating systems and is used to automate repetitive tasks such as backups, system updates, file synchronization, etc. In the context of web applications, cron jobs can be used for tasks such as database cleanup, report generation, maintenance script execution, among others.\n", "\n", "#### How to Configure a Cron Job on Render.com\n", "\n", "Render.com offers an easy way to configure and manage cron jobs for your applications. Here's how you can configure a cron job on Render:\n", "\n", "1. **Access Your Dashboard on Render**: \n", " - Log in to your Render.com account and access the dashboard.\n", "\n", "2. **Create a New Service**:\n", " - In the dashboard, select the option to create a new service. Render allows you to create different types of services, including cron jobs.\n", "\n", "3. **Configure Your Cron Job**:\n", " - When setting up a new service, select \"Cron Job\" as the service type.\n", " - Provide the necessary details for your cron job:\n", " - **Service Name**: Assign a descriptive name to your cron job.\n", " - **Command**: Define the command that will be executed. This should be the script or task you want to automate.\n", " - **Schedule**: You need to specify when the cron job will run. Render uses the standard cron syntax, which allows you to specify the frequency of execution (for example, `0 * * * *` to run every hour).\n", "\n", "4. **Additional Configurations**:\n", " - Depending on your need, you can set up environment variables and other specific settings for your cron job.\n", "\n", "5. **Deployment and Monitoring**:\n", " - Once configured, deploy your cron job. Render provides tools to monitor the execution of your cron jobs, allowing you to view logs and check the status of executions.\n", "\n", "6. **Updates or Changes**:\n", " - If you need to make changes or adjustments to your cron job, you can update the settings at any time from your dashboard on Render.\n", "\n", "#### Example of Cron Job Configuration\n", "\n", "Imagine you have a maintenance script called `daily-maintenance.sh` that you want to run every day at midnight. The configuration of your cron job in Render would be something like this:\n", "\n", "- **Command**: `./daily-maintenance.sh`\n", "- **Schedule**: `0 0 * * *` (This means \"at 0 hours, 0 minutes, of each day, of each month, of each day of the week\").\n", "\n", "Remember that the correct configuration and testing of your cron jobs are essential to ensure that automated tasks run as expected and without interruptions." ] }, { "cell_type": "markdown", "id": "cab00900-8a6a-4742-a221-77fd5c52c943", "metadata": {}, "source": [ "## Dev Environment" ] }, { "cell_type": "markdown", "id": "cb9da60b-bf8d-41e0-a23c-d7157c7b2176", "metadata": {}, "source": [ "* It's recommended to use the config included for a Github Codespace in the devcontiner.json file." ] }, { "cell_type": "markdown", "id": "2329e830-ddf6-4e41-b93e-7056be902df2", "metadata": {}, "source": [ "* Log into Github.\n", "* Go to the SEC Insights open source repository:\n", " * [Here](https://github.com/run-llama/sec-insights/tree/main) \n", "* Drop-down the \"Code\" button to create a Github Codespace.\n", "* At the bottom of the Codespace you have a remote terminal.\n", " * `cd frontend`\n", " * `ls`\n", " * You can see that the frontend is based on a basic vercel app.\n", " * `npm install`\n", " * source the frontend/.env.example folder to load the environment variables that are present here.\n", " * **the url there (NEXT_PUBLIC_BACKEND_URL) is the local one of the backend (localhost/8000), will have to be changed when we use the backend in the cloud**.\n", " * `set -a`\n", " * `source .env.example`\n", " * `npm run dev`\n", "* that starts running the app in localhost/3000 served by Codespace.\n", "* It comes with live reload, so if you edit any UI file it will show immediately. For example, you can edit the title in components/landing-page/TitleAndDropdown.tsx\n", "* With the app open in one terminal, you can open a second terminal in Codespace clicking on the plus sign and cd backend.\n", " * `cd backend`\n", " * `ls`\n", "* This is a fastAPI python backend app. Most of this app is based in the templates fastAPI offers." ] }, { "cell_type": "markdown", "id": "3ebf5b10-0253-434d-a027-00348bee0518", "metadata": {}, "source": [ "## Backend" ] }, { "cell_type": "markdown", "id": "d0f32411-7fd2-47e8-8953-18a25107cb94", "metadata": {}, "source": [ "* In the Codespace second terminal, go to /backend and read the readMe file.\n", "* `cd backend`\n", "* No need to install pyenv nor docker if you are running from the devcontainer image in Github Codespaces.\n", "* `cat .python-version`: confirms you have 3.11.3\n", "* `poetry shell`: activates a virtual environment." ] }, { "cell_type": "markdown", "id": "ef79e823-455c-44b4-b7e3-a5e25ef11560", "metadata": {}, "source": [ "The command `poetry shell` is used in the context of Python programming, specifically when managing Python projects with the tool Poetry. Poetry is a tool for dependency management and packaging in Python, allowing developers to declare, manage, and install dependencies of Python projects.\n", "\n", "Here's what `poetry shell` does:\n", "\n", "1. **Activates the Virtual Environment**: When you run `poetry shell`, it activates the project's virtual environment. A virtual environment is a self-contained directory that contains a Python installation for a particular version of Python, along with a number of additional packages.\n", "\n", "2. **Isolation of Project Dependencies**: This isolation ensures that the dependencies of the project do not interfere with the system-wide Python installation or other Python projects. It's a key practice in Python development to avoid dependency conflicts and maintain project consistency.\n", "\n", "3. **Interactive Shell**: Once the virtual environment is activated, you are placed into an interactive shell (like bash or Command Prompt) that is configured to use the project's Python interpreter. This means any Python commands you run in this shell will use the project's Python version and have access to its dependencies.\n", "\n", "4. **Convenience for Development**: The `poetry shell` command is convenient for development purposes. You can run Python scripts, start a Python interactive session, or use command-line tools that are part of your project's dependencies without needing to manually activate the virtual environment or adjust your system's `PATH`.\n", "\n", "5. **Temporary Activation**: The activation of the virtual environment using `poetry shell` is temporary. Once you exit the shell, the environment is deactivated, and your terminal returns to its previous state.\n", "\n", "In summary, `poetry shell` is a command to activate the virtual environment associated with your Poetry-managed Python project, providing an isolated and consistent development environment for that project." ] }, { "cell_type": "markdown", "id": "2bbe912d-58f8-4bc0-8370-039db80e697d", "metadata": {}, "source": [ "* As you can see in the terminal cursor, now in the github codespaces terminal you are in the (llama-app-backend-py3.11) virtual environment.\n", "* `poetry install`\n", "* Now you have to create the backend/.env file\n", " * `cp .env.development .env`\n", " * `set -a`\n", " * `source .env`" ] }, { "cell_type": "markdown", "id": "ed21f464-9e05-444f-aa52-c656a71f1dd9", "metadata": {}, "source": [ "The command `set -a` in a Unix/Linux shell environment is used to change the behavior of the shell with respect to how it handles variables and their visibility (exporting) to child processes. Here's what it does:\n", "\n", "1. **Auto-Export Variables**: When you use `set -a`, any variable that you subsequently define or modify in your shell session will be automatically exported. This means that these variables become environment variables and are inherited by any child processes or sub-shells spawned from your shell.\n", "\n", "2. **Child Process Inheritance**: Normally, when you create a variable in a shell, it's only known to that particular shell session. Child processes or scripts invoked from that shell don't have access to those variables unless they are explicitly exported using the `export` command. However, with `set -a`, this export is implicit for all variables set after the command.\n", "\n", "3. **Use Cases**: This command is particularly useful in scripts where you need to ensure that all defined variables are available to sub-processes without having to explicitly export each one. It's often used in startup scripts or in scripts that configure environment variables for a particular application or service.\n", "\n", "4. **Reversing the Effect**: If you want to revert to the normal behavior where variables are not automatically exported, you can use `set +a`. This command will stop the automatic export of variables defined after it.\n", "\n", "5. **Scope of Effect**: It's important to note that the effect of `set -a` is limited to the current shell session or script in which it is run. It does not affect other shell sessions or globally change the behavior of the shell.\n", "\n", "In summary, `set -a` is a shell command used to automatically export all variables set in the current shell session, making them available to any child processes. This can be useful in scripting scenarios where environment variable propagation is desired." ] }, { "cell_type": "markdown", "id": "08f90b78-38fe-4afe-907b-7cfbf9fa4102", "metadata": {}, "source": [ "* source .env" ] }, { "cell_type": "markdown", "id": "435f0ed3-62f6-4bf7-8a1d-10b970eb7479", "metadata": {}, "source": [ "The command `source .env` in a Unix/Linux shell is used to execute the contents of a file (in this case, a file named `.env`) in the current shell session. Here's what this command specifically does:\n", "\n", "1. **Loads Environment Variables**: The `.env` file typically contains environment variables. These are often key-value pairs that are used to configure the behavior of an application or script. By using `source .env`, you are effectively loading these variables into your current shell environment.\n", "\n", "2. **Executes in Current Shell**: The `source` command (which can also be represented as a dot `.` in some shells) executes the file in the context of the current shell, rather than starting a new shell to run the script. This means any changes made to the environment, such as setting variables, changing directories, etc., will persist in the current shell after the script completes.\n", "\n", "3. **Use in Application Configuration**: This is a common practice in application development, especially in web development, where `.env` files are used to set configuration variables that should not be hard-coded into the application, such as database passwords, API keys, and other sensitive information.\n", "\n", "4. **Security Note**: It's important to be cautious with `.env` files, especially regarding sensitive information. These files should not be included in version control (like Git) if they contain sensitive data.\n", "\n", "5. **Portability and Convenience**: This approach allows for easy customization of application behavior in different environments (like development, testing, production) by simply changing the contents of the `.env` file, rather than altering the application code.\n", "\n", "In summary, `source .env` is used to execute the contents of the `.env` file in the current shell, typically for the purpose of setting environment variables that configure the behavior of an application or script. This allows for a flexible and secure way to manage configuration settings." ] }, { "cell_type": "markdown", "id": "a8326a43-aaa8-4cc9-915a-c93d8a4df4e0", "metadata": {}, "source": [ "## Enter the environment variables in the backend/.env file" ] }, { "cell_type": "markdown", "id": "9838561e-34d1-4faa-94a9-8275d4127528", "metadata": {}, "source": [ "* DATABASE_URL=postgresql://user:password@127.0.0.1:5432/llama_app_db\n", " * change with the URL of the database you have created in Render.com\n", "* BACKEND_CORS_ORIGINS=\n", " * '[\"http://localhost\",\n", " * \"http://localhost:8000\",\n", " * \"http://localhost:3000\",\n", " * \"http://127.0.0.1:3000\",\n", " * \"https://llama-app-backend.onrender.com\",\n", " * change with the URL of your Render.com backend\n", " * \"https://llama-app-frontend.vercel.app\",\n", " * change with the URL of your Vercel frontend \n", " * \"http://secinsights.ai\",\n", " * change with the URL of your live app \n", " * \"http://www.secinsights.ai\",\n", " * change with the URL of your live app\n", " * \"https://secinsights.ai\",\n", " * change with the URL of your live app\n", " * \"https://www.secinsights.ai\"]'\n", " * change with the URL of your live app\n", "* OPENAI_API_KEY=sk-XXXXXXXXXXXXXXXXXXXXXXXX\n", " * enter your OpenAI API Key \n", "* LOG_LEVEL=debug\n", "* RENDER=False\n", "* S3_BUCKET_NAME=llama-app-backend-local\n", " * enter the name of your private bucket \n", "* S3_ASSET_BUCKET_NAME=llama-app-web-assets-local\n", " * enter the name of your public bucket \n", "* CDN_BASE_URL=http://llama-app-web-assets-local.s3-website.localhost.localstack.cloud:4566\n", " * By default, this is using a URL generated by LocalStack, that simulates AWS S3 in your local environment.\n", " * When you move to production, you will create a CDN using CloudFront from AWS. \n", " * This is a CDN to serve your buckets more efficiently\n", " * Read the price details carefully, this usually starts being free but it will turn into a paid version after you reach certain traffic volume (1TB).\n", " * Create a Cloudfront distribution\n", " * Associate it with your private bucket\n", " * Enter the URL\n", "* AWS_KEY=xxx\n", " * You can enter a fake key like 123abc123 in development. \n", " * In production, you will need to enter your AWS Key. Read the security recommendations of AWS about it.\n", "* AWS_SECRET=xxx\n", " * You can enter a fake key like 123abc123 in development. \n", " * In production, you will need to enter your AWS Key. Read the security recommendations of AWS about it.\n", "* POLYGON_IO_API_KEY=xxx\n", " * You can enter a fake key like 123abc123 in development. \n", " * In production, you will need to enter your Polygon API Key.\n", "* SEC_EDGAR_COMPANY_NAME=YourOrgName\n", " * You can enter a fake name like abc123 in development.\n", " * In production, you will need to enter your SEC Edgar Company Name.\n", "* SEC_EDGAR_EMAIL=you@example.com\n", " * You can enter a fake email like abc@abc.com in development.\n", " * In production, you will need to enter SEC Edgar account email." ] }, { "cell_type": "markdown", "id": "533fd7ea-bb02-4149-9857-d7fd52516b68", "metadata": {}, "source": [ "## Start the backend server" ] }, { "cell_type": "markdown", "id": "57651ac7-8608-4fc5-91ae-83b7e1826444", "metadata": {}, "source": [ "* `make migrate`: runs the database migrations.\n", " * In development, LocalStack will simulate AWS S3 in your local environment.\n", "* `make run`: starts the server locally.\n", " * This spins up the Postgres 15 DB & Localstack in their own docker containers.\n", " * The server will not run in a container but will instead run directly on your OS.\n", " * This is to allow for use of debugging tools like pdb" ] }, { "cell_type": "markdown", "id": "6b1df44d-fce6-49f1-8ebf-a27ee6a74236", "metadata": {}, "source": [ "## When you are ready, enter your final environment variables in the .env file" ] }, { "cell_type": "markdown", "id": "0f77d1fc-3421-48ca-8ec6-ae121fa4d766", "metadata": {}, "source": [ "* Open the .env file and replace the placeholder values with your own API keys.\n", "* Source the file again with `set -a` then `source .env`" ] }, { "cell_type": "markdown", "id": "da76b2ef-bfab-436e-a366-038121b533e0", "metadata": {}, "source": [ "## Populate your local database with some sample SEC filings" ] }, { "cell_type": "markdown", "id": "a08d4b5c-daa8-4f37-89b3-50d6093cd07a", "metadata": {}, "source": [ "* Run `make seed_db_local`\n", " * If this step fails, you may find it helpful to run `make refresh_db` to wipe your local database and re-start with emptied tables.\n", "* Done 🏁! You can run make run again and you should see some documents loaded at [http://localhost:8000/api/document](http://localhost:8000/api/document)" ] }, { "cell_type": "markdown", "id": "a210c216-1a75-4374-89d0-a5e01838d536", "metadata": {}, "source": [ "## To use this RAG app with your own private documents" ] }, { "cell_type": "markdown", "id": "e36d6107-ac1d-4e5e-af9c-63a1d4312ddb", "metadata": {}, "source": [ "* Example working with the app in your local environment.\n", "* Have the backend running in a second terminal window.\n", "* Go to localhost:8000/api/document\n", " * This will show the current documents in the database \n", "* To load a new document, in the /backend/scripts folder there are useful scripts to do that like upsert_document.py\n", "* In terminal, run `python upsert_document.py URLofYourPDFDocument\n", " * this loads a new document into the database our updates it if it already exists.\n", " * if you go to localhost:8000/api/document you will see the new document\n", " * then, if you run `make chat` you will be able to chat with your private documents from your terminal.\n", " * run `pick_docks`\n", " * pick > run `help` to see the available commands\n", " * pick > run `select_id idOfTheDocument`\n", " * pick > run `finish`\n", " * run `help`\n", " * run `create`\n", " * run `detail`\n", " * run `message WriteYourQuestionAboutTheDocumentHere`" ] }, { "cell_type": "markdown", "id": "e4459821-d126-4410-8dd3-4b0bdbda7c99", "metadata": {}, "source": [ "## For any issues\n", "* For any issues in setting up the above or during the rest of your development, you can check for solutions in the following places:\n", "\n", " * [backend/troubleshooting.md](https://github.com/run-llama/sec-insights/blob/main/backend/troubleshooting.md)\n", " * [Open & already closed Github Issues](https://github.com/run-llama/sec-insights/issues?q=is%3Aissue+is%3Aclosed)\n", " * [The #sec-insights discord channel](https://discord.com/channels/1059199217496772688/1150942525968879636)" ] }, { "cell_type": "markdown", "id": "1258a338-6ca3-4e61-9dea-0c7f4a33cda4", "metadata": {}, "source": [ "## SEC Document Downloader" ] }, { "cell_type": "markdown", "id": "fd1b8891-6c1f-4446-b107-9111a8eceb9d", "metadata": {}, "source": [ "We have a script to easily download SEC 10-K & 10-Q files! This is a single step of the larger seed script described in the next section. Unless you have some use for just running this step on it's own, you probably want to stick to the Seed script described in the section below. However, the setup instructions for this script are a pre-requisite for running the seed script.\n", "\n", "No API keys are needed to use this, it calls the SEC's free to use Edgar API.\n", "\n", "The instructions below explain a process to use the script to download the SEC filings, convert the to PDFs, and store them in an S3 bucket.\n", "\n", "## Setup / Usage Instructions\n", "Pre-requisite setup steps to use the downloader script to load the SEC PDFs directly into an S3 bucket.\n", "\n", "These steps assume you've already followed the steps above for setting up your dev workspace!\n", "\n", "#### Setup AWS CLI\n", "* Install AWS CLI\n", " * This step can be skipped if you're running from the devcontainer image in Github Codespaces\n", " * Steps:\n", " * curl \"https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip\" -o \"awscliv2.zip\"\n", " * unzip awscliv2.zip\n", " * sudo ./aws/install\n", "* Configure AWS CLI\n", " * This is mainly to set the AWS credentials that will later be used by s3fs\n", " * Run aws configure and enter the access key & secret key for a AWS IAM user that has access to the PDFs where you want to store the SEC files.\n", " * set the default AWS region to us-east-1 (what we're primarily using).\n", "\n", "#### Setup s3fs\n", "* Install s3fs\n", " * This step can be skipped if you're running from the devcontainer image in Github Codespaces\n", " * sudo apt install s3fs\n", "* Setup a s3fs mounted folder\n", " * Create the mounted folder locally mkdir ~/mounted_folder\n", " * s3fs llama-app-web-assets-preview ~/mounted_folder\n", " * You can replace llama-app-web-assets-preview with the name of the S3 bucket you want to upload the files to.\n", "\n", "#### Install wkhtmltopdf\n", "* This step can be skipped if you're running from the devcontainer image in Github Codespaces\n", "* Steps:\n", " * sudo apt-get update\n", " * sudo apt-get install wkhtmltopdf\n", "\n", "#### Get into your poetry shell with poetry shell from the project's root directory.\n", "\n", "#### Run the script! \n", "* python scripts/download_sec_pdf.py -o ~/mounted_folder --file-types=\"['10-Q','10-K']\"\n", "* Take a 🚽 break while it's running, it'll take a while!\n", "\n", "#### Go to AWS Console and verify you're seeing the SEC files in the S3 bucket." ] }, { "cell_type": "markdown", "id": "7a2b32a0-88b3-4547-b1ec-6ed9d74b938b", "metadata": {}, "source": [ "## Seed DB Script" ] }, { "cell_type": "markdown", "id": "cf33c50a-6320-41ee-b4d3-0b037f8db2f8", "metadata": {}, "source": [ "There are a collection of scripts we have for seeding the database with a set of documents. The script in `scripts/seed_db.py` is an attempt at consolidating those disparate scripts into one unified command.\n", "\n", "This script will:\n", "\n", "* Download a set of SEC 10-K & 10-Q documents to a local temp directory\n", "* Upload those SEC documents to the S3 folder specified by $S3_ASSET_BUCKET_NAME\n", "* Crawl through all the PDF files in the S3 folder and upsert a database row into the Document table based on the path of the file within the bucket\n", "\n", "#### Use Cases\n", "This is useful for times when:\n", "\n", "* You want to setup a local environment with your local Postgres DB to have a set of documents in the documents table\n", " * When running locally, this will use localstack to store the documents into a local S3 bucket instead of a real one.\n", "\n", "* You want to update the documents present in either Prod or Preview DBs\n", " * In fact, this is the very script that is run by the llama-app-cron cron job service that gets setup by the render.yaml blueprint when deploying this service to Render.com.\n", "\n", "#### Usage\n", "To run the script, make sure you've:\n", "\n", "* Activated your Python virtual environment using poetry shell\n", "* Installed all the pre-requisite dependencies for the SEC Document Downloader script.\n", "* Defined all the environment variables from `.env.development` within your shell environment according to the environment you want to execute the seed script (e.g. local, preview, prod environments)\n", "\n", "After that you can run `python scripts/seed_db.py` to start the seed process.\n", "\n", "To make things easier, the Makefile has some shorthand commands.\n", "\n", "* `make seed_db`\n", " * Just runs the seed_db.py script with no CLI args, so just based on what env vars you've set\n", "\n", "* `make seed_db_preview`\n", " * Same as make seed_db but only loads SEC documents from Amazon & Meta\n", " * We don't need to load that many company documents for Preview environments.\n", "\n", "* `make seed_db_local`\n", " * To be used for local database seeding\n", " * Runs seed_db.py just for `$AMZN & $META` documents\n", " * Sets up the localstack bucket to actually serve the documents locally as well, so you can load them in your local browser.\n", "\n", "* `make seed_db_based_on_env`\n", " * Automatically calls one of the above shorthands based on the RENDER & IS_PREVIEW_ENV environment variables" ] }, { "cell_type": "markdown", "id": "94a1ec67-af58-4ae5-9df4-1fb4406f5920", "metadata": {}, "source": [ "## Deep dive into the key LlamaIndex logic\n", "* Main placement: `/backend/app/chat/engine.py`\n", "* Secondary placements in same folder:\n", " * `messaging.py`\n", " * `pg_vector.py`\n", " * `a_response_synth.py`" ] }, { "cell_type": "markdown", "id": "d4966513-7131-4243-ae42-4deaed82fa88", "metadata": {}, "source": [ "## /backend/app/chat/engine.py" ] }, { "cell_type": "markdown", "id": "559ae2b8-59d7-4bf0-8cbd-b055acaa2e13", "metadata": {}, "source": [ "#### General summary" ] }, { "cell_type": "markdown", "id": "505355c3-df3f-449d-a589-366d984db5e8", "metadata": {}, "source": [ "This code is a backend process which interacts with documents and handles chat messages for an AI assistant service. \n", "\n", "It is using several Python libraries and custom modules to accomplish its tasks. Here's a simplified breakdown:\n", "\n", "1. **Import Statements**: The code starts by importing various Python modules and custom modules. These are used for handling different types of data, file operations, logging, and interacting with external services like AWS S3.\n", "\n", "2. **Logger Configuration**: It sets up logging to keep track of events and errors.\n", "\n", "3. **AsyncIO Patch**: A patch for AsyncIO (asynchronous input/output) is applied. This is used in Python for non-blocking I/O operations, which are important in web and network applications for efficiency.\n", "\n", "4. **S3 File System Setup**: A function `get_s3_fs` is defined to configure and return a connection to AWS S3, a cloud storage service. It uses credentials and settings from the application configuration.\n", "\n", "5. **Document Handling Functions**: Several functions are defined to fetch, read, and process documents. For example, `fetch_and_read_document` downloads a PDF document, reads it, and returns its contents.\n", "\n", "6. **Document Indexing and Querying**: Functions like `index_to_query_engine` and `build_doc_id_to_index_map` are defined for indexing documents and setting up a query engine. The application uses a document search and retrieval system.\n", "\n", "7. **Chat Message Processing**: The function `get_chat_history` processes chat messages, filtering and sorting them. This indicates the application might be handling chat data, possibly for customer support or an interactive AI assistant.\n", "\n", "8. **Service Context Setup**: The `get_tool_service_context` function sets up the context for various services used in the application, likely integrating different tools and APIs for processing requests.\n", "\n", "9. **Chat Engine Initialization**: The function `get_chat_engine` initializes a chat engine with various settings and tools. This engine is probably used to handle and respond to chat messages in a conversation, using AI models (like GPT-4).\n", "\n", "10. **Caching and Asynchronous Operations**: The code uses caching (with `cached` and `TTLCache`) and asynchronous operations (with `async` keyword) for efficiency, especially important in handling web requests and external API calls.\n", "\n", "11. **Application-Specific Logic**: The code includes specific logic related to the application's domain, like handling SEC documents, parsing chat messages, and generating responses.\n", "\n", "In summary, this code is a part of a backend service for an AI-powered chat application, dealing with document management, retrieval, and processing, as well as handling and responding to chat messages in a conversational context. It integrates various services and APIs, and employs modern Python programming techniques like asynchronous operations and caching for efficiency." ] }, { "cell_type": "markdown", "id": "5277c9f9-ae97-40de-bcdb-e4cfffe9f1b5", "metadata": {}, "source": [ "#### index_to_query_engine function" ] }, { "cell_type": "markdown", "id": "b0dabb84-2969-4989-8d60-748b4bfeb6e8", "metadata": {}, "source": [ "This code defines a function named `index_to_query_engine` in Python, which is designed to convert a document index into a query engine. Here's a simplified explanation:\n", "\n", "1. **Function Definition**: The function `index_to_query_engine` takes two parameters:\n", " - `doc_id`: A string representing the unique identifier of a document.\n", " - `index`: An object of type `VectorStoreIndex`, which is likely a class of the LlamaIndex library. This object represents an index of documents where each document is represented as a vector (a list of numbers) for efficient searching and comparison.\n", "\n", "2. **Setting Up Filters**: Inside the function, a `MetadataFilters` object is created with an `ExactMatchFilter`. This filter is configured to match documents with a specific ID (`DB_DOC_ID_KEY`) equal to the `doc_id` passed to the function. Essentially, this setup ensures that the query engine will only consider the document with the specified ID.\n", "\n", "3. **Query Engine Configuration**: The function then prepares a set of keyword arguments (`kwargs`) for the query engine:\n", " - `similarity_top_k`: This is set to 3, which means the query engine will return the top 3 most similar results when performing a semantic search.\n", " - `filters`: The filters configured earlier are passed here, restricting the search to the specified document.\n", "\n", "4. **Creating and Returning the Query Engine**: Finally, the function calls `as_query_engine` method on the `index` object, passing the configured arguments. This method converts the index into a query engine capable of performing search queries based on the settings provided. The resulting `BaseQueryEngine` object is returned.\n", "\n", "In simple terms, this function is part of a system that handles complex document searches. It takes a document's unique identifier and an index, and sets up a query engine specifically tailored to search for information related to that particular document, focusing on the top 3 most relevant results." ] }, { "cell_type": "markdown", "id": "911534c5-b670-4f0b-80d6-1c797acd812a", "metadata": {}, "source": [ "#### get_storage_context function" ] }, { "cell_type": "markdown", "id": "8bb5f324-3e88-4a50-912a-fd0bad74365b", "metadata": {}, "source": [ "This piece of Python code defines a function named `get_storage_context`. The purpose of this function is to create and return a `StorageContext` object, which is used for managing data storage in the application. Let's break down the function in simple terms:\n", "\n", "1. **Function Definition**: The function `get_storage_context` is defined with three parameters:\n", " - `persist_dir`: A string representing the directory where data should be persisted (stored). This is where the application will save or access its data.\n", " - `vector_store`: An object of type `VectorStore`. This refers to a storage system or structure where vectors are stored. Vectors are used to represent data, especially in the context of search and machine learning.\n", " - `fs`: An optional parameter of type `AsyncFileSystem`, which is a file system interface for asynchronous file operations. Asynchronous operations allow the application to handle file I/O (input/output) in a non-blocking way, which is more efficient for web and network applications. This parameter defaults to `None` if not provided.\n", "\n", "2. **Logging Information**: The function logs a message saying \"Creating new storage context.\" This is a standard practice in software development to record what the application is doing, which helps in debugging and monitoring.\n", "\n", "3. **Creating and Returning the Storage Context**: The function then creates a new `StorageContext` object by calling `StorageContext.from_defaults`. It passes the `persist_dir`, `vector_store`, and `fs` to this method. The `from_defaults` method likely sets up a standard storage context with the given parameters.\n", "\n", "4. **Return Value**: The newly created `StorageContext` object is returned. This object would encapsulate the details about how and where the application's data is stored and managed.\n", "\n", "In summary, `get_storage_context` is a utility function that sets up and returns a storage context for the application, based on the given directory for data persistence, a vector store for storing data vectors, and an optional asynchronous file system interface. This storage context is used in the application to handle data storage and retrieval operations efficiently." ] }, { "cell_type": "markdown", "id": "caecf6a7-8e47-4d5f-882c-58d23469c7de", "metadata": {}, "source": [ "#### build_doc_id_to_index_map function" ] }, { "cell_type": "markdown", "id": "444a09b4-d937-4650-aa35-bfec3659c951", "metadata": {}, "source": [ "This code defines an asynchronous function `build_doc_id_to_index_map` in Python. The function's purpose is to create a mapping between document IDs and their corresponding indices, which are used for efficient data retrieval and search operations. Here's a breakdown of the function in simple terms:\n", "\n", "1. **Function Definition**: \n", " - `async def`: Indicates that this is an asynchronous function, which can perform operations without blocking the execution of other parts of the program.\n", " - `service_context`: A parameter representing the context in which the service operates.\n", " - `documents`: A list of `DocumentSchema` objects, each representing a document in the system.\n", " - `fs`: An optional parameter of type `AsyncFileSystem`, used for handling file operations asynchronously. It defaults to `None` if not provided.\n", " - Returns a dictionary mapping string keys (document IDs) to `VectorStoreIndex` objects.\n", "\n", "2. **Setting Up Storage Context**:\n", " - The function defines a `persist_dir` variable, set to the name of an S3 bucket (a cloud storage location) from the application's settings.\n", " - It attempts to retrieve a singleton instance of a `vector_store` asynchronously using `await get_vector_store_singleton()`.\n", " - It tries to create a `storage_context` using this vector store and the file system provided. If it fails (due to a `FileNotFoundError`), it logs a message and creates a new storage context.\n", "\n", "3. **Loading Indices from Storage**:\n", " - The function attempts to load indices for each document from the storage. These indices are structures that enable efficient searching and accessing of documents.\n", " - If successful, it creates a mapping (`doc_id_to_index`) between each document's ID and its index.\n", "\n", "4. **Handling Errors and Creating New Indices**:\n", " - If there is a `ValueError`, suggesting that indices could not be loaded, the function logs an error message and proceeds to create new indices for each document.\n", " - It reads each document (using `fetch_and_read_document`), adds them to the storage context's document store, and creates a new index for each document.\n", " - These indices are then persisted in the storage and added to the `doc_id_to_index` mapping.\n", "\n", "5. **Returning the Mapping**:\n", " - Finally, the function returns the `doc_id_to_index` dictionary, which contains a mapping of document IDs to their corresponding indices.\n", "\n", "In summary, `build_doc_id_to_index_map` is an asynchronous function that creates and returns a mapping between document IDs and their indices. It handles both the scenario where indices can be loaded from existing storage and the case where new indices need to be created, ensuring efficient document management and retrieval in the application." ] }, { "cell_type": "markdown", "id": "3e82f37e-2bd9-411d-9b2f-b7b48f1c4e58", "metadata": {}, "source": [ "#### get_tool_service_context function" ] }, { "cell_type": "markdown", "id": "7f0f7a78-c3ab-40e4-b895-638926675ee5", "metadata": {}, "source": [ "This code defines a function named `get_tool_service_context`, which is designed to create and return a `ServiceContext` object. This `ServiceContext` is a configuration used to handle the main operations within the application. Here’s a breakdown in simpler terms:\n", "\n", "1. **Function Definition**:\n", " - The function takes one parameter, `callback_handlers`, which is a list of `BaseCallbackHandler` objects. These callback handlers are likely used to perform specific actions or processes when certain events or conditions are met in the application.\n", "\n", "2. **Setting Up OpenAI and Embedding Models**:\n", " - The function initializes an `OpenAI` object with specific settings, including a temperature of 0 (which controls the randomness of the AI's responses), the model `gpt-4-1106-preview` (indicating it uses a specific version of the GPT-4 AI model), and an API key for authentication.\n", " - It also creates a `CallbackManager` using the provided callback handlers.\n", " - An `embedding_model` is set up using `OpenAIEmbedding`. This model is configured for similarity mode, indicating it is likely used for tasks involving finding similarities in text data, and it uses a specific embedding model type (`TEXT_EMBED_ADA_002`).\n", "\n", "3. **Node Parser Configuration**:\n", " - A `node_parser` is initialized with default settings. This parser is configured with a chunk size and chunk overlap, parameters that might be relevant for breaking down and analyzing text data in smaller parts for more detailed or granular processing.\n", "\n", "4. **Creating the Service Context**:\n", " - The function then creates a `ServiceContext` object using the `from_defaults` method. This `ServiceContext` is configured with the previously created `callback_manager`, `llm` (the OpenAI object), `embedding_model`, and `node_parser`.\n", " - The `ServiceContext` is a comprehensive setup that includes components for AI processing (using OpenAI), handling callbacks, and text data processing.\n", "\n", "5. **Returning the Service Context**:\n", " - Finally, the function returns the `ServiceContext` object.\n", "\n", "In simple terms, `get_tool_service_context` is a setup function that prepares and returns a context (a collection of settings and tools) for a service. This context includes an AI model (GPT-4), mechanisms to handle callbacks (actions triggered by certain events), and tools for analyzing and processing text data. This setup is used in parts of the application where AI-powered text analysis and processing are needed." ] }, { "cell_type": "markdown", "id": "330357e7-c6d6-4202-bd44-f1fa52e5d273", "metadata": {}, "source": [ "#### get_chat_engine function" ] }, { "cell_type": "markdown", "id": "4cf1ceae-4cf3-4cdd-9d3a-f0697955cb0f", "metadata": {}, "source": [ "This code defines an asynchronous function named `get_chat_engine`, which is designed to set up and return a chat engine powered by OpenAI's technology for an application that handles interactive conversations. The function uses various tools and services to manage and respond to chat messages in a sophisticated manner. Here's a breakdown of the function in simpler terms:\n", "\n", "1. **Function Definition**:\n", " - It's an `async` function, meaning it's designed to perform asynchronous operations that don't block the execution of other parts of the program.\n", " - It takes two parameters: `callback_handler`, an object to handle certain types of events or actions, and `conversation`, a schema representing the structure of a conversation.\n", "\n", "2. **Setting Up the Service Context**:\n", " - The function initializes a service context using `get_tool_service_context`. This context includes various tools and settings for handling chat-related tasks, including AI models and callback mechanisms.\n", "\n", "3. **Document Indexing and Storage Setup**:\n", " - Retrieves a connection to an S3 file system (`s3_fs`).\n", " - Builds a map (`doc_id_to_index`) that links document IDs to their indices for efficient document retrieval. This is done using the `build_doc_id_to_index_map` function.\n", "\n", "4. **Creating Query Engines**:\n", " - Sets up two types of query engines:\n", " - **Qualitative Question Engine**: Handles qualitative aspects of the conversation, like sentiments, risks, etc., using documents.\n", " - **Quantitative Question Engine**: Deals with quantitative data, like financial metrics, from selected documents.\n", "\n", "5. **Integrating OpenAI Model**:\n", " - Initializes an `OpenAI` model with specific settings for generating or interpreting chat messages. This model uses OpenAI's GPT-4 with certain configurations.\n", "\n", "6. **Chat History and Document Titles**:\n", " - Processes the chat history from the `conversation` and prepares a list of document titles related to the conversation.\n", "\n", "7. **Assembling the Chat Engine**:\n", " - The chat engine (`OpenAIAgent`) is assembled using various tools, including the qualitative and quantitative query engines, the OpenAI model, chat history, and system prompts. The system prompt is formatted with the current date and document titles.\n", " - The chat engine is designed to use the assembled tools to handle and respond to chat messages in the context of the conversation.\n", "\n", "8. **Returning the Chat Engine**:\n", " - Finally, the function returns the fully assembled chat engine.\n", "\n", "In summary, the `get_chat_engine` function sets up a chat engine capable of handling and responding to conversations in a sophisticated manner. It integrates document management, AI-powered message generation and interpretation, and specialized query engines for handling different types of questions." ] }, { "cell_type": "markdown", "id": "1fdd6741-849a-48f3-86a4-b38bb5108be2", "metadata": {}, "source": [ "## backend/app/chat/pg_vector.py" ] }, { "cell_type": "markdown", "id": "9f9cf17e-efcc-4ea5-b5d2-25ab6809ff76", "metadata": {}, "source": [ "#### General Summary" ] }, { "cell_type": "markdown", "id": "b9e6ef08-1ccd-4b3b-9580-daee60545dc2", "metadata": {}, "source": [ "This code is part of a web application using FastAPI, and it deals with setting up and managing a specialized database store for vectors. Vectors are numerical representations of data, commonly used in machine learning and search applications. Let's break it down into simpler terms:\n", "\n", "1. **Import Statements**: \n", " - The code begins by importing necessary modules and classes. These include `VectorStore` (a type of data store for vectors), `PGVectorStore` (a PostgreSQL-based implementation of `VectorStore`), and various SQLAlchemy components (a Python SQL toolkit and ORM).\n", "\n", "2. **Singleton Setup**: \n", " - `singleton_instance` and `did_run_setup` are global variables. The `singleton_instance` will hold a single instance of the vector store (to ensure only one exists in the application), and `did_run_setup` is a flag to check if the setup has been done.\n", "\n", "3. **CustomPGVectorStore Class**: \n", " - This is a custom class that inherits from `PGVectorStore`. It's designed to integrate with the FastAPI application's database connections.\n", " - The `_connect` method overrides the parent class to set up the database engine and session using the application's existing database engine (`app_engine`) and session (`AppSessionLocal`). This ensures that the vector store uses the same database connection pool as the rest of the application.\n", " - The `close` method is an asynchronous function to close database sessions and dispose of the engine.\n", " - The `_create_tables_if_not_exists` and `_create_extension` methods are overridden but left empty, indicating they are not needed or their functionality is handled elsewhere.\n", " - The `run_setup` method is an asynchronous function that ensures the PostgreSQL extension for vector handling is created and initializes the required database tables. It uses the `did_run_setup` flag to avoid redundant setups.\n", "\n", "4. **Asynchronous Singleton Factory Function**:\n", " - `get_vector_store_singleton` is an asynchronous function that returns an instance of `VectorStore`. \n", " - It checks if `singleton_instance` already exists; if not, it creates a new instance of `CustomPGVectorStore` with parameters derived from the application's settings (like database URL and table name).\n", " - Once created, this instance is stored in `singleton_instance` and returned for use.\n", "\n", "In summary, this code is setting up and managing a custom vector store in a FastAPI application for storing and querying vector data efficiently. It integrates with the application's database settings and connection pool, and ensures that the vector store is set up only once (singleton pattern). The custom vector store is specifically tailored to work with PostgreSQL and is asynchronous, aligning with FastAPI's asynchronous nature." ] }, { "cell_type": "markdown", "id": "73cb3359-cd55-475c-96d6-636f11c1cf8b", "metadata": {}, "source": [ "## backend/app/chat/messaging.py" ] }, { "cell_type": "markdown", "id": "c2bd8099-0049-457f-a07c-b549a9b75b4a", "metadata": {}, "source": [ "#### General Summary" ] }, { "cell_type": "markdown", "id": "37801e2a-f8ba-4e58-9526-682f1f4a0bb7", "metadata": {}, "source": [ "This code focuses on handling chat messages and streaming responses. Let's break it down into simpler terms:\n", "\n", "1. **Import Statements and Class Definitions**:\n", " - The code begins by importing necessary Python modules and classes, including those for handling asynchronous operations (`asyncio`), logging, and data streaming.\n", " - It defines some data models (`StreamedMessage` and `StreamedMessageSubProcess`) using Pydantic, which are structured representations of the messages and subprocesses in the chat.\n", "\n", "2. **ChatCallbackHandler Class**:\n", " - This class extends `BaseCallbackHandler` and is designed to handle specific events that occur during the chat.\n", " - It defines methods like `on_event_start` and `on_event_end` to manage the start and end of certain events, creating asynchronous tasks to process these events.\n", " - The `async_on_event` method asynchronously handles specific events, sending information about these events (such as their metadata) to a channel for further processing.\n", " - The class also includes some no-operation methods (`start_trace` and `end_trace`), which are placeholders and do not perform any action.\n", "\n", "3. **Handle Chat Message Function**:\n", " - `handle_chat_message` is an asynchronous function that takes a conversation object, a user message, and a send channel as inputs.\n", " - The function sets up a chat engine by calling `get_chat_engine`, passing it an instance of `ChatCallbackHandler` and the conversation context.\n", " - It then sends a templated message (which includes the user's message) to the chat engine for processing.\n", " - The response from the chat engine is streamed back, and each part of the response is sent to the send channel as it is received.\n", " - If the response is empty, a default message is sent indicating that the system couldn't understand or answer the question.\n", "\n", "4. **Streaming Chat Response**:\n", " - The chat engine's response is handled in a streaming fashion, meaning the response is processed and relayed as it's being generated, rather than waiting for the entire response to be complete.\n", " - This is likely for efficiency and to provide a more interactive user experience.\n", "\n", "5. **Error Handling and Logging**:\n", " - The code includes checks to ensure that the send channel is not closed before sending messages.\n", " - It logs various events and errors, which is useful for debugging and monitoring the application.\n", "\n", "In summary, this code is part of a chatbot or an interactive system that handles user messages. It sets up a callback handler to process events during a chat, uses a chat engine to generate responses, and streams these responses back to the user. The system is designed to be asynchronous for efficiency and is equipped with error handling and logging for robust operation." ] }, { "cell_type": "markdown", "id": "9badb744-7c19-4c21-bfb8-00724436428f", "metadata": {}, "source": [ "## backend/app/chat/qa_response_synth.py" ] }, { "cell_type": "markdown", "id": "7984e7df-c310-49f7-ad6c-a80e8fdb068b", "metadata": {}, "source": [ "#### General Summary" ] }, { "cell_type": "markdown", "id": "82fee03f-d604-473b-b29c-ad929c5d32ff", "metadata": {}, "source": [ "This code defines a function named `get_custom_response_synth`, which creates a custom response synthesizer for a chatbot dealing with questions about SEC (U.S. Securities and Exchange Commission) filing documents. Here's a simplified explanation:\n", "\n", "1. **Import Statements**: The code imports necessary Python classes and functions, including ones related to response synthesis, service context, prompts, and document handling.\n", "\n", "2. **Function Definition**:\n", " - The function `get_custom_response_synth` takes two parameters: `service_context` (which likely provides context and settings for the service) and `documents` (a list of `DocumentSchema` objects, each representing an SEC filing document).\n", "\n", "3. **Building Document Titles**:\n", " - The function constructs a string (`doc_titles`) that contains a list of the titles of the provided SEC documents. This is done by joining the titles with newline characters, formatting each title with a hyphen prefix.\n", "\n", "4. **Refine Prompt Template**:\n", " - A template string `refine_template_str` is created for a type of prompt called \"Refine Prompt\". This template is structured to provide information about the selected SEC documents and an existing answer to a query. It then asks to refine the existing answer with additional context or to return the original answer if the context isn’t useful.\n", "\n", "5. **Question-Answer Prompt Template**:\n", " - Similarly, a template string `qa_template_str` is created for a \"Question-Answer Prompt\". This template provides the titles of the SEC documents and some context information, then asks to answer a query based on this information.\n", "\n", "6. **Creating Prompts**:\n", " - Two prompt objects (`refine_prompt` and `qa_prompt`) are created using the respective template strings. These prompts are configured with their types (`REFINE` and `QUESTION_ANSWER`).\n", "\n", "7. **Creating and Returning the Response Synthesizer**:\n", " - The function calls `get_response_synthesizer`, passing the `service_context`, the two prompts, and a flag for structured answer filtering (set to False, which might be specific to certain versions of GPT, like 3.5).\n", " - It returns the response synthesizer configured with these prompts. This synthesizer is used to generate responses to user queries in the chatbot.\n", "\n", "In summary, `get_custom_response_synth` sets up a custom response synthesizer for a system handling queries about SEC documents. It uses two types of prompts - one to refine existing answers and another to generate new answers from scratch based on provided document context." ] }, { "cell_type": "markdown", "id": "c5b2bd3d-713e-4fe6-b379-8538679ec5b4", "metadata": {}, "source": [ "## backend/app/chat/constants.py" ] }, { "cell_type": "markdown", "id": "4be1eaaa-6688-4db9-ad41-67a8a2081b0a", "metadata": {}, "source": [ "#### General Summary" ] }, { "cell_type": "markdown", "id": "f76abacf-b5ad-4efc-adf3-38d84f30f300", "metadata": {}, "source": [ "This code snippet is part of a larger application, an AI-driven chatbot or virtual assistant, designed to answer financial questions using specific guidelines and tools. Here's a breakdown of the components and their purposes:\n", "\n", "1. **Constant Definition (`DB_DOC_ID_KEY`)**:\n", " - `DB_DOC_ID_KEY = \"db_document_id\"`: This line defines a constant `DB_DOC_ID_KEY` with the value `\"db_document_id\"`. This constant is used as a key or identifier for document IDs in a database.\n", "\n", "2. **System Message Template (`SYSTEM_MESSAGE`)**:\n", " - The `SYSTEM_MESSAGE` variable holds a multiline string that serves as a template for the system message presented to the AI agent. This message defines the role and guidelines for the agent:\n", " - The AI is framed as an \"expert financial analyst\" who must use specific tools to answer financial questions.\n", " - It emphasizes the necessity of using these tools to find answers or relevant information, even if it seems the tools might not have a direct answer.\n", " - The AI should assume that user queries are related to the financial documents (SEC documents) they have selected.\n", " - For non-financial questions, the AI is instructed to respectfully decline to respond and prompt the user to ask a relevant question.\n", " - In cases where the tools don't find a direct answer, the AI should communicate this and still provide any useful insights obtained.\n", " - The message includes placeholders (`{doc_titles}` and `{curr_date}`) that will be replaced with actual document titles and the current date in the context of the conversation.\n", "\n", "3. **Node Parser Configuration (`NODE_PARSER_CHUNK_SIZE` and `NODE_PARSER_CHUNK_OVERLAP`)**:\n", " - These constants define parameters for a node parser, which is likely a component of the application used for processing text:\n", " - `NODE_PARSER_CHUNK_SIZE = 512`: This sets the size of text chunks that the node parser will process at a time. A chunk size of 512 characters is a common choice for balancing granularity and performance.\n", " - `NODE_PARSER_CHUNK_OVERLAP = 10`: This specifies the number of characters that will overlap between consecutive chunks. An overlap can help ensure that the context is not lost between chunks.\n", "\n", "In summary, this code provides configuration and instructions for an AI system designed to respond to financial queries. It sets up guidelines for how the AI should handle questions, emphasizing the use of specific tools and databases (especially for financial analysis related to SEC documents), and configures the text processing parameters for the node parser. The system is designed to be responsive and helpful, while also acknowledging its limitations and guiding users to ask appropriate questions." ] }, { "cell_type": "markdown", "id": "a3de862a-e5d4-4455-962b-93029dc5bccc", "metadata": {}, "source": [ "## Other interesting files" ] }, { "cell_type": "markdown", "id": "45449c12-b3b6-4a0c-843d-47d28e81c85f", "metadata": {}, "source": [ "## backend/app/api/crud.py" ] }, { "cell_type": "markdown", "id": "d7f72784-b9da-471a-9043-9973b6f84de5", "metadata": {}, "source": [ "#### General Summary" ] }, { "cell_type": "markdown", "id": "75da8842-31d8-479c-9409-57ea04229f90", "metadata": {}, "source": [ "This code is part of an asynchronous web application, built with Python and using SQLAlchemy for database operations. It defines several functions to interact with a database for managing conversations, messages, and documents, specifically in the context of an application like a chatbot. Here's a breakdown of the functions in simple terms:\n", "\n", "1. **Import Statements**:\n", " - The code imports necessary Python modules and classes, including those for typing, SQLAlchemy's ORM (Object-Relational Mapping), and the application's specific database and schema models.\n", "\n", "2. **`fetch_conversation_with_messages` Function**:\n", " - This function fetches a conversation and its associated messages (including sub-processes related to each message) from the database.\n", " - It takes an `AsyncSession` (for asynchronous database operations) and a `conversation_id`.\n", " - If the conversation exists, it returns a structured conversation object; otherwise, it returns `None`.\n", "\n", "3. **`create_conversation` Function**:\n", " - This function creates a new conversation in the database.\n", " - It takes the conversation payload (data) and inserts it into the database, along with any related documents.\n", " - After committing the new conversation to the database, it fetches and returns the full conversation with its messages.\n", "\n", "4. **`delete_conversation` Function**:\n", " - This function deletes a conversation from the database based on its ID.\n", " - It returns `True` if the deletion was successful (i.e., if any rows were affected), otherwise `False`.\n", "\n", "5. **`fetch_message_with_sub_processes` Function**:\n", " - This function fetches a specific message and its sub-processes (related operations or actions) based on the message's ID.\n", " - If the message exists, it returns a structured message object; otherwise, it returns `None`.\n", "\n", "6. **`fetch_documents` Function**:\n", " - This function fetches documents from the database based on different criteria like individual ID, a list of IDs, or URL.\n", " - It can also limit the number of documents returned.\n", " - The function returns a list of document objects or `None` if no documents are found.\n", "\n", "7. **`upsert_document_by_url` Function**:\n", " - This function performs an \"upsert\" operation for a document - it inserts a new document or updates it if it already exists, based on the document's URL.\n", " - It returns the upserted document object.\n", "\n", "Each of these functions is asynchronous (`async`), indicating they are designed for efficient I/O operations in a web application environment. They handle complex interactions with a database involving conversations, messages, and documents, which are central to applications like chatbots." ] }, { "cell_type": "markdown", "id": "3078ecc5-aee5-4044-a0a8-7ab0378055e6", "metadata": {}, "source": [ "## backend/app/models/main.py" ] }, { "cell_type": "markdown", "id": "d8c77539-051f-40f6-bca0-aa2cc0644d63", "metadata": {}, "source": [ "#### General Summary" ] }, { "cell_type": "markdown", "id": "b63f2ca6-23db-4fd1-9350-ea8b05617679", "metadata": {}, "source": [ "This code is part of a web application built using FastAPI. It includes setup, configuration, and launching instructions for the application. The application uses SQLAlchemy for database operations and integrates with Sentry for error tracking. Here's a breakdown of its main components:\n", "\n", "1. **Import Statements**: The code imports necessary modules for web application development, including FastAPI, SQLAlchemy, Alembic (for database migrations), and Uvicorn (an ASGI server for running the app).\n", "\n", "2. **Logging and Sentry Setup**:\n", " - The `__setup_logging` function configures logging for the application, setting the log level and format.\n", " - The `__setup_sentry` function initializes Sentry for error tracking, with configurations varying based on the environment (e.g., production vs. other environments).\n", "\n", "3. **Database and Vector Store Initialization**:\n", " - The `lifespan` async context manager ensures certain operations occur when the FastAPI application starts and stops. This includes checking and establishing a database connection, ensuring the database schema is up to date, initializing a vector store (for storing and managing vector data), and setting up a sentence tokenizer.\n", "\n", "4. **FastAPI Application Setup**:\n", " - An instance of `FastAPI` is created with configurations like the project name and OpenAPI URL.\n", " - CORS (Cross-Origin Resource Sharing) middleware is added to the app to allow requests from different origins, which is essential for a web application that interacts with various frontends or services.\n", "\n", "5. **Router and Middleware Configurations**:\n", " - The application includes various routers (like `api_router` and `loader_io_router`) for handling different URL endpoints.\n", " - The CORS settings are fine-tuned, including a special condition for GitHub Codespaces environments.\n", "\n", "6. **Application Start Function**:\n", " - The `start` function is the entry point for running the application. It sets up logging, initializes Sentry, and optionally runs database migrations depending on the deployment environment.\n", " - The application is then launched using Uvicorn, an ASGI server, with configurations for host, port, and worker count. The `reload` option is set based on the environment, enabling live reload in development.\n", "\n", "7. **Database Migration and App Environment Handling**:\n", " - In certain deployment environments (like Render.com), the app runs database migrations at startup. This ensures the database schema is always up to date.\n", " - The application environment (e.g., production, local) determines certain behaviors like logging level, Sentry setup, and whether to perform database migrations.\n", "\n", "In summary, this code sets up and launches a FastAPI web application with configurations for logging, error tracking, database connection, and CORS. It includes a lifespan context manager for initialization and cleanup tasks and is structured to handle different deployment environments and configurations." ] }, { "cell_type": "markdown", "id": "0891f846-7271-4ba2-9558-b6a2185aa7c2", "metadata": {}, "source": [ "## backend/app/tests/app/chat/test_engine.py" ] }, { "cell_type": "markdown", "id": "508fff03-306f-4e30-a078-9c7accade1c5", "metadata": {}, "source": [ "#### General Summary" ] }, { "cell_type": "markdown", "id": "0381c774-f91d-482e-a23f-5dc19c27b252", "metadata": {}, "source": [ "This code is for testing a chat application, specifically focusing on how the application handles and processes chat messages. It includes several components and functions to assist with these tests:\n", "\n", "1. **Imports and MockMessage Class**:\n", " - The code imports necessary modules and classes, including those for typing, UUIDs, dates, and chat-related models.\n", " - `MockMessage` is a mock class extending the `Message` schema, simulating a message in the chat application. It includes a conversation ID, message content, and associated sub-processes.\n", "\n", "2. **chat_tuples_to_chat_messages Function**:\n", " - This function converts a list of tuples representing chat messages into a list of `ChatMessage` objects. Each tuple contains a user message and an assistant message.\n", " - The function iterates through each tuple, creating `ChatMessage` objects for user and assistant messages if they exist.\n", "\n", "3. **TestGetChatHistory Class**:\n", " - This class contains multiple test methods to validate the functionality of `get_chat_history`, a function that retrieves chat history from messages.\n", " - Each test method simulates different scenarios to ensure `get_chat_history` behaves as expected:\n", " - `test_get_chat_history_happy_path`: Tests the standard case where user and assistant messages alternate.\n", " - `test_get_chat_history_multiple_consecutive_messages_from_same_role`: Tests how consecutive messages from the same role (user or assistant) are handled.\n", " - `test_get_chat_history_empty_input`: Checks the function's behavior with an empty message list.\n", " - `test_get_chat_history_error_status`: Tests handling of messages marked with an error status.\n", " - `test_get_chat_history_error_status_assistant_message`: Similar to the previous test, but checks for an error status in assistant messages.\n", " - `test_get_chat_history_strip_content`: Verifies how the function handles messages with only whitespace content.\n", " - `test_get_chat_history_unpaired_user_message`: Tests how the function handles unpaired user messages (i.e., user messages with no corresponding assistant message).\n", "\n", "In each test, `MockMessage` objects are created to simulate a conversation's messages. These messages are passed to `get_chat_history`, and the output is compared against an expected result generated by `chat_tuples_to_chat_messages`. These tests ensure that `get_chat_history` correctly processes and structures chat histories in various scenarios, which is crucial for maintaining a reliable and user-friendly chat application." ] }, { "cell_type": "code", "execution_count": null, "id": "8b5e9a34-e56d-4fe0-b6ec-646da6ac7b45", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.4" } }, "nbformat": 4, "nbformat_minor": 5 }