{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "1f9ced15-6222-4137-ab1f-bdc9466dcb83",
   "metadata": {},
   "source": [
    "## Detailed Study of Full Stack App Components"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e8f273f3-d018-4660-bb4b-cb58f27e0e49",
   "metadata": {},
   "source": [
    "## Basic understanding of\n",
    "* SEC Insight's Architecture\n",
    "* App file structure\n",
    "* LlamaIndex\n",
    "    * RAG\n",
    "    * Subquestions\n",
    "    * Use of API-based tools\n",
    "    * With Vercel\n",
    "    * With FastAPI\n",
    "    * With Postgres\n",
    "* create-llama\n",
    "* Llama Packs\n",
    "* Github Codespaces\n",
    "* Docker\n",
    "* Frontend\n",
    "    * NextJS, React, TypeScript\n",
    "    * Vercel\n",
    "    * Interaction with fastAPI\n",
    " * Backend\n",
    "     * FastAPI\n",
    "     * Render.com - similar to AWS but easier to use.\n",
    "     * Cron job\n",
    "         * Calls OpenAI Embedding API to get the embeddings of the given SEC documents\n",
    "         * Calls Edgar API\n",
    "         * Gets PDFs\n",
    "         * Stores PDFs in AWS Public Bucket\n",
    "         * Runs the embeddings\n",
    "         * Stores the embeddings in the Postgres database (we use PG vectorstore integration)\n",
    "         * Can run whatever you set in the render.yaml file\n",
    "     * Postgres 15 database\n",
    "     * render.yaml\n",
    "* AWS S3\n",
    "    * Setup AWS S3\n",
    "    * Public PDF Bucket\n",
    "    * Private StorageContext Bucket\n",
    "        * Metadata from LlamaIndex \n",
    "* External services\n",
    "    * OpenAI Service\n",
    "    * Sentry.io Monitoring\n",
    "        * Will ping you whenever there is an error in the backend service, threshold errors, etc.\n",
    "        * Can also do performance monitoring: what sections of the code take more time, etc.\n",
    "    * SEC's Edgar API\n",
    "    * Polygon.io Financial Data API"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc51ecd9-b0ae-41e4-b08e-bd7c6392da45",
   "metadata": {},
   "source": [
    "## Basic understanding of the setup process\n",
    "* Frontend server\n",
    "    * Basic Vercel App\n",
    "* Backend server\n",
    "    * Basic FastAPI\n",
    "    * Poetry shell\n",
    "    * Make migrate: runs database migrations\n",
    "    * Make run: starts the server locally\n",
    "        * spins up postgres 15 DB & Localstac in their own docker containers\n",
    "        * the server will not run in a container but on your OS. This is to allow the use of debugging tools like PDB."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b113617b-4d29-4280-84bc-8af5d0423426",
   "metadata": {},
   "source": [
    "## Basic understanding of the private keys introduction\n",
    "* OpenAI API key\n",
    "* Polygon API key\n",
    "* AWS key & AWS secret\n",
    "* SEC's Edgar API company name & email values\n",
    "* Source the file again with set -a\n",
    "* Then source .env\n",
    "* Add .env in the .gitignore file"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "85af852c-2124-4b8c-a5a3-95792afb4ddc",
   "metadata": {},
   "source": [
    "## Basic understanding of populate your local DB with sample SEC filings\n",
    "* make seed_db_local\n",
    "    * if it fails, make refresh_db to re-start with emptied tables\n",
    "* make run "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3ee147f-70d4-4ece-97be-735dd6f80c22",
   "metadata": {},
   "source": [
    "## Review the Forums for issues\n",
    "* backend/troubleshooting.md\n",
    "* Open & already closed Github Issues\n",
    "* The #sec-insights discord channel\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b17a23b-8aa4-4d9f-bd75-cca0972541a7",
   "metadata": {},
   "source": [
    "## Basic understanding of the SEC Document Downloader\n",
    "* AWS CLI setup\n",
    "* S3fs setup\n",
    "* wk_html_to_pdf install\n",
    "* poetry shell\n",
    "* run the script\n",
    "* AWS console: verify SEC files in S3 Bucket"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2832e42a-bd5a-42b7-9441-919465e8bebb",
   "metadata": {},
   "source": [
    "## Seed DB Script\n",
    "* script for seeding the DB with a set of documents\n",
    "    * upload to S3 folder "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d67bf067-3666-4d03-88e7-b790c7b87457",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}