{'title': 'FAQs', 'page': 1, 'content': 'FAQs Asking questions MyGPT answers Confidence metrics Library Creation Installation Customization General developers Asking questions 1. What kind of questions can I ask MyGPT? During MyGPT evaluation, we have used six different question types as follows: • Keyword search: An acronym or a definition from the papers. For example, “What is acetyl- CoA?” • Summarization: Question that summarize a topic or method. For example, “What is the proof that pathogenic gain-of-function mutations can shift the equilibrium towards the active state in FGFRs?” • Yes/no: The answer can be a simple yes or no. For example, “Does the regulatory domain inhibit the PAK4 kinase activity?” • Data query: Question to find data or other facts from a section or information from several papers. For example, “Where are the leukemia-related breakpoints located in CBP/p300?” • Complex question: A research question to collect information from different parts of a paper and synthesize a response. For example, “What ar', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 1, 'content': 'e the leukemia-related breakpoints located in CBP/p300?” • Complex question: A research question to collect information from different parts of a paper and synthesize a response. For example, “What are the biggest issues facing the development of better CAR-T therapies? Rank them in order of difficulty and provide examples with citations on all potential strategies to overcome these limitations.” • Irrelevant question: A question about a topic that is not covered by the publication library. For example, asking questions about Harry Potter to a library of GPCR documents: “What does Hagrid give Harry as a Christmas present?”. You can also ask questions that do not follow the above categories, and MyGPT will answer them if they are within the limits of LLMs and RAG concepts. Several tasks, such as performing statistical analysis, analyzing whole documents, or listing references from literature with high accuracy, are beyond the current capability of LLMs and MyGPT. 2. Can I ask any questi', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 1, 'content': 'ch as performing statistical analysis, analyzing whole documents, or listing references from literature with high accuracy, are beyond the current capability of LLMs and MyGPT. 2. Can I ask any questions about topics from my library? You can ask questions that can be answered using the information available within your document library. If the data is missing from the documents, MyGPT will not be able to answer it or will let you know that the information is missing from your papers. 3. Can I ask a question about information that is in tables or figures in the library documents? MyGPT will read tables as text and try to use that information to answer your question. However, it will not be able to parse the table and find the relationship between table columns and information from the table. Also, MyGPT will not perform statistical analysis for data present in the table. MyGPT can’t find information from the figures if it can only be answered by interpreting it or by performing the stat', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 1, 'content': '. Also, MyGPT will not perform statistical analysis for data present in the table. MyGPT can’t find information from the figures if it can only be answered by interpreting it or by performing the statistical analysis, although it will be able to read the legend of the figure. If the question can be answered using the legend, it can answer it. MyGPT answers 4. What information does MyGPT uses to answer my question? MyGPT uses information in the documents in your library as the primary source of information. If the information is missing from the library, MyGPT will use inherent knowledge of LLM, which was used for the training phase of specific LLM. 5. Is MyGPT using the internet to answer my question? MyGPT does not use the internet or any external API services to get information to answer your question. 6. Can I use MyGPT to chat with LLM without incorporating the documents pipeline (RAG), like ChatGPT? You can use MyGPT to chat directly with LLM without the RAG pipeline. At the top o', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 1, 'content': 'your question. 6. Can I use MyGPT to chat with LLM without incorporating the documents pipeline (RAG), like ChatGPT? You can use MyGPT to chat directly with LLM without the RAG pipeline. At the top of the Chat panel, you can use the drop-down to switch from "Chat with Documents" to "Direct chat with GPTs". In this case, MyGPT will use LLM’s inherent knowledge to answer users\' questions.', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 2, 'content': "7. What happens if the information to answer the question is not available in my library? If the information needed to answer the question is unavailable in your library, MyGPT will either acknowledge that it is missing from the library or, in rare cases, use the LLM’s inherent knowledge to answer it. It will also provide confidence matrices in the form of question relevance score (QRS), answer relevance score (ARS), and hallucination index (HI). Low QRS, QRS, and high HI should be interpreted as an indication to verify and cross-reference the generated answer with the retrieved context highlighted in the documents. Suppose the question is off-topic from the subjects covered in the papers. In that case, the QRS and ARS scores will be low, and HI will be high, indicating that the generated answer does not use any information from the document and is entirely generated using the LLM's inherent knowledge. 8. If I am certain my library has the necessary information to answer the question,", 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 2, 'content': "answer does not use any information from the document and is entirely generated using the LLM's inherent knowledge. 8. If I am certain my library has the necessary information to answer the question, but MyGPT is not able to find it, what should I do? We recommend several solutions if you are sure that the documents contain the information to answer the question. 1. The first thing to try is rephrasing it and providing more context with your question if it’s too short. 2. You can also adjust QRS and ARS cut-off values from the customizations. Certain words in embedding models have a higher distance than our perceived understanding of language as the specific domain knowledge was lacking in training of embedding models. Increasing the QCworst and Aworst values can aid in finding the correct context from documents. 3. We have also observed different embedding models provide different meanings to the same library of documents. So, if rephrasing and adjusting relevance scores doesn’t work,", 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 2, 'content': 'context from documents. 3. We have also observed different embedding models provide different meanings to the same library of documents. So, if rephrasing and adjusting relevance scores doesn’t work, we recommend creating the same library with a different embedding model. 9. What happens if the information to answer the question is contained in a figure or table? MyGPT will read tables as text and try to use that information to answer your question. However, it will not be able to parse the table and find the relationship between table column names and the data from the table. Also, MyGPT will not perform statistical analysis for data present in the table. MyGPT will be able to read the legend of the figure. If the question can be answered using the legend, it can answer it. However, if your question can only be answered by interpreting the figure or by comparing data present in your figure, MyGPT will not be able to answer it. 10. If there is information on my library that is outdate', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 2, 'content': 'our question can only be answered by interpreting the figure or by comparing data present in your figure, MyGPT will not be able to answer it. 10. If there is information on my library that is outdated or inconsistent with facts available in public domain, will MyGPT detect the inconsistency? MyGPT is designed to perform question-answering in the context of your library of documents and will hold the information from your documents as the highest truth. If the information in your document is outdated compared to facts in the public domain, MyGPT will answer them using only information from your documents. If the LLM has more up-to-date information about your question, the answer relevance score (ARS) and hallucination index (HI) may be able to guide you. MyGPT also provides answers generated without the RAG pipeline as the drop-down with the original MyGPT-generated answers. You can compare that answer with an original answer to verify the discrepancy in relevance scores. However, if t', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 2, 'content': 'ed without the RAG pipeline as the drop-down with the original MyGPT-generated answers. You can compare that answer with an original answer to verify the discrepancy in relevance scores. However, if the most up-to-date information about your topic is also missing from LLM training data, MyGPT will answer it only using information from your documents.', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 3, 'content': "11. If the answer is related to up-to-date information that is contained in my library but that was missing from the training data used for the LLM being used, will MyGPT be able to answer accurately? Yes, MyGPT uses the facts present in your documents as the highest truth and will be able to use them as context to answer your question. MyGPT does not rely on LLM training data and eliminates the need for periodic retraining of LLMs with new information. 12. Is MyGPT using any personal information about me as the knowledge base? No, MyGPT does not use any personal information. It uses login information for hosted instances and it's independent of the RAG pipeline. 13. Is there chat history available anywhere? Yes, MyGPT has a “History” menu to access chat history, which is available for any public library or your libraries after logging in. Confidence metrics 14. What are confidence metrics? Confidence metrics are quantitative scores that MyGPT uses to gauge the trustworthiness of a gen", 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 3, 'content': 'public library or your libraries after logging in. Confidence metrics 14. What are confidence metrics? Confidence metrics are quantitative scores that MyGPT uses to gauge the trustworthiness of a generated answer. QRS (Question Relevance Score) – measures how well the user’s question matches the content in the document library. A high QRS indicates the question is on-topic and likely supported by retrieved text. ARS (Answer Relevance Score) – evaluates how closely the answer aligns with the retrieved passages. A higher ARS indicates that the response is grounded in the source material. HI (Hallucination Index) – identifies potential hallucinations. A high HI signals that the answer may be largely generated from the LLM’s internal knowledge rather than the library, especially when QRS and ARS are low. Together, these metrics help users determine whether an answer is supported by actual documents or might be a hallucinated response, enabling better verification and trust in the system.', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 3, 'content': 'ARS are low. Together, these metrics help users determine whether an answer is supported by actual documents or might be a hallucinated response, enabling better verification and trust in the system.', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 4, 'content': '15. How are question relevance scores (QRS) and answer relevance scores (ARS) calculated, and how to interpret them? How QRS and ARS are computed Score Formula Question relevance scores (QRS) Answer relevance scores (ARS) : semantic similarity scores (vector-search). : keyword/BM25 scores. : cross-encoder rerank scores (0-1). The sums are taken over the top-k retrieved chunks that form the “golden context”. Interpretation Score Typical range What it indicates High QRS (close to 1) Question is well-matched to the library. The library likely contains the needed information. Low QRS (< 0.3) Question poorly matches the library. Information may be missing or irrelevant. High ARS (close to 1) Answer heavily relies on retrieved context. Generated answer is grounded in the documents. Low ARS (< 0.3) Answer uses little or no retrieved content. Possible hallucination; verify against source. The Hallucination Index (HI) combines QRS and ARS: low QRS/ARS → high HI, flagging potential hallucinatio', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 4, 'content': '3) Answer uses little or no retrieved content. Possible hallucination; verify against source. The Hallucination Index (HI) combines QRS and ARS: low QRS/ARS → high HI, flagging potential hallucinations. Use these metrics to decide whether a generated answer should be trusted or cross-checked with the documents. 16. How is the hallucination index (HI) calculated, and how to interpret it? Hallucination Index (HI) is a single-score metric that blends the Question Relevance Score (QRS) and the Answer Relevance Score (ARS) to flag how much an LLM’s response may be “hallucinated” (i.e., not supported by the retrieved library). p and q are weighting factors that can be determined by global sensitivity analysis. HI ≈ 0 – high QRS and ARS → the answer is strongly grounded in the library; hallucination unlikely. HI close to 1 – low QRS or ARS → the answer likely relies on the LLM’s internal knowledge rather than the document set; higher risk of hallucination. Thus, a lower HI signals a trust', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 4, 'content': 'nation unlikely. HI close to 1 – low QRS or ARS → the answer likely relies on the LLM’s internal knowledge rather than the document set; higher risk of hallucination. Thus, a lower HI signals a trustworthy, evidence-based reply, while a higher HI warns that the response may contain unsupported or fabricated content. 17. Can I change the calculations of QRS, ARS, and HI? Yes, the default formulas are provided in the paper and the software, but they are not hard-coded as the only possible choice. The weights (a,b,c) to calculate QRS and (x,y,z) to calculate ARS were chosen by a global-sensitivity analysis that maximized PR-AUC. If you prefer different weighting, you can re-run the same heat-map optimization with your own dataset or simply plug in new values from the chat settings menu. QRS = (a × ∑k i=1 Qsem,i k ) + (b × ∑k i=1 Qkey,i k ) + (c × ∑k i=1 Qrerank,i k ) ARS = (x × ∑k i=1 Asem,i k ) + (y × ∑k i=1 Akey,i k ) + (z × ∑k i=1 Arerank,i k ) Qsem, Asem Qkey, Akey Qrerank, Arerank H', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 4, 'content': '= (a × ∑k i=1 Qsem,i k ) + (b × ∑k i=1 Qkey,i k ) + (c × ∑k i=1 Qrerank,i k ) ARS = (x × ∑k i=1 Asem,i k ) + (y × ∑k i=1 Akey,i k ) + (z × ∑k i=1 Arerank,i k ) Qsem, Asem Qkey, Akey Qrerank, Arerank HI = 1 − ( p × QRS p + q ) − ( q × ARS p + q )', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 5, 'content': '18. I am asking a question, and it’s about a topic in the library, but it gives me 0% relevance. Is it possible to relax the cutoff values for QRS? Yes, MyGPT allows you to lower the QRS threshold so that documents with weaker matches can be retrieved. Here’s how: 1. Re-phrase the question: A longer or more specific query often yields higher semantic scores. 1. Adjust the cutoff parameters: In the MyGPT settings, you can change the weights (a, b, c). Setting c=0 removes reranking and provides all chunks from semantic and keyword searches. Or you can increase QCworst (the minimum cosine similarity allowed) and Aworst (the lowest keyword score accepted). Raising these values will relax the filter and bring more chunks into consideration. 2. Try a different embedding model: Some models encode domain terms better; switching to an alternative can improve relevance scores for your topic. If, after these steps, the QRS remains 0 % and no relevant passages appear, it’s likely that the library', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 5, 'content': 'domain terms better; switching to an alternative can improve relevance scores for your topic. If, after these steps, the QRS remains 0 % and no relevant passages appear, it’s likely that the library simply does not contain the needed information. In that case, you may need to add additional documents or verify that the content is present in the existing files. Library creation 19. What are the options for creating a library? Options for creating a MyGPT library 1. Direct upload through MyGPT 2. Simple upload: Add PDFs, Word files, etc., with default preprocessing. 3. Advanced settings: Customize the chunk size/method, overlap, embedding model, distance function, BM25, reranker, and other retrieval parameters before building the library. 4. Import from a Zotero collection 5. Create or use an existing private/shared Zotero library of publications. 6. Use MyGPT’s “Add Zotero collection” API to pull that collection into a new dataset/library within MyGPT. Both methods produce a pre-proces', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 5, 'content': "an existing private/shared Zotero library of publications. 6. Use MyGPT’s “Add Zotero collection” API to pull that collection into a new dataset/library within MyGPT. Both methods produce a pre-processed, searchable library that can be accessed from the MyGPT home page and used for RAG queries. 20. What document types are supported by MyGPT? MyGPT supports PDFs, DOC files ('.doc', '.docx'), and TEXT (.txt) files. MyGPT can only extract text from the PDF and DOC files; it can't answer questions from images in the files.", 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 6, 'content': '21. What is the library size limit supported by MyGPT? We have tested MyGPT on 800 PDFs totaling 10,000 pages. MyGPT has an upper limit of 10 GB for uploads and 10,000 files. The large library size will require more time for uploading, but the retrieval and answer-generation steps will be unaffected by library size. 22. How much time does it take to upload a library? The upload time varies with the number of pages in the library and the processing hardware. Device Pages PDFs Chunks Upload Retrieval (1st) Retrieval (Avg) Generation (1st) Generation (Avg) E2E (Avg) Mac Laptop (M1 chip) 10 1 70 2m10s 1.28s 1.06s 6.39s 2.99s 11s Mac Laptop (M1 chip) 100 2 235 2m33s 1.18s 0.62s 4.16s 6.03s 17s Mac Laptop (M1 chip) 1000 4 3211 5m53s 0.73s 0.59s 3.16s 3.9s 12s Mac Laptop (M1 chip) 10000 791 58869 95m19s 1.94s 0.64s 5.52s 5.82s 17s Mac Laptop (M4 chip) 10 1 70 2.05s 0.38s 0.225s 0.84s 1.205s 4.5s Mac Laptop (M4 chip) 100 2 235 4.9s 0.10s 0.0775s 2.04s 2.295s 4.5s Mac Laptop (M4 chip) 1000 4 32', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 6, 'content': '95m19s 1.94s 0.64s 5.52s 5.82s 17s Mac Laptop (M4 chip) 10 1 70 2.05s 0.38s 0.225s 0.84s 1.205s 4.5s Mac Laptop (M4 chip) 100 2 235 4.9s 0.10s 0.0775s 2.04s 2.295s 4.5s Mac Laptop (M4 chip) 1000 4 3211 1m18s 0.14s 0.11333s 1.9s 2.049s 4.2s Mac Laptop (M4 chip) 10000 791 58869 23m51s 0.69s 0.651s 1.86s 2.124s 5.0s SJ VM 10 1 70 11s 5.63s 0.80s 11.22s 0.51s 11s SJ VM 100 2 235 13s 0.77s 0.79s 1.2s 1.07s 6.75s SJ VM 1000 4 3211 2m12s 0.68s 0.73s 1.16s 1.49s 12.6s SJ VM 10000 791 58869 46m2s 0.77s 0.85s 1.19s 1.39s 8s Thus, a small library of about 10 pages can be uploaded in roughly two minutes on an M1 Mac, while a larger 1000-page library may take over one and a half hours on the same machine. Upload times drop dramatically when using higher-performance GPUs such as the M4 or A100. The exact duration depends on your hardware and library size. 23. How should I format my document library? MyGPT accepts all kinds of documents, such as Biomedical literature, SOPs, or policies. We recommend', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 6, 'content': "ct duration depends on your hardware and library size. 23. How should I format my document library? MyGPT accepts all kinds of documents, such as Biomedical literature, SOPs, or policies. We recommend not using scanned copies of documents, as MyGPT doesn't have OCR capability to extract text from photocopies. 24. Can MyGPT help me with library creation, management, or library expansion related to topics covered by the library? No, MyGPT doesn't allow creation or expansion of a library. It's the user's responsibility to collect all the documents they want to upload to MyGPT. Users can use third-party tools such as Zotero or EndNote to collect documents and PDFs, then either use MyGPT's Zotero plugin to add a Zotero library or export an EndNote library and upload the documents to MyGPT. 25. If a library is shared with other users, can they see my chat history? Yes, chat history is tied to the library, not to individual users.", 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 6, 'content': 'If a library is shared with other users, can they see my chat history? Yes, chat history is tied to the library, not to individual users.', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 7, 'content': 'When a library is marked public (or “shared” with other users), anyone with access to that library who logs in to MyGPT can view its chat history through the History menu. If the library remains private to you, only your own account will see the history. In short: shared/public libraries expose their chat history to all users who have permission; private libraries keep it visible only to you. 26. If I delete the library, will it delete chat history as well? Yes. When you delete a library, MyGPT removes all of its documents, vector embeddings, tokenizer, and associated chat history from the backend database. Installation 27. What are the options for installing and using MyGPT? MyGPT can be installed in the following environments: Personal Computer Server/VM with GPU Cloud services (Azure) Personal Computer MyGPT uses Ollama as the LLM server and requires at least 8 GB RAM (16 GB recommended for better response time) and 10 GB of disk space. Ollama provides direct installation support fo', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 7, 'content': 'ersonal Computer MyGPT uses Ollama as the LLM server and requires at least 8 GB RAM (16 GB recommended for better response time) and 10 GB of disk space. Ollama provides direct installation support for macOS and Linux. For Windows, use Docker to run Ollama. To run the pipeline on each platform, follow these instructions: Mac Basic Installation Linux Basic Installation Windows Basic Installation These instructions are simple to follow, and you can modify the bash scripts as needed. Server or VM with GPU MyGPT can be hosted on a server or VM with a GPU. For this setup, we recommend hosting the User Interface (UI), Backend server, and Ollama (LLM server) on three separate VMs. The Ollama VM should have a GPU with CUDA installed. To run the pipeline on a VM/server, follow: Linux Server Installation Cloud services (Azure) MyGPT can be hosted on any cloud provider, and Azure is provided as an example deployment. For this setup, we recommend hosting the User Interface (UI), Backend server, an', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 7, 'content': 'ation Cloud services (Azure) MyGPT can be hosted on any cloud provider, and Azure is provided as an example deployment. For this setup, we recommend hosting the User Interface (UI), Backend server, and Ollama (LLM server) on three separate VMs. The Ollama VM should have a GPU with CUDA installed. To run the pipeline on Azure, follow: Azure Installation 28. What is the minimum requirement for installing MyGPT on a laptop? The minimum specs for installing MyGPT on a laptop are: CPU: ≥ 8 cores RAM: ≥ 16 GB (the pipeline can run with 8 GB, but 16 GB is recommended for better response time) GPU: Required (any GPU that supports the chosen LLM via Ollama) Storage: ≈ 10 GB of free space These are the baseline requirements listed in the installation documentation. 29. How can I check if my computer/laptop is powerful enough to use MyGPT? The limitation of running MyGPT on personal devices is the GPU and RAM. To check if your lapotp has enough resources to run Ollama, you can run following co', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 7, 'content': 'ter/laptop is powerful enough to use MyGPT? The limitation of running MyGPT on personal devices is the GPU and RAM. To check if your lapotp has enough resources to run Ollama, you can run following command:', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 8, 'content': "This will print the total time it might take to run typical MyGPT queries. You can run the same command multiple times to get the average time. Ollama generally loads the model into memory, so the first query might take some time to run. But subsequent queries should be faster. The output should look like this: If it's taking more than a minute to run, your laptop might not have enough resources to run Ollama. 30. Is there any advantage to using MyGPT when installing it on my computer? Yes. Installing MyGPT locally gives several clear benefits: - Data privacy & security: All documents remain on your machine or internal network; no confidential information is sent to third-party servers, reducing the risk of leakage and enabling deployment behind a firewall. - Customizability: You can choose the best-performing LLMs, embedding models, distance functions, and fine-tune thresholds (QRS/ARS) to match your specific library or domain needs. - Ease of installation: A single script orchestrate", 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 8, 'content': 'e the best-performing LLMs, embedding models, distance functions, and fine-tune thresholds (QRS/ARS) to match your specific library or domain needs. - Ease of installation: A single script orchestrates pre-built Docker images, simplifying installation and maintenance; no complex debugging is required. - Offline operation: The entire RAG pipeline runs locally, so you can use it even without an internet connection. These advantages make local MyGPT a practical, secure, and cost-effective choice for many users. 31. Is there any advantage of using MyGPT by installing it on the server? Yes. Installing MyGPT on a server gives several clear benefits that are highlighted in the documentation: 1. Centralized, secure deployment: A server-based installation can be kept behind a firewall, eliminating the need to send confidential documents to third-party cloud services and reducing the risk of data leakage. 2. Scalability for many users: For departments, institutions, or industry groups with dozen', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 8, 'content': 'the need to send confidential documents to third-party cloud services and reducing the risk of data leakage. 2. Scalability for many users: For departments, institutions, or industry groups with dozens of users (e.g., 40 team members on an Nvidia DGX-1), hosting MyGPT on a server allows all users to share the same GPU resources and LLM model, which is more efficient than each user running it locally. 3. Cost-effectiveness: The server can run small- or medium-sized LLMs (e.g., Llama3, Gemma4) on commodity hardware or cloud VMs, often at lower cost than commercial cloud-hosted LLM services. 4. Simplified maintenance: MyGPT can be started using prebuilt Docker images; a single installation script handles environment setup, allowing system administrators to manage updates and backups centrally. 5. Performance gains: A server with a GPU (e.g., RTX 4090 or A100) delivers faster inference than most personal laptops, improving response times for all users. In short, a server installation offer', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 8, 'content': '. 5. Performance gains: A server with a GPU (e.g., RTX 4090 or A100) delivers faster inference than most personal laptops, improving response times for all users. In short, a server installation offers secure data handling, shared resources for many users, lower operating costs, easier maintenance, and better performance compared to running MyGPT on individual devices. 32. How much will it cost to host MyGPT on the cloud with minimum requirements cost? We have hosted a MyGPT instance on Azure, with an LLM server VM (NC8as T4 v3 - 8 vCPUs, 56 GB RAM) at $548.96/month and backend/frontend VM (E4ads v5 - 4 vCPUs, 32 GB RAM) at $325.58/month. This and similar estimates and configurations are in the following table. curl -o /dev/null -s -w \'Total: %{time_total}s\\n\' http://localhost:11434/api/generate -d \'{"model": "llama3", "stream": false, "prompt": "Despite the diversity among GPCRs, are there similarities in their activation pathways?", "system":"Use the following information to answer t', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 8, 'content': 'nerate -d \'{"model": "llama3", "stream": false, "prompt": "Despite the diversity among GPCRs, are there similarities in their activation pathways?", "system":"Use the following information to answer the question in less than 200 words, try not to use anything else: [tipsychotics). GPCR activation is facilitated by extracellular ligands, and leads to the recruitment of intracellular G proteins 3,6. Structural rearrangements of residue contacts in the transmembrane domain serve as ‘activation pathways’ that connect the ligand- binding pocket to the G protein-coupling region within the receptor. How similar are these activation pathways across different class A GPCRs? Here, we analysed 27 GPCRs from diverse subgroups for which structures of active and/or inactive states are available. We show that despite the diversity in activation pathways between receptors, the pathways converge near the G protein- coupling region. This convergence is mediated by a strikingly conserved structural rearr', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 8, 'content': 'that despite the diversity in activation pathways between receptors, the pathways converge near the G protein- coupling region. This convergence is mediated by a strikingly conserved structural rearrangement of, eptor activation are broadly similar (e.g. contraction of ligand binding site, opening of the cytosolic side due to relocation of TM6). However, receptor activation is mediated by diverse ligands and hence some aspects of ligand-induced GPCR activation must necessarily be receptor-specific. How similar are the activation pathways across different receptors? We carried out a comprehensive comparison of residue contacts of inactive and active state structures. Structural equivalence for residues across the different GPCRs was assigned using the GPCRdb numbering scheme 19 from GPCRdb 20 (www.gpcrdb.org ) (Methods). A contact between a pair of residues is defined to exist if the inter-atomic distance between any two atoms across the res, t reconsideration of the mechanisms involve', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 8, 'content': '20 (www.gpcrdb.org ) (Methods). A contact between a pair of residues is defined to exist if the inter-atomic distance between any two atoms across the res, t reconsideration of the mechanisms involved in cellular signaling diversi ﬁ cation. Despite their large numbers, GPCRs can only signal through the same limited number of Gproteins that they can activate. Previous studies indicated that signaling diversity is in part dictated by a combination of G pro- teins activated by individual GPCRs ( Inoue et al., 2019 ;Masuho (C) G a-selectivity ﬁ ngerprints of RGS13 WT (left), RGS18 WT (right), and the chimera (center). (D) G a-selectivity bar codes of RGS8 WT, RGS14 WT, and the RGS8/14-F chimera. (E) Ga-selectivity ﬁ ngerprints of RGS8 WT (left), RGS14 WT (right), and the RGS8/14-F chimera. Plotted values are means ±SEMs of 3 independent experiments. The PDB accession number 1AGR is used in (B) and (D), Diverse activation pathways in class A GPCRs converge near the G protein-coupling regio', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 8, 'content': 'otted values are means ±SEMs of 3 independent experiments. The PDB accession number 1AGR is used in (B) and (D), Diverse activation pathways in class A GPCRs converge near the G protein-coupling region A. J. Venkatakrishnan1,6,*, Xavier Deupi2, Guillaume Lebon3, Franziska M. Heydenreich2,4, Tilman Flock1, Tamara Miljus2,4, Santhanam Balaji1, Michel Bouvier5, Dmitry B. Veprintsev2,4, Christopher G. Tate1, Gebhard F. X. Schertler2,4, and M. Madan Babu1,* 1MRC Laboratory of Molecular Biology, Francis Crick Avenue, Cambridge CB2 0QH, United Kingdom 2Paul Scherrer Institute, Villigen, Switzerland 3Institut de Génomique Fonctionnelle, CNRS UMR 5203, INSERM U1191, Université Montpellier, Montpellier, France 4Department of Biology, ETH Zurich, Wolfgang-Pauli-Str. 27, Zurich, Switzerland 5Institute for Research in Immunology and Cancer, University of Mo, uscarinic receptor (M2R) 11, nucleoside-activatable A 2A receptor (A 2AR)12, and peptide-activatable µ-opioid receptor (µOR) 10. The remaining', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 8, 'content': 'for Research in Immunology and Cancer, University of Mo, uscarinic receptor (M2R) 11, nucleoside-activatable A 2A receptor (A 2AR)12, and peptide-activatable µ-opioid receptor (µOR) 10. The remaining structures have been determined only in either inactive or active states. The availability of structures of GPCRs from divergent subgroups (as low as ~20% sequence identity 18) that are bound to chemically diverse ligands and known to couple to different G proteins, allowed us to investigate activation pathways across class A GPCRs. Given that the GPCRs are structurally similar and activate a small set of G proteins, some structural aspects of receptor activation are broadly similar (e.g. contraction of ligand binding site, opening of the cytosolic side due to relocation of TM6). However, rece, ic residue contacts that mediate the convergence of activation pathways across class A GPCRs. Because the microenvironment (i.e. surrounding residues/second shell residues) in which such rearrangem', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 8, 'content': 'e, ic residue contacts that mediate the convergence of activation pathways across class A GPCRs. Because the microenvironment (i.e. surrounding residues/second shell residues) in which such rearrangement takes place diverges between receptor families, the detailed mechanism by which this common step is facilitated by diverse ligands is likely to be distinct for different sets of GPCRs. Remarkably, despite such differences, we find that the activation pathways ultimately converge to a common and very specific set of contact rearrangements between topologically equivalent residues near the G protein-coupling region. Future studies aimed at investigating residues at and around these positions can help uncover the unique steps that lead to]"}\' Total: 9.975889s', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 9, 'content': "Cloud Provider Infrastructure GPU Avg. Cost per Month (USD) Azure 24 vCPUs, 220 GB RAM 1 Nvidia A100 Tensor Core $2,681.29 Azure 8 vCPUs, 56 GB RAM 1 Nvidia T4 Tensor Core $548.96 Google 12 vCPUs, 85 GB RAM 1 Nvidia A100 Tensor Core $2,682.57 Google 8 vCPUs, 30 GB RAM 1 Nvidia T4 Tensor Core $374.03 Google 4 vCPUs, 16 GB RAM 1 Nvidia L4 Tensor Core $516.99 33. Is it possible to reduce hosting cost by sharing any infrastructure of MyGPT without compromising privacy? Yes, MyGPT uses third-party software- Ollama- as an LLM server to host and infer LLMs for the RAG pipeline. We don't store chats or PDFs to the LLM server. This component can be shared with several instances of MyGPT or other AI applications. Sharing this resource will not compromise data security or privacy. 34. Where are the PDFs I have uploaded located? The PDFs you upload to MyGPT are stored locally on the machine (or server) where MyGPT runs. When you add a document, MyGPT creates a private or shared “library” that stor", 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 9, 'content': 'DFs I have uploaded located? The PDFs you upload to MyGPT are stored locally on the machine (or server) where MyGPT runs. When you add a document, MyGPT creates a private or shared “library” that stores the file in the backend and automatically splits it into chunks. Those chunks are then converted into vectors by the chosen embedding model (e.g., nomic-embedding-text or multi-qa-MiniLM-L6-cos-v16) and stored in the Chroma vector database for fast retrieval. 35. Does MyGPT send any data to any external services or APIs? No. MyGPT never sends your chat or document data to external services or APIs. All embeddings are generated locally, and the vector store is kept in a local Chroma database. MyGPT does not use the internet or any external API services to obtain information for answering questions, nor does it rely on LLM training data or personal user information as its knowledge base. 36. Does MyGPT work offline? Yes, MyGPT is designed for offline use; it can be installed locally on a', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 9, 'content': 'stions, nor does it rely on LLM training data or personal user information as its knowledge base. 36. Does MyGPT work offline? Yes, MyGPT is designed for offline use; it can be installed locally on a laptop, personal computer, or an Edge AI device such as the NVIDIA Jetson Orin Nano. The pipeline runs entirely with local LLMs (e.g., Llama 3.2 1b/3b, Gemma 2b) and does not rely on external APIs or internet access to retrieve information. MyGPT does not use the internet or any external API services, and it can handle documents offline, making it suitable for remote or air-gapped environments. Customization 37. Which LLMs can I use with MyGPT? MyGPT can run any open-source LLM that is available through Ollama (or via the command-line interface). The most commonly used models are listed below: Small, edge-friendly: ideal for Jetson Orin Nano or other low-power devices Llama3.2 1 b / 3 b, Gemma2 2 b, DeepSeek-R1 1.5 b Mid-range: good balance of speed and accuracy. Gpt-oss:20 b (recommended', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 9, 'content': 'Small, edge-friendly: ideal for Jetson Orin Nano or other low-power devices Llama3.2 1 b / 3 b, Gemma2 2 b, DeepSeek-R1 1.5 b Mid-range: good balance of speed and accuracy. Gpt-oss:20 b (recommended for personal laptops), Llama2, Gemma, Vicuna, Mistral Large, high-performance: for servers or cloud deployments, Gpt-oss:120 b, plus any of the 100+ LLMs available on Ollama’s public catalog MyGPT’s UI lists all installed models and lets you switch between them in the RAG pipeline or in “Direct chat with GPTs.” You can also add custom models via the GitHub repository instructions (https://github.com/mb-group/MyGPT_public). 38. Can I use OpenAI, Claude, or Gemini LLMs with MyGPT? MyGPT is designed to run entirely locally and does not rely on any external API services. It uses pre-built Docker images with open-source LLMs that can be installed on a laptop, an edge device, an institutional server, or a cloud VM. So, MyGPT cannot directly call OpenAI, Claude, Gemini, or other commercial LLM AP', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 9, 'content': 'mages with open-source LLMs that can be installed on a laptop, an edge device, an institutional server, or a cloud VM. So, MyGPT cannot directly call OpenAI, Claude, Gemini, or other commercial LLM APIs. 39. Which are the embedding models I can use with MyGPT? MyGPT can use any open-source embedding model that is available through Hugging Face or Ollama. Here are some of the notable embedding models recommended by us: Model Source / Notes nomic-embed-text (default) Hosted via Ollama/Hugging Face; shown to perform best overall.', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 10, 'content': 'Model Source / Notes nomic-embed-text-v2-moe Multilingual variant, useful for non-English documents.bge-m3Multilingual embedding model from Hugging Face. Paraphrase-Multilingual Another multilingual option available on Hugging Face/Ollama. multi-qa-MiniLM-L6-cos-v1 Default sentence-transformer in the Chroma DB; can be chosen by the user. MedCPT-query-encoder Domain-specific biomedical model trained on PubMed query–article pairs. 40. What are the customizations offered by MyGPT? All of these components of MyGPT are modular; users can replace or tweak them through the MyGPT user interface at the pre-processing, real-time Q&A, and generation steps. Pre-processing (chunking): Fixed-size chunking or document-structure-preserving recursive chunking. Adjustable chunk overlap (number of characters shared between consecutive chunks) Embedding & retrieval: Choice of embedding model (e.g., from MTEB leaderboard) Selection of distance function for semantic search Vector database configuration and', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 10, 'content': 'shared between consecutive chunks) Embedding & retrieval: Choice of embedding model (e.g., from MTEB leaderboard) Selection of distance function for semantic search Vector database configuration and keyword-search methods Reranking / relevance: Reranker models to refine retrieved passages Relevancy scoring mechanisms Generation: Choice of LLM (e.g., from Chatbot Arena leaderboard) Creativity parameters (temperature, top-k/p) Context protocol: MCP (Model Context Protocol) settings for controlling context length and format Evaluation & transparency: Confidence metrics (QRS, ARS, HI scores) displayed with source citations Ability to swap any component via the UI', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 11, 'content': "41. Can you suggest any ideal customizations for MyGPT? The following settings strike a balance among performance, privacy, and resource usage while keeping MyGPT highly accurate across both public and private document libraries. 1. LLM selection: Pick a top-performing model from the Chatbot Arena leaderboard (e.g., GPT-oss:20b or equivalent). 2. Embedding model: Use a high-ranking MTEB model such as nomic-embedding-text (hosted via Ollama) or multi-qa-MiniLM-L6-cos-v16 from the Chroma database. For multilingual use cases, use an appropriate embedding model, such as nomic-text-v2-moe. 3. BM25: Use BM25 for better keyword search; it's not computationally expensive and is useful for data queries or keyword-search questions. 4. Chunking – Standard chunking usually suffices; advanced semantic chunking is computationally heavy and offers little extra accuracy. We recommand 1000 characters length for fixed-size chunking and use overlap for better accuracy with retrieval. 5. Relevance tuning", 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 11, 'content': 'chunking is computationally heavy and offers little extra accuracy. We recommand 1000 characters length for fixed-size chunking and use overlap for better accuracy with retrieval. 5. Relevance tuning – Adjust QRS/ARS cut-off values and raise QCworst/Aworst thresholds to pull in more context when needed. 6. Reranking & scoring – Enable the built-in reranker (qnli-electra-base) or switch to a multilingual reranker (gte-multiligual-reranker). 7. Creativity settings – Set creativity parameters according to your use case (low for factual queries, higher for brainstorming). General developers 42. Is there documentation on APIs available from MyGPT? Yes. The documentation is embedded in the context you provided and lists several MyGPT REST-API endpoints, along with their required parameters and expected responses. The documentation is available at https://github.com/stjude-c3d/MyGPT/blob/main/development.md Key APIs include: Endpoint Purpose Required keys (request) /api/get_documents/ Retriev', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 11, 'content': 'xpected responses. The documentation is available at https://github.com/stjude-c3d/MyGPT/blob/main/development.md Key APIs include: Endpoint Purpose Required keys (request) /api/get_documents/ Retrieve documents for a dataset dataset, user_email, user_group /api/get_context/ Get retrieval context (RAG) text, model_type, dataset, new_conversation, previous_query, no_context, use_default_qrs, question_best_distance, question_worst_distance, maximum_chunks_count', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 12, 'content': 'Endpoint Purpose Required keys (request) /api/save_answer/ Persist an answer and its metrics question_text, answer_text, answer_no_context_text, model_type, dataset, no_context, use_default_ars, answer_best_distance, answer_worst_distance, QRS_p, ARS_q, use_default_hi, temperature, top_k, top_p Each request/response shape is shown, including fields like context, relevance_score, semantic_score, keyword_score, and sources. The documentation also describes optional parameters (e.g., focused_document_titles, translated_text) and notes on dataset types (papers or videos). This should give you a clear reference for using MyGPT’s APIs. 43. How do I report a bug or feature for MyGPT? If you come across any bug or error, please report it in the issues section. 44. How can I contribute to the development of MyGPT? If you need access to the MyGPT source code to help with the development process, please contact Jaimin Patel (Email: jaimin.patel@stjude.org) or the appropriate person at St. Jude Ch', 'type': 'pagechunk'}
{'title': 'FAQs', 'page': 12, 'content': "opment of MyGPT? If you need access to the MyGPT source code to help with the development process, please contact Jaimin Patel (Email: jaimin.patel@stjude.org) or the appropriate person at St. Jude Children's Research Hospital.", 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 1, 'content': '1 Democratizing reliable knowledge-seeking with MyGPT: A Privacy-First, Open-Source Retrieval- Augmented Generation Platform Jaimin Patel1,2,†,*, Jude Downing1,2,†, Hugh Ferguson1,2, Thika You1,2, Duccio Malinverni1,2, Steve Mathew D. A.1,2,8, Inês Chen2, Madison Sluter1,2, Beth Moorefield1,2, Katalin Parej2, Mukundan Ragavan2, Cindy Morris3, Dinesh Keerthi3, Desiree Becerra Armada4, Balint Meszaros1,2, Vikas Trivedi1,2, Shahinur Alam5, Anthony Woodard5, Dan Alford5, Sagar Pathak1,2, Cai Li6, Renato Umeton7, Carlos Rodriguez-Galindo4, Catherine G. Lam4, Stephen Gottschalk3, Charalampos G. Kalodimos2, and M. Madan Babu1,2,7,* 1Center of Excellence for Data-Driven Discovery, Department of Structural Biology, St. Jude Children’s Research Hospital, Memphis, TN, USA 2Department of Structural Biology, St. Jude Children’s Research Hospital, Memphis, TN, USA 3Department of Bone Marrow Transplantation and Cellular Therapy, St. Jude Children’s Research Hospital, Memphis, TN', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 1, 'content': 'ent of Structural Biology, St. Jude Children’s Research Hospital, Memphis, TN, USA 3Department of Bone Marrow Transplantation and Cellular Therapy, St. Jude Children’s Research Hospital, Memphis, TN, USA 4Department of Global Pediatric Medicine, St. Jude Global, St. Jude Children’s Research Hospital, Memphis, TN, USA 5Research Information Services, St. Jude Children’s Research Hospital, Memphis, TN, USA. 6Biostatistics, St. Jude Children’s Research Hospital, Memphis, TN, USA. 7Office of Data Science, St. Jude Children’s Research Hospital, Memphis, TN, USA. 8College of Graduate Health Sciences, University of Tennessee Health Science Center, Memphis, TN, USA †J.P. and J.D. contributed equally to designing the initial prototype for MyGPT. * Corresponding authors: jaimin.patel@stjude.org, madan.babu@stjude.org ABSTRACT Large language models (LLMs) have transformed knowledge discovery, but their use in sensitive domains is limited by privacy concerns and risks of data leaka', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 1, 'content': '.org, madan.babu@stjude.org ABSTRACT Large language models (LLMs) have transformed knowledge discovery, but their use in sensitive domains is limited by privacy concerns and risks of data leakage. Moreover, LLMs may hallucinate when queried about information outside their training data, undermining their reliability for scientific, clinical, or policy related tasks. We present MyGPT, a fully open-source, privacy-preserving retrieval-augmented-generation (RAG) platform that uses the latest open-source LLMs and operates locally or within secure institutional environments. MyGPT empowers users to query user-defined libraries such as biomedical literature, confidential clinical documents, and multi-lingual global health policies, while ensuring that sensitive data never leaves the secure environment. We introduce new confidence metrics, including “relevance scores” and “hallucination index,” and provide explicit source citation from the library within a single interface to a', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 1, 'content': 'secure environment. We introduce new confidence metrics, including “relevance scores” and “hallucination index,” and provide explicit source citation from the library within a single interface to assess answer trustworthiness. MyGPT offers a cost-effective open-source alternative to commercial LLM services and democratizes knowledge-seeking across multiple domains, including use at remote facilities without internet access. KEYWORDS Large Language Models (LLMs), Generative AI, Generative Pre -trained Transformer (GPT), Artificial Intelligence, Retrieval-Augmented Generation (RAG), Hallucination Index (HI).', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 2, 'content': '2 INTRODUCTION Information seeking is fundamental to modern human activity. With the rise of Large Language Models (LLMs), users can now ask detailed questions that require synthesis, reasoning, and examination of multiple sources. However, LLMs have limitations that make them unreliable in areas requiring certainty , transparency, and reproducibility, such as scientific research1,2. They can generate plausible but incorrect responses, known as hallucination2, due to insufficient human feedback, biases in the training dataset3, and the exclusion of private and copyrighted material from training data. LLMs are also limited to data available up to their training cut-off date (e.g., December 2025 for GPT-5.5-Pro). Moreover, they may not perform well at tasks requiring synthesis of complex concepts for real-world, research-related purposes1,4 such as literature review, where LLMs may provide biased answers or hallucinated sources that undermine their reliability and transparen', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 2, 'content': 'complex concepts for real-world, research-related purposes1,4 such as literature review, where LLMs may provide biased answers or hallucinated sources that undermine their reliability and transparency5 (Supplementary Fig. 1). Use of commercial services such as ChatGPT and Gemini raises several important issues. First, they can collect user data, queries, and documents to train their AI/ML resources, posing a privacy concern6. While such services advise against sharing private, confidential, or sensitive information4,7, users in academic, clinical, and public institutions may inadvertently leak sensitive data8. Consequently, institutions either restrict access or opt for costly enterprise versions6. Importantly, commercial services replace their older LLMs with newer versions, hindering reproducible research8 (e.g., GPT-4o and GPT-4.1 were retired on Feb 2026). Moreover, institutes or scientists working in remote locations may be unable to use commercial services without re', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 2, 'content': 'dering reproducible research8 (e.g., GPT-4o and GPT-4.1 were retired on Feb 2026). Moreover, institutes or scientists working in remote locations may be unable to use commercial services without reliable internet and cloud connections, further complicating access and security concerns8. LLM-powered products designed with a focus on privacy and customization may overcome these limitations2,8,9. Currently, there are two approaches by which this can be achieved. One approach is to fine- tune the LLM using domain-specific information or private data, where the LLM parameters are reweighted to learn field-specific or private information. However, this approach faces similar issues of trustworthiness and reliability. It also requires periodic fine-tuning of the LLM to incorporate new information and close monitoring of the fine-tuned version for privacy, data-leakage, LLM redundancy, and reproducibility. The other approach is the retrieval-augmented generation (RAG) concept, wher', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 2, 'content': 'nformation and close monitoring of the fine-tuned version for privacy, data-leakage, LLM redundancy, and reproducibility. The other approach is the retrieval-augmented generation (RAG) concept, where LLMs trained on limited data can retrieve additional information from indexed documents to generate a response10. In simple terms, when the user asks a question, LLMs receive it with a set of instructions known as a ‘prompt’. When additional supporting information is provided as ‘context’ along with the prompt, improved responses and reduced hallucinations have been observed1. Such relevant context can be retrieved from user-defined libraries of documents to ensure that they remain private and to increase response accuracy. Providing exact sources, specific sections of the text within the sources, and confidence metrics for LLM- generated responses would be beneficial to address issues of transparency and trustworthiness and to identify instances of potential hallucinations11. S', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 2, 'content': 'n the sources, and confidence metrics for LLM- generated responses would be beneficial to address issues of transparency and trustworthiness and to identify instances of potential hallucinations11. Some commercial RAG solutions such as Microsoft Copilot, NotebookLM and ChatPDF provide source citing, but we are not aware of any that provide confidence metrics to identify potential hallucinations. In the past couple of years, rapid growth in Generative AI has enabled several open access RAG solutions for local installation, ranging from preliminary to advanced concepts. However, these solutions often depend on commercial API services, have limited document library sizes, and lack features (e.g., source surfacing, confidence metrics) to identify hallucinations. To circumvent these limitations, we have developed MyGPT, an open-access, locally run, and customizable end-to-end RAG solution that enables researchers to ask questions from a curated library of documents or publication', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 2, 'content': 'mitations, we have developed MyGPT, an open-access, locally run, and customizable end-to-end RAG solution that enables researchers to ask questions from a curated library of documents or publications. MyGPT presents the exact text sources (e.g., paragraphs) to increase transparency and accuracy and provides confidence metrics that help identify instances of likely hallucination. MyGPT features a user-friendly interface to customize various components within the pipeline. This modularity provides an easy and future-proof way of swapping components8 and allows users to upload documents', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 3, 'content': '3 and interact in their preferred languages, minimizing language barriers to information access. In this way, MyGPT democratizes reliable knowledge-seeking by filling a gap in the current arsenal of LLM-based tools. RESULTS Outline of MyGPT architecture and workflow We designed MyGPT as a software framework that (a) combines accurate and transparent context retrieval with source surfacing to validate LLM response s, (b) provides confidence metrics for validation, (c) incorporates a modular design to ensure future-proofing, and (d) is both domain- and language-agnostic. We also aimed to develop a software solution that creates a workflow with optimal RAG settings for a specific use case, enabling users to access a document library, validation tools, and a chat interface within a single browser window. MyGPT architecture is composed of three sections: a user interface (UI), a backend server, and an LLM server (Fig. 1). The UI and backend server function to pre-process', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 3, 'content': 'a single browser window. MyGPT architecture is composed of three sections: a user interface (UI), a backend server, and an LLM server (Fig. 1). The UI and backend server function to pre-process the library of documents to build a knowledge base and to perform real-time question-and-answer (Q&A) by providing relevant context using context engineering, while the LLM server is key for response generation. In the pre-processing step, the UI first receives a library of documents from the user via either a reference management tool (e.g., Zotero12; Methods) or direct upload. The backend server divides the documents into smaller chunks. Chunking is essential for effective information processing, as LLMs have a limited context window size and currently cannot accept entire or large documents. The chunks are then stored in two places: a BM2513 (Best Matching 25)-tokenized library for keyword search and an embedding-based vector library within a vector database for semantic sea', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 3, 'content': 'documents. The chunks are then stored in two places: a BM2513 (Best Matching 25)-tokenized library for keyword search and an embedding-based vector library within a vector database for semantic search. For semantic search, the chunks are converted into embeddings (numerical vectors) that capture similarity and thus acquire specific meanings (semantics) defined by the embedding model. Different embedding models, trained on domain-specific data, can enhance the response of an LLM by capturing specific relationships between words and sentences in that particular domain14. The vectorized representations of knowledge sources are then stored in a searchable database hosted through the backend (Fig. 1a). The embedding models organize semantically similar terms close to each other in the vector space14. This concept also applies to sentences and paragraphs, where entities with similar meanings are positioned closer to one another, enabling better performance in similarity searche', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 3, 'content': 'he vector space14. This concept also applies to sentences and paragraphs, where entities with similar meanings are positioned closer to one another, enabling better performance in similarity searches15, as illustrated in Supplementary Fig. 2. Similarly, for keyword search, the same chunks are tokenized and indexed using the BM25 algorithm during pre-processing. In our real-world use cases described below, domain-specific private documents were never used to train an embedding model or LLM. BM25 ensures that queries with domain-specific terms match relevant chunks of documents when semantic search fails. After pre-processing, the UI displays both the library as a list of documents and a chat box for real-time Q&A (Fig. 2), and users can upload additional documents and customize the various components of MyGPT (see below). When a user asks a question, it is sent to the backend server to retrieve the relevant context. For semantic search, the query is converted into a vector', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 3, 'content': 'the various components of MyGPT (see below). When a user asks a question, it is sent to the backend server to retrieve the relevant context. For semantic search, the query is converted into a vector embedding by the same embedding model used to preprocess the library and passed to the vector database (Fig. 1b). The vector database’s similarity search algorithm compares the question to the chunks from the library and identifies the most similar chunks. For keyword search, the query is submitted to the BM25 algorithm, which identifies and returns the chunks with the highest BM25 scores. Chunks generated from both approaches are then evaluated by a Reranker (such as cross-encoders16 developed by Sentence Transformers) to assess their relevance in addressing the query. Only those chunks that meet the Reranker threshold are retained as the final k chunks after duplicate entries have been consolidated. These k chunks are then provided to the LLM server as the ‘golden context’ for t', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 3, 'content': 's that meet the Reranker threshold are retained as the final k chunks after duplicate entries have been consolidated. These k chunks are then provided to the LLM server as the ‘golden context’ for the query, enhancing the LLM response17. The LLM, which runs locally on a personal computer or a secure server/cloud infrastructure, performs context-based Q&A, generates a', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 4, 'content': '4 response, and sends it back to the UI (Fig. 2). The UI then displays the source material (i.e., chunks from specific documents) used as context to generate the response and precisely specifies their location (i.e., as document pages with highlighted sentences or paragraphs). The UI additionally displays confidence metrics calculated by the backend server that can help identify instances of hallucinations (Fig. 2; see below and Methods). MyGPT workflow customization options The individual sections within MyGPT are comprised of modularized components (Fig. 3) that can be replaced or customized through the UI to provide a seamless user experience (Supplementary Fig. 3): • Chunking: MyGPT offers two chunking methods: fixed-size or document-structure-preserving recursive chunking (Methods). While both methods allow users to adjust chunk overlap (i.e., the number of characters shared between two consecutive chunks18,19), structure-preserving chunking additionally enables', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 4, 'content': 'ing (Methods). While both methods allow users to adjust chunk overlap (i.e., the number of characters shared between two consecutive chunks18,19), structure-preserving chunking additionally enables users to focus on a specific section within a document or on common sections across all documents (e.g. abstract or results for scientific publications). • Embedding models: MyGPT users can deploy any open-source embedding model available from Hugging Face (a platform to host and share open-source AI/ML resources) or via Ollama (an LLM hosting tool). Over 400 different20,21 embedding models are available, including general-purpose embedding models such as nomic-embed-text, or a domain-specific embedding model, such as MedCPT-query-encoder (trained on query-article pairs from PubMed search logs) for the biomedical domain. • Keyword search (BM25): MyGPT uses the BM25 algorithm13,22 to help queries with domain-specific terms find relevant document sections. BM25 is enabled by def', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 4, 'content': 'rch logs) for the biomedical domain. • Keyword search (BM25): MyGPT uses the BM25 algorithm13,22 to help queries with domain-specific terms find relevant document sections. BM25 is enabled by default for English, but users can select another language or skip this step as needed. • Reranker: To validate how relevant each chunk is to answer a user’s question, MyGPT uses a QNLI (Question Natural Language Inference) cross-encoder16 as a Reranker. This Reranker has been trained on Wikipedia passages paired with corresponding answerable questions. It provides a ranked score for each chunk based on its ability to answer the user’s query. Based on the score, the chunks can be sorted from the best to the worst. While it was originally trained on English, users can select different or multilingual Rerankers depending on their needs. Once the LLM generates an answer, MyGPT determines the textual entailment of chunks using a different Reranker trained on an NLI dataset. This Rerank', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 4, 'content': 'multilingual Rerankers depending on their needs. Once the LLM generates an answer, MyGPT determines the textual entailment of chunks using a different Reranker trained on an NLI dataset. This Reranker predicts whether two sentences (LLM answer and the chunk) contradict one another, entail one another, or are neutral. Based on these predictions, MyGPT calculates the Answer Relevance Score (ARS) and the Hallucination Index (HI), in addition to BM25 scores and vector distances. • Vector space distance function (i.e., semantic similarity search metric): To identify relevant chunks, MyGPT users can choose from three distance functions: Euclidian distance, cosine similarity, and Inner product (Methods). • Large language models (LLMs): MyGPT provides a UI for downloading several popular open-source LLMs via Ollama or any of the additional 100+ open-source LLMs using a command-line interface, enabling their local deployment and circumventing the need to connect to proprietary', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 4, 'content': 'pular open-source LLMs via Ollama or any of the additional 100+ open-source LLMs using a command-line interface, enabling their local deployment and circumventing the need to connect to proprietary online services8,23. Ollama’s performance depends on the available GPU, making it an essential component for running MyGPT. We performed comprehensive benchmarking to evaluate several LLMs in terms of ideal hosting environment, minimum requirements, average speed of response to a user query, and cost estimates of hosting in a cloud infrastructure (Supplementary Tables 1–3). For example,', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 5, 'content': '5 relatively small Llama3:8b and Llama2:7b24 (~8 billion and ~7 billion parameters, respectively) can run on a machine with a single GPU and generate answers quickly. In contrast, a large LLM such as Llama3:70b (~70 billion) must be hosted on a machine with a powerful GPU for optimal performance but can generate more detailed answers. Users can also interact directly with the LLMs by turning off the RAG pipeline through the UI. • Creativity vs precision and prompting: LLM models generate text by predicting the next words in the sequence, and this can be customized by adjusting specific LLM parameters : top-K (the most probable words to consider for the prediction), top-P (the threshold of probabilities of words), and temperature (the randomness of words selected by top-P and top-K). In this way, users can control the creativity or precision of the responses generated by the LLM. Users can also limit the output from the LLM by changing the default prompt through the UI, e.', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 5, 'content': 'p-K). In this way, users can control the creativity or precision of the responses generated by the LLM. Users can also limit the output from the LLM by changing the default prompt through the UI, e.g. answer in a single word (yes/no), to restrict creativity. • Model context protocol (MCP): This open standard developed by Anthropic25 facilitates integration of LLM applications with external public or private tools and data. MyGPT offers a basic implementation of the MCP server and client, enabling its integration with other private LLM applications and data sources hosted within the same AI ecosystem (Supplementary Information). Confidence metrics In RAG applications, the risk of hallucinations increases if relevant information is missing from the documents; users must therefore evaluate the answers provided by the LLM and verify their sources to ensure accuracy. To address this challenge, we developed and implemented ‘relevance scores’ both for user queries and MyGPT-', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 5, 'content': 'fore evaluate the answers provided by the LLM and verify their sources to ensure accuracy. To address this challenge, we developed and implemented ‘relevance scores’ both for user queries and MyGPT-generated answers, as well as a ‘hallucination index’. MyGPT provides built-in functionality to calculate these metrics, which can be further optimized for specific use cases. We define these metrics below: • Question Relevance Score (QRS): provided when the user submits the question, it serves as a metric to assess whether the user-provided library may have the necessary information to respond correctly to the question asked. To calculate Question Relevance Score (QRS), we define QRS as 𝑄𝑅𝑆= &𝑎 × ∑ 𝑄!"#,% & %’( 𝑘 ++&𝑏 × ∑ 𝑄&"),% & %’( 𝑘 ++&𝑐 × ∑ 𝑄*"*+,&,% & %’( 𝑘 + (1) where Qsem,i is the Semantic score, Qkey,i is the Keyword score and Qrerank,i is the Rerank score for the ith chunk, and a, b and c are weighting factors for the 3 contributions (domain-specific a,b,c can', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 5, 'content': 'Qsem,i is the Semantic score, Qkey,i is the Keyword score and Qrerank,i is the Rerank score for the ith chunk, and a, b and c are weighting factors for the 3 contributions (domain-specific a,b,c can be determined by global sensitivity analysis of a reference curated Q&A dataset, see Methods; Finding the optimal settings for QRS, ARS, and HI). k denotes the number closest retrieved chunks after applying thresholds from semantic search, keyword search and reranking (see Methods). The semantic score Qsem, i for chunk i is given by, 𝑄!"#,%=1− 𝑄𝐶𝑠𝑒𝑚% −𝑄𝐶𝑠𝑒𝑚-"!. 𝑄𝐶𝑠𝑒𝑚/0*!.−𝑄𝐶𝑠𝑒𝑚-"!. (2) where QCsem,i is the distance between the embeddings of the question and the of chunk i in the vector database. The normalized distance is subtracted from 1 because lower distances correlate to higher semantic similarities. QCsembest and QCsemworst are the empirical highest and lowest QC distances that were observed for 627 questions and retrieved context chunks for the benchmark datase', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 5, 'content': 'o higher semantic similarities. QCsembest and QCsemworst are the empirical highest and lowest QC distances that were observed for 627 questions and retrieved context chunks for the benchmark dataset (details in Supplementary Data). Both vary for different embedding models. The method to calculate these values as well as precalculated values for five populat embedding models are', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 6, 'content': '6 provided for the user. We have calculated QCsembest and QCsemworst for several embedding models and provide resources to calculate them for other embedding models (see Methods). The Keyword score Qkey,i similarly defined as 𝑄&"),%= 𝑄𝐶𝑘𝑒𝑦% −𝑄𝐶𝑘𝑒𝑦-"!. 𝑄𝐶𝑘𝑒𝑦/0*!.−𝑄𝐶𝑘𝑒𝑦-"!. (3) where QCkey,i is the score for the question and the ith chunk calculated by the BM25 algorithm. QCkeybest and QCkeyworst are the empirical highest and lowest QCkey BM25 scores that were observed for 627 questions and retrieved context chunks for the benchmark dataset (details in Supplementary Data). The rerank score Qrerank,i for chunk i (described above) is directly generated by the Reranker model and range from 0 to1. The cross-encoder library provides an option to use the sigmoid function to produce min-max-normalized scores between 0 and 1. • Answer Relevance Score (ARS): measures the extent to which MyGPT used the context (i.e., source material) retrieved from the documents to g', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 6, 'content': 'to produce min-max-normalized scores between 0 and 1. • Answer Relevance Score (ARS): measures the extent to which MyGPT used the context (i.e., source material) retrieved from the documents to generate the answer. ARS is defined as 𝐴𝑅𝑆= &𝑥 × ∑ 𝐴!"#,% & %’( 𝑘 ++&𝑦 × ∑ 𝐴&"),% & %’( 𝑘 ++&𝑧 × ∑ 𝐴",.+%1,% & %’( 𝑘 + (4) where Asem,i is the Semantic score, Akey,i is the Keyword score and Aentail,i is the Textual entailment score using the Reranker for the ith source. x,y,z are weighting factors for the 3 contributions (domain- specific values can be determined by global sensitivity analysis of a reference curated Q&A dataset, see Methods) Similarly to Qsem,i, Semantic score Asem,i is defined as, 𝐴!"#,%=1− 𝐴𝐶𝑠𝑒𝑚% −𝐴𝐶𝑠𝑒𝑚-"!. 𝐴𝐶𝑠𝑒𝑚/0*!.−𝐴𝐶𝑠𝑒𝑚-"!. (5) where ACsemi is the distance between the embeddings of the question and the ith chunk in the vector database. We subtract the normalized distance from 1 because the lower the distance, the better the semantic similari', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 6, 'content': 'distance between the embeddings of the question and the ith chunk in the vector database. We subtract the normalized distance from 1 because the lower the distance, the better the semantic similarity. ACsembest and ACsemworst are the empirical highest and lowest AC distances that were observed for 627 answers and retrieved context chunks for the benchmark dataset (details in Supplementary Data). Both vary for different embedding models. The method to calculate these values as well as precalculated values for five populat embedding models are provided for the user. Similarly, the Keyword score Akey,i is defined as 𝐴&"),%= 𝐴𝐶𝑘𝑒𝑦% −𝐴𝐶𝑘𝑒𝑦-"!. 𝐴𝐶𝑘𝑒𝑦/0*!.−𝐴𝐶𝑘𝑒𝑦-"!. (6) where ACkeyi is the score for the answer and the ith chunk calculated by the BM25 algorithm. ACkeybest and ACkeyworst are empirical highest and lowest ACkey that were observed for 627 answers and retrieved context chunks for the benchmark dataset (details in Supplementary Data). We calculated Aentail', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 6, 'content': 'ACkeyworst are empirical highest and lowest ACkey that were observed for 627 answers and retrieved context chunks for the benchmark dataset (details in Supplementary Data). We calculated Aentail using a Reranker that provides a textual entailment label for each of the retrieved chunks and the LLM generated answer. This entailment label is either ‘entailment’, ‘contradiction’ or', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 7, 'content': '7 ‘neutral’. To calculate the Aentail score, we converted the label to a score for each chunk, either 1, -1, or 0.5 values for entailment, contradiction, and neutral, respectively, and averaged these values across the k chunks. • Hallucination Index (HI): is a weighted linear combination of QRS and ARS. This score helps users identify likely instances of hallucination and serves as a confidence metric for the accuracy and reliability of the generated answers. In this way, MyGPT can flag likely instances of hallucinations, either because the library is less relevant to the question or because the context retrieved from the document lacks sufficient information for the LLM to generate an accurate response. We defined HI by combining QRS and ARS, given by 𝐻𝐼=1−B𝑝 × 𝑄𝑅𝑆 𝑝+𝑞E−B𝑞 × 𝐴𝑅𝑆 𝑝+𝑞E (7) where p and q are weighting factors that can be determined by global sensitivity analysis on a reference Q&A dataset (see Methods). HI is normalized to [0,1], such that when 𝑄𝑅𝑆', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 7, 'content': '× 𝐴𝑅𝑆 𝑝+𝑞E (7) where p and q are weighting factors that can be determined by global sensitivity analysis on a reference Q&A dataset (see Methods). HI is normalized to [0,1], such that when 𝑄𝑅𝑆 and A𝑅𝑆 are 1, 𝐻𝐼 will be zero (and conversely). 𝐻𝐼 values closer to 1 (or 100%) indicates likely hallucination and values closer to 0 (0%) indicates less hallucination and more certainty in the response. In addition to ensuring that the implemented RAG concept works as expected ( Supplementary Information), we assessed the broader utility of our RAG solution by comparing the performance of MYGPT against other RAG solutions. Specifically, we benchmarked MyGPT and its confidence metrics using three publicly available benchmarking datasets as described below. Benchmarking MyGPT and other open-source RAG solutions for public datasets We directly compared the performance of MyGPT with that of four open-source RAG solutions: RAGFlow, Dify, PrivateGPT, and Kotaemon, using three p', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 7, 'content': 'ther open-source RAG solutions for public datasets We directly compared the performance of MyGPT with that of four open-source RAG solutions: RAGFlow, Dify, PrivateGPT, and Kotaemon, using three public evaluation datasets: PubMedQA26, BioASQ27, and Open-rag-bench. To do so, we collected 294 questions (294 documents) from PubMedQA (a biomedical question-answering dataset collected from PubMed abstracts), 163 questions (397 documents) from BioASQ (a manually curated evaluation dataset on large-scale biomedical semantic indexing and question answering), and 170 questions (165 documents) from the Open RAG Benchmark datasets designed to evaluate RAG systems using arXiv PDFs (Methods). MyGPT achieved 81.6%, 91.4%, and 92.9% accuracy on PubMedQA, BioASQ, and Open RAG Benchmark, respectively (Fig. 4a), outperforming RAGFlow (77.5%, 90.2% and 90% across the 3 datasets), Dify (68%, 65.6% and 78.8%), PrivateGPT (51%, 74.8% and 78.2%) and Kotaemon (45.6%, 76.1% and 55.8%) in both a', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 7, 'content': 'ectively (Fig. 4a), outperforming RAGFlow (77.5%, 90.2% and 90% across the 3 datasets), Dify (68%, 65.6% and 78.8%), PrivateGPT (51%, 74.8% and 78.2%) and Kotaemon (45.6%, 76.1% and 55.8%) in both accuracy, document processing, and source highlighting. The confidence metrics (QRS, ARS, and HI) obtained using MyGPT differentiated between correct and incorrect answers and conclusively identified instances of hallucination (Fig. 4b-d). We observed that both QRS (p = 4x10-3; effect size = 0.11 (biserial rank correlation r, small); Mann Whitney U test) and ARS (p = 1x10-5; effect size = 0.17 (r, small); Mann Whitney U test) were significantly higher for relevant questions and correct answers, and HI (p = 1x10-5; effect size = 0.17 (r, small); Mann Whitney U test) was significantly higher for irrelevant questions and incorrect answers. These findings demonstrate that MyGPT (and the confidence metrics) works as anticipated and that MyGPT outperforms the open -source RAG solutions RAG', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 7, 'content': 'for irrelevant questions and incorrect answers. These findings demonstrate that MyGPT (and the confidence metrics) works as anticipated and that MyGPT outperforms the open -source RAG solutions RAGFlow, Dify, PrivateGPT, and Kotaemon by generating more correct answers in our benchmark. There is growing evidence and concern that LLMs that are trained on public data, including research articles28, making it challenging to determine which specific publications were included in the training datasets. To address this concern, rather than evaluating the LLM as an independent component, we evaluated the entire RAG pipeline of MyGPT with other open-source RAG platforms with the same LLM. Due to our limited access to pdf versions of published articles and the manual effort required for validation,', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 7, 'content': 'validation,', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 8, 'content': '8 we used subsets of these datasets. However, power calculations were used to guide and interpret the benchmarking sample sizes (Methods), and both the number of Q&A pairs and the document count reflect those considerations. Real-world use cases and performance of MyGPT using public and confidential, private documents To highlight real world applications of MyGPT in the different domains, we describe three examples of use cases in clinical sciences, basic sciences, and global health at St. Jude Children’s Research Hospital as below: Private clinical documents: MyGPT can be securely deployed within hosting environments that require private and confidential protocols, such as those maintained within hospitals or clinical settings. For example, we implemented MyGPT for the Department of Bone Marrow Transplantation and Cellular Therapy (BMTCT) at St. Jude Children’s Research Hospital (Methods) to allow staff to easily access and query Standard Operating Procedures (SOP) do', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 8, 'content': 'rtment of Bone Marrow Transplantation and Cellular Therapy (BMTCT) at St. Jude Children’s Research Hospital (Methods) to allow staff to easily access and query Standard Operating Procedures (SOP) documents. Doctors and nurses use d MyGPT to query specific information from a library of SOPs (Figure 5) and verified the MyGPT responses against the original documents for validating the responses. Since SOPs are meticulously prepared and approved through a rigorous process at St. Jude, responses generated by the language model should be accurate, institution- specific, and verifiable with source citations. MyGPT showed 94.85% accuracy in answering the questions for the evaluation dataset we prepared for this use case, with significantly higher relevance scores for correct answers than for incorrect answers and a lower hallucination index (p = 1.4x10-4; effect size = 0.33 (r, medium); Mann Whitney U test). MyGPT may also be utilized for other confidential clinical documents, such as', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 8, 'content': 'r incorrect answers and a lower hallucination index (p = 1.4x10-4; effect size = 0.33 (r, medium); Mann Whitney U test). MyGPT may also be utilized for other confidential clinical documents, such as patient reports and Electronic Health Records (EHRs). Biomedical literature: MyGPT is useful for querying biomedical literature, particularly for exploring new literature that is not included in LLM training or for querying bespoke libraries of published articles collected by research groups. MyGPT was implemented in the BlueSky Kinase project of the Structural Biology department at St. Jude to test its ability to retrieve technical and highly specific details of biochemical assays and expression systems that monitor substrate phosphorylation and kinase activity and to determine whether these effects were referenced in any figure or table descriptions. To validate its performance, we compiled 83 biomedical publications for 26 different protein kinases and created 282 Q&A pairs wi', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 8, 'content': 'er these effects were referenced in any figure or table descriptions. To validate its performance, we compiled 83 biomedical publications for 26 different protein kinases and created 282 Q&A pairs with input from experts in the field (principal and senior scientists working on kinases). The answers produced by MyGPT were reviewed by a biocurator, who reported 73% (206/282) accuracy. When we repeated the experiment with 15 chunks per question rather than the default value of 5 chunks, we obtained 21 more correct answers with 80.5% (227/282) accuracy with the same customization showing significant improvement (p = 0.7x10-2; effect size = 0.34 (phi coefficient F, medium); McNemar chi-square test). We also repeated the experiment with structure-preserving chunking and 5 chunks, obtaining 75.8% (214/282) accuracy, which did not provide a significant improvement (p = 0.27; effect size=0.15 (F, small); McNemar chi-square test) over fixed chunking. Although structure-preserving chunki', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 8, 'content': 'g 75.8% (214/282) accuracy, which did not provide a significant improvement (p = 0.27; effect size=0.15 (F, small); McNemar chi-square test) over fixed chunking. Although structure-preserving chunking does not provide improvement here, it should be useful when the user wants to focus on a specific section or avoid specific sections while generating a response. For publications containing the requisite information, MyGPT provided highly relevant answers with minimal hallucination, with exact paragraphs highlighted and a significantly lower hallucination index for correct than for incorrect answers (Supplementary Fig. 4; Methods). Moreover, the MyGPT pipeline completed the task in less than 2 hours, a process that would have taken a biocurator approximately 30-40 hours to finish (about 25-30 minutes per article) by reading and extracting data from 83 articles. Thus, MyGPT’s confidence metrics and source-surfacing capability enable researchers to assess the trustworthiness and a', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 8, 'content': 'ut 25-30 minutes per article) by reading and extracting data from 83 articles. Thus, MyGPT’s confidence metrics and source-surfacing capability enable researchers to assess the trustworthiness and accuracy of LLM-generated answers while keeping the knowledge base constrained to a selected library or even to a single document. Multi-lingual global health policies: MyGPT allows users to interact with documents in various languages without concerns about language compatibility. This is possible due to its modular design, which enables users to switch between different embedding models and BM25 algorithms, including those trained on', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 9, 'content': '9 multiple languages, such as bge-m3, Paraphrase-Multilingual, and nomic-embed-text-v2-moe. When a user uploads a document in a language other than English, the embedding model can categorize the content alongside other chunks with similar meanings, irrespective of the original language. Although a user may not be familiar with the context in an unfamiliar language, the language model used to generate the final answer can recognize and translate it to the language in which the user asks their question (Methods). This can be especially beneficial where rapid and accurate data extraction and interpretation are warranted from length and compled document sets such as national health policies. Such documents guide critical decision making and funding, and yet are often laden with jargon, interdependencies across sections, and where interpretation is highly dependent on country, context and subtext. We leveraged this capability to implement MyGPT with the department of Global Pedi', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 9, 'content': 'n, interdependencies across sections, and where interpretation is highly dependent on country, context and subtext. We leveraged this capability to implement MyGPT with the department of Global Pediatric Medicine, which has developed a Policy Monitor with national health policy documents from 197 countries and territories around the world, in 46 different languages. To validate the use case, we compiled 20 health policies: 10 each for Spanish and French. We comipled 16 questions to evauate each policy across five topics commonly queried by ministry teams and counterparts in national policy planning. These include topics in health and non-communicable diseases in children, pediatric and adolescent cancer types, existence of a staffed national cancer program, access to vaccination to prevent cervical cancer in adults and access to essential imaging (Methods and Supplementary Data). The answers produced by MyGPT were independently reviewed by an expert team of health policy an', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 9, 'content': 'o prevent cervical cancer in adults and access to essential imaging (Methods and Supplementary Data). The answers produced by MyGPT were independently reviewed by an expert team of health policy analysts with the department of St. Jude Global Pediatric Medicine using structured rubrics and tools, and found to be 80% (128/160 ) accurate for Spanish and 88% (141/160) for French (Supplementary Fig. 5). Machine learning-based optimization of hallucination index As we observed significantly higher HI scores for incorrectly answered questions in the 3 real -world examples used to assess MyGPT, we asked whether the information contained in ARS/QRS could be used to train a machine learning classifier to explicitly predict whether answers are likely to be correct or incorrect. To achieve this, we trained a random forest algorithm using 20 input features derived from QRS and ARS calculations (Methods). We observed 87.35% (±0.4%) train and 74.17% (±4.16%) test ROC-AUC (Supplementar', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 9, 'content': 've this, we trained a random forest algorithm using 20 input features derived from QRS and ARS calculations (Methods). We observed 87.35% (±0.4%) train and 74.17% (±4.16%) test ROC-AUC (Supplementary Fig. 6) for a random forest algorithm on 1045 Q&A, combining public and private datasets (80:20 train-test split; 10 replicates; Methods). We also observed that this optimized HI score showed marked improvement in separating correct from incorrect responses for 3 public datasets (p = 2.2´10-16; effect size = 0.38 (r, medium); Mann Whitney U test) and private datasets: Bone Marrow Transplantation SOPs (p = 1.4´10-4; effect size = 0.33 (r, medium); Mann Whitney) and Protein Kinases (p = 8.8´10-16; effect size = 0.59 (r, large); Mann Whitney). While we present the Random Forest model as the default to predict HI, users can either retrain the model using the scripts that we provide and replace it in the source code or can train their own model with their own custom settings and librar', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 9, 'content': 'e default to predict HI, users can either retrain the model using the scripts that we provide and replace it in the source code or can train their own model with their own custom settings and libraries to obtain optimized HI scores. The latter option requires generating a document library and at least 130 domain-specific questions and answers, each with correct vs. incorrect labels (see Methods). LLM hallucination can be independently evaluated by other methods such as SelfCheckGPT 29 and semantic entropy30 through generation of multiple responses. However, these are not well-suited for RAG contexts, as SelfCheckGPT and semantic entropy both require substantial GPU resources, and SelfCheckGPT is ineffective for smaller LLMs such as GPT-3 and Llama29. Moreover, if the correct context is provided, even smaller LLMs can produce accurate answers. Thus, by providing QRS, ARS and HI metrics and explicit source citing, MyGPT offers a comprehensive and generalizable method to quant', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 9, 'content': 'ext is provided, even smaller LLMs can produce accurate answers. Thus, by providing QRS, ARS and HI metrics and explicit source citing, MyGPT offers a comprehensive and generalizable method to quantify the correctness of LLM-generated answers, thereby increasing the reliability and transparency of RAG applications. We anticipate that MyGPT can be used across range of environments, including routine tasks in hospitals, human resources, administration, legal fields, and commercial contexts or by students and the general public, providing a comprehensive, end-to-end, seamless user experience from a single browser window (Figure 6).', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 10, 'content': '10 Considerations and limitations MyGPT uses the textual content from documents to create a vector database, and it is currently unable to interpret tabular data, figures, mathematical equations, formulas, or chemical structures. This limitation can be addressed in the future with multimodal LLMs (e.g., text + images using Gemme3; text + audio using Gemma4). The open-source nature of MyGPT makes such future extensions easily implementable by the community. Currently, the user needs to provide the documents to create the library, as MyGPT does not automatically retrieve all relevant publications on a topic of interest; if supporting or contradicting information is absent from the user-provided library and/or was not included in the LLM training data, MyGPT could generate a biased response by citing only the available source in the library. While ARS, QRS, and HI metrics can help identify instances of hallucination, they do not eliminate them and may need to be optimized t', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 10, 'content': 'response by citing only the available source in the library. While ARS, QRS, and HI metrics can help identify instances of hallucination, they do not eliminate them and may need to be optimized to distinguish between correct and incorrect responses for specific libraries and scientific domains. These metrics rely on the properties of user input, such as question type and clarity and on MyGPT parameters such as the embedding model used. In addition, users need to install MyGPT on an individual device for personal use and on Edge AI devices, a server, or a secure cloud for institutional use, requiring maintenance as an internal web resource. The size of the library can influence the number of chunks per library and impact the uploading time, but it does not affect the duration of different steps in real-time Q&A once uploading is complete (Supplementary Table 4). In our ongoing efforts, we also observed that advanced semantic chunking, which is computationally expensive, does', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 10, 'content': 'ifferent steps in real-time Q&A once uploading is complete (Supplementary Table 4). In our ongoing efforts, we also observed that advanced semantic chunking, which is computationally expensive, does not provide a substantial performance improvement31. Despite these considerations, local installation capabilities allow users to customize the pipeline according to their requirements, to securely deploy it within a firewall, eliminates the need to send confidential information to third parties, minimizes data leakage, and makes MyGPT more affordable than commercial cloud-hosted LLM products for individuals, research groups, and organizations. DISCUSSION MyGPT is an open-source, easy to install, portable, and customizable RAG platform that provides confidence metrics based on context engineering and a machine learning algorithm to infer correctness of LLM-generated answers. MyGPT’s implementation of RAG concepts consistently delivers above 80% accuracy across public and priva', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 10, 'content': 'xt engineering and a machine learning algorithm to infer correctness of LLM-generated answers. MyGPT’s implementation of RAG concepts consistently delivers above 80% accuracy across public and private datasets, and its unique modular framework enables users to both integrate the latest LLMs and embedding models and to perform multilingual Q&A in their preferred language, thereby minimizing language barriers to information access. Users can search their private, curated document library and obtain reliable answers from multiple documents simultaneously. Thus, MyGPT fills a crucial gap in knowledge discovery tools by enabling users to efficiently extract verified insights from multiple private documents. For all public and private datasets, we evaluated MyGPT with various customizations and selected the best- performing setting (Methods). To identify the optimal combination of MyGPT components for a user’s own library or use case (LLMs, embedding model, distance function), we', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 10, 'content': 'tions and selected the best- performing setting (Methods). To identify the optimal combination of MyGPT components for a user’s own library or use case (LLMs, embedding model, distance function), we suggest selecting top-performing LLMs from leaderboards such as Chatbot Arena32,33 and embedding models from the Massive Text Embedding Benchmark (MTEB) Leaderboard20,21 recommended by sentence transformer studies 34. For other customizations, domain-specific LLMs or embedding models beyond the leaderboards, users can also utilize automated evaluation frameworks such as Retrieval -Augmented Generation Assessment (Ragas)35,36 with MyGPT APIs. We note that MyGPT includes solutions similar to those offered by commercial products such as ChatPDF and NotebookLM or by open-source products such as PrivateGPT, Kotaemon, RAGFlow, Dify, Verba, LlamaIndex, and BioChatter. However, in contrast to those products, MyGPT is a complete, end-to-end solution built with open-source tools that are', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 10, 'content': 'uch as PrivateGPT, Kotaemon, RAGFlow, Dify, Verba, LlamaIndex, and BioChatter. However, in contrast to those products, MyGPT is a complete, end-to-end solution built with open-source tools that are small enough for local installation on a laptop/personal computer, Edge AI devices, institutional servers/VMs, or the cloud. MyGPT can be installed and initiated by a script orchestrating prebuilt Docker images, eliminating complex debugging and environment', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 11, 'content': '11 dependencies for less tech-savvy users. When compared to end-to-end solutions such as PrivateGPT, Kotaemon, RAGFlow, and Dify, MyGPT offers greater accuracy and broader range of features, including confidence metrics, library management, and document highlighting (Figure 6b). MyGPT also offers users complete control over their data. In research , healthcare, and industry, a substantial amount of knowledge is locked in private documents, necessitating a solution for a contextual, retrievable, and scalable knowledge base. Future RAG development with agentic architecture must access this private knowledge based on user permissions. Agentic workflows can effectively integrate MyGPT’s’ confidence metrics to filter out low-relevance references and to rephrase less relevant questions, thereby enhancing decision-making and improving accuracy. Moreover, the parameters collected during each Q&A session are used by MyGPT to train a low-latency machine learning model to improve co', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 11, 'content': 'ns, thereby enhancing decision-making and improving accuracy. Moreover, the parameters collected during each Q&A session are used by MyGPT to train a low-latency machine learning model to improve confidence metrics, which can be further refined based on the types of questions in different domains. Current commercial Deep Research applications either do not support private documents or they require a subscription to commercial services. In contrast, MyGPT offers an open-source API with built-in user access control that enhances agentic workflows, as well as Model Context Protocol (MCP) to enhance ongoing efforts such as Open Deep Research37 and multi-agent systems for healthcare (MASH)38. With its ability to run on Edge AI devices such as Nvidia Orin Nano (Supplementary Fig. 7) and to handle documents offline, MyGPT provides a cost-effective alternative to commercial LLM services, democratizing access to reliable knowledge-seeking tasks, particularly in remote medical and r', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 11, 'content': 'nd to handle documents offline, MyGPT provides a cost-effective alternative to commercial LLM services, democratizing access to reliable knowledge-seeking tasks, particularly in remote medical and research facilities without internet access. As vector embeddings for specific libraries are independent of GPT models, sharing pre- made MyGPT libraries with the community may accelerate and broaden opportunities for knowledge seeking and scientific discovery. This capability empowers researchers to use publications from open research initiatives such as PubMed Central39 or openRxiv40, to create private and open-source instances of AI-powered literature research tools such as Consensus41 or essential RAG-component of specialized LLM-based agentic tools such as CRISPR-GPT42 and PrimeGen43. As we look to the future of generative AI, personalized GPT products are increasingly likely to become a reality. An open-source, privacy-focused retrieval-augmented generation framework presents a', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 11, 'content': 'As we look to the future of generative AI, personalized GPT products are increasingly likely to become a reality. An open-source, privacy-focused retrieval-augmented generation framework presents a compelling path for generative AI that balances accessibility, accountability, and user autonomy. REFERENCES 1 Cascella, M., Montomoli, J., Bellini, V. & Bignami, E. Evaluating the Feasibility of ChatGPT in Healthcare: An Analysis of Multiple Clinical and Research Scenarios. J Med Syst 47, 33 (2023). https://doi.org/10.1007/s10916-023-01925-4 2 Eloundou, T., Manning, S., Mishkin, P. & Rock, D. GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models. ArXiv abs/2303.10130 (2023). 3 Alkaissi, H. & McFarlane, S. I. Artificial Hallucinations in ChatGPT: Implications in Scientific Writing. Cureus 15, e35179 (2023). https://doi.org/10.7759/cureus.35179 4 OpenAI. How does OpenAI use my personal data?, https://openai.com/policies/row-privacy- pol', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 11, 'content': 'tGPT: Implications in Scientific Writing. Cureus 15, e35179 (2023). https://doi.org/10.7759/cureus.35179 4 OpenAI. How does OpenAI use my personal data?, https://openai.com/policies/row-privacy- policy/ (2024). 5 Kwong, J. C. C., Wang, S. C. Y., Nickel, G. C., Cacciamani, G. E. & Kvedar, J. C. The long but necessary road to responsible use of large language models in healthcare research. NPJ Digit Med 7, 177 (2024). https://doi.org/10.1038/s41746-024-01180-y 6 Lin, Z. Techniques for supercharging academic writing with generative AI. Nat Biomed Eng 9, 426–431 (2025). https://doi.org/10.1038/s41551-024-01185-8 7 Gemini. Gemini Apps Privacy Notice, https://support.google.com/gemini/answer/13594961?visit_id=638586578410247935- 92666435&p=privacy_notice&rd=1#privacy_notice (2024). 8 Hutson, M. Forget ChatGPT: why researchers now run small AIs on their laptops. Nature, Technology Feature (2024). 9 Xu, R., Feng, Y. & Chen, H. ChatGPT vs. Google: A Comparative Study of Search Perf', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 11, 'content': 'utson, M. Forget ChatGPT: why researchers now run small AIs on their laptops. Nature, Technology Feature (2024). 9 Xu, R., Feng, Y. & Chen, H. ChatGPT vs. Google: A Comparative Study of Search Performance and User Experience. ArXiv abs/2307.01135 (2023).', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 12, 'content': '12 10 Lewis, P. et al. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. ArXiv abs/2005.11401 (2020). 11 Nowogrodzki, J. ChatGPT for science: how to talk to your data. Nature 631, 924–925 (2024). https://doi.org/10.1038/d41586-024-02386-6 12 Zotero.org. https://www.zotero.org/ 13 Kamphuis, C., de Vries, A. P., Boytsov, L. & Lin, J. Which BM25 Do You Mean? A Large-Scale Reproducibility Study of Scoring Variants. Advances in Information Retrieval, Ecir 2020, Pt Ii 12036, 28–34 (2020). https://doi.org/10.1007/978-3-030-45442-5_4 14 AWS. What are embeddings in machine learning?, https://aws.amazon.com/what-is/embeddings- in-machine-learning/ 15 Jing, Z. et al. When Large Language Models Meet Vector Databases: A Survey. arXiv preprint arXiv:2402.01763 (2024). 16 Reimers, N. & Gurevych, I. Sentence-BERT: Sentence Embeddings using Siamese BERT- Networks. 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 12, 'content': 'eimers, N. & Gurevych, I. Sentence-BERT: Sentence Embeddings using Siamese BERT- Networks. 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (Emnlp-Ijcnlp 2019), 3982–3992 (2019). 17 Omar, M., Ullanat, V., Loda, M., Marchionni, L. & Umeton, R. ChatGPT for digital pathology research. The Lancet Digital Health 6, e595–e600 (2024). https://doi.org/10.1016/S2589- 7500(24)00114-6 18 Peter, A. What Chunk Size and Chunk Overlap Should You Use?, https://dev.to/peterabel/what- chunk-size-and-chunk-overlap-should-you-use-4338 (2023). 19 Hoang, L. How Chunk Sizes Affect Semantic Retrieval Results, https://ai.plainenglish.io/investigating-chunk-size-on-semantic-results-b465867d8ca1 20 Muennighoff, N., Tazi, N., Magne, L. & Reimers, N. MTEB: Massive text embedding benchmark. arXiv preprint arXiv:2210.07316 (2022). 21 Massive Text Embedding Benchmark (MTEB) Leaderboard, https://huggingface.co/spa', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 12, 'content': 'Tazi, N., Magne, L. & Reimers, N. MTEB: Massive text embedding benchmark. arXiv preprint arXiv:2210.07316 (2022). 21 Massive Text Embedding Benchmark (MTEB) Leaderboard, https://huggingface.co/spaces/mteb/leaderboard 22 Lù, X. H. BM25S: Orders of magnitude faster lexical search via eager sparse scoring. arXiv (2024). https://doi.org/https://doi.org/10.48550/arXiv.2407.03618 23 Jeffery Morgan, M. C. Ollama, https://ollama.com/ (2021). 24 Touvron, H. et al. Llama 2: Open Foundation and Fine-Tuned Chat Models. ArXiv abs/2307.09288 (2023). 25 Model Context Protocol (MCP) (GitHub https://github.com/modelcontextprotocol, 2024). 26 Jin, Q., Dhingra, B., Liu, Z. P., Cohen, W. W. & Lu, X. H. PubMedQA: A Dataset for Biomedical Research Question Answering. 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (Emnlp- Ijcnlp 2019), 2567–2577 (2019). 27 Balikas, G., Krithara, A., Partalas, I. & P', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 12, 'content': 'thods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (Emnlp- Ijcnlp 2019), 2567–2577 (2019). 27 Balikas, G., Krithara, A., Partalas, I. & Paliouras, G. BioASQ: A Challenge on Large-Scale Biomedical Semantic Indexing and Question Answering. Lect Notes Comput Sc 9059, 26–39 (2015). https://doi.org/10.1007/978-3-319-24471-6_3 28 Gibney, E. Has your paper been used to train an AI model? Almost certainly. Nature 632, 715– 716 (2024). https://doi.org//10.1038/d41586-024-02599-9 29 Manakul, P., Liusie, A. & Gales, M. J. F. SELFCHECKGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models. 2023 Conference on Empirical Methods in Natural Language Processing (Emnlp 2023), 9004–9017 (2023). 30 Farquhar, S., Kossen, J., Kuhn, L. & Gal, Y. Detecting hallucinations in large language models using semantic entropy. Nature 630 (2024). https://doi.org/10.1038/s41586-024-07421-0 31 Qu, R., Tu, R. & Bao', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 12, 'content': 'ar, S., Kossen, J., Kuhn, L. & Gal, Y. Detecting hallucinations in large language models using semantic entropy. Nature 630 (2024). https://doi.org/10.1038/s41586-024-07421-0 31 Qu, R., Tu, R. & Bao, F. S. Is Semantic Chunking Worth the Computational Cost? Findings of the Association for Computational Linguistics: NAACL 2025, pages 2155–2177, Albuquerque, New Mexico. Association for Computational Linguistics. 32 LMSYS Chatbot Arena Leaderboard, https://lmarena.ai/?leaderboard 33 Chiang, W.-L. et al. Chatbot arena: An open platform for evaluating llms by human preference. arXiv preprint arXiv:2403.04132 (2024). 34 Pretrained Models https://www.sbert.net/docs/sentence_transformer/pretrained_models.html 35 Ragas (RAG ASsement) https://docs.ragas.io/en/stable/index.html 36 Es, S., James, J., Espinosa-Anke, L. & Schockaert, S. Ragas: Automated evaluation of retrieval augmented generation. arXiv preprint arXiv:2309.15217 (2023). 37 Open Deep Research (GitHub https://github.com', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 12, 'content': 'James, J., Espinosa-Anke, L. & Schockaert, S. Ragas: Automated evaluation of retrieval augmented generation. arXiv preprint arXiv:2309.15217 (2023). 37 Open Deep Research (GitHub https://github.com/langchain-ai/open_deep_research).', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 13, 'content': '13 38 Moritz, M., Topol, E. & Rajpurkar, P. Coordinated AI agents for advancing healthcare. Nat Biomed Eng 9, 432–438 (2025). https://doi.org/10.1038/s41551-025-01363-2 39 Central, P. PubMed Central, https://pmc.ncbi.nlm.nih.gov/ (2000). 40 OpenRxiv. OpenRxiv, https://openrxiv.org/ (2025). 41 consensus. Consensus - AI-powered search engine for scientific research papers, https://consensus.app/home/about-us/ (2021). 42 Qu, Y. et al. CRISPR-GPT for agentic automation of gene-editing experiments. Nat Biomed Eng (2025). https://doi.org/10.1038/s41551-025-01463-z 43 Wang, Y. et al. Accelerating primer design for amplicon sequencing using large language model- powered agents. Nat Biomed Eng (2025). https://doi.org/10.1038/s41551-025-01455-z 44 Trychroma.com. https://www.trychroma.com/ 45 Lopez, P. GROBID: Combining Automatic Bibliographic Data Recognition and Term Extraction for Scholarship Publications. Research and Advanced Technology for Digital Libraries, Proceedings 5714,', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 13, 'content': '45 Lopez, P. GROBID: Combining Automatic Bibliographic Data Recognition and Term Extraction for Scholarship Publications. Research and Advanced Technology for Digital Libraries, Proceedings 5714, 473–474 (2009). 46 Breiman, L. Random forests. Mach Learn 45, 5–32 (2001). https://doi.org/Doi 10.1023/A:1010933404324 47 Pedregosa, F. et al. Scikit-learn: Machine Learning in Python. J Mach Learn Res 12, 2825–2830 (2011). 48 Lundberg, S. M. & Lee, S. I. A Unified Approach to Interpreting Model Predictions. Advances in Neural Information Processing Systems 30 (Nips 2017) 30 (2017).', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 15, 'content': '15 Figure 1. Overview of the MyGPT workflow. a Top-level view. Blue arrows represent processes followed by user-uploaded documents. Documents are pre-processed, converted from text into chunks, and stored in the backend for “context engineering”. Orange arrows show processes followed by user queries. In the real-time Q&A step, queries are compared against a knowledge base created during pre-processing, appropriate chunks identified as the “golden” context are supplied to the LLM as context for the user query to produce a response (green arrows). The pipeline provides the user with an explicit context source from the library with the generated answer and relevancy scores for question (QRS), answer (ARS), and hallucination index (HI) to identify likely instances of hallucination in the response. b In-depth view of MyGPT context engineering. After document chunks are prepared (blue arrow), they are stored in a BM25 (Best Matching 25)-tokenized library for keyword search and', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 15, 'content': 'n the response. b In-depth view of MyGPT context engineering. After document chunks are prepared (blue arrow), they are stored in a BM25 (Best Matching 25)-tokenized library for keyword search and in an embedding-based vector library for semantic search. During the Q&A step (orange arrow), the user\'s query is tokenized to retrieve the top BM25-scored chunks (keyword based retrieval). In parallel, the query is embedded to find the closest vectors (chunk) using Euclidean distance (semantic search based retrieval). The top n candidates from each method are selected and are passed to a Reranker (cross-encoder), which assigns a relevance score to each chunk based on both the query and chunk content and filters out chunks with low scores, resulting in l chunks from keyword search and m chunks from semantic search. The top k scored and re-ranked chunks, with duplicates merged, serve as the "golden context", and are supplied to the LLM to generate the response. QRS (Query Relevance', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 15, 'content': 'chunks from semantic search. The top k scored and re-ranked chunks, with duplicates merged, serve as the "golden context", and are supplied to the LLM to generate the response. QRS (Query Relevance Score) is calculated using Euclidean distances from vector search, BM25 scores from keyword search, and reranking scores from the cross -encoder. For correctness prediction (green arrow), the same process is repeated for k chunks using BM25 and vector embeddings but compared with LLM response rather than the user’s query. For this step, we used a different Reranker to assess how well the “golden context" supports or contradicts the generated answer, resulting in the Answer Relevance Score (ARS) and Hallucination Index (HI).', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 16, 'content': '16 Figure 2. MyGPT user interface. The MyGPT user interface presents three panels. The left-most panel is used for real-time Q&A and displays the user query with a question relevance score (QRS), the LLM generated answer with an answer relevance score (ARS), and a hallucination index (HI). The answer is also accompanied by sources, including the document and specific paragraphs (with exact page numbers) used to generate the answers. Clicking on the source activates the document from the middle panel, which displays all the documents from the library, while the right-most panel displays the document page and the highlighted context. The sources with a green background and green highlights correspond to highly relevant context for the user’s questions and the LLM-generated answer, and medium-relevant sources are displayed with an amber background and yellow highlights, helping users to find pages and paragraphs with high significance.', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 16, 'content': 'r, and medium-relevant sources are displayed with an amber background and yellow highlights, helping users to find pages and paragraphs with high significance.', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 17, 'content': '17 Figure 3. MyGPT pipeline customizations. Users can modify the MyGPT pipeline via the user interface at all three steps of the MyGPT workflow: pre-processing (blue), real-time Q&A (orange), and generation (green). Colored arrows represent the progression of each step. The dark blue dots indicate customization features such as chunking method, embedding models, vector and keyword search methods, Rerankers, relevancy scores, LLMs, creativity parameters and MCP (model context protocol; Supplementary Information) capabilities. Dotted lines indicate the customization options at individual steps, with the range of options presented in the connected boxes in the lower panel.', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 18, 'content': '18 Figure 4. Benchmarking MyGPT with other open-source RAG solutions using public datasets. a Frequency of answer correctness for MyGPT versus other open-source RAG solutions. A Q&A dataset was collected using three publicly available evaluation datasets: PubMedQA, BioASQ, and Open RAG Benchmark. Using the ground truth provided by the dataset, answers were manually evaluated as “Correct” (dark grey/blue) or “Incorrect” (light grey/orange). A total of 294 questions (294 documents) were collected from PubMedQA, 163 questions (397 documents) from BioASQ, and 170 questions (165 documents) from Open RAG Benchmark. MyGPT performs better than RAGFlow and significantly better than Dify, PrivateGPT and Kotaemon, achieving more than 80% accuracy across all three datasets. b MyGPT’s Question relevancy score (QRS) distribution for questions of correct and incorrect answers (p = 4x10-3; effect size = 0.11 (biserial rank correlation r, small); Mann Whitney U test) for all three da', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 18, 'content': 'Question relevancy score (QRS) distribution for questions of correct and incorrect answers (p = 4x10-3; effect size = 0.11 (biserial rank correlation r, small); Mann Whitney U test) for all three datasets combined shows higher relevance for questions that were answered correctly. c MyGPT’s Answer relevancy score (ARS) distribution for correct and incorrect answers (p = 1x10-5; effect size = 0.17 (r, small); Mann Whitney U test) combined for all three datasets shows higher scores for the correct than for incorrect answers. d MyGPT’s default Hallucination Index (HI) for correct and incorrect answers using arithmetic means between QRS and ARS (p = 1x10-5; effect size = 0.17 (r, small); Mann Whitney U test) combined for all three datasets shows significantly higher value for the hallucination index for incorrect than for correct answers.', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 18, 'content': 'ination index for incorrect than for correct answers.', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 19, 'content': '19 Figure 5. Use case: Standard Operating Procedures (SOPs) for Bone Marrow Transplantation. a Example of a question and answer using the library of private SOP documents from the Bone Marrow Transplantation Library at St. Jude. The example shows the user’s question and answer, along with the highlighted section of the SOP used by LLM (gpt-oss:20b) to generate the answer. The library was created using the “nomic-embed-text” embedding model, which incorporates BM25 for precise keyword search, in addition to context-based semantic search via a vector database and reranked using “cross-encoder/qnli- electra-base” Reranker to filter out less relevant chunks. b Of the 136 questions (with ground truth) collected from 68 documents for this evaluation, MyGPT was able to answer 129 questions correctly with 94.85% accuracy. Answers were also collected using Zerank Reranker, which was able to answer 119 answers correctly with 87.5% accuracy, showing that “qnli-electra-base” is', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 19, 'content': '9 questions correctly with 94.85% accuracy. Answers were also collected using Zerank Reranker, which was able to answer 119 answers correctly with 87.5% accuracy, showing that “qnli-electra-base” is significantly better (p = 0.01; effect size = 0.56 (phi coefficient F, large); McNemar chi-square test). c MyGPT’s Question Relevance Score (QRS) distribution for questions with correct or incorrect answers (p = 5x10-4; effect size = 0.3 (r, small); Mann', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 20, 'content': '20 Whitney U test) shows higher relevance for questions that were answered correctly. Similarly, MyGPT’s Answer Relevancy Score (ARS) distribution for correct and incorrect answers (p = 4x10-3; effect size = 0.25 (r, small); Mann Whitney U test) shows higher scores for the correct than incorrect answers. MyGPT’s Default Hallucination Index (HI) for correct vs incorrect answers (p = 1.4x10-4; effect size = 0.33 (r, medium); Mann Whitney U test) shows significantly higher hallucination for incorrect answers.', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 21, 'content': '21 Figure 6. MyGPT applications and comparison to current GPT solutions. a Example use cases of MyGPT for information-seeking tasks performed by individual users or groups of users from academia, industry, and the general public. Users have the flexibility to cusromize and modify MyGPT parameters based on their specific use case. b Comparative summary table of MyGTP’s features with those of various commercial and open-source RAG solutions. MyGPT provides a software solution that fills the unmet needs of existing GPT versions in a single platform.', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 22, 'content': '22 METHODS MyGPT architecture The architecture of the MyGPT workflow comprises three sections: (i) a user interface (UI), (ii) a backend server, and (iii) an LLM server (Fig. 1). The UI and backend serve a dual purpose of library pre-processing and real-time question and answer. For library pre-processing, the UI receives the documents and customizations for the chunking method, the selected embedding model, and the distance function and sends them to the backend. The backend then converts documents into chunks and the chunks into embeddings using a user-selected embedding model and stores them in a vector database for semantic search. For different libraries, the backend creates separate sections/collections and stores them in the same vector database along with necessary metadata about the document type, title of documents, and user. This enables multiple users to use the same library and vector database without the need for access control and prevents leakage of in', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 22, 'content': 'etadata about the document type, title of documents, and user. This enables multiple users to use the same library and vector database without the need for access control and prevents leakage of information among libraries of individual or multiple users. In addition to semantic search, MyGPT also creates BM25 tokenizer for each library for keyword search. The UI also provides an interface to manage libraries: users can add, switch, or delete the documents in the libraries. Deleting the library delete s all the documents, vector embeddings and tokenizer from the vector database and removes the library chat history from the backend database. For real-time question-and-answer, the UI receives the question and sends it to the backend with a default prompt. The backend server converts the question into an embedding (using the same embedding model as the one used for processing the library) and sends the encoded question to the vector database to retrieve the nearest chunks (s', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 22, 'content': 'rts the question into an embedding (using the same embedding model as the one used for processing the library) and sends the encoded question to the vector database to retrieve the nearest chunks (see below). MyGPT also collects chunks identified by the BM25 algorithm that match the keywords in the question and then combines chunks identified by both methods by merging duplicates, filtering and re-ranking them using Reranker (see below). The final set of chunks is provided to the LLM server as context for a response. We have incorporated several open-access LLMs into the MyGPT pipeline, including Gpt-oss, Llama3, Llama2, Mistral, Vicuna, and Gemma, with Gpt-oss set as the default option (Supplementary Table 1). The main purpose of these LLMs is to receive a prompt, a user’s query, and the context and to respond with an LLM -generated answer with source citing from the documents in the library (Fig. 1). The UI also provides options to change the default prompt or the creativity', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 22, 'content': 'and the context and to respond with an LLM -generated answer with source citing from the documents in the library (Fig. 1). The UI also provides options to change the default prompt or the creativity vs precision settings before the LLM generates the answer. The LLM section of the pipeline can run independently of MyGPT and be used for purposes beyond MyGPT. In addition, a single LLM instance can power multiple instances of MyGPT to reduce cost and maintenance. The MyGPT pipeline uses the following third -party, open-access tools (Fig. 3) to orchestrate user interaction with LLMs for the three sections mentioned above: • Reference management tool (UI): We provide the option to create the document library using Zotero12, a free reference management tool for collecting and annotating publications. Zotero users can create a private or shared library of publications. Alternatively, MyGPT users can upload documents directly without using any reference management tool. • Embed', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 22, 'content': 'ting publications. Zotero users can create a private or shared library of publications. Alternatively, MyGPT users can upload documents directly without using any reference management tool. • Embedding model (Backend): We utilize “sentence transformer”, a Python library that converts text, paragraphs, or images into vectors and Ollama’s capability to host and inference embedding models. In our analysis, nomic-embedding-text hosted using Ollama showed ideal performance for different datasets, and we use it as a default. MyGPT also provides an option to use the multi-qa-MiniLM-L6-cos- v16 sentence transformer, an open-access embedding model available in the Chroma database. MyGPT can also be used with other embedding models available via Ollama and Hugging Face. • Vector database (Backend): We use Chroma database44, an AI-native open-source vector embedding database, to store all the chunks from user-provided documents locally as vectors, allowing for easy and fast searching wi', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 22, 'content': 'ckend): We use Chroma database44, an AI-native open-source vector embedding database, to store all the chunks from user-provided documents locally as vectors, allowing for easy and fast searching within the vector database.', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 23, 'content': "23 • BM25 (Backend): We use a lightweight Python library, BM25S, to create a keyword search tokenizer for the uploaded libraries. We use PyStemmer's stemming functionality with BM25S to support English (default) and other languages such as Spanish, French, and Portuguese. If the language of the document is selected as “unknown” when uploading the library, MyGPT will skip the BM25 step. • Reranker (Backend): We used the Cross-encoder library from “sentence transformer” for reranking. We used two different Rerankers: one for reranking chunks based on the user’s question and another for computing textual entailment between the retrieved chunks and the LLM-generated answer. To compare questions with consolidated chunks from semantic and keyword searches, we used “cross-encoder/qnli- electra-base” trained on Wikipedia passages and questions that could be answered by using those passages. To get textual entailment of those same chunks with the LLM-generated answer, we used “cross-", 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 23, 'content': 'lectra-base” trained on Wikipedia passages and questions that could be answered by using those passages. To get textual entailment of those same chunks with the LLM-generated answer, we used “cross-encoder/nli-deberta-v3-base” as the default Reranker. We also tried several other Rerankers available via Huggingface, supported by the Cross-encoder library, and available via UI. The Cross- encoder library allows downloading and use of a local Reranker, enabling both faster performance than querying the Hugging Face-hosted Reranker for each real-time Q&A as well as offline usage of Reranker after initial download. • LLM hosting library (LLM server): We use Ollama23, an LLM hosting library that enables users to employ open-source LLMs locally without the need to connect to proprietary online services. MyGPT employs an API architecture to utilize open-source LLMs that are currently available via Ollama or will be made available in the future. As of this publication, we recommend Gpt', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 23, 'content': 'line services. MyGPT employs an API architecture to utilize open-source LLMs that are currently available via Ollama or will be made available in the future. As of this publication, we recommend Gpt-oss:20b for personal laptops and Gpt-oss:120b for Ollama installed on servers/VMs. MyGPT also provides a user interface (UI) to install other LLMs such as Llama2, Gemma, Gemma2, Vicuna, and Mistral. Once installed, the MyGPT UI will display the various LLMs available for use in the pipeline (Supplementary Fig. 3). Document chunking methods MyGPT provides two methods for chunking text: • Fixed-size chunking: MyGPT divides the text from the user-provided documents into chunks of a predetermined number of characters from start to end. By default, MyGPT uses a chunk size of 1000 characters, but users can choose from options of 500, 750, 1000, or 1200 characters. Additionally, users can adjust the chunk overlap, which refers to the number of characters shared between two consecuti', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 23, 'content': "cters, but users can choose from options of 500, 750, 1000, or 1200 characters. Additionally, users can adjust the chunk overlap, which refers to the number of characters shared between two consecutive chunks. If this option is selected, MyGPT automatically applies a 20% overlap between adjacent chunks to maintain the meaning of sentences at the end of each chunk. Document-structure-preserving recursive chunking: MyGPT first identifies major hierarchical structures within the documents, including sections, subsections, and other semantic elements. This is achieved by leveraging the table of contents available in the PDF's metadata, or, in the absence of this information, by utilizing the GROBID45 document parsing library. GROBID's XML outputs provide structured boundaries that preserve the organization of academic publications. Once the sections are identified, each section is treated as an individual entity for subsequent chunking. Unlike fixed -size chunking, this approac", 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 23, 'content': 'eserve the organization of academic publications. Once the sections are identified, each section is treated as an individual entity for subsequent chunking. Unlike fixed -size chunking, this approach first identifies paragraphs within each section. The recursive approach attempts to split the text at subsection boundaries, then paragraph breaks, and finally sentence boundaries. If a paragraph exceeds the chosen chunk size (e.g., 1000 characters), it will be further split without breaking sentences mid-way. This method also allows users to configure overla p for adjacent chunks. By considering the document hierarchy, this approach maintains semantic coherence at the most significant structural levels while adhering to the defined token limits. Vector embedding of textual chunks and identification of chunks for semantic search MyGPT converts chunks of text from the library into high-dimensional vectors. MyGPT splits the text from the provided documents with one of the methods', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 23, 'content': 'd identification of chunks for semantic search MyGPT converts chunks of text from the library into high-dimensional vectors. MyGPT splits the text from the provided documents with one of the methods mentioned in the prior section, then converts these chunks into vectors using multi-qa-MiniLM-L6-cos-v1 or a user-selected embedding model. The vector dimensions are different for different embedding models; for example, it is 384 for multi-qa-MiniLM-L6-cos-v1 and 768', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 24, 'content': '24 for nomic-ai/nomic-embed-text-v1. When a question is asked, MyGPT converts the question into a vector using the same embedding model, embeds it in the same vector space as the library, and then finds the nearest n chunks (Fig. 1). The number of chunks, n, is set to 5 by default, and users can change the value of n in the user interface settings. If MyGPT can’t find all n chunks, or if not all nearest chunks meet the cutoff criteria of vector distances, chunks returned from semantic search will be less than n, where n is the maximum number of chunks from semantic search. MyGPT converts vector distances to Semantic score Qsem via min-max normalization using the observed minimum and maximum distances for public datasets (see main text). Once the LLM generates the answer, MyGPT converts the answer into a vector using the same embedding model, finds the distance between the answer and k chunks from the “golden context”, and calculates Semantic score Asem via min-max normaliza', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 24, 'content': 'nverts the answer into a vector using the same embedding model, finds the distance between the answer and k chunks from the “golden context”, and calculates Semantic score Asem via min-max normalization as above for Qsem. Note that the k chunks for the “golden context” are finalized during Reranking and the consolidation of chunks step, as discussed in the following sections, and, once finalized, the “golden context” with k chunks is used for the remaining steps in the workflow. Keyword search for textual chunks using BM25 MyGPT offers the option to use the BM25 method to identify relevant chunks through keyword search, ensuring that no keywords are overlooked. If selected alongside vector-based contextual search, MyGPT will collect n chunks with the highest BM25 scores for the user query. Similar to semantic search, if MyGPT can’t find n chunks or not all n chunks meet the BM25 cutoff score, returning chunks from keyword search will be fewer than n, where n is the maximum', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 24, 'content': 'user query. Similar to semantic search, if MyGPT can’t find n chunks or not all n chunks meet the BM25 cutoff score, returning chunks from keyword search will be fewer than n, where n is the maximum number of chunks returning from keyword search, making the maximum number of chunks returning from both methods 2n (Fig. 1). MyGPT converts BM25 raw scores to Keyword score Qkey via min-max normalization using the observed minimum and maximum BM25 scores for public datasets (see main text). Once the LLM generates the answer, MyGPT calculates the BM25 scores between the answer and k chunks from the “golden context” and generates Keyword score Akey via min-max normalization as above for the Qkey. Reranking and consolidation of chunks for LLM context Once the semantic and keyword search returns chunks, those chunks are provided to Reranker along with the user’s query. The Reranker assigns a rerank score, Qrerank, and a rank based on the probability of answering the user’s query us', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 24, 'content': 'returns chunks, those chunks are provided to Reranker along with the user’s query. The Reranker assigns a rerank score, Qrerank, and a rank based on the probability of answering the user’s query using that chunk. Chunks that do not meet the minimum cutoff for Rerank scores will be removed, leaving l chunks from keyword search and m chunks from semantic search. These chunks from both of these methods are consolidated by merging duplicates, leaving k chunks to be returned as the “golden context” for the rest of the MyGPT workflow (Fig. 1). Sample size calculations using Power analysis Because the RAG systems are evaluated on the same questions, and multiple questions may come from the same document, standard independent-sample power analysis is not appropriate. We therefore perform a sample size calculation (i.e., number of documents) that accounts for both paired evaluations and clustering within documents.To determine the ideal sample size for performance comparisons betwe', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 24, 'content': 'm a sample size calculation (i.e., number of documents) that accounts for both paired evaluations and clustering within documents.To determine the ideal sample size for performance comparisons between different RAG-applications or RAG pipeline settings, we used the following formula, 𝑐= B𝑧(2 4 5 +𝑧(2 6E 5 I𝑝((1− 𝑝()+ 𝑝5 (1− 𝑝5)−2𝜌K𝑝((1− 𝑝() 𝑝5 (1− 𝑝5)L {1+(𝑚−1)𝜌7} 𝑚 (𝑝(− 𝑝5)5 (8) Where z is the z-score a is Type I error rate; e.g., a = 0.05 and 𝑧(2 ! " = 1.96, b is Type II error rate = 1 – Power, e.g., b = 0.1 and 𝑧(2 6 = 1.28, p1 and p2 are model accuracies for models 1 and 2; e.g., p1 = 0.6 and p2 = 0.4, m is number of questions per document, e.g., m = 5,', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 25, 'content': '25 c is number of documents, Y1ij and Y2ij are binary correctness indicators for models 1 and 2, respectively, on question j from document i, where 1= correct and 0= incorrect, r is correlation between models on the same question (i.e., how similarly the models succeed or fail on a question), defined as Corr(Y1ij, Y2ij), where i is document index and j is question index; e.g., r = 0.8, rc is the intra-document correlation of the paired differences across questions within the same document, where Dij = Y1ij - Y2ij, and rc = Corr(Dij, Dij’); e.g., rc = 0.5, This formula contains two correlation adjustments: one for pairing across models (r) and one for clustering within documents (rc). The pairing correction term 2𝜌K𝑝((1− 𝑝() 𝑝5 (1− 𝑝5) accounts for pairing between the two models when evaluated on the same question. When r =0, there is no pairing effect, and the setup behaves like two independent sets of questions assigned to two independent models. Positive pairing co', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 25, 'content': 'odels when evaluated on the same question. When r =0, there is no pairing effect, and the setup behaves like two independent sets of questions assigned to two independent models. Positive pairing correlation reduces the variance of the paired difference and therefore decreases the required sample size. The clustering correction term {1+(𝑚−1)𝜌7} accounts for clustering within documents, where multiple questions are nested within the same document. When rc = 0, questions within documents are independent, indicating no clustering and reducing to an independent-sample design. When m=1, there is only one question per document, so clustering does not occur. Positive clustering inflates the variance of the estimated mean difference and therefore increases the required sample size. In the absence of both clustering and pairing, the formula reduces to the standard sample size calculation for comparing two proportions under an independent-sample design. While p1, p2, r, rc and', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 25, 'content': 'the absence of both clustering and pairing, the formula reduces to the standard sample size calculation for comparing two proportions under an independent-sample design. While p1, p2, r, rc and m may vary across evaluation datasets, use-cases, MyGPT settings and comparisons with other RAG solutions, we fix the Type I and Type II error rates at 0.05 and 0.1, respectively, corresponding to a 5% significance level and 90% power. These choices provide a conservative basis for determining the number of documents required across datasets. The parameter values for individual datasets are specified in their corresponding sections (Benchmarking MyGPT and other open-source RAG solutions using public datasets, MyGPT libraries for the different use cases, and Supplementary Table 5). Running and hosting MyGPT The MyGPT pipeline can be run on a local computer for personal use or hosted on a server/cloud infrastructure for group, department, or institution-wide usage. To run MyGPT', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 25, 'content': 'ing and hosting MyGPT The MyGPT pipeline can be run on a local computer for personal use or hosted on a server/cloud infrastructure for group, department, or institution-wide usage. To run MyGPT locally, a user needs a device with at least 8 CPUs, 16 GB RAM, and a GPU. We have provided benchmarking instructions for testing Ollama on personal devices (see GitHub repository for instructions). If users can run Ollama satisfactorily, they can install MyGPT and run the entire pipeline on the same device. Detailed descriptions of the steps for all approaches are available in the GitHub repository (https://github.com/mb-group/MyGPT_public). We have tested installation and LLM performance of MyGPT on various personal devices (Supplementary Table 2). During our testing, we were also able to install and run MyGPT on an affordable Edge AI compute device – NVIDIA Jetson Orin Nano (cost - $250), with small LLMs such as Llama 3.2 (1b, 3b) and Gemma (2b) (Supplementary Fig. 7). In ca', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 25, 'content': 'to install and run MyGPT on an affordable Edge AI compute device – NVIDIA Jetson Orin Nano (cost - $250), with small LLMs such as Llama 3.2 (1b, 3b) and Gemma (2b) (Supplementary Fig. 7). In cases with many MyGPT users, such as departments, institutions, or industry settings, hosting MyGPT on a server or cloud platform may be advantageous. To optimally run the MyGPT pipeline in such environments, the installer needs to understand the operation and maintenance of a web application server. We have tested MyGPT on local servers with various GPUs, including Nvidia GPU GeForce RTX 4090 and Nvidia GPU A100, as well as on cloud infrastructures such as Amazon Web Services. As resource', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 26, 'content': '26 requirements and configurations will vary for each use case (details in Supplementary Table 1), involving Information Technology personnel to assist in hosting MyGPT for these instances may be helpful. Cost and resource usage To illustrate the range of deployments and resource considerations, we hosted MyGPT on personal devices, on commodity hardware for group use, in the cloud for review purposes, and on a VM for institutional use. • Personal devices: Installation on Mac laptops with M1 to M4 chips ranged from $2,000 - $4,500 and were suitable for experimental scientists and AI engineers. Users followed GitHub instructions and maintained their installations by periodically updating code from GitHub. • Commodity hardware: MyGPT was hosted for 40 team members on Nvidia DGX -1s at $4,000. Installation by a senior engineer took 2 hours. Recommended similar hardware and LLM hosting configurations are listed in Supplementary Table 6. • On Cloud: MyGPT was hosted on Azure,', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 26, 'content': '1s at $4,000. Installation by a senior engineer took 2 hours. Recommended similar hardware and LLM hosting configurations are listed in Supplementary Table 6. • On Cloud: MyGPT was hosted on Azure, with an LLM server VM (NC8as T4 v3 - 8 vCPUs, 56 GB RAM) at $548.96/month and backend/frontend VM (E4ads v5 - 4 vCPUs, 32 GB RAM) at $325.58/month. This and similar estimates and configurations are in Supplementary Table 3. Installation by three team members took 12 hours, and maintenance requires 20-30 minutes every 3 months. • Institutional deployment: MyGPT was deployed at St. Jude Children’s Research Hospital on on- premises VMs approved for Protected health information (PHI) data storage. The process began with obtaining necessary approvals from institutional stakeholders. Our team of 4 professionals (2 Sr. Software engineers, a Software Architect, and an AI engineer) performed installation, testing, and vulnerability and security checks. The Information Security and infrastru', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 26, 'content': 'of 4 professionals (2 Sr. Software engineers, a Software Architect, and an AI engineer) performed installation, testing, and vulnerability and security checks. The Information Security and infrastructure teams reviewed compliance. Deployment spanned 4 weeks for these steps. Maintenance involves updating code and LLMs/embedding models every 2-3 months, requiring 0.5 - 1 hour. We use 4 A100 GPUs for the LLM server and a VM with 4 CPUs, 64 GB of RAM, and 1 TB of storage for the backend and frontend. Due to confidentiality and vendor contracting, VM costs cannot be shared publicly. Cost comparisons with other commercial and open-source RAG solutions are provided in Supplementary Table 7. Finding the optimal setting of QRS, ARS, and HI For public datasets and individual use case, we calculated optimal weights for QRS, ARS, and HI equations using global sensitivity analysis. Global sensitivity analysis explores how outputs vary across the entire range of possible input values', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 26, 'content': 'we calculated optimal weights for QRS, ARS, and HI equations using global sensitivity analysis. Global sensitivity analysis explores how outputs vary across the entire range of possible input values. • QRS is calculated by multiplying optimal weights a, b, and c by Qsem (Semantic score), Qkey (Keyword score), and Qrerank (Rerank score), respectively, and summing the three values, as per the equation (1). To obtain the optimal weights a, b and c, we assigned 10 different weights (-4 to 5) to Qsem, Qkey and Qrerank and calculated the area under Precision-Recall Curve (PR-AUC) for each combination of these three variables. We plotted them in ten (10x10) heatmaps, each for individual rerank weights, x-axis with 10 vector weights, and y-axis with 10 BM25 weights (Supplementary Fig. 8). We selected optimal weights for all 3 variables with the highest PR-AUC value. For 3 public datasets combined (n=627), the optimal weights a, b, and c are 5, -2, and -2, respectively, with PR-AUC', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 26, 'content': '. We selected optimal weights for all 3 variables with the highest PR-AUC value. For 3 public datasets combined (n=627), the optimal weights a, b, and c are 5, -2, and -2, respectively, with PR-AUC of 90%. • ARS is calculated by multiplying optimal weights x, y, and z by Asem (Semantic score), Akey (Keyword score), and Arerank (Rerank score), respectively, and adding the three values per equation (4). We calculated optimal weights of 3 variables using the same method as for QRS by plotting 10 different weights in ten 10x10 heatmaps (Supplementary Fig. 8). For 3 public datasets combined, the optimal weights x, y and z are 5, -2, and 0, respectively, with PR-AUC of 92%. • HI is calculated by multiplying optimal weights p and q by QRS and ARS, respectively, per equation (7). We calculated them using the same method, giving optimal weights of 2 for p and 3 for q to for the public datasets combined.', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 26, 'content': 'alculated them using the same method, giving optimal weights of 2 for p and 3 for q to for the public datasets combined.', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 27, 'content': '27 Benchmarking MyGPT and other open-source RAG solutions using public datasets We used 3 different public datasets to benchmark MyGPT and other RAG solutions: • PubMedQA: The dataset contains 1.0k Q&A pairs, with answers labeled as yes, no, or maybe, generated by humans and suitable for automated evaluation. Each of the questions can be answered from a single published article. With institutional access to published articles, we collected 294 articles in PDF format. The number of documents (294) was ideal for comparison with other RAG solutions, such as Dify, PrivateGPT, and Kotaemon, based on sample-size calculations using power analysis (Supplementary Table 5). Given the close correctness rates for MyGPT and RAGFlow, the required number of documents (806) is higher than the number we used, and we are unable use more documents due to limitations on the number of published articles we can obtain with institutional access. • BioASQ: BioASQ provides human-curated pairs of Q', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 27, 'content': 'umber we used, and we are unable use more documents due to limitations on the number of published articles we can obtain with institutional access. • BioASQ: BioASQ provides human-curated pairs of Q&A with exact answers and necessary context as snippets from the original published article. We used Training dataset 13b to collect 163 questions and answers. We created 210 PDFs with abstracts/snippets provided by the datasets and used them as a library of documents. The questions were independent, and no clustering was present as a BioASQ dataset design criterion. Similar to PubMedQA, the number of documents (210) was ideal for comparison with other RAG solutions, such as Dify, PrivateGPT, and Kotaemon, but for RAGFlow we will need 5106 documents based on sample-size calculations. It is possible to create additional PDFs using the same method we used for the 210 PD F documents and perform the evaluation, but effort s for manual evaluation are beyond the scope of this manuscript.', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 27, 'content': 'is possible to create additional PDFs using the same method we used for the 210 PD F documents and perform the evaluation, but effort s for manual evaluation are beyond the scope of this manuscript. • Open RAG benchmark: This dataset provides Q&A pairs from 1.0k Arxiv documents, with 400 positive and 600 negative documents. As a subset, we collected 170 questions and 165 PDFs corresponding to the ArXiv documents referenced. This dataset provides 2 types of text-only query questions: abstractive and extractive. Abstractive queries require generating a summary or a rephrased response using understanding and synthesis, while extractive queries require fact-based extraction of an answer directly from the given text. Of 170 Q&A pairs, we used 78 abstractive and 92 extractive queries. Similar to the two prior datasets, the number of documents (170) was ideal for comparison with other RAG solutions such as Dify, PrivateGPT, and Kotaemon, but for RAGFlow, we would need 1063 documents', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 27, 'content': 'milar to the two prior datasets, the number of documents (170) was ideal for comparison with other RAG solutions such as Dify, PrivateGPT, and Kotaemon, but for RAGFlow, we would need 1063 documents based on sample-size calculations. Per the dataset design, we can increase the document count up to 400 with 1900 Q&A, and the time and effort of manual evaluation is beyond the scope of this manuscript. Benchmarking and usability testing of the MyGPT pipeline To test the MyGPT pipeline for technical and usability aspects, we asked 10 participants with varying domain knowledge and technical skills to install and test MyGPT on their personal computers and to use it with their respective libraries. We further assessed the performance of MyGPT by running it in various environments such as on a personal computer, a local server with GPU inferencing, and cloud infrastructure. Differences in response times are reported in Supplementary Table 2. During our benchmarking test, we noted', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 27, 'content': 'as on a personal computer, a local server with GPU inferencing, and cloud infrastructure. Differences in response times are reported in Supplementary Table 2. During our benchmarking test, we noted that the Gemma model was the fastest in all environments, but the answers were generally shorter and provided fewer details than those generated by larger models such as Llama3:70b. Comparing different environments, the performance of MyGPT on an Apple laptop with M1 GPU (8 CPUs and 64 GB RAM) was equivalent to or better than Google Cloud Platform (GCP) with a T4, L4 or A100 GPU and cloud infrastructure such as Amazon Web Services (AWS) with A10G GPU. Users without access to GPU via a PC or laptop or who have less powerful laptops can use either compact Edge AI devices such as Nvidia Jetson Orin (starts at $250) or use Cloud service with T4 GPU, which can cost around $0.42 per GPU per hour or around $306 per GPU per month on-demand (in US market for AWS as of April 2026). Librar', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 27, 'content': 'son Orin (starts at $250) or use Cloud service with T4 GPU, which can cost around $0.42 per GPU per hour or around $306 per GPU per month on-demand (in US market for AWS as of April 2026). Library size benchmarking We tested different library sizes and their impact on the MyGPT workflow. Rather than focusing on the number of documents, we examined pages per library, creating four libraries with 10, 100, 1,000, and 10,000 pages. These were tested on two Apple MacBooks with M1 and M4 GPUs (8 CPUs, 64 GB RAM) and a St. Jude Server with A100 GPU (64 CPUs, 2 TB RAM). Uploading and retrieval times were managed', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 28, 'content': '28 by the MyGPT backend, while generation time was handled by the LLM server. The end-to-end timing captured the complete Q&A process. Uploading was performed using a fixed-chunking method (1,000- character chunks) with the nomic-embed-text-v1.5 model for embedding. We used llama3:8b for generation on both laptops and the server. Results are reported in Supplementary Table 4. MyGPT libraries for the different use cases To demonstrate the application of MyGPT for biomedical research, clinical tasks and global health policy at St. Jude Children’s Research Hospital, we created multiple instances of MyGPT. • For the literature review of biomedical research articles, we created a library of articles on various kinases. The library contained 83 biomedical publications for 26 different kinases (99 kinase-publication pairs), and we created 297 Q&A pairs with input from an expert in the field (provided as Supplementary material). The library was created using the nomic-embed-text:', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 28, 'content': 't kinases (99 kinase-publication pairs), and we created 297 Q&A pairs with input from an expert in the field (provided as Supplementary material). The library was created using the nomic-embed-text:latest embedding model with a fixed chunking method of 1000 characters and an overlap of 20%. The Squared L2 distance function and the BM25 algorithm were used for keyword matching, and retrieved chunks were reranked using “cross- encoder/qnli-electra-base” Reranker. We utilized gpt-oss:20b LLM to generate an answer. For the first experiment, a default chunk count of 5 was used as the maximum number of chunks returned by the semantic and keyword search. For the second experiment to test the hypothesis that complex questions benefit from a larger number of chunks, we used n=15, which yielded more chunks for most questions and ultimately more correct answers (Supplementary Fig. 4). • For the clinical document, we created a MyGPT library with Standard Operating Procedures (SOPs) from', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 28, 'content': 'ed more chunks for most questions and ultimately more correct answers (Supplementary Fig. 4). • For the clinical document, we created a MyGPT library with Standard Operating Procedures (SOPs) from the Department of Bone Marrow Transplantation and Cellular Therapy (BMTCT) at St. Jude. The library contained 136 questions (with ground truth) from 68 SOP documents and was created using the nomic-embed-text:latest embedding model with a fixed chunking method of 1000 characters, an overlap of 20% between chunks, a Squared L2 distance function, and the BM25 algorithm for keyword matching. Retrieved chunks were reranked using “cross-encoder/qnli-electra-base” Reranker, and gpt-oss:20b LLM was used to generate an answer. • For Q&A on multilingual global health policy documents, we created Spanish, French, and English document libraries, each containing 10 health policies from the St. Jude Global Pediatric Medicine Department. Using nomic-text-v2-moe:latest, we embedded texts with a f', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 28, 'content': "Spanish, French, and English document libraries, each containing 10 health policies from the St. Jude Global Pediatric Medicine Department. Using nomic-text-v2-moe:latest, we embedded texts with a fixed chunk size of 1000 characters and 20% overlap, using the Squared L2 distance and the BM25 algorithm for each language. Due to limited foreign language expertise and manual validation, we omitted the Reranker from this evaluation. For answers, we used gpt-oss:20b. To support multiple languages, we modified MyGPT in several ways: during document upload, users must select the language of documents, prompting MyGPT to choose the appropriate embedding model and BM25 version. User questions are translated into the document's language using TranslateGemma LLM before semantic and keyword search for context retrieval. Using this context, the user-selected LLM generates a response in the user's language, and MyGPT uses translated answers from TranslateGemma to compute ARS and HI. Curren", 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 28, 'content': "arch for context retrieval. Using this context, the user-selected LLM generates a response in the user's language, and MyGPT uses translated answers from TranslateGemma to compute ARS and HI. Current testing confirms this modified pipeline works as intended, though future evaluation of Reranker suitability is needed. Data sources and preprocessing for a machine learning model to compute Hallucination Index We built a random forest classifier46 to predict the incorrectness of each question-answer pair based on various associated scores, such as semantic, keyword, and reranking scores. The training dataset consists of 3 public and 3 private datasets (see the above section and the main text), totaling 1045 question-answer pairs. Each question-answer pair has semantic, keyword, and reranking scores for the question and the answer, as well as a binary correctness label. The correctness labels for public datasets were provided by two independent researchers based on ground-truth", 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 28, 'content': 'reranking scores for the question and the answer, as well as a binary correctness label. The correctness labels for public datasets were provided by two independent researchers based on ground-truth answers available with the datasets, and the labels for private datasets were provided by experts in the domain for real-world use cases.', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 29, 'content': '29 Semantic scores, BM25 scores, and rerank scores were provided as a list for each Q&A, with each element containing the score for the corresponding chunk. For each list, we computed the summary statistics mean, maximum, minimum, and standard deviation. These statistics were computed for question semantic scores, question BM25 scores, question rerank scores, answer semantic scores, and answer BM25 scores, totalling 20 features for the training data. Q&A were dropped if any of the following fields were missing: semantic scores for questions, BM25 scores for questions, reranked question scores, semantic scores for answers, BM25 scores for answers, or reranking sentiment labels for answers. The binary outcome variable (y_true) was defined as incorrect = 1 and correct = 0, derived from the correctness label. Architecture of a random forest model and model Interpretability with SHAP Using the training data mentioned in the above section, we trained a random forest classifier t', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 29, 'content': 'om the correctness label. Architecture of a random forest model and model Interpretability with SHAP Using the training data mentioned in the above section, we trained a random forest classifier to predict incorrectness (binary classification). As we observed significantly higher HI for incorrect answers using the HI equation, we used incorrectness as the primary prediction target rather than correctness. The model was implemented using scikit-learn47. To assess robustness to data splitting, we performed 10 independent replications using different random seeds. In each replication, the dataset was split into training (80%) and test (20%) sets using stratified sampling. A Random Forest model was then trained on the training set and evaluated on the test set. In addition to the default probability cutoff of 0.50, we explored optimized decision thresholds. For each test set, predicted probabilities were thresholded across values from 0.01 to 0.99. For each threshold, an ROC-AU', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 29, 'content': 'ault probability cutoff of 0.50, we explored optimized decision thresholds. For each test set, predicted probabilities were thresholded across values from 0.01 to 0.99. For each threshold, an ROC-AUC was computed using thresholded predictions. The threshold yielding the highest valid test-set ROC-AUC across replications was recorded as the best cutoff. The best-performing model across replications was selected based on test ROC-AUC. Feature importance analyses were performed using the built-in feature importance function of Random Forest in scikit-learn, and SHAP48 (SHapley Additive exPlanations) feature importance values using the selected model. To assess feature stability, we computed feature importances for each replication and evaluated pairwise correlations between importance vectors across replications. Model interpretability was evaluated using SHAP with a TreeExplainer optimized for ensemble tree models. SHAP values were computed for all samples, focusing on class', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 29, 'content': 'ce vectors across replications. Model interpretability was evaluated using SHAP with a TreeExplainer optimized for ensemble tree models. SHAP values were computed for all samples, focusing on class 1 (incorrect prediction) contributions.', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 30, 'content': '30 Acknowledgments We thank ALSAC for funding and support. We also acknowledge support from the Blue-Sky Kinase initiative (J.P., J.D., D.M., B.Me., K.P., M.R., C.G.K., and M.M.B) and GPCR Collaborative (J.D., V.T., and M.M.B) funding mechanisms within St Jude. We thank the domain experts, Drs. Balaji Santhanam, Katarina Nemec, Preethi Badrinarayan, Ben Leslie, Kalyan Immadisetty, and Benjamin Lang for their assistance in creating the Q&A datasets used for benchmarking the biomedical literature and public datasets, Jino John for helping with evaluation of the kinase literature, and Nancy Gough and Jeyanthy Eswaran for their comments on the evaluation. We gratefully acknowledge Andrea Williams, Marcia Sabino de Freitas, Heather Fosburgh, Patricia Loggetto and Melisssa Santiago together with D.B.A. and C.G.L. from St. Jude Global for their evaluation of health policies.We thank Pranay Kumar Anugu, Roshan Shrestha, and Piby Paul for their help with deploying MyGPT on the VMs a', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 30, 'content': 'ether with D.B.A. and C.G.L. from St. Jude Global for their evaluation of health policies.We thank Pranay Kumar Anugu, Roshan Shrestha, and Piby Paul for their help with deploying MyGPT on the VMs approved for PHI data. We also thank Drs. Jim Downing, Paul Taylor, Elly Park, Babu group and Blue-Sky Kinase team for comments on this work as well as Ashok Kumar Reddy Boddu, Ed Suh, Michael Brewer, Darrell Gentry, Chad Burdyshaw, and Keith Perry from the Information Services department, and Andrew Dedmon, Valentina Perez, and Brian Elrod from the Information Security team of St. Jude for their support. Author Contributions J.P. and M.M.B. conceived the MyGPT concept and all confidence metrics. J.P. and J.D. contributed equally to designing a prototype for MyGPT software. J.P. developed the MyGPT software and pipeline. B.Me., V.T., and D.M. helped with multiple rounds of manual evaluation of the benchmark datasets. D.M. reviewed the code. H.F. implemented the recursive chunking', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 30, 'content': 'ped the MyGPT software and pipeline. B.Me., V.T., and D.M. helped with multiple rounds of manual evaluation of the benchmark datasets. D.M. reviewed the code. H.F. implemented the recursive chunking approach and evaluated MyGPT and PrivateGPT on public datasets. T.Y. evaluated MyGPT, Dify, and Kotaemon on public datasets and ran evaluation pipelines for MyGPT on health policies and the kinase literature. J.D. collected open RAG Benchmark datasets and set up an evaluation pipeline for MyGPT. Cai L. provided methods and guidance for sample size calculations for RAG applications; R.U. and J.P. verified their use with MyGPT and other RAG solutions. R.U. and J.P. conducted a cost analysis of MyGPT and other similar solutions. S.P. designed a visual settings menu for uploading and customizing chat settings in the MyGPT interface. With the guidance of C.G.K., K.P., M.R., and B.Me. collected Q&A datasets and performed validation of MyGPT answers for the kinase Q&As. D.B.A. and C.G.L.', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 30, 'content': 'izing chat settings in the MyGPT interface. With the guidance of C.G.K., K.P., M.R., and B.Me. collected Q&A datasets and performed validation of MyGPT answers for the kinase Q&As. D.B.A. and C.G.L. defined and collected Q&A datasets and coordinated and performed validated MyGPT answers for health policies with the St. Jude Health Systems team, facilitated by C.G.L. and C.R.G. C.M. collected Q&A datasets and performed validation of MyGPT answers for BMTCT SOPs with the guidance of D.K. and S.G. S.M.D.A. assisted in analyzing confidence metric values and created a random forest model for Hallucination Index. S.A. installed and supported Ollama on a server with an A100 GPU, D.A. set the automation pipeline and GitHub repository management, A.W. implemented frontend unit tests, and A.W. and D.A. set up cloud Azure VMs. J.P., I.C., M.S., B. Mo., and M.M.B. wrote the manuscript. J.P. led the project. M.M.B. managed the project and set its overall vision and specific directions.', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 30, 'content': 'W. and D.A. set up cloud Azure VMs. J.P., I.C., M.S., B. Mo., and M.M.B. wrote the manuscript. J.P. led the project. M.M.B. managed the project and set its overall vision and specific directions. Competing interests The authors declare no competing interests. Additional information Supplementary Information is available for this paper. Correspondence and request for materials should be addressed to Jaimin Patel or M. Madan Babu. Code availability All code, data, and tools are available at https://github.com/stjude-c3d/MyGPT. [This link is currently private and will be made public upon acceptance for publication. The source code is provided as a zip file in Supplementary Information for the review process. We have also provided a URL with username and password for each reviewer to the journal editor; please contact the editor if you prefer to evaluate the', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 30, 'content': 'ch reviewer to the journal editor; please contact the editor if you prefer to evaluate the', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 31, 'content': '31 hosted version of MyGPT]. We performed vulnerability testing using Snyk to evaluate best practices in software development to eliminate security weaknesses. Our code and data are licensed under GPL-3.0 license, while LLM models and LLM tools used for the project are available under the licenses of the respective sources.', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 32, 'content': '32 SUPPLEMENTARY TABLES Supplementary Table 1: Example LLMs that can be used with MyGPT via Ollama. LLM Parameters Size Developed by Note Ideal hosting environment Minimum requirements Llama2:7b 6.7B 3.8 GB Meta Released on 18th July 2023 PC/Laptop, Cloud service 1 GPU, 8 GB RAM Llama3:8b 8.03B 4.7 GB PC/Laptop, Cloud service 1 GPU, 8 GB RAM Llama3:70b 70.6B 40 GB VM, server, Cloud service 1 GPU, 64 GB RAM Llama3.1:8b 8.03B 4.9 GB Released on 23rd July, 2024 PC/Laptop, Cloud service 1 GPU, 8 GB RAM Llama3.2:1b 1.24B 1.3 GB Released on 25th September, 2024 Edge AI device, PC/Laptop, Cloud service 1 GPU, 8 GB RAM Llama3.3:70b 70.6B 43 GB Released on 7th December, 2024 VM, server, Cloud service 1 GPU, 64 GB RAM Gemma:7b 8.54B 5 GB Google DeepMind Released on 21st February 2024 PC/Laptop, Cloud service 1 GPU, 8 GB RAM Gemma2:9b 9.24B 5.4 GB Released on 27th June 2024 PC/Laptop, Cloud service 1 GPU, 8 GB RAM Gemma2:2b 2.61B', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 32, 'content': 'pMind Released on 21st February 2024 PC/Laptop, Cloud service 1 GPU, 8 GB RAM Gemma2:9b 9.24B 5.4 GB Released on 27th June 2024 PC/Laptop, Cloud service 1 GPU, 8 GB RAM Gemma2:2b 2.61B 1.6 GB Edge AI device, PC/Laptop, Cloud service 1 GPU, 8 GB RAM Gemma3:1b, 4b 1B, 4.3B 0.8GB, 3.3GB Released on 12th March 2025 Edge AI device, PC/Laptop, Cloud service 1 GPU, 8 GB RAM Gemma3:12b, 27b 12.2B, 27.4B 8.1GB, 17GB Released on 12th March 2025 PC/Laptop, Cloud service 1 GPU, 32 GB RAM Gemma4:26b, 31b 25.8B, 31.3B 18GB, 20GB Released on 2nd April 2026 PC/Laptop, Cloud service 1 GPU, 32 GB RAM Gpt-oss:20b 20.9B 14GB OpenAI Released on 5th August 2025 (Thinking model) PC/Laptop, Cloud service 1 GPU, 32 GB RAM DeepSeek- R1:8b, 14b 8.19B, 14.8B 5.2GB, 9GB DeepSeek Released on January 20, 2025 (Thinking model) PC/Laptop, Cloud service 1 GPU, 32 GB RAM Qwen3:8b, 14b 8.19B, 14.8B 5.2GB, 9.3GB Alibaba Released on April', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 32, 'content': '14.8B 5.2GB, 9GB DeepSeek Released on January 20, 2025 (Thinking model) PC/Laptop, Cloud service 1 GPU, 32 GB RAM Qwen3:8b, 14b 8.19B, 14.8B 5.2GB, 9.3GB Alibaba Released on April 29 th, 2025 (Thinking model) PC/Laptop, Cloud service 1 GPU, 16 GB RAM Mistral:7b 7.2B 4.1 GB Mistral AI Released on 27th September 2023 PC/Laptop, Cloud service 1 GPU, 8 GB RAM Vicuna:7b 6.74B 3.8 GB LMSYS Org, UC Berkeley Finetuned Llama2 PC/Laptop, Cloud service 1 GPU, 8 GB RAM', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 33, 'content': '33 Supplementary Table 2: LLM performance for different environments. LLM size category LLM Host environment Infrastructure GPU Avg. Speed (seconds) Mid-size LLMs (most common) Llama2:7b Google Cloud Platform 2 vCPUs, 83 GB RAM 1 Nvidia A100 Tensor Core 3.80 Google Cloud Platform 2 vCPUs, 12 GB RAM 1 Nvidia T4 Tensor Core 6.46 St. Jude internal server 64 CPUs, 2 TB RAM 1 Nvidia A100 Tensor Core 2.89 Apple Mac m1 8 CPUs, 64 GB RAM 1 Apple M1 Metal 9.26 Apple Mac m3 8 CPUs, 64 GB RAM 1 Apple M3 Metal 5.21 Llama3:8b Google Cloud Platform 2 vCPUs, 83 GB RAM 1 Nvidia A100 Tensor Core 3.34 Google Cloud Platform 2 vCPUs, 12 GB RAM 1 Nvidia T4 Tensor Core 4.96 St. Jude internal server 64 CPUs, 2 TB RAM 1 Nvidia A100 Tensor Core 3.40 Apple Mac M1 8 CPUs, 64 GB RAM 1 Apple M1 Metal 7.47 Apple Mac M3 8 CPUs, 64 GB RAM 1 Apple M3 Metal 6.00 Llama3.1:8b Google Cloud Platform 2 vCPUs, 83 GB RAM 1 Nvidia A100 Tensor Core 2.49', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 33, 'content': 'ple Mac M1 8 CPUs, 64 GB RAM 1 Apple M1 Metal 7.47 Apple Mac M3 8 CPUs, 64 GB RAM 1 Apple M3 Metal 6.00 Llama3.1:8b Google Cloud Platform 2 vCPUs, 83 GB RAM 1 Nvidia A100 Tensor Core 2.49 Google Cloud Platform 2 vCPUs, 12 GB RAM 1 Nvidia T4 Tensor Core 6.31 St. Jude internal server 64 CPUs, 2 TB RAM 1 Nvidia A100 Tensor Core 2.05 Apple Mac M1 8 CPUs, 64 GB RAM 1 Apple M1 Metal 7.05 Apple Mac M3 8 CPUs, 64 GB RAM 1 Apple M3 Metal 4.45 Gemma:7b Google Cloud Platform 2 vCPUs, 83 GB RAM 1 Nvidia A100 Tensor Core 2.74 Google Cloud Platform 2 vCPUs, 12 GB RAM 1 Nvidia T4 Tensor Core 4.38 St. Jude internal server 64 CPUs, 2 TB RAM 1 Nvidia A100 Tensor Core 1.61 Apple Mac M1 8 CPUs, 64 GB RAM 1 Apple M1 Metal 5.52 Apple Mac M3 8 CPUs, 64 GB RAM 1 Apple M3 Metal 2.80 Gemma2:9b Google Cloud Platform 2 vCPUs, 83 GB RAM 1 Nvidia A100 Tensor Core 2.72 Google Cloud Platform 2 vCPUs, 12 GB RAM 1 Nvidia T4 Tensor Core 4.57 St. J', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 33, 'content': 'RAM 1 Apple M3 Metal 2.80 Gemma2:9b Google Cloud Platform 2 vCPUs, 83 GB RAM 1 Nvidia A100 Tensor Core 2.72 Google Cloud Platform 2 vCPUs, 12 GB RAM 1 Nvidia T4 Tensor Core 4.57 St. Jude internal server 64 CPUs, 2 TB RAM 1 Nvidia A100 Tensor Core 1.46 Apple Mac M1 8 CPUs, 64 GB RAM 1 Apple M1 Metal 6.01 Apple Mac M3 8 CPUs, 64 GB RAM 1 Apple M3 Metal 3.28 Mistral:7b Google Cloud Platform 2 vCPUs, 83 GB RAM 1 Nvidia A100 Tensor Core 3.37 Google Cloud Platform 2 vCPUs, 12 GB RAM 1 Nvidia T4 Tensor Core 5.69 St. Jude internal server 64 CPUs, 2 TB RAM 1 Nvidia A100 Tensor Core 2.67 Apple Mac M1 8 CPUs, 64 GB RAM 1 Apple M1 Metal 7.87', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 34, 'content': '34 Apple Mac M3 8 CPUs, 64 GB RAM 1 Apple M3 Metal 5.28 Vicuna:7b Google Cloud Platform 2 vCPUs, 83 GB RAM 1 Nvidia A100 Tensor Core 3.96 Google Cloud Platform 2 vCPUs, 12 GB RAM 1 Nvidia T4 Tensor Core 8.73 St. Jude internal server 64 CPUs, 2 TB RAM 1 Nvidia A100 Tensor Core 3.56 Apple Mac M1 8 CPUs, 64 GB RAM 1 Apple M1 Metal 9.51 Apple Mac M3 8 CPUs, 64 GB RAM 1 Apple M3 Metal 8.93 Small LLMs Llama3.2:3b Apple Mac M1 8 CPUs, 64 GB RAM 1 Apple M1 Metal 3.79 Apple Mac M3 8 CPUs, 64 GB RAM 1 Apple M3 Metal 2.28 Nvidia Orin-nano 1 CPU, 8 GB RAM 1 1024-core NVIDIA Ampere 19.88 Llama3.2:1b Apple Mac M1 8 CPUs, 64 GB RAM 1 Apple M1 Metal 2.15 Apple Mac M3 8 CPUs, 64 GB RAM 1 Apple M3 Metal 1.52 Nvidia Orin-nano 1 CPU, 8 GB RAM 1 1024-core NVIDIA Ampere 9.82 Gemma:2b Apple Mac M1 8 CPUs, 64 GB RAM 1 Apple M1 Metal 1.25 Apple Mac M3 8 CPUs, 64 GB RAM 1 Apple M3 Metal 0.80 Nvidia Orin-nano 1 CPU, 8 GB RAM 1 1024-core NVIDIA Ampere 5.', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 34, 'content': 'Ampere 9.82 Gemma:2b Apple Mac M1 8 CPUs, 64 GB RAM 1 Apple M1 Metal 1.25 Apple Mac M3 8 CPUs, 64 GB RAM 1 Apple M3 Metal 0.80 Nvidia Orin-nano 1 CPU, 8 GB RAM 1 1024-core NVIDIA Ampere 5.26 Large LLMs Llama3:70b Google Cloud Platform 2 vCPUs, 83 GB RAM 1 Nvidia A100 Tensor Core 11.17 St. Jude internal server 64 CPUs, 2 TB RAM 1 Nvidia A100 Tensor Core 31.11 Apple Mac M1 8 CPUs, 64 GB RAM 1 Apple M1 Metal 60.27 Apple Mac M3 8 CPUs, 64 GB RAM 1 Apple M3 Metal 49.53 Supplementary Table 3. Cost of hosting LLMs on Cloud infrastructure. Cloud service provider Infrastructure GPU Avg. cost per month (USD) Azure 24 vCPUs, 220 GB RAM 1 Nvidia A100 T ensor Core $2,681.29 Azure 8 vCPUs, 56 GB RAM 1 Nvidia T4 T ensor Core $548.96 Google 12 vCPUs, 85 GB RAM 1 Nvidia A100 T ensor Core $2,682.57 Google 8 vCPUs, 30 GB RAM 1 Nvidia T4 T ensor Core $374.03 Google 4 vCPUs, 16 GB RAM 1 Nvidia L4 T ensor Core $516.99 Supplementary Table 4. Benchmarkin', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 34, 'content': 'AM 1 Nvidia A100 T ensor Core $2,682.57 Google 8 vCPUs, 30 GB RAM 1 Nvidia T4 T ensor Core $374.03 Google 4 vCPUs, 16 GB RAM 1 Nvidia L4 T ensor Core $516.99 Supplementary Table 4. Benchmarking of various MyGPT steps for different library sizes. Device Pages PDF count Chunks count (fixed chunk size) Upload time Retrieval time (1st) Retrieval time (avg) Generation time (1st) llama3:7b Generation time (avg) end- to- end time (avg) Mac Laptop (M1 chip) 10 1 70 2m10s 1.28s 1.06s 6.39s 2.99s 11s 100 2 235 2m33s 1.18s 0.62s 4.16s 6.03s 17s 1000 4 3211 5m53s 0.73s 0.59s 3.16s 3.9s 12s 10000 791 58869 95m19s 1.94s 0.64s 5.52s 5.82s 17s Mac Laptop 10 1 70 2.05s 0.38s 0.225s 0.84s 1.205s 4.5s 100 2 235 4.9s 0.1s 0.0775s 2.04s 2.295s 4.5s', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 35, 'content': '35 (M4 chip) 1000 4 3211 1m18s 0.14s 0.11333s 1.9s 2.049s 4.2s 10000 791 58869 23m51s 0.69s 0.651s 1.86s 2.124s 5.0s SJ VM 10 1 70 11s 5.63s 0.80s 11.22s 0.51s 11s 100 2 235 13s 0.77s 0.79s 1.2s 1.07s 6.75s 1000 4 3211 2m12s 0.68s 0.73s 1.16s 1.49s 12.6s 10000 791 58869 46m2s 0.77s 0.85s 1.19s 1.39s 8s Supplementary Table 5: Sample size calculations for MyGPT vs other RAG solutions. Dataset MyGPT correction rate (p1) Other RAG Tools Correction rate (p2) MyGPT-vs-RAG- tool corrections correlation (r) Required number of docs (c) PubMedQA Q&A=294, docs used=294) 0.8163 RAGFlow 0.7755 0.608 806 PrivateGPT 0.5102 0.062 43 Dify 0.6803 0.146 179 Kotaemon 0.4558 0.134 28 BioASQ (Q&A=163, docs used=210) 0.9141 RAGFlow 0.9018 0.561 5106 PrivateGPT 0.7485 0.327 72 Dify 0.6564 0.009 48 Kotaemon 0.7607 0.085 108 Open RAG Benchmark (Q&A=170, docs used =165) 0.9294 RAGFlow 0.9 0.444 1063 PrivateGPT 0.7824 0.299 84 Dify 0.7882 0.306 89 Kotae', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 35, 'content': '485 0.327 72 Dify 0.6564 0.009 48 Kotaemon 0.7607 0.085 108 Open RAG Benchmark (Q&A=170, docs used =165) 0.9294 RAGFlow 0.9 0.444 1063 PrivateGPT 0.7824 0.299 84 Dify 0.7882 0.306 89 Kotaemon 0.5588 0.218 20 Supplementary Table 6: Cost of hosting MyGPT on commodity hardware for group or department usage. Hardware Configuration GPU Price Best LLM to host Mac Mini 64 GB RAM, 512 GB Storage M4 Pro chip $1,839 Gpt-os:20b, Gemma4:31b Framework Desktop 64 GB RAM, 512 GB Storage AMD Ryzen™ AI Max+ 395 $2,213 Gpt-os:20b, Gemma4:31b Mac Studio 128 GB, 512 GB storage M4 Max chip $3,149 Gpt-oss:120b, Llama3.3:70b Framework Desktop 64 GB RAM, 512 GB Storage AMD Ryzen™ AI Max+ 395 $3,253 Gpt-oss:120b, Llama3.3:70b Nvidia DGX Spark 128 GB, 1 TB storage Nvidia Blackwell Architecture $4,000 Gpt-oss:120b, Llama3.3:70b Supplementary Table 7: Cost comparison with various commercial and open-source RAG solutions. Tool Type Pricing structure Cost No', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 35, 'content': 'dia Blackwell Architecture $4,000 Gpt-oss:120b, Llama3.3:70b Supplementary Table 7: Cost comparison with various commercial and open-source RAG solutions. Tool Type Pricing structure Cost Notes Sources Microsoft 365 Copilot (Business) Commercial, SaaS Per user/month $18/user/mo (through Jun 2026), then $21/user/mo Requires a qualifying M365 license on top of the Copilot fee. RAG grounded in M365 Graph data. https://copilot- experts.com/microsoft- copilot-pricing-guide/ Microsoft 365 Copilot (Enterprise) Commercial, SaaS Per user/month $30/user/mo on top of M365 E3 ($39) or E5 ($60) True all-in cost ranges from $66 - $87/user/month when base license is included. https://redresscomplia nce.com/microsoft- copilot-licensing-guide- 2026.html, https://www .directionsonmicrosoft. com/microsoft-to- increase-office-suite- prices-across-the- board-starting-july- 2026/', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 35, 'content': '.directionsonmicrosoft. com/microsoft-to- increase-office-suite- prices-across-the- board-starting-july- 2026/', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 36, 'content': '36 Gemini for Google Workspace Commercial, SaaS Per user/month Now bundled into Workspace plans ($7-- $22/user/mo after 17--22% price increase) Previously sold as a $20 - $30/user/mo add-on; standalone Gemini add-ons discontinued for new purchases. Vertex AI API access is separate. https://www.lineserve.n et/blog/google- workspace-pricing- 2026, https://www.cum ulusglobal.com/google- workspace-gemini-ai- features-and-pricing- changes/ Gemini Enterprise (Google Cloud) Commercial, SaaS Per user/month + consump- tion $30/user/mo as Workspace add-on for existing subscribers; volume discounts at 500+ and 1000+ seats. Vertex AI: $1.88/M input tokens, $7.50/M output tokens. Includes agentic platform, compliance controls (HIPAA, FedRAMP). https://atonementlicens ing.com/blog/gemini- enterprise/, https://wor kspace.google.com/pri cing ChatGPT Business Commercial, SaaS Per user/month $25/user/mo (annual) $30/user/mo (monthly) Includes admi', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 36, 'content': 'mentlicens ing.com/blog/gemini- enterprise/, https://wor kspace.google.com/pri cing ChatGPT Business Commercial, SaaS Per user/month $25/user/mo (annual) $30/user/mo (monthly) Includes admin controls, SOC 2, SAML SSO. Data not used for training. https://chatgpt.com/pri cing/, https://tldv.io/blo g/chatgpt-pricing/ ChatGPT Enterprise Commercial, SaaS Custom quote Negotiated Adds custom data retention, volume licensing, dedicated support. https://openai.com/busi ness/chatgpt-pricing/ Glean Commercial, SaaS Per user/month (quote- based) $45-$50+ user/mo base; AI add-on ~$15/user/mo extra 100-seat minimum; enterprise contracts often exceed $200K/year. No public pricing. Renewal increases of 7--12% reported. https://fritz.ai/glean- review/, https://www.go search.ai/blog/glean- pricing-explained/ Vectara Commercial, SaaS/VPC Usage- based (credits) Enterprise plans: Small $100K/yr, Medium $250K/yr, Large $500K/yr 30-day free trial. Pricing', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 36, 'content': 'go search.ai/blog/glean- pricing-explained/ Vectara Commercial, SaaS/VPC Usage- based (credits) Enterprise plans: Small $100K/yr, Medium $250K/yr, Large $500K/yr 30-day free trial. Pricing based on index size and query volume, not per-seat. VPC and on-prem options available. https://www.vectara.co m/pricing, https://www. extruct.ai/hub/vectara- com-funding/ PrivateGPT (Abstracta) Commercial, self-hosted (AWS) Custom quote No public list price; request private offer via AWS Marketplace Self-hosted on EKS/ECS. Includes SSO, per-user spend caps, RAG pipelines, usage analytics. https://aws.amazon.co m/marketplace/pp/prod view-j3dxmeroxeqju Kotaemon Open source (Apache 2.0) Free (self- hosted) $0 (software) Self-hosted via Docker. Supports OpenAI, Azure, Ollama, local LLMs. Ongoing costs are infra + LLM API tokens. https://github.com/Cinn amon/kotaemon, https: //www.marktechpost.c om/2024/09/01/kotaem on-an-open-source- rag-based-tool-for- chatti', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 36, 'content': 'Ollama, local LLMs. Ongoing costs are infra + LLM API tokens. https://github.com/Cinn amon/kotaemon, https: //www.marktechpost.c om/2024/09/01/kotaem on-an-open-source- rag-based-tool-for- chatting-with-your- documents/ LlamaIndex / LangChain Open source Free $0 (framework) Developer frameworks, not turnkey products. Total cost depends on hosting, vector DB, and LLM API spend. https://www.meilisearc h.com/blog/rag-tools Meilisearch Open source + Cloud Freemium / usage- based Build plan $30/mo (50K searches, 100K docs); Search engine with RAG integration, not a full copilot/assistant. https://www.meilisearc h.com/blog/rag-tools', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 37, 'content': '37 self-hosted is free Key takeaways • Per-user tools carry hidden costs: Microsoft Copilot requires a qualifying M365 base license, so the real per-seat cost is roughly double the headline add-on price. Glean layers on support fees (~10% of contract value) and reported renewal increases. • Open-source options shift cost to operations. Kotaemon, LlamaIndex, and LangChain have zero licensing fees but require engineering time, infrastructure, and LLM API bills. • Gemini pricing restructured in 2025. Google folded Gemini into base Workspace plans with a 17--22% price increase, eliminating the separate add-on SKU for new customers. Existing subscribers on legacy Gemini add-ons should evaluate whether they are double-paying for capabilities also available through Vertex AI. • Vectara is the outlier in pricing model. It charges by usage credits (index size + queries) rather than seats, which can benefit organizations with many documents but relatively few heavy users.', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 37, 'content': 'tara is the outlier in pricing model. It charges by usage credits (index size + queries) rather than seats, which can benefit organizations with many documents but relatively few heavy users.', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 38, 'content': '38 Supplementary Figures Supplementary Figure 1. Information-seeking tasks and solutions. Keyword searches from a single document, multiple documents, a private intranet, or the entire web are routine tasks for most knowledge workers that have been optimized over several decades to provide accurate information. With ChatGPT and similar products, users can get answers to complex questions requiring processing of related material, but accuracy and factuality are lower than those of typical search solutions. MyGPT and other RAG solutions fill a void in the landscape of available discovery tools. The orange-to-green color of the framed boxes indicates the level of accuracy of the information obtained from the indicated task.', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 39, 'content': '39 Supplementary Figure 2. Visualization of chunks from library of documents in the vector space of embeddings. The PCA plot visualizes vector embeddings of chunks from two different libraries of publicly available biomedical articles on (a) G protein-coupled receptors (GPCRs; 27 articles) in blue, (b) intrinsically disordered regions (IDRs; 8 articles) in brown, an d text from (c) Harry Potter -Book 1 in purple (Supplementary Data). Each dot in the plot denotes a chunk from the three different libraries. A question related to the IDR library is converted into a vector (red square). MyGPT then identifies the chunks (orange squares) that are closest to the question and provides them as context for the LLM to generate a response. The generated answer (green square), when converted into a vector using the same embedding model, is closer to the context, highlighting similarity.', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 39, 'content': 'ed into a vector using the same embedding model, is closer to the context, highlighting similarity.', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 40, 'content': '40 Supplementary Figure 3. User interface for MyGPT a MyGPT UI home screen. The home screen displays a chat box (left) with options to switch between pre-loaded document libraries and LLMs, and a dropdown menu that lets users chat directly with the LLM or with documents. The “Your document library” panel (middle) displays a list of documents in the library, with the selected document loaded and visible in', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 41, 'content': '41 the document viewer (right). The top navigation bar displays icons to enable user login to private libraries (e.g., group- or department-level access). Users can also check chat history for various libraries and access the settings menu from this area. Users can verify whether the Backend and LLM server are properly connected and functional from the page footer. b MyGPT upload and chat menu. From the Upload menu, users can upload documents and create a library; each preprocessed library is accessible from the MyGPT home page. When uploading, users can either select a Simple upload with default customization pre- selected or can go to Advanced settings to customize various components such as chunk size, chunk method, chunk overlap, embedding model, distance function, BM25, and Reranker. In the chat settings, users can change the LLM, prompt, precision vs. creativity parameters, chunk retrieval customization, and relevance score customization.', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 41, 'content': 'ranker. In the chat settings, users can change the LLM, prompt, precision vs. creativity parameters, chunk retrieval customization, and relevance score customization.', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 42, 'content': '42 Supplementary Figure 4. Use case 1: Q&A with large library of biomedical literature. a Example of a complex question and answer for the library of Biomedical literature on protein kinases. The user’s question is shown along with the highlighted section of the articles used by the LLM (gpt-oss:20b) to generate the answer. This use case addresses the challenge of retrieving related information from multiple sections of an article and synthesizing an answer after analyzing the data. The library was created using the nomic-', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 43, 'content': '43 embed-text embedding model, which utilizes the reasoning capabilities of the LLM (gpt -oss:20b) to generate an answer. b 282 questions (with ground truth) were collected from 83 documents for this evaluation. MyGPT was able to correctly answer 206 questions with 73% accuracy. The same experiment performed with 15 chunks per question (n=15, maximum chunks per semantic and keyword search) rather than the default 5 chunks yielded more correct answers with 80.13% (238/297) accuracy, showing significant improvement (p = 0.7x10-2; effect size = 0.34 (F, medium); McNemar chi-square test). Also, changing the chunking strategy from fixed to structure-preserving (with n=5) yielded 75.8% (214/282) accuracy, which did not provide a significant improvement (p = 0.27; effect size=0.15 (F, small); McNemar chi-square test) over fixed chunking with 5 chunks. c MyGPT’s Question relevance score (QRS) distribution for questions with correct or incorrect answers (p = 0.13; effect size = 0.08', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 43, 'content': 'all); McNemar chi-square test) over fixed chunking with 5 chunks. c MyGPT’s Question relevance score (QRS) distribution for questions with correct or incorrect answers (p = 0.13; effect size = 0.08 (r, small); Mann Whitney U test) is not statistically different, but the Answer relevancy score (ARS) distribution for correct and incorrect answers (p = 5x10-12; effect size = 0.41 (r, medium); Mann Whitney U test) shows higher scores for the correct answers. MyGPT’s Default Hallucination Index (HI) for correct and incorrect answers (p = 6x10-13; effect size = 0.43 (r, medium); Mann Whitney U test) shows significantly higher hallucination index values for incorrect answers.', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 45, 'content': '45 Supplementary Figure 5. Use case 2: Global health policies in multiple languages. a Example of a question and answer for global health policies with documents in Spanish from the Policy Monitor, a global set of national cancer-related Health Policies library curated by St. Jude Gobal. A user’s question, posed in English, was compared across various document sections using an embedding model with multilingual support (nomic-text-v2-moe:latest). The context in Spanish was utilized by the language model (GPT - oss:20b) to generate the answer in English. b 128 and 160 questions (with ground truth) were collected from 10 health policies in Spanish or French, respectively, providing 320 total Q&A for this evaluation. MyGPT correctly answered 110 (80%) of questions in Spanish and 141 (88.1%) of questions in French from the library. c MyGPT uses the user-provided document language to select the appropriate embedding model and BM25 version. User questions were translated into the d', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 45, 'content': "88.1%) of questions in French from the library. c MyGPT uses the user-provided document language to select the appropriate embedding model and BM25 version. User questions were translated into the document's language using the TranslateGemma LLM before performing semantic and keyword searches for context retrieval. Using this context, the user-selected LLM generates a response in the user's language, and MyGPT uses translated answers from TranslateGemma to compute ARS and HI. Modifications to MyGPT's workflow for this use case are indicated in orange (backend) or green (LLM) bold font.", 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 46, 'content': '46 Supplementary Figure 6. Random forest model for generalizable HI predictor. a Performance of 10 replicates of the random forest model trained and tested on 1045 Q&A from public and private datasets with an 80:20 split to predict incorrect answers. For the best perfoming replicate out of 10, the ROC-AUC is 86.70% for training and 79.72% for testing. b The precision-recall curve of the same best perfroming replicate of the model shows 60.3% for training and 50.95% for testing. The baseline of 17% represents the proportion of incorrect answers for 1045 samples, which will be the performance of a random predictor. Precision-recall of 50.95% for the random forest model signifies a 3-fold better performance than that of a random predictor. c SHAP summary plot of 20 features used to train and test the random forest model to predict incorrect answers. Features are ranked by mean absolute SHAP value, with positive SHAP values pushing the model toward predicting “incorrect”', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 46, 'content': 'ed to train and test the random forest model to predict incorrect answers. Features are ranked by mean absolute SHAP value, with positive SHAP values pushing the model toward predicting “incorrect” and negative SHAP values to ward predicting “correct” answers. The high feature values are marked in pink and the low feature values in blue. d The HI values of all 1045 Q&A using the random forest model. The 3 public datasets combined (p = 2.2´10-16; effect size = 0.38 (r, medium); Mann Whitney U test), BMT (Bone Marrow Transplantation) SOPs (p = 1.4´10-4; effect size = 0.33 (r, medium); Mann Whitney U test) and Protein kinases (p = 8.8´10-16; effect size = 0.59 (r, large); Mann Whitney U test) datasets display significant differences in HI scores between correct and incorrect Q&A pairs with large effect size.', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 46, 'content': 'with large effect size.', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 47, 'content': '47 Supplementary Figure 7. MyGPT running on Nvidia Orin Nano with small-size LLMs. Small LLMs include Llama3.2:3b, Llama3.2:1b, Gemma2:2b, and DeepSeek-R1:1.5b that are ideal for remote locations with limited or no internet access or for air-gapped deployments.', 'type': 'pagechunk'}
{'title': 'MyGPT Manuscript 2026', 'page': 48, 'content': '48 Supplementary Figure 8. Global sensitivity analysis to calculate QRS and ARS for public datasets. a Heatmaps showing classification performance (AUC) across combinations of weight parameters used in QRS computation for a range of vector, BM25, and reranking weight cutoffs. The weights marked in red box are the optimal weights for QRS calculations, 5 for vector, -2 for BM25 and -2 for Reranker. b Corresponding sensitivity maps for ARS across weight configurations for a range of vector, BM25, and reranking weight cutoffs. Color scale indicates AUC, highlighting the robustness and dependence of QRS and ARS on parameter choices. The weights marked in red box are the optimal weights for ARS calculations, 5 for vector, -2 for BM25 and 0 for Reranker.', 'type': 'pagechunk'}
{'title': 'MyGPT Supplementary Information 2026', 'page': 1, 'content': '1 Supplementary Information 1 2 for 3 4 Democratizing Reliable Knowledge-Seeking with MyGPT: 5 A Privacy-First, Open-Source Retrieval-Augmented 6 Generation Platform 7 8 Jaimin Patel1,2,†,*, Jude Downing1,2,†, Hugh Ferguson1,2, Thika You1,2, Duccio Malinverni1,2, Steve 9 Mathew D. A.1,2,8, Inês Chen2, Madison Sluter1,2, Beth Moorefield1,2, Katalin Parej2, Mukundan 10 Ragavan2, Cindy Morris3, Dinesh Keerthi3, Desiree Becerra Armada4, Balint Meszaros1,2, Vikas 11 Trivedi1,2, Shahinur Alam5, Anthony Woodard5, Dan Alford5, Sagar Pathak1,2, Cai Li6, Renato 12 Umeton7, Carlos Rodriguez-Galindo4, Catherine G. Lam4, Stephen Gottschalk3, Charalampos 13 G. Kalodimos2, and M. Madan Babu1,2,7,* 14 15 1Center of Excellence for Data-Driven Discovery, Department of Structural Biology, St. Jude Children’s Research 16 Hospital, Memphis, TN, USA 17 2Department of Structural Biology, St. Jude Children’s Research Hospital, Memphis, TN, USA 18 3Department of Bone Marrow Transplantatio', 'type': 'pagechunk'}
{'title': 'MyGPT Supplementary Information 2026', 'page': 1, 'content': ', St. Jude Children’s Research 16 Hospital, Memphis, TN, USA 17 2Department of Structural Biology, St. Jude Children’s Research Hospital, Memphis, TN, USA 18 3Department of Bone Marrow Transplantation and Cellular Therapy, St. Jude Children’s Research Hospital, Memphis, 19 TN, USA 20 4Department of Global Pediatric Medicine, St. Jude Global, St. Jude Children’s Research Hospital, Memphis, TN, 21 USA 22 5Research Information Services, St. Jude Children’s Research Hospital, Memphis, TN, USA. 23 6Biostatistics, St. Jude Children’s Research Hospital, Memphis, TN, USA. 24 7Office of Data Science, St. Jude Children’s Research Hospital, Memphis, TN, USA. 25 8College of Graduate Health Sciences, University of Tennessee Health Science Center, Memphis, TN, USA 26 †J.P. and J.D. contributed equally to designing the initial prototype for MyGPT. 27 * Corresponding authors: jaimin.patel@stjude.org, madan.babu@stjude.org 28 Table of Contents 29 30 1. Model Context Protocol ...................', 'type': 'pagechunk'}
{'title': 'MyGPT Supplementary Information 2026', 'page': 1, 'content': 'y to designing the initial prototype for MyGPT. 27 * Corresponding authors: jaimin.patel@stjude.org, madan.babu@stjude.org 28 Table of Contents 29 30 1. Model Context Protocol ..................................................................................................................... 2 31 Figure 1. MCP server and client capabilities in MyGPT. ...................................................................... 2 32 Figure 2. MyGPT MCP server exposing information using MCP tools. ............................................... 3 33 Figure 3. MyGPT MCP Client User Interface (UI) and Customization Menu. ...................................... 4 34 2. Verifying MyGPT performance on biomedical Q&A using public literature ................................. 5 35 Figure 4. Verification of MyGPT-generated answers on the dataset of public biomedical articles. ..... 5 36 Figure 5. QRS, ARS and HI distribution of correct vs incorrect Q&A for biomedical dataset. ............. 6', 'type': 'pagechunk'}
{'title': 'MyGPT Supplementary Information 2026', 'page': 1, 'content': 'ication of MyGPT-generated answers on the dataset of public biomedical articles. ..... 5 36 Figure 5. QRS, ARS and HI distribution of correct vs incorrect Q&A for biomedical dataset. ............. 6 37 3. Verifying MyGPT performance on a private document .................................................................. 7 38 Figure 6. Verifying MyGPT (Llama2 + RAG) generated answers for a private document. .................. 8 39 References .............................................................................................................................................. 8 40 41 42', 'type': 'pagechunk'}
{'title': 'MyGPT Supplementary Information 2026', 'page': 2, 'content': '2 1. Model Context Protocol 43 Model Context Protocol (MCP)1 is an open standard developed by Anthropic that enables the integration of 44 language model (LLM) applications with various external tools and data sources, both public and private. 45 MCP operates on a client-server architecture. This means that information from external applications, local 46 databases, or local files can be made available through an MCP server, and that LLMs from an AI 47 application can, in turn, access this information using the MCP client. 48 49 MyGPT provides a basic implementation of the MCP server and client, thus facilitating integration with other 50 private LLM applications and data sources within the same AI ecosystem. The MyGPT MCP server exposes 51 data from the MyGPT backend using MCP tools, which any third-party MCP client can utilize. Conversely, 52 the MyGPT MCP client enables connections to the MCP server, whether it is running locally or in a private 53 network (Fig. 1). Th', 'type': 'pagechunk'}
{'title': 'MyGPT Supplementary Information 2026', 'page': 2, 'content': 'tools, which any third-party MCP client can utilize. Conversely, 52 the MyGPT MCP client enables connections to the MCP server, whether it is running locally or in a private 53 network (Fig. 1). This feature allows the client to use private database information as context in addition to 54 information from documents. 55 56 57 Figure 1. MCP server and client capabilities in MyGPT. 58 a MCP server and b MCP client workflow with MyGPT. 59 60 The MyGPT server is built using a simple Python application that leverages the FastMCP2 library, which is 61 available as a Docker image for quick setup like all components of the MyGPT pipeline. For prototyping 62 purposes, we have developed two MCP tools: get_mygpt_datasets and get_mygpt_documents. To 63 demonstrate how to use these tools, we employed the private third-party MCP Client for Ollama (ollmcp)3. 64 As shown in Figure 2, users can inquire about the available libraries and documents from MyGPT. The 65 LLM first executes t', 'type': 'pagechunk'}
{'title': 'MyGPT Supplementary Information 2026', 'page': 2, 'content': "s, we employed the private third-party MCP Client for Ollama (ollmcp)3. 64 As shown in Figure 2, users can inquire about the available libraries and documents from MyGPT. The 65 LLM first executes the tool and then uses the information generated to answer the user's questions. 66 67", 'type': 'pagechunk'}
{'title': 'MyGPT Supplementary Information 2026', 'page': 3, 'content': '3 68 69 Figure 2. MyGPT MCP server exposing information using MCP tools. 70 The MCP server exposes data from the MyGPT backend to the third-party MCP Client for Ollama. 71 72 The MyGPT MCP Client allows users to connect to either locally running or privately hosted MCP servers. 73 Users follow instructions on the MyGPT GitHub page to establish a connection and, once linked, the 74 customization menu displays details about the server and lists the available tools. Users can select tools 75 and compatible LLMs such as Llama3.1, Gpt-oss, or Qwen3 to prepare for real-time Q&A. For example, 76 an MCP server was created using MCP tools to utilize the Protein Data Bank (PDB). After connecting to 77 the server and selecting Llama3.1:8b, users posed a complex question that required data from both the 78 document and the PDB. As shown in Fig. 3, MyGPT successfully collected the PDB ID from the document, 79 used that information to retrieve additional data from the PDB database,', 'type': 'pagechunk'}
{'title': 'MyGPT Supplementary Information 2026', 'page': 3, 'content': 'data from both the 78 document and the PDB. As shown in Fig. 3, MyGPT successfully collected the PDB ID from the document, 79 used that information to retrieve additional data from the PDB database, and finally generated an answer, 80 illustrating its capability to integrate various data sources to enhance answers. 81 82', 'type': 'pagechunk'}
{'title': 'MyGPT Supplementary Information 2026', 'page': 4, 'content': '4 83 84 Figure 3. MyGPT MCP Client User Interface (UI) and Customization Menu.85', 'type': 'pagechunk'}
{'title': 'MyGPT Supplementary Information 2026', 'page': 5, 'content': '5 2. Verifying MyGPT performance on biomedical Q&A using public literature 86 To ensure that MyGPT works as expected, we curated a dataset consisting of questions and correct 87 answers (ground truths) from domain experts, along with a relevant set of publications to answer the 88 questions. To do so, we asked 10 participants to provide us with at least 20 questions from 8-56 publications 89 in their area of expertise. We thus collected 232 questions from 5 publication libraries (see Supplementary 90 Data 1 for full dataset) covering different scientific domains within the area of drug discovery (G protein-91 coupled receptors (GPRCs), intrinsically disordered regions (IDRs), Chimeric Antigen Receptor T Cells 92 (CAR-T), protein kinases, and Post-Translational Modifications (PTMs)). This evaluation was conducted 93 from February to April 2024, during which time Llama2:7b was the latest available open-source LLM. Of 94 120 articles considered for the evaluation, 66 were publi', 'type': 'pagechunk'}
{'title': 'MyGPT Supplementary Information 2026', 'page': 5, 'content': '). This evaluation was conducted 93 from February to April 2024, during which time Llama2:7b was the latest available open-source LLM. Of 94 120 articles considered for the evaluation, 66 were published before the training cut-off date of 2022 for 95 llama2:7b, while 54 were published afterward, thus assuring that not all the questions could be answered 96 with LLM alone and would need RAG to work. This study design ensured that (a) not all answers to 97 questions or information on the broad topic were expected to be found within the library (e.g., given the 98 library size of documents is small); (b) the LLM models may have been partially trained on information in 99 these documents; and (c) the LLM models alone would be unable to answer certain questions that rely on 100 information from articles published after their training cut-off date. 101 102 The expert-generated questions were answered by MyGPT using Llama2 with the relevant library and 103 Llama2:7b (alone, withou', 'type': 'pagechunk'}
{'title': 'MyGPT Supplementary Information 2026', 'page': 5, 'content': 'ion from articles published after their training cut-off date. 101 102 The expert-generated questions were answered by MyGPT using Llama2 with the relevant library and 103 Llama2:7b (alone, without context). We used the all-MiniLM-L6-v2 embedding model and chunk size of 104 1000 characters without overlaps. The squared Euclidean distance metric was used to find the distance 105 between the chunks and the question and Ollama’s default setting for temperate (0.8), top-P (0.9) and top-106 K (40) to generate the answers. Participants were blinded to the response and were asked to provide 107 feedback on whether the individual answers generated from MyGPT and Llama2 were correct, incorrect or 108 ambiguous. We observed a 31.7% improvement in the number of correct answers generated by Llama2 109 when used within the MyGPT pipeline compared to Llama2 on its own for our dataset (Fig. 4). 110 111 112 Figure 4. Verification of MyGPT-generated answers on the dataset of public biome', 'type': 'pagechunk'}
{'title': 'MyGPT Supplementary Information 2026', 'page': 5, 'content': 'lama2 109 when used within the MyGPT pipeline compared to Llama2 on its own for our dataset (Fig. 4). 110 111 112 Figure 4. Verification of MyGPT-generated answers on the dataset of public biomedical articles. 113 232 questions were collected from 10 domain experts from 5 publication libraries. Answers were generated 114 using Llama2 (no context) and MyGPT (Llama2:7b + RAG). Frequency of answer correctness by 115 standalone LLM and MyGPT. Users were blinded and asked if the generated answers were correct or not, 116 and they provided feedback with “Yes” (dark grey), indicating the answer generated was correct, and “No” 117 (light grey), indicating the answer generated was incorrect. Answers were considered “Ambiguous” if 118 domain experts felt the generated answer was too vague to answer the intended question or if the answer 119 was only partially correct. 120 121', 'type': 'pagechunk'}
{'title': 'MyGPT Supplementary Information 2026', 'page': 5, 'content': 'e to answer the intended question or if the answer 119 was only partially correct. 120 121', 'type': 'pagechunk'}
{'title': 'MyGPT Supplementary Information 2026', 'page': 6, 'content': '6 We then analyzed the QRS, ARS, and the default HI distribution for correct and incorrect answers (Fig. 5). 122 For our dataset of 232 questions from 5 publication libraries, we observed that the QRS scores were 123 comparable for the questions with correct and incorrect answers. This suggested that for our dataset, the 124 library of documents was generally relevant for the kinds of questions asked. In contrast, we observed that 125 the ARS was higher for correct answers compared to incorrect answers (p = 5´10-4), suggesting that correct 126 responses tended to incorporate specific material from the library of documents while generating the 127 response. Consistently, we also observed that the HI (default) was lower for correct compared to incorrect 128 answers (p = 1.2´10-2). The relationship between QRS, ARS, and HI for each question in the dataset shows 129 how either higher QRS, ARS, or both result in lower HI for correct compared to incorrect answers. We wish 130 to c', 'type': 'pagechunk'}
{'title': 'MyGPT Supplementary Information 2026', 'page': 6, 'content': 'he relationship between QRS, ARS, and HI for each question in the dataset shows 129 how either higher QRS, ARS, or both result in lower HI for correct compared to incorrect answers. We wish 130 to clarify that we only used Semantic scores (Qsem, Asem) for QRS and ARS calculations in this exercise to 131 validate whether the RAG concepts4 proposed by Lewis et al. work as expected. Specifically, we excluded 132 the Keyword and Re-rank score-based hybrid retrieval methods that are introduced in the manuscript as 133 advanced RAG approaches. 134 135 Taken together, these observations suggest that the QRS, ARS, and HI can serve as a quantitative guide 136 to help identify whether the question is relevant to the material in the library, whether the answer has 137 incorporated information from the library, and whether the response is likely to have been a hallucination. 138 To enhance trustworthiness in AI-powered tools, users can manually evaluate their datasets, optimize the 139', 'type': 'pagechunk'}
{'title': 'MyGPT Supplementary Information 2026', 'page': 6, 'content': 'on from the library, and whether the response is likely to have been a hallucination. 138 To enhance trustworthiness in AI-powered tools, users can manually evaluate their datasets, optimize the 139 HI scoring function, and use statistical learning approaches to leverage QRS, ARS, and HI scores to predict 140 and classify responses as correct or incorrect. HI optimization and classification require generating a library, 141 domain-specific questions, and labels indicating whether LLM-generated answers are correct or incorrect. 142 By leveraging QRS, ARS, and HI, users can significantly improve the reliability and accuracy of answers 143 generated by LLMs. We want to emphasize that correctness labels for LLM -generated answers are 144 subjective. A user’s preference for what constitutes a correct answer may result in varying ratios of correct 145 to incorrect labels. 146 147 148 149 Figure 5. QRS, ARS and HI distribution of correct vs incorrect Q&A for biomedical dataset.', 'type': 'pagechunk'}
{'title': 'MyGPT Supplementary Information 2026', 'page': 6, 'content': 'tes a correct answer may result in varying ratios of correct 145 to incorrect labels. 146 147 148 149 Figure 5. QRS, ARS and HI distribution of correct vs incorrect Q&A for biomedical dataset. 150 a Question relevance score (QRS) distribution for correct and incorrect answers using Semantic scores 151 (Qsem). b Answer relevance score (ARS) distribution using Semantic scores (Asem) for correct and incorrect 152 answers (p = 5´10-4; effect size = 0.23, Wilcoxon rank-sum test). c Hallucination Index for correct and 153 incorrect answers using default parameters (p = 1.2´10-2; effect size=0.17; Wilcoxon rank-sum test). 154', 'type': 'pagechunk'}
{'title': 'MyGPT Supplementary Information 2026', 'page': 7, 'content': '7 3. Verifying MyGPT performance on a private document 155 156 There is growing concern and evidence that LLMs are trained on public data, including research articles5, 157 making it challenging to determine which specific publications were included in the training datasets for the 158 various LLMs. To circumvent this concern, we used the previous version of MyGPT manuscript (which 159 could not have been included in the LLM training dataset) as a private document, allowing us to test how 160 the RAG answers questions regarding a private document. 161 162 We curated a dataset of 40 questions related to the MyGPT manuscript from 2 domain experts: 25 were 163 related to information present in the manuscript (relevant questions), and 15 were related to the topic but 164 the manuscript lacked the information to answer them (irrelevant questions) . MyGPT answered all 25 165 relevant questions correctly and failed to answer the 15 irrelevant questions (Fig. 6a-c). In contras', 'type': 'pagechunk'}
{'title': 'MyGPT Supplementary Information 2026', 'page': 7, 'content': 'nuscript lacked the information to answer them (irrelevant questions) . MyGPT answered all 25 165 relevant questions correctly and failed to answer the 15 irrelevant questions (Fig. 6a-c). In contrast, the 166 LLM alone generated incorrect (or hallucinated) answers for all 40 questions. The confidence metrics (ARS, 167 QRS, and HI) obtained using MyGPT could differentiate between correct and incorrect answers and identify 168 instances of hallucination decisively (Fig. 6d-f). Like previous study, we have used only Semantic scores 169 (Qsem, Asem) for QRS and ARS calculations. We observed that QRS (p = 1x10-3) and ARS (p = 1.9x10-2) were 170 significantly higher for relevant questions and correct answers, and HI (p = 1.7x10-3) were significantly 171 higher for irrelevant questions and incorrect answers. These findings demonstrate that MyGPT (and the 172 confidence metrics) works as expected and outperforms standalone LLMs when using private documents. 173 174 175', 'type': 'pagechunk'}
{'title': 'MyGPT Supplementary Information 2026', 'page': 7, 'content': 'd incorrect answers. These findings demonstrate that MyGPT (and the 172 confidence metrics) works as expected and outperforms standalone LLMs when using private documents. 173 174 175', 'type': 'pagechunk'}
{'title': 'MyGPT Supplementary Information 2026', 'page': 8, 'content': '8 Figure 6. Verifying MyGPT (Llama2 + RAG) generated answers for a private document. 176 a Frequency of answer correctness with Llama2:7b alone and MyGPT (Llama2:7b + RAG). Users were 177 asked if the generated answers were correct or not, and they provided feedback with “Yes” (dark grey), 178 indicating the answer generated was correct, and “No” (light grey), indicating the answer generated was 179 incorrect. The 40 questions were designed to have 25 questions where the necessary information to answer 180 was in the private document and 15 irrelevant questions (i.e., questions lacking the necessary information 181 in the document to answer them correctly). b Example of relevant question and correct answer, indicated 182 by high QRS, ARS, and low HI. c Example of irrelevant questions and incorrect answers, indicated by low 183 QRS, ARS, and high HI. d Question relevancy score (QRS, using Semantic scores (Qsem)) distribution for 184 questions of correct and incorrect answers', 'type': 'pagechunk'}
{'title': 'MyGPT Supplementary Information 2026', 'page': 8, 'content': 'ions and incorrect answers, indicated by low 183 QRS, ARS, and high HI. d Question relevancy score (QRS, using Semantic scores (Qsem)) distribution for 184 questions of correct and incorrect answers (p=1x10-3; effect size=0.52; Wilcoxon rank-sum test). e Answer 185 relevancy score (ARS, using Semantic scores (Asem)) distribution for correct and incorrect answers 186 (p=1.9x10-2; effect size=0.37; Wilcoxon rank-sum test). f Default Hallucination Index (HI) for correct and 187 incorrect answers using arithmetic means between QRS and ARS (p=1.7x10-3; effect size=0.5; Wilcoxon 188 rank-sum test). 189 190 References 191 1 Model Context Protocol (MCP) (GitHub https://github.com/modelcontextprotocol, 2024). 192 2 FastMCP (GitHub https://github.com/jlowin/fastmcp, 2024). 193 3 MCP Client for Ollama (ollmcp) (GitHub https://github.com/jonigl/mcp-client-for-ollama, 2025). 194 4 Lewis, P. et al. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. ArXiv 195 abs/2005.11', 'type': 'pagechunk'}
{'title': 'MyGPT Supplementary Information 2026', 'page': 8, 'content': 'ent for Ollama (ollmcp) (GitHub https://github.com/jonigl/mcp-client-for-ollama, 2025). 194 4 Lewis, P. et al. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. ArXiv 195 abs/2005.11401 (2020). 196 5 Gibney, E. Has your paper been used to train an AI model? Almost certainly. Nature 632, 715–716 197 (2024). https://doi.org//10.1038/d41586-024-02599-9 198 199', 'type': 'pagechunk'}
{'title': 'Installation on Windows', 'page': 1, 'content': 'Installation on Windows Requirements To Run the MyGPT pipeline, we will need the following minimum specifications for your system: 8 CPUs 8 GB Memory (16 GB or more for better response time) 10 GB hard-drive storage We will also need several tools to run the pipeline: Python Chocolatey Git Docker Ollama Requirements installation We will install these two tools in the following steps: 1. install Python and Pip Check if you have python installed on your system, by running following command from Command Prompt. python -v If python is not isntalled on your PC, install it by downloading latest release from this page: https://www.python.org/downloads/windows/ While Python installtion, make sure to select the following option: 2. Chocolatory installation', 'type': 'pagechunk'}
{'title': 'Installation on Windows', 'page': 2, 'content': 'Check if you have choco installed on your system by running following command. choco list If you get error that choco not found , you can install choco by folliwng command from the new Command Prompt window: @"%SystemRoot%\\System32\\WindowsPowerShell\\v1.0\\powershell.exe" -NoProfile - InputFormat None -ExecutionPolicy Bypass -Command " [System.Net.ServicePointManager]::SecurityProtocol = 3072; iex ((New-Object System.Net.WebClient).DownloadString(\'https://community.chocolatey.org/install.ps1\'))" && SET "PATH=%PATH%;%ALLUSERSPROFILE%\\chocolatey\\bin" 3. Git installation Check if you have git installed on your system by running following command. git --help if you get message git not found, you can install it using choco. choco install git After you get success message, you can close the Command Prompt window and open new Command Prompt as Administrator to run next command. 4. Docker installation To run MyGPT with CPUs-only, the entire pipeline will run as a single unit from Docker. Go to', 'type': 'pagechunk'}
{'title': 'Installation on Windows', 'page': 2, 'content': 'and Prompt window and open new Command Prompt as Administrator to run next command. 4. Docker installation To run MyGPT with CPUs-only, the entire pipeline will run as a single unit from Docker. Go to the official Docker installation page for Windows and install the appropriate Docker on your system: https://docs.docker.com/desktop/install/windows-install/ After installation, to verify if Docker is running on your system, run the following code: docker --help If you get an error that docker not found , Go to the official Docker installation page for Linux and install the appropriate Docker on your system: https://docs.docker.com/desktop/install/linux-install/ 5. Ollama installation Finally, download and install Ollama by following instructions from this offical Ollama site You can check if Ollama is running by visiting http://localhost:11434/ in your default browser. [!CAUTION] After installing Ollama, close any open Terminal/Command Prompt before you pull Llama3. Once you start Ollama', 'type': 'pagechunk'}
{'title': 'Installation on Windows', 'page': 2, 'content': 'Ollama is running by visiting http://localhost:11434/ in your default browser. [!CAUTION] After installing Ollama, close any open Terminal/Command Prompt before you pull Llama3. Once you start Ollama, you have to pull Llama3 model by running following command: ollama pull llama3 Also, get the nomic embedding model, which is best performing embedding model for MyGPT pipeline, by running following command: ollama pull nomic-embed-text', 'type': 'pagechunk'}
{'title': 'Installation on Windows', 'page': 3, 'content': "MyGPT installation 1. Get MyGPT source code If you have zip file with MyGPT installation instructions, you can unzip it and copy it on your desktop or your desired location. You can skip remaining instructions from below and go to Step 2. Or you can get installation instructions by running following command from Terminal. It will create MyGPT_public folder on your Desktop or desired folder. To get this from GitHub, we will need GitHub username and GitHub access token as the GitHub repo is private. When you run the following command, it will ask for this credentials. git clone https://github.com/stjude-c3d/MyGPT.git [!NOTE] If you don't have GitHub access token, you can genearte classic token using this guideline: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your- personal-access-tokens#creating-a-personal-access-token-classic [!CAUTION] While generating you GitHub Access Token, make sure you check access for repo and read:packages similar", 'type': 'pagechunk'}
{'title': 'Installation on Windows', 'page': 3, 'content': 'cure/managing-your- personal-access-tokens#creating-a-personal-access-token-classic [!CAUTION] While generating you GitHub Access Token, make sure you check access for repo and read:packages similar to image below. 2. login to GitHub Docker registry As the GitHub repository is private rightnow, we have to login to GitHub Docker registry to use the prebuilt images. To login to GitHub Docker registry, run following command. It will ask for your GitHub username and password. [!CAUTION] Make sure Docker Desktop application is open and running before running the command. the password is your access token (same token you used in step 1), not your github password you use to login in github account.', 'type': 'pagechunk'}
{'title': 'Installation on Windows', 'page': 4, 'content': 'docker login ghcr.io 3. Run docker containers We will run following script to run docker containers: cd MyGPT_public\\installation\\windows run_docker.bat This script should take around 5-10 minutes to run. Once the script finish running, open following pages in your default browser. You can see status of different components of MyGPT pipeline on these pages. backend: http://localhost:8000/ frontend: http://localhost:3000/', 'type': 'pagechunk'}
{'title': 'Installation on Windows', 'page': 5, 'content': 'MyGPT pipeline usage Upload publications You can upload documents or publications as PDF file by going to the settings, and selecting first tab: "Publication libraries" and selecting "Upload documents" menu as screeshot below. start by giving your library a "Library Name" and start uploading documents. You can upload upto 40 PDFs at a time. You can upload multiple times to same library. It will take some time to process the documents and once it\'s done, you can see your new library in the list with success message. Add Zotero library You can add Zotero library by going to the settings, and selecting first tab: "Publication libraries" and selecting "Add Zotero library" menu as screeshot below. You must provide Zotero AI key, Zotero Group ID/User ID, and Zotero Collection ID to add Zotero library.', 'type': 'pagechunk'}
{'title': 'Installation on Windows', 'page': 6, 'content': "If you need help getting any of these, you can click on the help button next to the input field or check section after screenshot. Zotero API key: You can generate an API key in your profile settings https://www.zotero.org/settings/keys Zotero User ID: You can get it from Zotero prfile page by visiting https://www.zotero.org/settings/keys. It's 6-7 digit number. Zotero Group ID: You can get it from URL of the group, for example, here is URL for BABU group and group ID is 4982570 : https://www.zotero.org/groups/4982570/babu_group/ Zotero Collection ID: You can get it from URL of the collection, for example, here is URL for BABU group and collection ID is YTPMLXYY : https://www.zotero.org/groups/4982570/babu_group/collections/YTPMLXYY Other optional tasks for MyGPT pipeline Run MCP server", 'type': 'pagechunk'}
{'title': 'Installation on Windows', 'page': 7, 'content': "To run MCP server, run following command if you are using pre-built docker images: cd MyGPT_public\\installation\\windows run_mcp_server.bat This will start the MCP server and you can access it at http://localhost:5001/sse/ Configure MCP Client in MyGPT To setup MCP client in MyGPT, you need to stop the frontend, and then update the environment variable in the .env file. To stop the frontend, run following command if you are using pre-built docker images: cd MyGPT_public\\installation\\windows stop_docker.bat Change following line in .env file: REACT_APP_MCP_SERVER_URL = 'http://localhost:5001/sse' REACT_APP_MCP_SHOW_MCP_MENU=true You can change MCP server URL to your custom MCP server URL if you are using a different server. Then, you can start the frontend again by running following command if you are using pre-built docker images: cd MyGPT_public\\installation\\windows start_docker.bat Once the frontend is running, you can access it at http://localhost:3000/ and you should see MCP menu", 'type': 'pagechunk'}
{'title': 'Installation on Windows', 'page': 7, 'content': 'd if you are using pre-built docker images: cd MyGPT_public\\installation\\windows start_docker.bat Once the frontend is running, you can access it at http://localhost:3000/ and you should see MCP menu in the settings page. Select the MCP tools you want to use from the customization page. Once you select the tools, go to the main page and select LLM with tool support and you should see the MCP tools in the chat interface.', 'type': 'pagechunk'}
{'title': 'Installation on Windows', 'page': 8, 'content': 'stop docker containers To stop docker containers, run following command: stop_docker.bat', 'type': 'pagechunk'}
{'title': 'Installation on Mac OS', 'page': 1, 'content': 'Installation on Mac OS Requirements To Run the MyGPT pipeline, we will need the following minimum specifications for your system: 8 CPUs 8 GB Memory (16 GB or more for better response time) 10 GB hard-drive storage We will also need several tools to run the pipeline: Brew Git Docker Ollama Requirements installation We will install these required tools in the following steps: 1. Brew installation Check if you have brew installed on your system by running following command. brew --version If you get error that brew not found , you can install brew by folliwng command from the new terminal window (similar to Jupyter installation but open new tab for terminal without closing existing window that\'s running Jupyter). For more details about brew check this page /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" 2. Brew packages installation We will install Brew packages by running following commands: brew install git 3. Docker installation To run', 'type': 'pagechunk'}
{'title': 'Installation on Mac OS', 'page': 1, 'content': '//raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" 2. Brew packages installation We will install Brew packages by running following commands: brew install git 3. Docker installation To run MyGPT with CPUs-only, the entire pipeline will run as a single unit from Docker. Check if you have docker installed on your system by running following command. docker --version If you get an error that docker not found , go to the official Docker installation page for Mac and install the appropriate Docker on your system: https://docs.docker.com/desktop/install/mac-install/ You can change Docker setting to match requirements for MyGPT:', 'type': 'pagechunk'}
{'title': 'Installation on Mac OS', 'page': 2, 'content': "4. Ollama installation Finally, download and install Ollama by following instructions from this offical Ollama site You can check if Ollama is running by visiting http://localhost:11434/ in your default browser. [!CAUTION] After installing Ollama, close any open Terminal/Command Prompt before you pull Llama3. Once you start Ollama, you have to pull Llama3 model by running following command: ollama pull llama3 Also, get the nomic embedding model, which is best performing embedding model for MyGPT pipeline, by running following command: ollama pull nomic-embed-text MyGPT installation You have 2 options to install MyGPT pipeline: 1. Use pre-built docker images (faster and easier to install, but you can't modify source code) 2. Build docker images from source code (slower, but you can modify source code) Option 1: Use pre-built docker images 1. Get MyGPT source code If you have zip file with MyGPT installation instructions, you can unzip it and copy it on your desktop or your desired locat", 'type': 'pagechunk'}
{'title': 'Installation on Mac OS', 'page': 2, 'content': 'ource code) Option 1: Use pre-built docker images 1. Get MyGPT source code If you have zip file with MyGPT installation instructions, you can unzip it and copy it on your desktop or your desired location. You can skip remaining instructions from below and go to Step 2.', 'type': 'pagechunk'}
{'title': 'Installation on Mac OS', 'page': 3, 'content': "Or you can get installation instructions by running following command from Terminal. It will create MyGPT folder on your Desktop or desired folder. To get this from GitHub, we will need GitHub username and GitHub access token as the GitHub repo is private. When you run the following command, it will ask for this credentials. git clone https://github.com/stjude-c3d/MyGPT.git [!NOTE] If you don't have GitHub access token, you can genearte classic token using this guideline: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your- personal-access-tokens#creating-a-personal-access-token-classic [!CAUTION] While generating you GitHub Access Token, make sure you check access for repo and read:packages similar to image below. 2. login to GitHub Docker registry As the GitHub repository is private rightnow, we have to login to GitHub Docker registry to use the prebuilt images. To login to GitHub Docker registry, run following command. It will ask for yo", 'type': 'pagechunk'}
{'title': 'Installation on Mac OS', 'page': 3, 'content': 'istry As the GitHub repository is private rightnow, we have to login to GitHub Docker registry to use the prebuilt images. To login to GitHub Docker registry, run following command. It will ask for your GitHub username and password. [!CAUTION] Make sure Docker Desktop application is open and running before running the command. the password is your access token (same token you used in step 1), not your github password you use to login in github account. docker login ghcr.io 3. Run docker containers We will run following script to run docker containers:', 'type': 'pagechunk'}
{'title': 'Installation on Mac OS', 'page': 4, 'content': 'cd MyGPT/installation/macOS/prebuilt_images bash run_docker.sh This script should take around 5-10 minutes to run. While above script is running, it will open several pages in your default browser. You can see status of different components of MyGPT pipeline on these pages. backend: http://localhost:8000/ frontend: http://localhost:3000/ Option 2: Build docker images from source code 1. Get MyGPT source code', 'type': 'pagechunk'}
{'title': 'Installation on Mac OS', 'page': 5, 'content': "If you have zip file with MyGPT source code, you can unzip it and copy it on your desktop or your desired location. You can skip remaining instructions from below and go to Step 2. If you want to get source code from GitHub we will first get MyGPT source code by running following command. It will create MyGPT folder on your Desktop. To get the source code from GitHub, we will need GitHub username and GitHub access token as the GitHub repo is private. When you run the following command, it will ask for this credentials. git clone https://github.com/stjude-c3d/MyGPT.git Note: If you don't have GitHub access token, you can genearte classic token using this guideline: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your- personal-access-tokens#creating-a-personal-access-token-classic 2. Build docker images We will run following script to build docker images: cd MyGPT bash installation/macOS/build_images/build_docker.sh 3. Run docker containers We", 'type': 'pagechunk'}
{'title': 'Installation on Mac OS', 'page': 5, 'content': 'g-a-personal-access-token-classic 2. Build docker images We will run following script to build docker images: cd MyGPT bash installation/macOS/build_images/build_docker.sh 3. Run docker containers We will run following script to run docker containers: bash installation/macOS/build_images/run_docker.sh This script should take around 5-10 minutes to run. While above script is running, it will open several pages in your default browser. You can see status of different components of MyGPT pipeline on these pages. backend: http://localhost:8000/ frontend: http://localhost:3000/', 'type': 'pagechunk'}
{'title': 'Installation on Mac OS', 'page': 6, 'content': 'MyGPT pipeline usage Upload publications You can upload documents or publications as PDF file by going to the settings, and selecting first tab: "Publication libraries" and selecting "Upload documents" menu as screeshot below. start by giving your library a "Library Name" and start uploading documents. You can upload upto 40 PDFs at a time. You can upload multiple times to same library. It will take some time to process the documents and once it\'s done, you can see your new library in the list with success message.', 'type': 'pagechunk'}
{'title': 'Installation on Mac OS', 'page': 7, 'content': 'Add Zotero library You can add Zotero library by going to the settings, and selecting first tab: "Publication libraries" and selecting "Add Zotero library" menu as screeshot below. You must provide Zotero AI key, Zotero Group ID/User ID, and Zotero Collection ID to add Zotero library. If you need help getting any of these, you can click on the help button next to the input field or check section after screenshot.', 'type': 'pagechunk'}
{'title': 'Installation on Mac OS', 'page': 8, 'content': "Zotero API key: You can generate an API key in your profile settings https://www.zotero.org/settings/keys Zotero User ID: You can get it from Zotero prfile page by visiting https://www.zotero.org/settings/keys. It's 6-7 digit number. Zotero Group ID: You can get it from URL of the group, for example, here is URL for BABU group and group ID is 4982570 : https://www.zotero.org/groups/4982570/babu_group/ Zotero Collection ID: You can get it from URL of the collection, for example, here is URL for BABU group and collection ID is YTPMLXYY : https://www.zotero.org/groups/4982570/babu_group/collections/YTPMLXYY Other optional tasks for MyGPT pipeline create super user To create super user, run following command if you are using pre-built docker images:", 'type': 'pagechunk'}
{'title': 'Installation on Mac OS', 'page': 9, 'content': 'cd MyGPT/installation/macOS/prebuilt_images/ bash create_superuser.sh Or run following command if you are building docker images: bash MyGPT/installation/macOS/build_images/create_superuser.sh You can check backend database at http://localhost:8000/admin/ with username and password you created in above step. Run MCP server To run MCP server, run following command if you are using pre-built docker images: cd MyGPT/installation/macOS/prebuilt_images/ bash run_mcp_server.sh Or run following command if you are building docker images: bash MyGPT/installation/macOS/build_images/run_mcp_server.sh This will start the MCP server and you can access it at http://localhost:5001/sse/ Configure MCP Client in MyGPT To setup MCP client in MyGPT, you need to stop the frontend, and then update the environment variable in the .env file. To stop the frontend, run following command if you are using pre-built docker images: bash stop_docker.sh Or run following command if you are building docker images: bas', 'type': 'pagechunk'}
{'title': 'Installation on Mac OS', 'page': 9, 'content': "nt variable in the .env file. To stop the frontend, run following command if you are using pre-built docker images: bash stop_docker.sh Or run following command if you are building docker images: bash MyGPT/installation/macOS/build_images/stop_docker.sh Change following line in .env file: REACT_APP_MCP_SERVER_URL = 'http://localhost:5001/sse' REACT_APP_MCP_SHOW_MCP_MENU=true You can change MCP server URL to your custom MCP server URL if you are using a different server. Then, you can start the frontend again by running following command if you are using pre-built docker images: bash run_docker.sh ``` Or run following command if you are building docker images: bash MyGPT/installation/macOS/build_images/run_docker.sh Once the frontend is running, you can access it at http://localhost:3000/ and you", 'type': 'pagechunk'}
{'title': 'Installation on Mac OS', 'page': 10, 'content': 'should see MCP menu in the settings page. Select the MCP tools you want to use from the customization page. Once you select the tools, go to the main page and select LLM with tool support and you should see the MCP tools in the chat interface. <img src="../../images/MyGPT_MCP_UI.png?raw=true" width="1000px"> ### stop docker containers To stop docker containers, run following command if you are using pre-built docker images: cd MyGPT/installation/macOS/prebuilt_images/ bash stop_docker.sh Or run following command if you are building docker images: bash MyGPT/installation/macOS/build_images/stop_docker.sh ```', 'type': 'pagechunk'}
{'title': 'Installation on Linux', 'page': 1, 'content': 'Installation on Linux Requirements This readme is an example for a Linux with Ubuntu platform. Only difference between Ubuntu and other platform is how packages are installed, for example, we will be using apt in this readme but you can use your own installer like dnf or pacman as per your Linux platform. To Run the MyGPT pipeline, we will need the following minimum specifications for your system: 8 CPUs 8 GB Memory (16 GB or more for better response time) 10 GB hard-drive storage We will also need several tools to run the pipeline: Apt Git Docker Ollama Requirements installation We will install these two tools in the following steps: 1. Apt installation Check if you have apt installed on your system by running following command. apt --help If you get error that apt not found, you can check what package manager your platform is using as default. Some of the default package managers for Linux system are dnf, yum and pacman. If you have any of these package managers, you might want t', 'type': 'pagechunk'}
{'title': 'Installation on Linux', 'page': 1, 'content': 'check what package manager your platform is using as default. Some of the default package managers for Linux system are dnf, yum and pacman. If you have any of these package managers, you might want to edit the commands in the cell below according to your needs. We will be using these package manager to install Git and Docker. 2. Apt packages installation We will install Apt packages by running following commands: sudo apt install git 3. Docker installation The MyGPT pipeline will run as a single unit from Docker. Check if you have docker installed on your system by running following command. docker --version If you get an error that docker not found , Go to the official Docker installation page for Linux and install the appropriate Docker on your system: https://docs.docker.com/desktop/install/linux- install/ You can change Docker setting to match requirements for MyGPT:', 'type': 'pagechunk'}
{'title': 'Installation on Linux', 'page': 1, 'content': 'nstall/linux- install/ You can change Docker setting to match requirements for MyGPT:', 'type': 'pagechunk'}
{'title': 'Installation on Linux', 'page': 2, 'content': "4. Ollama installation Finally, download and install Ollama by following instructions from this offical Ollama site After installation, you can run following command in Terminal to start Ollama server, if you haven't done it already: sudo systemctl start ollama You can check if Ollama is running by visiting http://localhost:11434/ in your default browser. [!CAUTION] After installing Ollama, close any open Terminal/Command Prompt before you pull Llama3. Once you start Ollama, you have to pull Llama3 model by running following command: ollama pull llama3 Also, get the nomic embedding model, which is best performing embedding model for MyGPT pipeline, by running following command: ollama pull nomic-embed-text MyGPT installation You have 2 options to install MyGPT pipeline: 1. Use pre-built docker images (faster and easier to install, but you can't modify source code) 2. Build docker images from source code (slower, but you can modify source code)", 'type': 'pagechunk'}
{'title': 'Installation on Linux', 'page': 2, 'content': "cker images (faster and easier to install, but you can't modify source code) 2. Build docker images from source code (slower, but you can modify source code)", 'type': 'pagechunk'}
{'title': 'Installation on Linux', 'page': 3, 'content': "Option 1: Use pre-built docker images 1. Get MyGPT source code If you have zip file with MyGPT installation instructions, you can unzip it and copy it on your desktop or your desired location. You can skip remaining instructions from below and go to Step 2. Or you can get installation instructions by running following command from Terminal. It will create MyGPT_public folder on your Desktop or desired folder. To get this from GitHub, we will need GitHub username and GitHub access token as the GitHub repo is private. When you run the following command, it will ask for this credentials. git clone https://github.com/stjude-c3d/MyGPT.git [!NOTE] If you don't have GitHub access token, you can genearte classic token using this guideline: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your- personal-access-tokens#creating-a-personal-access-token-classic [!CAUTION] While generating you GitHub Access Token, make sure you check access for repo and rea", 'type': 'pagechunk'}
{'title': 'Installation on Linux', 'page': 3, 'content': 'account-and-data-secure/managing-your- personal-access-tokens#creating-a-personal-access-token-classic [!CAUTION] While generating you GitHub Access Token, make sure you check access for repo and read:packages similar to image below. 2. login to GitHub Docker registry As the GitHub repository is private rightnow, we have to login to GitHub Docker registry to use the prebuilt images. To login to GitHub Docker registry, run following command. It will ask for your GitHub username and password. [!CAUTION] Make sure Docker Desktop application is open and running before running the command. the password is your access token (same token you used in step 1), not your github password you use to login in github account.', 'type': 'pagechunk'}
{'title': 'Installation on Linux', 'page': 4, 'content': 'docker login ghcr.io 3. Run docker containers We will run following script to run docker containers: cd MyGPT/installation/linux/prebuilt_images bash run_docker.sh This script should take around 5-10 minutes to run. While above script is running, it will open several pages in your default browser. You can see status of different components of MyGPT pipeline on these pages. backend: http://localhost:8000/ frontend: http://localhost:3000/', 'type': 'pagechunk'}
{'title': 'Installation on Linux', 'page': 5, 'content': "Option 2: Build docker images from source code 1. Get MyGPT source code If you have zip file with MyGPT source code, you can unzip it and copy it on your desktop or your desired location. You can skip remaining step from below and go to Step 2. If you want to get source code from GitHub we will first get MyGPT source code by running following command. It will create MyGPT folder on your Desktop. To get the source code from GitHub, we will need GitHub username and GitHub access token as the GitHub repo is private. When you run the following command, it will ask for this credentials. git clone https://github.com/mb-group/MyGPT.git Note: If you don't have GitHub access token, you can genearte classic token using this guideline: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your- personal-access-tokens#creating-a-personal-access-token-classic 2. Build docker images We will run following script to build docker images: cd MyGPT bash installation/l", 'type': 'pagechunk'}
{'title': 'Installation on Linux', 'page': 5, 'content': '-and-data-secure/managing-your- personal-access-tokens#creating-a-personal-access-token-classic 2. Build docker images We will run following script to build docker images: cd MyGPT bash installation/linux/build_images/build_docker.sh 3. Run docker containers We will run following script to run docker containers: bash installation/linux/build_images/run_docker.sh This script should take around 5-10 minutes to run. While above script is running, it will open several pages in your default browser. You can see status of different components of MyGPT pipeline on these pages.', 'type': 'pagechunk'}
{'title': 'Installation on Linux', 'page': 6, 'content': 'backend: http://localhost:8000/ frontend: http://localhost:3000/ MyGPT pipeline usage Upload publications You can upload documents or publications as PDF file by going to the settings, and selecting first tab: "Publication libraries" and selecting "Upload documents" menu as screeshot below. start by giving your library a "Library Name" and start uploading documents. You can upload upto 40 PDFs at a time. You can upload multiple times to same library. It will take some time to process the documents and once it\'s done, you can see your new library in the list with success message.', 'type': 'pagechunk'}
{'title': 'Installation on Linux', 'page': 7, 'content': 'Add Zotero library You can add Zotero library by going to the settings, and selecting first tab: "Publication libraries" and selecting "Add Zotero library" menu as screeshot below. You must provide Zotero AI key, Zotero Group ID/User ID, and Zotero Collection ID to add Zotero library. If you need help getting any of these, you can click on the help button next to the input field or check section after screenshot.', 'type': 'pagechunk'}
{'title': 'Installation on Linux', 'page': 8, 'content': "Zotero API key: You can generate an API key in your profile settings https://www.zotero.org/settings/keys Zotero User ID: You can get it from Zotero prfile page by visiting https://www.zotero.org/settings/keys. It's 6-7 digit number. Zotero Group ID: You can get it from URL of the group, for example, here is URL for BABU group and group ID is 4982570 : https://www.zotero.org/groups/4982570/babu_group/ Zotero Collection ID: You can get it from URL of the collection, for example, here is URL for BABU group and collection ID is YTPMLXYY : https://www.zotero.org/groups/4982570/babu_group/collections/YTPMLXYY Other optional tasks for MyGPT pipeline create super user To create super user, run following command if you are using pre-built docker images:", 'type': 'pagechunk'}
{'title': 'Installation on Linux', 'page': 9, 'content': 'cd MyGPT/installation/linux/prebuilt_images/ bash create_superuser.sh Or run following command if you are building docker images: bash MyGPT/installation/linux/build_images/create_superuser.sh You can check backend database at http://localhost:8000/admin/ with username and password you created in above step. Run MCP server To run MCP server, run following command if you are using pre-built docker images: cd MyGPT/installation/linux/prebuilt_images/ bash run_mcp_server.sh Or run following command if you are building docker images: bash MyGPT/installation/linux/build_images/run_mcp_server.sh This will start the MCP server and you can access it at http://localhost:5001/sse/ Configure MCP Client in MyGPT To setup MCP client in MyGPT, you need to stop the frontend, and then update the environment variable in the .env file. To stop the frontend, run following command if you are using pre-built docker images: bash stop_docker.sh Or run following command if you are building docker images: bas', 'type': 'pagechunk'}
{'title': 'Installation on Linux', 'page': 9, 'content': "nt variable in the .env file. To stop the frontend, run following command if you are using pre-built docker images: bash stop_docker.sh Or run following command if you are building docker images: bash MyGPT/installation/linux/build_images/stop_docker.sh Change following line in .env file: REACT_APP_MCP_SERVER_URL = 'http://localhost:5001/sse' REACT_APP_MCP_SHOW_MCP_MENU=true You can change MCP server URL to your custom MCP server URL if you are using a different server. Then, you can start the frontend again by running following command if you are using pre-built docker images: bash run_docker.sh ``` Or run following command if you are building docker images: bash MyGPT/installation/linux/build_images/run_docker.sh Once the frontend is running, you can access it at http://localhost:3000/ and you", 'type': 'pagechunk'}
{'title': 'Installation on Linux', 'page': 10, 'content': 'should see MCP menu in the settings page. Select the MCP tools you want to use from the customization page. Once you select the tools, go to the main page and select LLM with tool support and you should see the MCP tools in the chat interface. <img src="../../images/MyGPT_MCP_UI.png?raw=true" width="1000px"> ### stop docker containers To stop docker containers, run following command if you are using pre-built docker images: cd MyGPT/installation/linux/prebuilt_images/ bash stop_docker.sh Or run following command if you are building docker images: bash MyGPT/installation/linux/build_images/stop_docker.sh', 'type': 'pagechunk'}
{'title': "Developer's Guide", 'page': 1, 'content': 'Developer\'s Guide Backend database The backend database is used to store the documents, questions, answers, and user information. The backend database is built using Django and PostgreSQL. The database can be accessed by creating a superuser and logging in to the admin panel. create super user To create super user, run following command if you are using pre-built docker images: cd MyGPT/installation/macOS/prebuilt_images/ bash create_superuser.sh Or run following command if you are building docker images: bash MyGPT/installation/macOS/build_images/create_superuser.sh You can check backend database at http://localhost:8000/admin/ with username and password you created in above step. API Endpoints These endpoints are implemented in backend/testdb/views/apis.py and routed in backend/testdb/urls.py . Minimal frontend payloads (5 most used) Use these examples as a quick integration reference. 1) POST /api/get_datasets/ Request: { "user_email": "abc@xyz.com", "user_group": "" } Resp', 'type': 'pagechunk'}
{'title': "Developer's Guide", 'page': 1, 'content': 'rls.py . Minimal frontend payloads (5 most used) Use these examples as a quick integration reference. 1) POST /api/get_datasets/ Request: { "user_email": "abc@xyz.com", "user_group": "" } Response: [ { "dataset_name": "GPCR", "dataset_size": 3366, "user_email": "abc@xyz.com" } ] 2) POST /api/get_documents/', 'type': 'pagechunk'}
{'title': "Developer's Guide", 'page': 2, 'content': 'Request: { "dataset": "GPCR", "user_email": "abc@xyz.com", "user_group": "" } Response: { "documents": [], "dataset_type": "papers" } 3) POST /api/get_context/ Request: { "text": "Which cohorts are available from Survivorship portal?", "model_type": "llama3:latest", "dataset": "Survivorship", "new_conversation": true, "previous_query": "", "no_context": false, "use_default_qrs": true, "question_best_distance": 0.2, "question_worst_distance": 1.7, "maximum_chunks_count": 5, "no_cutoff": false } Response (shape): { "context": "...", "relevance_score": 61, "semantic_score": 58, "keyword_score": 42, "rerank_score": 0, "sources": [] } 4) POST /api/save_answer/ Request: { "question_text": "Which cohorts are available from Survivorship portal?",', 'type': 'pagechunk'}
{'title': "Developer's Guide", 'page': 2, 'content': 're available from Survivorship portal?",', 'type': 'pagechunk'}
{'title': "Developer's Guide", 'page': 3, 'content': '"answer_text": "...", "answer_no_context_text": "...", "model_type": "llama3:latest", "dataset": "Survivorship", "no_context": false, "use_default_ars": true, "answer_best_distance": 0.12, "answer_worst_distance": 1.42, "QRS_p": 1, "ARS_q": 2, "use_default_hi": true, "temperature": 0.4, "top_k": 20, "top_p": 0.7 } Response (shape): { "saved": true, "mean_distance_a": 0.402, "relevance_score": 73, "semantic_score": 71, "keyword_score": 0, "hallucination_index_by_equation": 22, "hallucination_index_by_ml": 19, "sources": [] } 5) GET /api/get_question_details/?question_id=<id> Request example: GET /api/get_question_details/?question_id=1 Response (shape): { "question": "...", "relevance_score": 66, "ground_truth": "-", "question_type": "other", "keywords": "", "llm": "llama3:latest", "answers": [], "sources": [] } Core chat APIs', 'type': 'pagechunk'}
{'title': "Developer's Guide", 'page': 3, 'content': '"ground_truth": "-", "question_type": "other", "keywords": "", "llm": "llama3:latest", "answers": [], "sources": [] } Core chat APIs', 'type': 'pagechunk'}
{'title': "Developer's Guide", 'page': 4, 'content': '1) Get datasets Method/Path: POST /api/get_datasets/ Body: { "user_email": "abc@xyz.com", "user_group": "" } Response: array of dataset field objects (sorted by dataset_name ). [ { "dataset_name": "GPCR", "dataset_size": 3366, "user_email": "abc@xyz.com" } ] 2) Get dataset details Method/Path: POST /api/get_dataset_details/ Body: { "dataset": "GPCR", "user_email": "abc@xyz.com", "user_group": "" } Response: single dataset field object. 3) Update dataset settings Method/Path: POST /api/update_dataset/ Body: dataset is required. You can send one or more updatable fields. { "dataset": "GPCR", "system_prompt": "You are a biomedical assistant.", "Qsem_a": 4, "Qkey_b": -4, "Qrank_c": -1, "Asem_x": 5, "Akey_y": -2, "Arank_z": 0, "QRS_p": 1, "ARS_q": 2,', 'type': 'pagechunk'}
{'title': "Developer's Guide", 'page': 4, 'content': '"Arank_z": 0, "QRS_p": 1, "ARS_q": 2,', 'type': 'pagechunk'}
{'title': "Developer's Guide", 'page': 5, 'content': '"HI_by_equation": true } Response: { "saved": true, "dataset": { "dataset_name": "GPCR" } } 4) Get documents in a dataset Method/Path: POST /api/get_documents/ Body: { "dataset": "GPCR", "user_email": "abc@xyz.com", "user_group": "" } Response: { "documents": [], "dataset_type": "papers" } dataset_type is papers or videos . 5) Get retrieval context (RAG) Method/Path: POST /api/get_context/ Body (required keys): { "text": "Which cohorts are available from Survivorship portal?", "model_type": "llama3:latest", "dataset": "Survivorship", "new_conversation": true, "previous_query": "", "no_context": false, "use_default_qrs": true, "question_best_distance": 0.2, "question_worst_distance": 1.7, "maximum_chunks_count": 5,', 'type': 'pagechunk'}
{'title': "Developer's Guide", 'page': 6, 'content': '"no_cutoff": false } Optional keys: focused_document_titles , focused_section , keywords , translated_text , skip_highlight . Response includes: context , relevance_score , semantic_score , keyword_score , rerank_score , sources . 6) Save generated answer + metrics Method/Path: POST /api/save_answer/ Body (required keys): { "question_text": "Which cohorts are available from Survivorship portal?", "answer_text": "...", "answer_no_context_text": "...", "model_type": "llama3:latest", "dataset": "Survivorship", "no_context": false, "use_default_ars": true, "answer_best_distance": 0.12, "answer_worst_distance": 1.42, "QRS_p": 1, "ARS_q": 2, "use_default_hi": true, "temperature": 0.4, "top_k": 20, "top_p": 0.7 } Optional key: translated_answer_text . Response (when no_context=false ) includes: saved , mean_distance_a , relevance_score , semantic_score , keyword_score , hallucination_index_by_equation , hallucination_index_by_ml', 'type': 'pagechunk'}
{'title': "Developer's Guide", 'page': 6, 'content': 'ated_answer_text . Response (when no_context=false ) includes: saved , mean_distance_a , relevance_score , semantic_score , keyword_score , hallucination_index_by_equation , hallucination_index_by_ml , sources . Response (when no_context=true ): saved , relevance_score . 7) Get question details Method/Path: GET /api/get_question_details/?question_id=<id> Response includes: question , relevance_score , ground_truth , question_type , keywords , llm , answers , sources . answers[] fields: answer relevance_score hallucination_index_by_equation hallucination_index_by_ml answer_no_context sources[] fields include vector/BM25/rerank values and rank fields.', 'type': 'pagechunk'}
{'title': "Developer's Guide", 'page': 7, 'content': '8) Get conversation history by dataset Method/Path: GET /api/get_conversation_history/?dataset=<dataset_name> Response: { "conversations": [ { "conversation_id": 6664, "questions_answers": [ { "question_id": 7547, "question": "What kind of analysis can be performed?", "relevance_score": 58 } ] } ] } 9) Save answer feedback Method/Path: POST /api/feedback/ Body: { "answer_text": "...", "rating": 5, "user_comment": "Helpful answer" } Response: { "saved": true } Dataset/library management APIs 10) Upload documents Method/Path: POST /api/upload_documents/ Content type: multipart form-data Behavior: creates/updates dataset via uploaded files, optionally indexes BM25, then adds vectors to Chroma. Response: { "uploaded": true } 11) Add Zotero collection Method/Path: POST /api/add_zotero_collection/ Content type: multipart form-data', 'type': 'pagechunk'}
{'title': "Developer's Guide", 'page': 7, 'content': 'ed files, optionally indexes BM25, then adds vectors to Chroma. Response: { "uploaded": true } 11) Add Zotero collection Method/Path: POST /api/add_zotero_collection/ Content type: multipart form-data Body fields: api_key , library_id , library_id_type , collection_id , embedding_model , distance_function , chunking_method , use_bm25 , user , user_email , user_group .', 'type': 'pagechunk'}
{'title': "Developer's Guide", 'page': 8, 'content': 'Response: { "added": true, "datasets": ["..."] } or { "error": true, "error_message": "..." } . 12) Add video library Method/Path: POST /api/add_video_library/ Content type: multipart form-data Body fields: dataset_name , embedding_model , video_urls (comma-separated), playlist_url , user , user_email , user_group . Response: { "added": true } . 13) Delete dataset Method/Path: GET /api/delete_dataset/?dataset=<name>&user_email=<email> Response: { "deleted": true } or error JSON. 14) Add demo library Method/Path: GET /api/add_demo_library/?embedding_model=<model> Response: { "added": true } 15) Get dataset sections Method/Path: POST /api/get_sections/ Body: { "dataset_name": "GPCR" } Response: { "sections": [{ "section_title": "Methods", "section_count": 42 }] } Embedding/analytics APIs 16) Get vector embeddings (PCA) Method/Path: GET /api/get_vector_embeddings/?datasets=<d1,d2>&question_id= <optional_id> Response: { "pca_embeddings": [...] } 17) Add dataset embeddings/PCA Method', 'type': 'pagechunk'}
{'title': "Developer's Guide", 'page': 8, 'content': 'PIs 16) Get vector embeddings (PCA) Method/Path: GET /api/get_vector_embeddings/?datasets=<d1,d2>&question_id= <optional_id> Response: { "pca_embeddings": [...] } 17) Add dataset embeddings/PCA Method/Path: GET /api/add_dataset_embeddings/?dataset=<dataset_name> Response: { "added": true } 18) Get embedding model details for a dataset Method/Path: GET /api/get_embedding_model_details/?dataset=<dataset_name> Response: { "embedding_model": { ...distance ranges... } } 19) Get distance between two answers Method/Path: POST /api/get_distance_between_answers/ Body:', 'type': 'pagechunk'}
{'title': "Developer's Guide", 'page': 9, 'content': '{ "sentence1": "...", "sentence2": "...", "embedding_model": "multi-qa-MiniLM-L6-cos-v1" } Response: { "distances": [...] } LLM model APIs 20) Get installed Ollama models Method/Path: POST /api/get_ollama_models/ Response: { "added": true, "models": [...] } 21) Register Ollama models in DB Method/Path: POST /api/add_ollama_models/ Body: { "llms": [ { "name": "llama3:latest", "size": 4661224676 } ] } Response: { "added": true } 22) Register embedding models in DB Method/Path: POST /api/add_embedding_models/ Body: { "embedding_models": [ { "name": "multi-qa-MiniLM-L6-cos-v1", "size": 1, "source": "sentence- transformers" } ] } Response: { "added": true } 23) Ollama text generation Method/Path: POST /api/ollama_generate/ Supports: non-streaming JSON response (stream=false ) streaming NDJSON response (stream=true , content-type application/x-ndjson ) Body fields: model , prompt , optional system , stream , think , options.temperature , options.to', 'type': 'pagechunk'}
{'title': "Developer's Guide", 'page': 9, 'content': 'response (stream=false ) streaming NDJSON response (stream=true , content-type application/x-ndjson ) Body fields: model , prompt , optional system , stream , think , options.temperature , options.top_k , options.top_p . 24) Ollama chat completion', 'type': 'pagechunk'}
{'title': "Developer's Guide", 'page': 10, 'content': 'Method/Path: POST /api/ollama_chat/ Supports: non-streaming JSON response (stream=false ) streaming NDJSON response (stream=true , content-type application/x-ndjson ) Body fields: model , messages , optional stream , think , and sampling options. 25) Ollama pull model (streaming progress) Method/Path: POST /api/ollama_pull_model/ Body: { "name": "llama3:latest" } Response: NDJSON progress events (status , progress , layer_complete , final done=true ). Settings/auth/media APIs 26) Frontend settings Method/Path: GET /api/frontend_settings/ Response shape: { "settings": { "show_no_context_switch": false, "restriction_without_login": false, "azure_login": false, "django_login": false, "disable_chat_without_login": false } } Settings meaning: show_no_context_switch : allow direct chat mode toggle. restriction_without_login : prevent uploads without login. azure_login : enable Azure/Microsoft SSO. django_login : enable Django authentication', 'type': 'pagechunk'}
{'title': "Developer's Guide", 'page': 10, 'content': 'show_no_context_switch : allow direct chat mode toggle. restriction_without_login : prevent uploads without login. azure_login : enable Azure/Microsoft SSO. django_login : enable Django authentication. disable_chat_without_login : block chat usage without login. 27) Resolve username from access token Method/Path: POST /api/get_username/ Body: { "access_token": "<jwt_access_token>" } Response: username , user_email , user_group (or empty username when invalid). 28) Submit disclaimer agreement Method/Path: POST /api/submit_disclaimer_agreement/ Body: { "username": "<django_username>" }', 'type': 'pagechunk'}
{'title': "Developer's Guide", 'page': 11, 'content': 'Response: { "agreed": true } 29) Secure media file access Method/Path: GET /media/<path:file_path>/ Behavior: serves files from MEDIA_ROOT ; may require auth token/user login depending on frontend settings. 30) Logout (JWT refresh token blacklist) Method/Path: POST /logout/ Body: { "refresh_token": "<jwt_refresh_token>" } Permission: authenticated user required.', 'type': 'pagechunk'}
{'title': 'Dependencies', 'page': 1, 'content': 'Dependencies This document summarizes project dependencies for MyGPT. Backend Dependencies Source: backend/requirements.txt Django>=5.0.14 psycopg2-binary>=2.8 requests>=2.32.5 djangorestframework django-cors-headers djangorestframework-simplejwt>=5.3.1 sacremoses transformers>=4.57.2 sentence_transformers>=5.1.2 pypdf pandas pyzotero pymupdf==1.24.14 numpy chromadb==0.5.7 tqdm python-dotenv pytest-django pytube youtube_transcript_api pdfkit ollama>=0.6.1 duckdb openpyxl langchain_community langchain==1.0.0a1 huggingface-hub>=0.36.0 pycryptodome==3.15.0 sqlparse>=0.5.0 bm25s pystemmer zipp>=3.19.1 Frontend Dependencies Source: frontend/package.json Runtime Dependencies (dependencies ) @azure/msal-react@^2.0.22 @heroicons/react@^2.1.5 @modelcontextprotocol/sdk@^1.21.1 @react-pdf-viewer/bookmark@3.12.0', 'type': 'pagechunk'}
{'title': 'Dependencies', 'page': 1, 'content': 'mark@3.12.0', 'type': 'pagechunk'}
{'title': 'Dependencies', 'page': 2, 'content': '@react-pdf-viewer/core@^3.12.0 @react-pdf-viewer/default-layout@^3.12.0 @react-pdf-viewer/highlight@3.12.0 @react-pdf-viewer/page-navigation@3.12.0 @react-three/drei@^9.122.0 @react-three/fiber@^8.18.0 @testing-library/jest-dom@^5.17.0 @testing-library/react@^15.0.7 @testing-library/user-event@^13.5.0 @types/d3@^7.1.0 @types/node@^16.18.108 @types/three@^0.180.0 better-react-mathjax@^2.3.0 d3@7.9.0 env-cmd@^10.1.0 http-proxy-middleware@^3.0.5 node-forge@^1.3.2 nth-check@2.1.1 react@^18.3.1 react-dom@^18.3.0 react-markdown@^8.0.7 react-router-dom@^6.30.1 react-scripts@5.0.1 rehype-katex@^7.0.1 remark-math@^6.0.0 three@^0.181.2 typescript@^4.9.5 web-vitals@^2.1.4 Development Dependencies (devDependencies ) @types/jest@^29.5.12 @types/react@^18.3.1 @types/react-dom@^18.3.0 autoprefixer@^10.4.4 jest@^29.7.0 postcss@^8.4.31 react-test-renderer@^18.3.1 tailwindcss@^3.0.23 ts-jest@^29.1.2 Environment-Specific Dependencies and Setup For OS, VM/server, and cloud-specific prerequisites (for exam', 'type': 'pagechunk'}
{'title': 'Dependencies', 'page': 2, 'content': '4 jest@^29.7.0 postcss@^8.4.31 react-test-renderer@^18.3.1 tailwindcss@^3.0.23 ts-jest@^29.1.2 Environment-Specific Dependencies and Setup For OS, VM/server, and cloud-specific prerequisites (for example, Docker, GPU/CUDA, and Ollama installation differences), use the installation guides: Mac: installation/macOS/README.md Linux: installation/linux/README.md Windows: installation/windows/README.md VM/Server: installation/vm/README.md Cloud (Azure): installation/azure/README.md', 'type': 'pagechunk'}