# Gaia > Decentralizing Generative AI Chatbot and Agent This file contains all documentation content in a single document following the llmtxt.org standard. ## Anything LLM # Anything LLM Anything LLM is the all-in-one Desktop & Docker AI application with full RAG and AI Agent capabilities. You can configure Anything LLM using the Gaia node as the LLM backend. It's recommended to start a node without any snapshots, like [this one](https://github.com/GaiaNet-AI/node-configs/tree/main/llama-3-8b-instruct). ## Steps First, we will need to add Gaia Node as the LLM chat model and embedding model. * Go to Settings * Choose Local AI as the LLM provider in the LLM inference section. Then copy and paste the Gaia node API base URL in the LocalAI Base URL. The chat model and embedding model along with your node will be loaded automatically. Choose the chat model here and input the context length. Remember to click on Save Changes to make the changes take effect. ![](/img/docs/anything-llm-01.png) * Then go to the Embedding Preference section and choose LocalAI as the embedding provider. Then copy and paste the Gaia node API base URL in the LocalAI Base URL. The chat model and embedding model along with your node will be loaded automatically. Choose the embedding model here and input the max embedding chunk length. Don't forget to click on Save Changes to make the changes take effect. The above operations make the Gaia node as the LLM backend. Second, let's set up the data source. You can upload a file or use the data connectors provided by Anything LLM. ![](/img/docs/anything-llm-02.png) When you move a data source to the workspace, Anything LLM will call the Gaia node's embedding API to chunk and compute embeddings for your documentation, which may take some minutes. That's it. When the embedding is done, go back to your workspace and ask a question. ![](/img/docs/anything-llm-03.png) > You can check out the `start-llamaedge.log` to check what happens. --- ## [IDE] CodeGPT # [IDE] CodeGPT CodeGPT is a pair-programming partner for developers. It offers AI chat assistance, auto-completion, code explanation, error-checking, and much more. You can find the CodeGPT extension in VScode and Jetbrains. You can easily configure it to use Gaia nodes as LLM backends. ## Prerequisites You will need a Gaia node ready to provide LLM services through a public URL. You can * [run your own node](../../getting-started/quick-start) * [use a public node](../../nodes) In this tutorial, we will use the public CodeStral nodes to power the CodeGPT plugin. | Model type | API base URL | Model name | |-----|--------|-----| | Chat | https://coder.gaia.domains/v1/v1/ | coder | > For some reason, CodeGPT requires the API endpoint to include an extra `v1/` at the end. ## Install CodeGPT Download [the CodeGPT for VScode](https://marketplace.visualstudio.com/items?itemName=DanielSanMedium.dscodegpt&ssr=false#overview) and [Jetbrains](https://plugins.jetbrains.com/plugin/21056-codegpt). Once you install it successfully, you can find the plugin on the right sidebar. You don't need to sign up for an account here. ![](/img/docs/codegpt-01.png) ## Configure CodeGPT Click the CODEGPT on the right sidebar and enter the settings page for CodeGPT. 1. Go through the **Select Your AI** and choose the **Custom** PROVIDER. Copy and paste `codestral` into the MODEL field. ![](/img/docs/codegpt-02.png) 3. Click the Connect button to configure the model base url and API key. Again, note the extra `v1\` at the end of the URL. | Attribute | Value | |-----|--------| | API endpoint URL | https://coder.gaia.domains/v1/v1/ | | API Key | gaia | ![](/img/docs/codegpt-03.png) Save the above settings. > If you're using a Domain service, not your own node, you will [need to get an API key from Gaia](../../getting-started/authentication/authentication.md). ## Use the plugin You can summon the coding assistant using slash commands defined in the plugin. You can ask the coding assistant to fix bugs, explain codes, write documentation, refactor the docs, and create unit test cases for the specific code. ![](/img/docs/codegpt-04.png) ## Video Guide --- ## [IDE] Continue # [IDE] Continue [Continue](https://github.com/continuedev/continue) is the leading open-source AI code assistant. It is a copilot-like plugin for VSCode and JetBrains to provide custom autocomplete and chat experiences inside those IDEs. You can easily configure it to use Gaia nodes as LLM backends. In fact, you can choose different Gaia nodes for * The autocomplete model for coding tasks. * The chat model for understanding and discussing code. * The embedding model to provide chat context based on local files. ## Prerequisites You will need a Gaia node ready to provide LLM services through a public URL. You can * [Run your own node](../../getting-started/quick-start) * [Use a public node](../../nodes/nodes.md) In this tutorial, we will use public nodes to power the Continue plugin. | Model type | API base URL | Model name | |-----|--------|-----| | Chat | https://llama8b.gaia.domains/v1/ | llama | | Embedding | https://llama8b.gaia.domains/v1/ | nomic | | Autocompletion | https://coder.gaia.domains./v1/ | codestral | > It is important to note that Continue requires the API endpoint to include a `/` at the end. ## Install Continue [Load this link](https://marketplace.visualstudio.com/items?itemName=Continue.continue) to install the Continue IDE plugin. It will open up your VSCode when you click on the **Install** button on the web page. When you are asked to configure Continue, just click on **Skip** and finish the installation without selecting a local model. ## Configure Continue Click on the gear icon on the toolbar to load the `config.json` file for the Continue plugin. The file is located in your own home directory `$HOME/.continue/config.json`. You can now change the `config.json` file as follows. It asks the Continue plugin to use different public Gaia nodes and models for chat, code autocomplete and embeddings. ``` { "models": [ { "model": "llama", "title": "LlamaEdge", "apiBase": "https://llama8b.gaia.domains/v1/", "provider": "openai" } ], "tabAutocompleteModel": { "title": "Autocomplete", "apiBase": "https://coder.gaia.domains/v1/", "model": "codestral", "provider": "openai" }, "embeddingsProvider": { "provider": "openai", "model": "nomic-embed", "apiBase": "https://llama8b.gaia.domains/v1/" }, "customCommands": [ { "name": "test", "prompt": "{{{ input }}}\n\nWrite a comprehensive set of unit tests for the selected code. It should setup, run tests that check for correctness including important edge cases, and teardown. Ensure that the tests are complete and sophisticated. Give the tests just as chat output, don't edit any file.", "description": "Write unit tests for highlighted code" } ], "allowAnonymousTelemetry": true } ``` Save the `config.json` file and you are done! ## Use the plugin The following screenshot shows how you can chat with an error message inside the IDE. ![](/img/docs/continue-01.png) --- ## [IDE] Cursor # [IDE] Cursor :::warning You will need at least the Pro or Ultra [plans](https://cursor.com/pricing) on cursor to use your own Gaia node in cursor. ::: [Cursor](https://www.cursor.com/) is an AI-powered code editor / IDE. Using LLMs to generate and review code, Cursor is an alternative to the very popular GitHub Copilot. You can use Cursor with your own Gaia node as the LLM backend. There are two big reasons for that * Your Gaia node could be supplemented by a knowledge base that is specific to your proprietary code repository, programming language choices, and coding guidelines / styles. * Your Gaia node could ensure that your code stays private within your organization. ## Prerequisites You will need a Gaia node to provide LLM API services. You can * [Run your own node](../../getting-started/quick-start) * [Use a public node](../../nodes) In this tutorial, we will use public [Qwen 2.5 Coder](https://github.com/QwenLM/Qwen2.5-Coder) nodes to power Cursor. | Model type | API base URL | Model name | |-----|--------|-----| | General coding assistant | `https://coder.gaia.domains/v1` | coder | | Coding assistant with Rust knowledge | `https://rustcoder.gaia.domains/v1` | rustcoder | | Rust expert (slower but more accurate) | `https://rustexpert.gaia.domains/v1` | rustexpert | > A limitation of Cursor is that it does not support local LLM services. A Gaia node comes with a default networking tunnel that turns your local LLM service into a HTTPS service accessible from the Internet. That allows Cursor to use your own private LLM for coding. Start your own [Qwen Coder](https://github.com/GaiaNet-AI/node-configs/tree/main/qwen-2.5-coder-7b-instruct) or [Qwen Coder with Rust](https://github.com/GaiaNet-AI/node-configs/tree/main/qwen-2.5-coder-7b-instruct_rustlang) nodes today! ## Configure Cursor First, download and install [Cursor](https://www.cursor.com/). Click on the **Settings** button on the top right. Then, click on **Models** to configure the backend LLM service. Second, add a model named `coder` and turn off all the other models like `gpt-4o`. Third, go to the OpenAI API Key section, * Click on **Override OpenAI Base URL**. Type `https://coder.gaia.domains/v1` here. * For the OpenAI API key, you can use any random chars such as `GAIA`. Click on **Verify** to test if the connection is correct. ![](/img/docs/cursor-01.png) ## Use Cursor You can use * **command + K** to edit the highlighted code * **command + L** to open the chat room and ask questions about the code. ![](/img/docs/cursor-02.png) ## Video Guide --- ## Dify + Gaia # Dify + Gaia You can configure the Dify framework using any Gaia node as the backend LLM API. That allows you to use your own or community Gaia nodes in any application built on Dify. It supports * The hosted [Dify.ai](https://dify.ai/) service. * Products and services with embedded Dify framework, such as the [Terminus](https://www.jointerminus.com/) project. * Any product that is built on the open source [Dify framework](https://github.com/langgenius/dify). ## Steps First, log into Dify's web portal and select `Settings | Model Provider`. From the list, you can add an OpenAI-API-compatible provider. Add an LLM model with the model name and API endpoint listed on your Gaia node's web dashboard. Or, you can just add [a popular Gaia node](../../nodes/nodes.md). Leave the API Key field empty. ![Configure a Gaia Llama3 8b model in Dify](/img/docs/dify_chat.png) Most Dify applications also require an embedding model to search text in the vector space. Add an embedding model with the model name and API endpoint listed on your Gaia node's web dashboard. Or, you can just add [a popular Gaia node](../../nodes). Leave the API Key field empty. ![Configure a Gaia embedding model in Dify](/img/docs/dify_embedding.png) That's it. You can now see that the new models are available at the top panel of Dify for every chatbot or agent. Just select your Gaia models for chat or embedding, and the Dify app will automatically use it! ![Select a Gaia node as backend model in Dify](/img/docs/dify_select.png) ![Chat with the Gaia Llama3 8b model in Dify](/img/docs/dify_chatbot_ui.png) --- ## FlowiseAI tool call # FlowiseAI tool call FlowiseAI is a low-code tool for developers to build customized LLM orchestration flows & AI agents. You can configure the FlowiseAI tool to use a Gaia node that supports [LLM tool calling](https://github.com/LlamaEdge/LlamaEdge/blob/main/llama-api-server/doc/ToolUse.md). ## Prerequisites You will need a Gaia node ready to provide LLM services through a public URL. In this tutorial, you will need to [set up a public node with tool call support](https://github.com/GaiaNet-AI/node-configs/blob/main/mistral-0.3-7b-instruct-tool-call/README.md). ## Start a FlowiseAI server Follow [the FlowiseAI guide](https://docs.flowiseai.com/getting-started) to install Flowise locally ``` npm install -g flowise npx flowise start ``` After running successfully, you can open `http://localhost:3000` to check out the Flowise AI tool. ## Build a chatbot for realtime IP lookup Step 1: Create a new **Chatflow** from the UI. ![](/img/docs/flowise-tool-01.png) Step 2: On the **Chatflow** canvas, add a node called **ChatLocalAI**. ![](/img/docs/flowise-tool-02.png) Step 3: Configure the **ChatLocalAI** widget to use the Gaia node with tool call support you have created. * Base path: `https://YOUR-NODE-ID.gaia.domains/v1` * Model name: e.g., `Mistral-7B-Instruct-v0.3.Q5_K_M` Step 4: Add a node called **Custom Tool** Create a function named `get_ip_address_geo_location`. The function requires a `string` parameter called `ip`. The **Tool description** field is the "prompt" that tells the LLM when to use this function. In this example, if the LLM detects that the user is asking about the city or country of an IP address, it will return a tool call response asking FlowiseAI to perform this function call first. ![](/img/docs/flowise-tool-03.png) Now you can add JavaScript code for this function. It looks up the location of the input `ip` parameter. ``` const fetch = require("node-fetch") const url = "http://ipwho.is/"+$ip try { const response = await fetch(url) const result = await response.text() console.log(result) return result } catch(error) { console.error(error) } ``` ![](/img/docs/flowise-tool-04.png) Step 5: Add a node called **Buffer Memory** to the canvas. Step 6: Add a node called **Tool Agent**. Step 7: Connect the nodes. Connect the **Custom Tool** and **Buffer Memory** nodes to the appropriate connectors on the **Tool Agent** node. Connect the **ChatLocalAI** node to the **Custom Tool**. ![](/img/docs/flowise-tool-05.png) Step 8: Save the **Chatflow**. ## Give it a try From the FlowiseAI UI, you can open a chat window to chat with the **ChatLocalAI** you just created. Let's ask a question: ``` What's the location of this address 35.222.115.181 ``` The LLM understands that the request is to find a location for an IP address, and sees that we have a function called `get_ip_address_geo_location` in tools, which has a description that matches this task. So, it responds with a JSON message to call this function with the IP address it extracts from the user query. This tool calling JSON message is NOT displayed to the user in the chatbot. Instead, the FlowiseAI **Custom Tool** node captures it and executes the JavaScript code associated with this tool call. The result of the tool call is then sent back to the LLM together with the original query, which is why we need the **Buffer Memory** node BTW, and the LLM formulates a human readable response to the original question. ![](/img/docs/flowise-tool-06.png) --- ## FlowiseAI RAG chat # FlowiseAI RAG chat FlowiseAI is a low-code tool for developers to build customized LLM orchestration flows & AI agents. You can configure the FlowiseAI tool to use Gaia nodes as LLM service providers. ## Prerequisites You will need a Gaia node ready to provide LLM services through a public URL. You can * [run your own node](../../getting-started/quick-start) * [use a public node](../../nodes/nodes.md) In this tutorial, we will use public nodes to power the Continue plugin. | Model type | API base URL | Model name | |-----|--------|-----| | Chat | https://llama8b.gaia.domains/v1 | llama | | Embedding | https://llama8b.gaia.domains/v1 | nomic | ## Start a FlowiseAI server Follow [the FlowiseAI guide](https://docs.flowiseai.com/getting-started) to install Flowise locally ``` npm install -g flowise npx flowise start ``` After running successfully, you can open http://localhost:3000 to check out the Flowise AI tool. ## Build a documents QnA chatbot FlowiseAI allows you to visually set up all the workflow components for an AI agent. If you're new to FlowiseAI, it's recommended to use a template quick start. In fact, there are lots of templates around OpenAI in the Flowise marketplace. All we need to do is to replace the ChatOpenAI component with the ChatLocalAI component. Let's take the **Flowise Docs QnA** as an example. You can build a QnA chatbot based on your documents. In this example, we would like to chat with a set of documents in a GitHub repo. The default template was built with OpenAI and we will now change it to use an open-source LLM on a Gaia node. ### Get the **Flowise Docs QnA** template ![](/img/docs/flowise-01.png) Click on Marketplaces on the left tab to browse all the templates. The template **Flowise Docs QnA** we will use is the first one. ![](/img/docs/flowise-02.png) Then, click on Use this template button on the left top corner to open the visual editor. ### Connect the chat model API You will need to delete the ChatOpenAI component and click the + button to search ChatLocalAI, and then drag the ChatLocalAI to the screen. ![](/img/docs/flowise-03.png) Then, you will need to input * the Gaia node base URL `https://llama8b.gaia.domains/v1` * the model name `llama` Next, connect the ChatLocalAI component with the field `Chat model` in the **Conversational Retrieval QA Chain** component. ### Connect the embedding model API The default template uses the OpenAI Embeddings component to create embeddings for your documents. We need to replace the **OpenAI Embeddings** component with the **LocalAI Embeddings** component. * Use the Gaia node base URL `https://llama8b.gaia.domains/v1` in the Base Path field. * Input the model name `nomic-embed-text-v1.5.f16` in the Model Name field. Next, connect the **LocalAI Embeddings** component with the field `embedding` in the **In-Memory Vector Store** component. ### Set up your documents Then, let's go through the GitHub component to connect the chat application to our documents on GitHub. You will need to put your docs GitHub link into the **Repo Link** field. For example, you can put Gaia's docs link: `https://github.com/GaiaNet-AI/docs/tree/main/docs`. ## Give it a try You can send a question like "How to install a Gaia node" after saving the current chatflow. ![](/img/docs/flowise-04.png) You will get the answer based on the Gaia docs, which are more accurate. ## More examples There are lots of examples on the Flowise marketplace. To build a Flowise agent based on Gaia, simply replace the **Chat OpenAI** and **OpenAI Embeddings** component with the Gaia base URL. --- ## OpenAI Ecosystem Apps import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; # OpenAI Ecosystem Apps Since each Gaia node provides an OpenAI-compatible API service, it can be a drop-in replacement for OpenAI in almost all LLM applications and frameworks. Check out the articles in this section for instructions and examples for how to use Gaia in popular LLM apps. ## The OpenAI Python library :::note Make sure to replace `YOUR_API_KEY_GOES_HERE` with your **own API key**. To get your own API key, follow [this](../getting-started/authentication) tutorial. ::: You can install the [official OpenAI Python library](https://pypi.org/project/openai/) as follows. ``` pip install openai ``` When you create an OpenAI client using the library, you can pass in the API endpoint point as the `base_url`. Remember to append the `/v1` after the host name. You can find a list of public nodes [here](../nodes/nodes.md). ``` import openai client = openai.OpenAI(base_url="https://YOUR-NODE-ID.gaia.domains/v1", api_key="YOUR_API_KEY_GOES_HERE") ``` Alternatively, you could set an environment variables at the OS level. ``` export OPENAI_API_BASE=https://YOUR-NODE-ID.gaia.domains/v1 export OPENAI_API_KEY=YOUR_API_KEY_GOES_HERE ``` Then, when you make API calls from the `client`, make sure that the `model` is set to the model name available on your node. ```py response = client.chat.completions.create( model="Meta-Llama-3-8B-Instruct-Q5_K_M", messages=[ {"role": "system", "content": "You are a strategic reasoner."}, {"role": "user", "content": "What is the purpose of life?"} ], temperature=0.7, max_tokens=500 ] ) ``` That's it! You can now take any application built with the official OpenAI Python library and use a Gaia node as its backend! ## The OpenAI Node API library :::note Make sure to replace `YOUR_API_KEY_GOES_HERE` with your **own API key**. To get your own API key, follow [this](../getting-started/authentication) tutorial. ::: You can install the [OpenAI Node library](https://www.npmjs.com/package/openai) which provides convenient access to the OpenAI REST API from TypeScript or JavaScript as follows: ``` npm install openai ``` Import it into your project as: ```js // Example usage in Node.js const OpenAI = require('openai'); ``` Create an OpenAI client with a custom base URL. Remember to append the `/v1` after the host name. ```js const client = new OpenAI({ baseURL: 'https://YOUR-NODE-ID.gaia.domains/v1', apiKey: 'YOUR_API_KEY_GOES_HERE' }); ``` Alternatively, you can set an environment variable using `dotenv` in Node. ``` process.env.OPENAI_API_BASE = 'https://YOUR-NODE-ID.gaia.domains/v1'; ``` Then, when you make API calls from the `client`, make sure that the `model` is set to the model name available on your node. ```js async function callOpenAI() { try { const response = await client.chat.completions.create({ model: "Meta-Llama-3-8B-Instruct-Q5_K_M", messages: [ { role: "system", content: "You are a strategic reasoner." }, { role: "user", content: "What is the purpose of life?" } ], temperature: 0.7, max_tokens: 500 }); console.log(response.choices[0].message.content); } catch (error) { console.error('Error:', error); } } //Usage callOpenAI(); ``` --- ## Langchain Integration import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; # Langchain Integration :::note Integrations with Langchain can be done with Python or Javascript. ::: LangChain is a framework for developing large language model (LLM) powered applications. You can configure Langchain to use any Gaia node as the LLM backend, that way you can build any AI agent or AI-powered application that uses Gaia for inferencing. ## Prerequisites You will need a Gaia node ready to provide LLM services through a public URL. You can: - [Run your own node](../../getting-started/quick-start/quick-start.md) - [Use a public node](../../nodes/nodes.md) If you are using a public node, you will need an [API key](https://www.gaianet.ai/setting/gaia-api-keys). **Gaia offers free 50,000 API credits to use with available services such as public nodes when you apply for a developer account**. ### Setup - Project setup on machine (JavaScript or Python) - Langchain Installation: ```bash npm install @langchain/openai @langchain/core dotenv ``` ```bash pip install langchain langchain-openai python-dotenv ``` ## Integration with Gaia To get started with running your Gaia node, you can follow the guide on the [Setting up your own node](../../getting-started/quick-start/quick-start.md) page for a quickstart. In this guide, we will be running our Gaia node locally so we do not need an API key, you can use a string like: "Gaia" as a placeholder. Create a `.env` file and store your API key: ```bash GAIANET_API_KEY="Gaia" ``` Integrations with Langchain and Gaia can be done with any JavaScript or Python. There are code snippets below that show how integration looks like in both languages: ```js import { ChatOpenAI, OpenAI } from "@langchain/openai"; import dotenv from "dotenv"; dotenv.config(); const model = new ChatOpenAI({ configuration: { apiKey: process.env.GAIANET_API_KEY, model: "Llama-3-Groq-8B-Tool", baseURL: "gaia-node-url/v1", }, }); const response = await model.invoke("Hello, world!"); console.log(response) ``` ```python from langchain_openai import ChatOpenAI, OpenAI import os model = ChatOpenAI( api_key=os.environ.get("GAIANET_API_KEY"), model="Llama-3-Groq-8B-Tool", base_url="gaia-node-url/v1" ) response = model.invoke("Hello, world!") print(response) ``` ## Invoking Gaia models Once you have the basic connection established, you can start using Langchain's powerful features. Start by making invocations to the model. ```js // ... const response = await model.invoke("Hello, world!"); console.log(response) ``` ```python # ... response = model.invoke("Hello, world!") print(response) ``` The LangChain support also opens up integrations with [LangGraph](https://www.langchain.com/langgraph) and [LangSmith](https://www.langchain.com/langsmith). --- ## LobeChat # LobeChat You can configure [LobeChat](https://lobehub.com/) to use a Gaia node as its backend LLM API. It provides a richer and more customizable UI than the default Gaia chatbot UI. ## Steps **Step 1: Set up the Gaia node API base url as the OpenAI provider** Go to the [Language Model Setting page](https://chat-preview.lobehub.com/settings/modal?agent=&session=inbox&tab=llm) and choose OpenAI. 1. Enter a random string in the OpenAI API Key field. It does not matter what you enter here since we are going to ignore it on the backend. 2. Enter the Gaia node API base URL in the API Proxy Address field. For example, you can use `https://llama8b.gaia.domains/v1` here. 3. Enable Use Client-Side Fetching Mode 4. Click on the Get Model List text and it will automatically fetch LLMs available on the Gaia node. Choose the chat model `llama` here. 5. Optional: click on the Check button to check the connection status. ![](/img/docs/lobechat-gaianet-01.png) **Step 2: Start chatting via the LobeChat UI** Next, let's go back to the chat page. Before starting chatting, choose the model you just chose in the previous step around **Just Chat**. Now you can chat with the Gaia node via LobeChat. ![](/img/docs/lobechat-gaianet-02.png) --- ## Obsidian # Obsidian Obsidian is a note-taking application that enables users to create, link, and visualize ideas directly on their devices. With Obsidian, you can seamlessly sync notes across devices, publish your work, and collaborate with others. The app is highly customizable, allowing users to enhance functionality through a wide range of plugins and themes. Its unique features include a graph view to visualize connections between notes, making it ideal for managing complex information and fostering creativity. Obsidian also emphasizes data privacy by storing notes locally. **Obsidian-local-gpt is a plugin that** allows users to run a local large language model within Obsidian note-taking application. This plugin enables various AI-powered features directly in Obsidian, such as text generation, summarization, spelling and grammar checks, and task extraction. A key feature of this plugin is that it supports a large number of open source LLMs. You can choose an LLM that is finetuned for your specific task — eg if you take a lot of coding notes, you could choose a Codestral or CodeLlama or DeepSeek LLM. Furthermore, if you choose to run the LLM locally on your own computer, the plugin would support private and offline use of the LLM features. For more details, you can visit the [obsidian-local-gpt GitHub page](https://github.com/pfrankov/obsidian-local-gpt). This guide explains how to set up and use the plugin with a Gaia node as an alternative to OpenAI or Ollama. ## Prerequisites You will need a Gaia node ready to provide LLM services through a public URL. You can * [Run your own node](../../getting-started/quick-start/quick-start.md) * [Use a public node](../../nodes/nodes.md) In this tutorial, we will use a public node. | Attribute | Value | |-----|--------| | API endpoint URL | https://llama8b.gaia.domains/v1 | | Model Name | llama | ## Obsidian-local-gpt Plugin Setup Make sure you have already installed the Obsidian app on your device. ### Install the Obsidian-local-gpt Plugin * Open Obsidian settings, navigate to "Community plugins", and search for `obsidian-local-gpt`. * Install the plugin by clicking “Install”. ![](/img/docs/obsidian-enable.png) Then click “Enable”. ### **Configure the Plugin** 1. Go to the plugin settings. 2. Select "AI Provider" as "OpenAI compatible server". 3. Set the server URL. Use https://llama8b.gaia.domains/ if you are using a public Gaia node. Or, use http://localhost:8080/ if you are running a local Gaia node. 4. Configure API key to Gaia. ![](/img/docs/obsidian-configure.png) Make sure to click the refresh button and choose the **llama** model if you’re using the public Gaia node url and **Phi-3-mini-4k-instruct** if you’re using the local Gaia node. ![](/img/docs/obsidian-model.png) ### Configure Obsidian Hotkey 1. Open Obsidian Settings. 2. Go to Hotkeys. 3. Filter "Local" and you should see "Local GPT: Show context menu". 4. Click on `+` icon and press hotkey (e.g. `⌘ + M`). ![](/img/docs/obsidian-hotkey.png) As long as you have set the hotkey, while writing or editing a note, select the text you want to interact with, and press the hotkey you have set to use this LLM powered plugin! ## Use Cases ### **Text Continuation** * Select a text segment, right-click, and choose "Continue writing". The model will generate the continuation (displayed in the screenshot in grey). ![](/img/docs/obsidian-text-continuation.png) ### **Summarization** * Select a longer text segment, right-click, and choose "Summarize text". The model provides a summary as below of the CNAI report content I selected. ![](/img/docs/obsidian-summarization.png) Here is a concise summary of the key points: * Cloud Native (CN) refers to well-balanced systems built using microservices, promoting modular design and reusability. * Kubernetes has become the de facto cloud operating system, offering scalability, resilience, and DevOps best practices. * Every Cloud Service Provider offers Kubernetes as a service, facilitating access to infrastructure and support services for various workloads, including AI/ML. * The Cloud Native Computing Foundation defines Cloud Native as empowering organizations to build scalable applications in modern environments using containers, microservices, and declarative APIs. * Cloud Native Artificial Intelligence (CNAI) is an evolving extension of Cloud Native, focusing on building and deploying AI applications and workloads using Cloud Native principles. ### **Spelling and Grammar Check** * Select text, right-click, and choose "Fix spelling and grammar". The model will correct errors. ![](/img/docs/obsidian-grammar.png) ### **Extract Action Items** * For notes with multiple tasks, select the text and click "Find action items". The model will list the tasks. ![](/img/docs/obsidian-extract.png) The Generated content are displayed below your own text: `Here are the action items extracted from the document in Markdown checkbox format:` * `[ ] Add feedback on WasmEdge Q3 Roadmap Discussion` * `[ ] Provide input on Update on WasmEdge Community Governance` * `[ ] Discuss any additional topics at the meeting (add to [https://docs.google.com/document/d/1iFlVl7R97Lze4RDykzElJGDjjWYDlkI8Rhf8g4dQ5Rk/edit#](https://docs.google.com/document/d/1iFlVl7R97Lze4RDykzElJGDjjWYDlkI8Rhf8g4dQ5Rk/edit))` `Let me know if you'd like me to help with anything else!` ### **General Assistance** * Select any text and click "General help" to get contextually relevant information from the model. I entered and selected some information on KubeCon + CloudNativeCon + Open Source Summit + AI_dev China 2024. Because llama3 has not been trained with info on this conference, so the output is not very helpful: `The information you're looking for is not present in this context.` `If you need to know the format and dates of KubeCon + CloudNativeCon + Open Source Summit + AI_dev China 2024, I suggest searching for official announcements or websites related to these events.` ## Try it now! Ready to supercharge your note-taking with AI? Get started with the Obsidian-local-gpt plugin and Gaia today: 1. Set up the Obsidian-local-gpt plugin in your Obsidian app. 2. Explore the various AI-powered features to enhance your productivity. Start your journey towards smarter, more efficient note-taking now! --- ## Open WebUI # Open WebUI You can configure the Open WebUI framework, a self-hosted WebUI, using any Gaia node as the backend LLM API. That allows you to use your own or community Gaia nodes in any application built on Open WebUI. ## Prerequisites You will need a Gaia node ready to provide LLM services through a public URL. You can * [run your own node](../../getting-started/quick-start/quick-start.md) * [use a public node](../../nodes/nodes.md) In this tutorial, we will use public nodes to power the Continue plugin. | Model type | API base URL | Model name | |-----|--------|-----| | Chat | https://llama8b.gaia.domains/v1 | llama | | Embedding | https://llama8b.gaia.domains/v1 | nomic | ## Start the Open WebUI on your machine After successfully starting the Gaia node, you can use `docker run` to start the Open WebUI. ``` docker run -d -p 3000:8080 \ -v open-webui:/app/backend/data \ -e OPENAI_API_BASE_URL="https://llama8b.gaia.domains/v1" \ -e OPENAI_API_KEYS="gaianet" \ --name open-webui \ --restart always \ ghcr.io/open-webui/open-webui:main ``` Then, open `http://localhost:3000` in your browser and you will see the Open WebUI page. You can also configure your own node when the webUI is started. * Click on your profile on the top right corner and choose **Setting**. * Then choose Connections. In the OpenAI API field, type your node base URL and enter several random characters. * Click on Save to make the change take effective. ![](/img/docs/openwebui-02.png) ## Use Open WebUI as a Chatbot UI Simply choose the chat model under **Select a model** and then you can send messages to the Gaia node. ![](/img/docs/openwebui-01.png) ## Use Open WebUI as a client-side RAG tool Open WebUI also offers a way to implement RAG application. Since the Gaia nodes have OpenAI-compatible embedding APIs, you can also use this feature. However, to use this feature, it's recommend to start a node without any snapshots, like [this one](https://github.com/GaiaNet-AI/node-configs/tree/main/llama-3-8b-instruct). **Step 1:** Use Gaia node as the embedding API * Click on **Workspace** on left top and choose **Documents** tab. This is where you manage the uploaded documents. * Click on **Document Settings** to configure the embedding setting. * In the **General Settings**, choose OpenAI as the Embedding Model Engine. Enter the node API base URL and several random characters. Then, enter the embedding model name in the Embedding Model field. Click Save to apply the changes. ![](/img/docs/openwebui-04.png) **Step 2:** Use Gaia node as the embedding API Click on **+** to upload your documentations. **Step 3:** Chat Then go back to the chat page. Before you send a message, type **#** to choose the document you want to as the context. ![](/img/docs/openwebui-05.png) That's it. --- ## [IDE] Zed # [IDE] Zed [Zed](https://zed.dev/) is a next-generation code editor designed for high-performance collaboration with humans and AI, and it is written in Rust. You can use Zed with your own Gaia node as the LLM backend. There are two big reasons for that * Your Gaia node could be supplemented by a knowledge base that is specific to your proprietary code repository, programming language choices, and coding guidelines/styles. * Your Gaia node could ensure that your code stays private within your organization. ## Prerequisites You will need a Gaia node to provide LLM services to Zed. You can * [run your own node](../../getting-started/quick-start/quick-start.md) * [use a public node](../../nodes/nodes.md) In this tutorial, we will use public [Qwen 2.5 Coder](https://github.com/QwenLM/Qwen2.5-Coder) nodes to power Cursor. | Model type | API base URL | Model name | |-----|--------|-----| | General coding assistant | `https://coder.gaia.domains/v1` | coder | | Coding assistant with Rust knowledge | `https://rustcoder.gaia.domains/v1` | rustcoder | | Rust expert (slower but more accurate) | `https://rustexpert.gaia.domains/v1` | rustexpert | > A limitation of Cursor is that it does not support local LLM services. A Gaia node comes with a default networking tunnel that turns your local LLM service into a HTTPS service accessible from the Internet. That allows Cursor to use your own private LLM for coding. Start your own [Qwen Coder](https://github.com/GaiaNet-AI/node-configs/tree/main/qwen-2.5-coder-7b-instruct) or [Qwen Coder with Rust](https://github.com/GaiaNet-AI/node-configs/tree/main/qwen-2.5-coder-7b-instruct_rustlang) nodes today! ## Configure Zed First, download and install [Zed](https://zed.dev/). Click on your profile on the top right and choose **Setting**. Then a new tab called `settings.json` will be opened. You can configure your Zed by editing this file. ![](/img/docs/zed-01.png) Below is the `settings.json` we used. You can copy and paste sections `language_models` and `assistant` to your own. They configure Zed to use an OpenAI-compatible API provider and then specify the API endpoint URL and model name for that provider. ``` { "features": { "inline_completion_provider": "none" }, "language_models": { "openai": { "version": "1", "api_url": "https://rustcoder.gaia.domains/v1", "low_speed_timeout_in_seconds": 60, "available_models": [ { "name": "yicoder9b", "max_tokens": 8096 } ] } }, "assistant": { "provider": "openai", "default_model": { "provider": "openai", "model": "yicoder9b" }, "version": "2" }, "ui_font_size": 16, "buffer_font_size": 16, "theme": { "mode": "system", "light": "One Light", "dark": "One Dark" } } ``` Next we will configure the API key to access this Gaia node. Go back to the folder you opened. Click on the Star icon at the bottom to turn on the Assistant panel. ![](/img/docs/zed-02.png) Click on **Open configuration** to set up the API Key. ![](/img/docs/zed-03.png) Since we are using a free public Gaia node, you can use any API key in the OpenAI section at the bottom of the screen. For example, you can enter `GAIA`. Now, we have everything ready. # Use Zed You can * Edit the highlighted code by selecting the code and clicking on the **Inline Assistant** button. ![](/img/docs/zed-04.png) * Open the Assistant panel by clicking on the **Assistant** icon at the bottom to turn on the Assistant panel. ![](/img/docs/zed-05.png) --- ## Quick Start with Launching Gaia Domain # Quick Start with Launching Gaia Domain This guide provides all the information you need to quickly set up and run a Gaia Domain. > **Note:** Ensure that you are the owner of a Gaia Domain Name before proceeding. You can verify your Gaia Domain Name in the "Assets" section of your profile. Gaia simplifies the process for domain operators to launch and host a Gaia Domain service in just a few clicks. ### Steps to Launch Your Gaia Domain 1. **Access the Create Gaia Domain Page** Click **LAUNCH DOMAIN** in the "Domain" or "Assets" section under [your profile](https://www.gaianet.ai/). This will take you to the Create Gaia Domain page. 2. **Fill in Domain Details** Enter the general information for your domain, including: - Domain profile - Domain Name - Description - System Prompt 3. **Choose a Gaia Domain Name** Select a Gaia domain name from your assets. 4. **Select a Supplier** Currently, **Gaia Cloud** is the only supplier. 5. **Pick a Gaia Domain Tier** Choose a tier to enhance your domain's rewards, which is necessary. 6. **Configure Server and Management Options** - Confirm the server configuration for running your domain. - Set management preferences, such as whether nodes can join automatically and the specific LLM to use. After completing these six steps, your Gaia Domain will be successfully launched and other nodes can join your domain. --- ## ❓ Frequently Asked Questions # ❓ Frequently Asked Questions ### What is Gaia? {#hidden-headings} What is Gaia? Gaia is a platform for creating and deploying AI agents. It provides tools and infrastructure to build, train, and manage custom AI solutions for various applications. ### Is Gaia suitable for enterprise use? {#hidden-headings} How do I create my own knowledge base? You can create a knowledge base in Gaia through several methods: 1. [From a plain text file](./knowledge-bases/how-to/text) 2. [From a markdown file](./knowledge-bases/how-to/markdown) 3. [From Source/Summary Pairs (or a CSV)](./knowledge-bases/how-to/csv) 4. [From a PDF file](./knowledge-bases/how-to/pdf) 5. [From a URL](./knowledge-bases/how-to/firecrawl) Each method is detailed in our ["How to create a knowledge base"](./knowledge-bases) guide. ### Is Gaia suitable for enterprise use? {#hidden-headings} Is Gaia suitable for enterprise use? Yes, Gaia is designed with enterprise needs in mind. It offers: - Scalable infrastructure - Security features for sensitive data - Integration capabilities with existing systems - Customization options for specific industry needs --- ## Start a node on AWS using AMI images # Start a node on AWS using AMI images We have created a series of public AMIs for you to start Gaia nodes in AWS with just a few clicks. Now we have three AMI images available in Asia Pacific (Osaka) and all the US regions including N. Virginia, Ohio, N. California, and Oregon. | AMI Images Name | Architecture | Regions | |----------------------------------|-------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------| | GaiaNet_ubuntu22.04_amd64_cuda12 | GPU | N. Virginia, Ohio, N. California, Oregon, and Osaka | | GaiaNet_ubuntu22.04_amd64 | x86 CPU machines | N. Virginia, Ohio, N. California, Oregon, and Osaka | | GaiaNet_ubuntu22.04_arm64 | ARM CPU machines | N. Virginia, Ohio, N. California, Oregon, and Osaka | ## Running an Nvidia GPU-enabled AWS instance Load the [AWS console](https://aws.amazon.com/console/) and sign into your account. Go to EC2 | instances and click on the "Launch instance" button. In the "Application and OS Images" section, search the AMI catalog and select the image named `GaiaNet_ubuntu22.04_amd64_cuda12`. ![](/img/docs/aws_ami.png) In the "Instance type" section, select any of the `g4dn` types. Those are EC2 VMs with Nvidia T4 GPUs. ![](/img/docs/aws_instance_type.png) In the "Network settings", make sure that you allow SSH connections. ![](/img/docs/aws_network.png) Click on the "Launch instance" button and wait for the instance to start up. Once the instance is ready, SSH into its public IP address. Once you are in the VM, run the following two commands. ```bash gaianet init gaianet start ``` The node is ready when it shows `The Gaia node is started at: https://...` on the console. You can go to that URL from your browser to interact with the Gaia node. You can [customize your Gaia node](../../customize/customize.md) with your own choice of LLMs and knowledge base snapshots. ## Running a CPU-only AWS instance Load the [AWS console](https://aws.amazon.com/console/) and sign into your account. Go to EC2 | instances and click on the "Launch instance" button. In the "Application and OS Images" section, search the AMI catalog and select the image named * `GaiaNet_ubuntu22.04_amd64` for x86 CPU machines * `GaiaNet_ubuntu22.04_arm64` for ARM CPU machines In the "Instance type" section, select an instance with at least 8GB of RAM. For example, we recommend `t2.large` or `t2.xlarge` instances. In the "Network settings", make sure that you allow SSH connections. Click on the "Launch instance" button and wait for instance to start up. Once the instance is ready, SSH into its public IP address. Once you are in the VM, run the following two commands. ```bash gaianet init gaianet start ``` The node is ready when it shows `The Gaia node is started at: https://...` on the console. You can go to that URL from your browser to interact with the Gaia node. You can [customize your Gaia node](../../customize/customize.md) with your own choice of LLMs and knowledge base snapshots. --- ## Install CUDA on Linux # Install CUDA on Linux If you are using an Nvidia-enabled VM instance from a public cloud, you should probably use the VM image provided by the cloud. It typically has the correct versions of Nvidia driver and CUDA toolkit already installed. Read on if you need to install Nvidia driver and CUDA toolkit on your own machine. ## Ubuntu 22.04 ### 1 Install the Nvidia driver. Rebuild the grub configuration: ``` sudo apt-get install -y gcc make linux-headers-$(uname -r) cat << EOF | sudo tee --append /etc/modprobe.d/blacklist.conf blacklist vga16fb blacklist nouveau blacklist rivafb blacklist nvidiafb blacklist rivatv EOF sudo sed -i 's/GRUB_CMDLINE_LINUX=""/GRUB_CMDLINE_LINUX="rdblacklist=nouveau"/' /etc/default/grub sudo update-grub ``` Download and install the Nvidia driver ``` wget https://storage.googleapis.com/nvidia-drivers-us-public/GRID/vGPU16.1/NVIDIA-Linux-x86_64-535.104.05-grid.run sudo sh NVIDIA-Linux-x86_64-535.104.05-grid.run ``` Confirm the driver is installed successfully ``` nvidia-smi -q | head ==============NVSMI LOG============== Timestamp : Fri Oct 27 21:54:05 2023 Driver Version : 535.104.05 CUDA Version : 12.2 Attached GPUs : 1 GPU 00000000:00:1E.0 Product Name : NVIDIA A10G ``` Disable GSP and reboot. ``` sudo touch /etc/modprobe.d/nvidia.conf echo "options nvidia NVreg_EnableGpuFirmware=0" | sudo tee --append /etc/modprobe.d/nvidia.conf sudo reboot ``` ### 2 Install the CUDA toolkit. ``` wget https://developer.download.nvidia.com/compute/cuda/12.2.2/local_installers/cuda_12.2.2_535.104.05_linux.run sudo sh cuda_12.2.2_535.104.05_linux.run --silent --override --toolkit --samples --toolkitpath=/usr/local/cuda-12 --samplespath=/usr/local/cuda --no-opengl-libs ``` Confirm that CUDA is installed. ``` /usr/local/cuda/bin/nvcc --version nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2023 NVIDIA Corporation Built on Tue_Aug_15_22:02:13_PDT_2023 Cuda compilation tools, release 12.2, V12.2.140 Build cuda_12.2.r12.2/compiler.33191640_0 ``` After that, use the following two commands to set up the environment path. You should probably add these two lines to your `~/.bashrc` and `~/.bash_profile` (or `~/.zshrc` and `~/.profile`) files so that new terminals and future logins will still be able to find these CUDA library files. ``` export LD_LIBRARY_PATH=/usr/local/cuda/lib64:${LD_LIBRARY_PATH} export PATH=/usr/local/cuda/bin:${PATH} ``` ## More resources Here are more scripts that could help you in case you are stuck. * The [Nvidia official install guide](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/) for Linux. * [Yam Peleg's popular script](https://x.com/yampeleg/status/1751823896800583924) for Ubuntu 22.04 * [Make CUDA available in Docker containers](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) --- ## Start a node with Docker # Start a node with Docker You can run all the commands in this document without any change on any machine with the latest Docker and at least 8GB of RAM available to the container. By default, the container uses the CPU to perform computations, which could be slow for large LLMs. For GPUs, * Mac: Everything here works on [Docker Desktop for Mac](https://docs.docker.com/desktop/install/mac-install/). However, the Apple GPU cores will not be available inside Docker containers until [WebGPU is supported by Docker](https://github.com/LlamaEdge/LlamaEdge/blob/main/docker/webgpu.md) later in 2024. * Windows and Linux with Nvidia GPU: You will need to install [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html#installation) for Docker. In the instructions below, replace the `latest` tag with `cuda12` or `cuda11` to use take advantage of the GPU, and add the `--device nvidia.com/gpu=all` flag. If you need to build the images yourself, replace `Dockerfile` with `Dockerfile.cuda12` or `Dockerfile.cuda11`. Find [Gaia Docker images](https://hub.docker.com/?namespace=gaianet) you can run! ## Quick start Start a Docker container for the Gaia node. It will print running logs from the Gaia node in this terminal. ``` docker run --name gaianet \ -p 8080:8080 \ -v $(pwd)/qdrant_storage:/root/gaianet/qdrant/storage:z \ gaianet/phi-3-mini-instruct-4k_paris:latest ``` The node is ready when it shows `The Gaia node is started at: https://...` on the console. You can go to that URL from your browser to interact with the Gaia node. The docker image contains the LLM and embedding models required by the node. However, the vector collection snapshot (i.e., knowledge base) is downloaded and imported at the time when the node starts up. That is because the knowledge based could be updated frequently. The `qdrant_storage` directory on the host machine stores the vector database content. Alternatively, the command to run the Gaia on your Nvidia CUDA 12 machine is as follows. ``` docker run --name gaianet \ -p 8080:8080 --device nvidia.com/gpu=all \ -v $(pwd)/qdrant_storage:/root/gaianet/qdrant/storage:z \ gaianet/phi-3-mini-instruct-4k_paris:cuda12 ``` ## Stop and re-start You can stop and re-start the node as follows. Every time you re-start, it will re-initialize the vector collection (knowledge base). ``` docker stop gaianet docker start gaianet ``` NOTE: When you restart the node, the log messages will no longer be printed to the console. You will need to wait for a few minutes before the restarted node comes back online. You can still see the logs by logging into the container as follows. ``` docker exec -it gaianet /bin/bash tail -f /root/gaianet/log/start-llamaedge.log ``` You can also delete the node if you no longer needs it. ``` docker stop gaianet docker rm gaianet ``` ## Make changes to the node You can update the configuration parameters of the node, such as context size for the models, by executing the `config` command on the `gaianet` program inside the container. For example, the following command changes the chat LLM's context size to 8192 tokens. ``` docker exec -it gaianet /root/gaianet/bin/gaianet config --chat-ctx-size 8192 ``` Then, restart the node for the new configuration to take effect. You will need to wait for a few minutes for the server to start again, or you can monitor the log files inside the container as discussed above. ``` docker stop gaianet docker start gaianet ``` ## Change the node ID You can update the node ID (Ethereum address) associated with the node. Start the node and copy the `nodeid.json` file, as well as the keystore file defined in `nodeid.json` into the container. ``` docker cp /local/path/to/nodeid.json gaianet:/root/gaianet/nodeid.json docker cp /local/path/to/1234-abcd-key-store gaianet:/root/gaianet/1234-abcd-key-store ``` Then, restart the node for the new address and keystore to take effect. ``` docker stop gaianet docker start gaianet ``` ## Build a node image locally Each Gaia is defined by a `config.json` file. It defines the node's required LLM and embedding models, model parameters, prompts, and vector snapshots (e.g., knowledge base). The following command builds a Docker image with two platforms for a node based on the specified `config.json` file. ``` docker buildx build . --platform linux/arm64,linux/amd64 \ --tag gaianet/phi-3-mini-instruct-4k_paris:latest -f Dockerfile \ --build-arg CONFIG_URL=https://raw.githubusercontent.com/GaiaNet-AI/gaianet-node/main/config.json ``` > The `Dockerfile` is available [here](https://raw.githubusercontent.com/GaiaNet-AI/gaianet-node/main/docker/Dockerfile). Feel free to change it to Nvidia [CUDA versions](https://raw.githubusercontent.com/GaiaNet-AI/gaianet-node/main/docker/Dockerfile.cuda12) if your Docker is enabled with the [Nvidia container toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html). You can publish your node for other people to use it. ``` docker push gaianet/phi-3-mini-instruct-4k_paris:latest ``` --- ## Run a local-only node # Run a local-only node By default, the Gaia node registers itself with a Gaia domain and is accesible from the public. For many users, it could also be important to start a local server for testing. To do that, you just need to pass the `--local-only` option. ``` gaianet start --local-only ``` --- ## Install multiple nodes on a single machine # Install multiple nodes on a single machine The [default Gaia installer](../quick-start/quick-start.md) installs the node into the `$HOME/gaianet` base directory. You could install multiple nodes on the same machine. Each node has its own "base directory". To do that, you just need to use the `--base` option. Let's say that the base directory for your second node is `$HOME/node-2`. You can install Gaia node software using the following command. ``` curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash -s -- --base $HOME/node-2 ``` After that, you can specify the `gaianet` CLI to operate on `node-2` by appending the `--base` option as well. ``` gaianet init --base $HOME/node-2 gaianet start --base $HOME/node-2 ``` --- ## Ensuring Gaia Node Reliability # Ensuring Gaia Node Reliability ## Protect the server process Sometimes, the OS could kill the `wasmedge` process on the Gaia node if it consumes too many resources. For production servers, you should protect the server process. ## Use Supervise The `supervise` tool can help us monitor the `wasmedge` process, and automatically restart the process in case the process dies or is killed. The `gaianet` CLI will make use of `supervise` automatically if it is installed on the system. For macOS users, you can install `supervise` via the daemontools tool by running the following command. ``` brew install daemontools ``` For Linux users, please refer to [the installation guide](https://cr.yp.to/daemontools/install.html) to install the `daemontools` tool. ## Reduce the nice value If the `supervise` tool is too heavy handed, we could also increase the priority of the `wasmedge` process. The OS will try NOT to kill high priority processes until it absolutely has to. We do that by reducing the `nice` value of the `wasmedge` process. ``` sudo renice -n -19 $(cat $HOME/gaianet/llamaedge.pid) ``` --- ## Install Gaia on Windows # Install Gaia on Windows Here is the complete guide to installing and running your own decentralised AI inference using Gaia from a Windows machine. ## Prerequisites - Windows Sub-system for Linux - Ubuntu (Latest distribution) Learn more about [system requirements](../system-requirements/system-requirements.md). ## Step 1 Install WSL by opening your Command Prompt or Powershell in your windows machine and running the following command: ``` wsl --install Ubuntu-24.04 ``` Following is the progress that you should notice when WSL and Ubuntu-24.04 is being installed. ![](/img/docs/wsl-install-ubuntu.png) ![](/img/docs/wsl-installer.png) ![](/img/docs/launching-ubuntu.png) ![](/img/docs/installation-complete.png) ## Step 2 Make sure to: - [ ] Set your Unix user account - [ ] Set a password - [ ] Re-enter the password for confirmation Once WSL is installed and your choice of Ubuntu is installed, you should see the following: ![Ubuntu-24.04 Installation Complete](/img/docs/installation-complete.png) ## Step 3 Once you see the user account logged in as shown in the above screenshot, you can follow the Gaia CLI installation steps from [here](https://docs.gaianet.ai/getting-started/quick-start/#installing-the-node). ![Gaia CLI Installation](/img/docs/gaia-cli-installation.png) ![Gaia CLI Installation Complete](/img/docs/cli-installation.png) ![Gaianet Init](/img/docs/gaianet-init.png) ![Gaianet Start](/img/docs/gaianet-start.png) > That's it! Gaia now runs on your Windows machine! --- ## API Reference # API Reference ## Introduction Each Gaia node is an OpenAI compatible API server. You can build your application based on the Gaia node API. You can also replace OpenAI API configuration with the Gaia node API in other AI agent frameworks. The base URL to send all API requests is `https://node_id.gaia.domains/v1`. :::note Make sure to replace `YOUR_API_KEY_GOES_HERE` with your **own API key**. To get your own API key, follow [this](./authentication) tutorial. ::: ## Endpoints ### Chat The `chat/completions` endpoint returns an LLM response based on the system prompt and user query. #### Non-streaming By default, the API responds with a full answer in the HTTP response. **Request** ``` curl -X POST https://node_id.gaia.domains/v1/chat/completions \ -H 'accept:application/json' \ -H 'Content-Type: application/json' \ -H 'Authorization: Bearer YOUR_API_KEY_GOES_HERE' \ -d '{"messages":[{"role":"system", "content": "You are a helpful assistant."}, {"role":"user", "content": "What is the capital of France?"}], "model": "model_name"}' ``` **Response:** ``` {"id":"chatcmpl-bcfeebe0-5372-42c0-ac92-0615213e1c97","object":"chat.completion","created":1716380086,"model":"Llama-3-8B-Instruct-262k-Q5_K_M","choices":[{"index":0,"message":{"role":"assistant","content":"Paris."},"finish_reason":"stop"}],"usage":{"prompt_tokens":61,"completion_tokens":4,"total_tokens":65}}% ``` #### streaming Add `"stream":true` in your request to make the API send back partial responses as the LLM generates its answer. **Request:** ``` curl -X POST https://node_id.gaia.domains/v1/chat/completions \ -H 'accept:application/json' \ -H 'Content-Type: application/json' \ -H 'Authorization: Bearer YOUR_API_KEY_GOES_HERE' \ -d '{"messages":[{"role":"system", "content": "You are a helpful assistant."}, {"role":"user", "content": "What is the capital of France?"}], "model": "model_name", "stream":true}' ``` **Response:** ``` data: {"id":"chatcmpl-73a1f57d-185e-42c2-b8a6-ba0bae58f3b4","choices":[{"index":0,"delta":{"role":"assistant","content":"I"},"logprobs":null,"finish_reason":null}],"created":1716381054,"model":"Llama-3-8B-Instruct-262k-Q5_K_M","system_fingerprint":"fp_44709d6fcb","object":"chat.completion.chunk"} data: {"id":"chatcmpl-73a1f57d-185e-42c2-b8a6-ba0bae58f3b4","choices":[{"index":0,"delta":{"role":"assistant","content":" am"},"logprobs":null,"finish_reason":null}],"created":1716381054,"model":"Llama-3-8B-Instruct-262k-Q5_K_M","system_fingerprint":"fp_44709d6fcb","object":"chat.completion.chunk"} data: {"id":"chatcmpl-73a1f57d-185e-42c2-b8a6-ba0bae58f3b4","choices":[{"index":0,"delta":{"role":"assistant","content":" a"},"logprobs":null,"finish_reason":null}],"created":1716381054,"model":"Llama-3-8B-Instruct-262k-Q5_K_M","system_fingerprint":"fp_44709d6fcb","object":"chat.completion.chunk"} ... data: {"id":"chatcmpl-73a1f57d-185e-42c2-b8a6-ba0bae58f3b4","choices":[{"index":0,"delta":{"role":"assistant","content":" an"},"logprobs":null,"finish_reason":null}],"created":1716381055,"model":"Llama-3-8B-Instruct-262k-Q5_K_M","system_fingerprint":"fp_44709d6fcb","object":"chat.completion.chunk"} data: {"id":"chatcmpl-73a1f57d-185e-42c2-b8a6-ba0bae58f3b4","choices":[{"index":0,"delta":{"role":"assistant","content":" AI"},"logprobs":null,"finish_reason":null}],"created":1716381055,"model":"Llama-3-8B-Instruct-262k-Q5_K_M","system_fingerprint":"fp_44709d6fcb","object":"chat.completion.chunk"} data: {"id":"chatcmpl-73a1f57d-185e-42c2-b8a6-ba0bae58f3b4","choices":[{"index":0,"delta":{"role":"assistant","content":"."},"logprobs":null,"finish_reason":null}],"created":1716381055,"model":"Llama-3-8B-Instruct-262k-Q5_K_M","system_fingerprint":"fp_44709d6fcb","object":"chat.completion.chunk"} data: [DONE] ``` #### Request body | Field | Type | Required | Description | Default | Example | |-------------------|---------|----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | messages | List | Required | A list of messages for the conversation.1 . System message (depends on the large language mode you use) * `content` of the system messages is required * `"role":"system"` is required 2. User message (required) * `content` is required. * `"role":"user"` is required | N/A | "messages": ["role": "system","content": "You are a helpful assistant."},{"role": "user","content": "Hello!"}] | | model | String | Required | The chat model you used | N/A | Llama-3-8B-262k-Q5_K_M | | top_p | Number | Optional | An alternative to sampling with temperature. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. | 1 | Number between 0 and 1. | | temperature | Number | Optional | Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. | 1 | Number between 0 and 2. | | presence_penalty | Number | Optional | Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics. | 0 | Number between -2.0 and 2.0. | | stream | boolean | Optional | Make the answer streaming output | FALSE | "stream":true | | frequency_penalty | Number | Optional | Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood of repeating the same line verbatim. | 0 | Number between -2.0 and 2.0. | #### Response body | Field | Type | Streaming or non-streaming | Description | Default | Example | |---------|---------|----------------------------|------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | id | string | Both | A unique identifier for the chat completion. | Generated randomly | chatcmpl-73a1f57d-185e-42c2-b8a6-ba0bae58f3b4 | | object | string | Both | The object type | `chat.completion.chunk` in the streaming mode. `chat.completion` in the non-streaming mode. | `chat.completion.chunk` in the streaming mode. `chat.completion` in the non-streaming mode. | | choices | array | Both | A list of chat completion choices. | | "choices":[{"index":0,"message":{"role":"assistant","content":"Paris."},"finish_reason":"stop"}] | | created | integer | Both | The Unix timestamp (in seconds) of when the chat completion was created. | N/A | 1716380086 | | model | string | Both | The model used for the chat completion. | Depends on the model you use. | Llama-3-8B-Instruct-Q5_K_M | | usage | object | Both | Usage statistics for the completion request, including completion_tokens, prompt_tokens, and total_tokens. | N/A | "usage":{"prompt_tokens":61,"completion_tokens":4,"total_tokens":65} | ### Embedding The `embeddings` endpoint computes embeddings for user queries or file chunks. **Request** ``` curl -X POST https://node_id.gaia.domains/v1/embeddings \ -H 'accept:application/json' \ -H 'Content-Type: application/json' \ -H 'Authorization: Bearer YOUR_API_KEY_GOES_HERE' \ -d '{"model": "nomic-embed-text-v1.5.f16", "input":["Paris, city and capital of France, ..., for Paris has retained its importance as a centre for education and intellectual pursuits.", "Paris’s site at a crossroads ..., drawing to itself much of the talent and vitality of the provinces."]}' ``` **Response:** ``` { "object": "list", "data": [ { "index": 0, "object": "embedding", "embedding": [ 0.1428378969, -0.0447309874, 0.007660218049, ... -0.0128974719, -0.03543198109, 0.03974733502, 0.00946635101, -0.01531364303 ] }, { "index": 1, "object": "embedding", "embedding": [ 0.0697753951, -0.0001159032545, 0.02073983476, ... 0.03565846011, -0.04550019652, 0.02691745944, 0.02498772368, -0.003226313973 ] } ], "model": "nomic-embed-text-v1.5.f16", "usage": { "prompt_tokens": 491, "completion_tokens": 0, "total_tokens": 491 } } ``` ### Retrieve The `retrieve` endpoint can retrieve text from the node's vector collection based on the user's query. **Request:** ``` curl -X POST https://node_id.gaia.domains/v1/retrieve \ -H 'accept:application/json' \ -H 'Content-Type: application/json' \ -H 'Authorization: Bearer YOUR_API_KEY_GOES_HERE' \ -d '{"messages":[{"role":"system", "content": "You are a helpful assistant."}, {"role":"user", "content": "What is the location of Paris?"}], "model":"nomic-embed-text-v1.5.f16"}' ``` **Response:** ``` { "points": [ { "source": "\"Paris is located in northern central France, in a north-bending arc of the river Seine whose crest includes two islands, the Île Saint-Louis and the larger Île de la Cité, which form the oldest part of the city. The river's mouth on the English Channel is about 233 mi downstream from the city. The city is spread widely on both banks of the river. Overall, the city is relatively flat, and the lowest point is 35 m above sea level. Paris has several prominent hills, the highest of which is Montmartre at 130 m.\\n\"", "score": 0.74011195 }, { "source": "\"The Paris region is the most active water transport area in France, with most of the cargo handled by Ports of Paris in facilities located around Paris. The rivers Loire, Rhine, Rhône, Me\\n\"", "score": 0.63990676 }, { "source": "\"Paris\\nCountry\\tFrance\\nRegion\\nÎle-de-France\\r\\nDepartment\\nParis\\nIntercommunality\\nMétropole du Grand Paris\\nSubdivisions\\n20 arrondissements\\nGovernment\\n • Mayor (2020–2026)\\tAnne Hidalgo (PS)\\r\\nArea\\n1\\t105.4 km2 (40.7 sq mi)\\n • Urban\\n (2020)\\t2,853.5 km2 (1,101.7 sq mi)\\n • Metro\\n (2020)\\t18,940.7 km2 (7,313.0 sq mi)\\nPopulation\\n (2023)\\n2,102,650\\n • Rank\\t9th in Europe\\n1st in France\\r\\n • Density\\t20,000/km2 (52,000/sq mi)\\n • Urban\\n (2019)\\n10,858,852\\n • Urban density\\t3,800/km2 (9,900/sq mi)\\n • Metro\\n (Jan. 2017)\\n13,024,518\\n • Metro density\\t690/km2 (1,800/sq mi)\\nDemonym(s)\\nParisian(s) (en) Parisien(s) (masc.), Parisienne(s) (fem.) (fr), Parigot(s) (masc.), \\\"Parigote(s)\\\" (fem.) (fr, colloquial)\\nTime zone\\nUTC+01:00 (CET)\\r\\n • Summer (DST)\\nUTC+02:00 (CEST)\\r\\nINSEE/Postal code\\t75056 /75001-75020, 75116\\r\\nElevation\\t28–131 m (92–430 ft)\\n(avg. 78 m or 256 ft)\\nWebsite\\twww.paris.fr\\r\\n1 French Land Register data, which excludes lakes, ponds, glaciers > 1 km2 (0.386 sq mi or 247 acres) and river estuaries.\\n\"", "score": 0.62259054 }, { "source": "\" in Paris\\n\"", "score": 0.6152092 }, { "source": "\"The Parisii, a sub-tribe of the Celtic Senones, inhabited the Paris area from around the middle of the 3rd century BC. One of the area's major north–south trade routes crossed the Seine on the île de la Cité, which gradually became an important trading centre. The Parisii traded with many river towns (some as far away as the Iberian Peninsula) and minted their own coins.\\n\"", "score": 0.5720232 } ], "limit": 5, "score_threshold": 0.4 } ``` ### Get the model The `models` endpoint provides the chat and embedding models available on the node. **Request:** ``` curl -X POST https://node_id.gaia.domains/v1/models ``` **Response:** ``` {"object":"list","data":[{"id":"Llama-3-8B-Instruct-262k-Q5_K_M","created":1716383261,"object":"model","owned_by":"Not specified"},{"id":"nomic-embed-text-v1.5.f16","created":1716383261,"object":"model","owned_by":"Not specified"}]}% ``` ### Get the node info The `info` endpoint provides detailed information about the node. **Request:** ``` curl -X POST https://node_id.gaia.domains/v1/info ``` **Response:** ``` { "version": "0.5.0", "plugin_version": "b2694 (commit 0d56246f)", "port": "8080", "models": [ { "name": "Llama-2-7b-chat-hf-Q5_K_M", "type": "chat", "prompt_template": "Llama2Chat", "n_predict": 1024, "n_gpu_layers": 100, "ctx_size": 4096, "batch_size": 512, "temperature": 1.0, "top_p": 1.0, "repeat_penalty": 1.1, "presence_penalty": 0.0, "frequency_penalty": 0.0 }, { "name": "all-MiniLM-L6-v2-ggml-model-f16", "type": "embedding", "prompt_template": "Llama2Chat", "n_predict": 1024, "n_gpu_layers": 100, "ctx_size": 384, "batch_size": 512, "temperature": 1.0, "top_p": 1.0, "repeat_penalty": 1.1, "presence_penalty": 0.0, "frequency_penalty": 0.0 } ], "qdrant_config": { "url": "http://localhost:6333", "collection_name": "default", "limit": 5, "score_threshold": 0.4 } } ``` ## Status Codes | HTTP response code | Description | Reason | Solutions | |--------------------|-----------------------|-----------------------------|----------------------------------| | 404 | Not found | The endpoint URL is invalid | Please check the endpoint URL | | 500 | Internal Server Error | Model is not found. | Please check out the model name. | | 400 | Bad request | | | :::info Head over to the [Troubleshooting](./troubleshooting/troubleshooting.md) page for more information into common issues and solutions. ::: --- ## Get your API key # Get your API key We're introducing API keys for authentication. You can create API keys by following the steps below: 1. Go to https://gaianet.ai and click on **Launch App** 2. Click **CONNECT** and connect your Metamask Wallet 3. After connecting your wallet, click on the profile drop down and then click **Setting** ![](/img/docs/settings-for-api.png) 4. Under **Setting**, click on **Gaia API Keys** and then **Create API Key** ![Go to settings to get your API key](/img/docs/settings-for-api-keys.png) 5. Give your API Key a name and click **Create** :::danger Important Remember that your API key is a secret! Do not share it with others or expose it in any client-side code (browsers, apps). Production requests must be routed through your own backend server where your API key can be securely loaded from an environment variable or key management service. ::: ![Gaia API keys option under settings](/img/docs/create-api-key.png) ![API key is now created](/img/docs/api-key-created.png) :::tip Tip Your API Key is like a password and helps verify your usage. This will be one of the last times you‘ll see it displayed, so remember to save it in a safe place. ::: :::warning #### API Keys are now available for all your applications and usage. Currently, we don't charge anything for usage or API key creation. However, it is highly recommended that you start using and updating your existing applications or new ones with your own API keys to avoid any disruption in the future. Please refer to the [API Reference](../../getting-started/api-reference.md) page for the updated examples with the API keys usage. ::: ### Frequently Asked Questions When do I need an API key? There are two types of API Keys that you'll need to interact with Gaia Nodes & Gaia Domains. - API Key for [Public Gaia Domains](../../nodes/nodes.md) (these are maintained by Gaia) - API Key for Domains on the network An API Key is required for all requests to the Gaia Node's OpenAI compatible endpoints. Error: Your API key doesn't match the type. Please apply for a Developer API key. To interact with the public [Gaia Domains](../../nodes/nodes.md), you'll need to apply for a Developer API key with Free Developer Credits. Apply for Developer Free Trial by clicking on the image as shown below: ![Free Developer Credits on Gaia](/img/docs/developer-free-trial.png) :::info All API keys created through an account that has been approved for free developer credits cannot be used to interact with other Gaia Domains on the network. Create a different account to interact with the Gaia Domains on the network (other than the public nodes maintained by Gaia) ::: --- ## Gaia CLI options # Gaia CLI options After installing the Gaia software, you can use the `gaianet` CLI to manage the node. The following are the CLI options. ## help You can use `gaianet --help` to check all the available CLI options. ``` gaianet --help ## Output Usage: gaianet {config|init|run|stop|OPTIONS} Subcommands: config Update the configuration. init Initialize the GaiaNet node. run|start Start the GaiaNet node. stop Stop the GaiaNet node. info Show the device_id and node_id. Options: --help Show this help message ``` ## version You can use `gaianet --version` to check your GaiaNet version. ``` gaianet --version ``` ## init The `gaianet init` command initializes the node according to the `$HOME/gaianet/config.json` file. You can use some of our [pre-set configurations](https://github.com/GaiaNet-AI/node-configs). * `gaianet init` will init the default node. It's an RAG application with Gaia knowledge. * `gaianet init --config mua` will init a node with the MUA project knowledge. * `gaianet init --base ` will init a node in an alternative directory. You can also use `gaianet init url_your_config_json` to init your customized settings for the node. You can customize your node using the Gaia node link. If you're familiar with the Gaia config.json, you can create your own manually. See an example [here](https://github.com/GaiaNet-AI/gaianet-node/blob/main/config.json). ``` gaianet init --config https://raw.githubusercontent.com/GaiaNet-AI/node-configs/main/pure-llama-3-8b/config.json ``` ## start The `gaianet start` is to start running the node. * Use `gaianet start` to start the node according to the `$HOME/gaianet/config.json` file. * Use `gaianet start --base $HOME/gaianet-2.alt ` to start the node according to the `$HOME/gaianet-2/config.json` file. * Use `gaianet start --local-only` to start the node for local use according to the `$HOME/gaianet/config.json` file. ## stop The `gaianet stop` is to stop the running node. * Use `gaianet stop` to stop running the node. * Use `gaianet stop --force` to force stop the Gaia node. * Use `gaianet stop --base $HOME/gaianet-2.alt` to stop the node according to the `$HOME/gaianet-2/config.json` file. ## config The `gaianet config` can update the key fields defined in the `config.json` file. * `gaianet config --help` will list all the available arguments * `gaianet config --chat-url ` will change the download link of the chat model. * `gaianet config --prompt-template ` will change the prompt_template of the chat model. * `gaianet config --chat-ctx-size ` will change the context size of the chat model. The default value is 4096. * `gaianet config --embedding-url ` will change the download link of the embedding model. * `gaianet config --embedding-ctx-size ` will change the context size of the embedding model. The value here is associated with the embedding model you choose. * `gaianet config --port ` will change the port of the Gaia node API server. * `gaianet config --system-prompt ""` will change the system prompt. * `gaianet config --rag-prompt ""` will change the rag prompt. * `gaianet config --reverse-prompt ""` will change the reverse prompt. * `gainet config --base ` will modify the `/config.json` parameters. After you use `gaianet config` to change some parameters, please 1. use `gaianet init` to make your settings take effect. 2. use `gaianet start` to start your new node. If you use `gaianet config --base $HOME/gaianet-2.alt` to update some settings, please 1. use `gaianet init --base $HOME/gaianet-2.alt` to make your settings take effect. 2. use `gaianet start --base $HOME/gaianet-2.alt` to start your new node. The `gaianet config` supports multiple parameters in one command. The example below will change the download link and prompt template of the chat model at the same time. ``` gaianet config --chat-url https://huggingface.co/gaianet/gemma-1.1-2b-it-GGUF/resolve/main/gemma-1.1-2b-it-Q5_K_M.gguf --prompt-template gemma-chat ``` The output is the following. ``` [+] Updating the url of chat model ... * Old url: https://huggingface.co/gaianet/Llama-3-8B-Instruct-GGUF/resolve/main/Meta-Llama-3-8B-Instruct-Q5_K_M.gguf * New url: https://huggingface.co/gaianet/gemma-1.1-2b-it-GGUF/resolve/main/gemma-1.1-2b-it-Q5_K_M.gguf [+] Updating the prompt template of chat model ... * Old template: llama-3-chat * New template: gemma-chat [+] COMPLETED! The config.json is updated successfully. ``` ## info You can use `gaianet info` to display the id of the current running node and device running the node. ``` gaianet info ``` Running the command gives this output: ``` Node ID: 0x1234567890abcdef0987654321fedcbabcdef123 Device ID: device-fedcbabcdef0123456789098 ``` ## base The `--base` option is global. You can combine it with other subcommands to specify a base directory for the Gaia node other than the `$HOME/gaianet`. --- ## Customizing Your Gaia Node # Customizing Your Gaia Node A key goal of the Gaia project is to enable each individual to create and run his or her own agent service node using finetuned LLMs and proprietary knowledge. In all likelihood, you are not going to run a node with the [default](../quick-start) Llama 3.2 LLM and Paris guidebook knowledge base. In this chapter, we will discuss ways to customize your node. ## Pre-set configurations All the node configuration options, such as LLM settings, vector collection for the knowledge base, and prompts, are all in the `gaianet/config.json` file. You can edit this file directly to use your models and vector collections. Or, you can select a different `config.json` when you initialize the node. Just pass in a URL to the `config.json` file in your `gaianet init` command. We have several pre-set `config.json` files to choose from [in this repo](https://github.com/GaiaNet-AI/node-configs). For example, the following command initialize a Gaia node with a Llama 3 8B model. ``` gaianet init --config https://raw.githubusercontent.com/GaiaNet-AI/node-configs/main/llama-3-8b-instruct/config.json ``` > The URL to the `config.json` must point to the actual text file. (i.e., the `raw.githubusercontent.com` URL for GitHub links) instead of the GitHub HTML page for that file. ## The config subcommand After you have initialized the node, you can still make changes to its configuration by editing the `config.json` file directly. But it is easier and safer to use the `gaianet` CLI to make changes. :::note You MUST run `gaianet init` and `gaianet start` again after you make any changes to the node configuration. ::: The following command shows the `config.json` fields you can make changes to. ``` gaianet config list ``` Now, let's look at some examples. ### Select an LLM There are over 10,000 finetuned open-source LLMs you can choose from on Huggingface. They each have different sizes (larger models are more capable but more expensive to run), unique capabilities (e.g., uncensored, to excel in math or reasoning, to support large context length etc), domain expertise (e.g., medicine, coding), and / or styles (e.g., to speak like a teacher or a pirate, to respond in code, to follow conversations). To replace Gaia node's default LLM with an alternative finetuned model, you will need to make changes to the model file, prompt template, and model context length parameters. Those parameters vary depending on the model, but they can be found on the [Gaia Huggingface organization's](https://huggingface.co/gaianet) model cards. For example, the following command changes the LLM to a Llama 3 8B model. ``` gaianet config \ --chat-url https://huggingface.co/gaianet/Llama-3-8B-Instruct-GGUF/resolve/main/Meta-Llama-3-8B-Instruct-Q5_K_M.gguf \ --chat-ctx-size 4096 \ --prompt-template llama-3-chat ``` > The llama 3 8B model requires at least 16GB of RAM. If none of the published finetuned models are perfect for your use case, you can also finetune your own LLM by following [these guides](../../tutorial/llamacpp). Your Gaia node can run your own finetuned models. > The `--chat-url` argument could point to a local file under `$HOME/gaianet` instead of a public URL. That allows you to use a privately trained or finetuned LLM model file. ### Select a knowledge base A key feature of Gaia is that users can create and deploy proprietary knowledge base on the node to supplement the LLM. Each knowledge base is a snapshot file for a vector collection. We encourage you to [create your own knowledge base](../../knowledge-bases/how-to). But you can also use ready-made knowledge bases. You will need to do the following. * specify the URL to the vector collection (i.e., the `snapshot` or `snapshot.tar.gz` file) in the `snapshot` option. * use the same embedding model that generated this vector collection. * modify the `system_prompt` to give the model background knowledge. * modify the `rag_prompt` to instruct the model to answer the question when context is retrieved from the vector collection. The following example changes the knowledge base in the node from "Paris guidebook" to "London guidebook". ``` gaianet config \ --snapshot https://huggingface.co/datasets/gaianet/london/resolve/main/london_768_nomic-embed-text-v1.5-f16.snapshot.tar.gz \ --embedding-url https://huggingface.co/gaianet/Nomic-embed-text-v1.5-Embedding-GGUF/resolve/main/nomic-embed-text-v1.5.f16.gguf \ --embedding-ctx-size 8192 \ --system-prompt "You are a tour guide in London, UK. Please answer the question from a London visitor accurately." \ --rag-prompt "The following text is the context for the user question.\n----------------\n" ``` > The `--snapshot` could point to a local file under `$HOME/gaianet` instead of a public URL. That allows you to use a private vector collection snapshot. Depending on the quality and size of the vectors, you might also need to change the `qdrant-` options to customize the retrieval behavior. * `qdrant-limit` sets the max number of relevant context to add to the prompt. If your knowledge base consists of large sections of text (i.e., each book chapter is a vector), you should probably make this 1 or 2 to limit the prompt length to a reasonable size. * `qdrant-score-threshold` is the min match "score" the knowledge content must meet in order to be considered "relevant". This depends on the quality of the knowledge text and the embedding model. In general, this score should be over 0.5 to reduce irrelevant context in the prompt. > The embedding model encodes and transforms text into vectors so that the can be stored, searched and retrieved. For different context material, you might need a different embedding model to achieve the optimal performance. The [MTEB leaderboard](https://huggingface.co/spaces/mteb/leaderboard) is a good place to see the performance benchmarks of embedding models. You can find many of them in the [gaia organization on Huggingface](https://huggingface.co/gaianet). ### Customize prompts In `config.json`, you can also customize the prompts. Prompts are often tailored for the finetuned LLM or the knowledge base to generate optimal responses from the node. The `--system-prompt` option sets a system prompt. It provides the background and "personality" of the node. Each API request can set its own system prompt. The `--rag-prompt` is the prompt to be appended after the system prompt (or user query). It introduces the RAG context retrieved from the vector database, which follows it. The `--rag-policy` option specifies where the `rag-prompt` and context should go. By default, its value is `system-message` and it puts the context in the system prompt. But you could also set it to `last-user-message`, which puts the `rag-prompt` and context in front of the latest message from the user. ## Next steps Remember to re-initialize and re-start the node after you make configuration changes. ``` # If the node is running # gaianet stop gaianet init gaianet start ``` Next, you can * [Create a knowledge base](../../knowledge-bases/how-to) from your proprietary knowledge or skills. * [Finetune](../../tutorial/llamacpp) your own LLM. Have fun! --- ## Install or Uninstall the CLI # Install or Uninstall the CLI The Gaia node utilizes version control from [its source GitHub repo](https://github.com/GaiaNet-AI/gaianet-node). You can check out the Gaia node versions from [the release page](https://github.com/GaiaNet-AI/gaianet-node/releases). ## Install You can install the WasmEdge Runtime on any generic Linux and MacOS platform. ### Install the latest version of Gaia node To install the most recent version of Gaia node, run the following command line. ```bash curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash ``` The Gaia node will be installed in your `$HOME/gaianet` folder by default. > If you want to install gaianet in a different directory, you can use `curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash -s -- --base ` to specify where you want to install GaiaNet. Once you use `--base` to define a different directory, you should always add `--base ` to init and start your node. > Here is an example: > ``` > # Assume that you're in the root directory > mkdir test > curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash -s -- --base $HOME/test > gaianet init --base $HOME/test > gaianet start --base $HOME/test > gaianet stop --base $HOME/test > ``` ### Install the specific version of Gaia Node If you want to install a particular Gaia node version, change the version number in the following command line. ``` curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/download/0.1.3/install.sh' | bash ``` Check out the release log [here](https://github.com/GaiaNet-AI/gaianet-node/releases). ## Update the current Gaia node Simply run the following command to upgrade your node. ``` curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash -s -- --upgrade ``` :::tip The `upgrade` option will keep your node id. To ensure your node ID and device ID are properly protected, it's recommended to back up the following files in a secure location. If anything goes wrong, you can use these files to recover your node: * nodeid.json * deviceid.txt * Keystore file (e.g., 0278996e-5dad-4xy9-b3xu7-be3xuxxxaac94) ::: ## What's installed If you install the Gaia node in the `$HOME/gaianet` directory by default, you will have the following directories and files after installation: * The `$HOME/gaianet/bin` directory contains the Gaia CLI tool, frpc binary and Qdrant Vector database binary. * The `$HOME/gaianet/` directory contains the `llamaedge-api-server.wasm` and `rag-api-server.wasm` for the LLM inference, dashboard (chatbot ui), nodeid.json for the registering your node, and gaianet-domain binary. * The `$HOME/.wasmedge/bin` directory contains the WasmEdge Runtime CLI executable files, which serve as the LLM runtime. ## CLI options for the installer You can use the following command line to check out all the available CLI options ``` curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash -s -- --help ``` The output should be as follows. You can use the following options to customize your installation. ``` Usage: ./install.sh [Options] Options: --config Specify a url to the config file --base Specify a path to the gaianet base directory --reinstall Install and download all required deps --tmpdir Specify a path to the temporary directory [default: /tmp] --ggmlcuda [11/12] Install a specific CUDA enabled GGML plugin version [Possible values: 11, 12]. --enable-vector: Install vector log aggregator --version Print version --help Print usage ``` ## Uninstalling the Gaia CLI To uninstall or clear the environment, run the following command. ``` curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/uninstall.sh' | bash ``` :::danger **Important Reminder: This command will remove all the Gaia-related files, including the `nodeid.json`. It's your responsibility to keep your nodeid.json safe. If you want to run the same node after reinstalling, please save the `nodeid.json` file and `frpc.toml` file carefully.** ::: --- ## Using your Gaia Node # Using your Gaia Node When you [start a Gaia node](../quick-start/quick-start.md), or you find a node on the web, you could use it as a web-based chatbot UI and an OpenAI compatible web service. Just load the node's public URL in the browser to open its dashboard. Let's say the URL is as follows. ``` https://0x1234...xyz.gaia.domains/ ``` > Please refer to the [agent apps](../../agent-integrations/intro) section to see how to use the Gaia node API in your favorite agent frameworks or apps. ## Web-based chatbot On the Gaia node dashboard, you will see a "Chat with this node" button. ![Gaia Node's Default Dashboard](/img/docs/chat_button.png) ## OpenAI API replacement The Gaia node is a drop-in replacement for OpenAI API in [agent and LLM apps](../../agent-integrations/intro). On the Gaia node dashboard, you will see a table that shows how to replace OpenAI parameters in those apps. ![Options to use with OpenAI Compatible APIs](/img/docs/openai_api_options.png) --- ## Setting up your own node # Setting up your own node This guide provides the requisite knowledge necessary to quickly get started with installing a Gaia node. ### Prerequisites Before you get started, ensure that you have the following on your system: | System | Minimum Requirements | |---|---| | OSX with Apple Silicon (M1-M4 chip) | 16GB RAM (32GB recommended) | | Ubuntu Linux 20.04 with Nvidia CUDA 12 SDK | 8GB VRAM on GPU | | Azure/AWS | Nvidia T4 GPU Instance | Learn more about [system requirements](../system-requirements/). **For Windows Users: Important Note on WSL (Windows Subsystem for Linux)** If you are using a Windows system, you **must** have [Windows Subsystem for Linux (WSL)](https://learn.microsoft.com/en-us/windows/wsl/install) installed and configured with an Ubuntu distribution. The installation commands for the Gaia node are Linux-based and require a Linux environment to run, which WSL provides on Windows. ### Installing the node 1. Use the following command to download the latest version of the Gaia node: ``` curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash ``` 2. Run the command printed on the terminal to set up the environment path, it is started with `source`. ![](/img/docs/quick-start.png) 3. Use the following command to initialize the Gaia node according to the configuration options in `$HOME/gaianet/config.json`. By default, the Gaia is initialized with a [Llama 3.2](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct) LLM. This command could take some time since it will download a very large LLM file. ``` gaianet init ``` 4. Use the following command to start your node: ``` gaianet start ``` #### After starting your node - A successful start prints a public URL for the node. Opening a browser to that URL will display the node information and allow you to chat with the AI agent on the node. ``` ... ... https://0xf63939431ee11267f4855a166e11cc44d24960c0.gaia.domains ``` - To stop the node: `gaianet stop` ## Video Guide ## Next steps - [Customize](../customize) your node. Here are some knowledge bases you can try by customizing your node: - [Rust](https://huggingface.co/datasets/gaianet/learn-rust) - [Solidity](https://huggingface.co/datasets/harishkotra/solidity) - [Paris](https://huggingface.co/datasets/gaianet/paris) - [Vyper Lang Developer Docs](https://huggingface.co/datasets/meowy-ai/vyper-lang) - [Monad Developer Docs](https://huggingface.co/datasets/harishkotra/monad-docs) - [Web3 Knowledge Base](https://huggingface.co/datasets/meowy-ai/web3-knowledge-base) - [Join the Gaia Protocol](../register) and join the Gaia protocol network to receive payments. - [Ensure Node Reliability](../advanced-deployment-options/protect.md) the node server to ensure stable service. --- ## Joining the Gaia Protocol # Joining the Gaia Protocol After successfully running a Gaia node on your machine, it's time to join the Gaia protocol network and get rewards for sharing computing power with the world by binding your node ID and device ID to a Metamask account. When you run a node with the Getting Started guide, you may notice that the Gaia software has generated a node ID for you. The node ID is an ETH address. The easiest way to find the node ID is to use `gaianet info` to print the node ID and device ID on the terminal. ``` gaianet info ``` The output will be the following: ``` Node ID: 0x80642b1----9a76a284efeb2bb6d142e Device ID: device-e77---1446f1b51 ``` ![](/img/docs/register-01.png) To receive rewards, bind your node and device IDs with a Metamask account using the Gaia web portal: 1. Open https://www.gaianet.ai/ on your browser and click **"Launch App."** 2. click **Connect** to log into the website with Metamask. 3. Hover on your profile and click **"Setting"** and click **"Nodes."** 4. Click **"Connect new node"** and enter your node and device IDs in the boxes. 5. Click **"Join"** ![](/img/docs/node-register.png) After your node has successfully joined the network, it will be displayed in the list of nodes on the Node Page. :::note If you are running multiple nodes, you can bind the node IDs and their corresponding device IDs to your MetaMask wallet. ::: ### Protect your node ID and device ID The Gaia installer generates a pair of an ETH address and keystore and password for your node automatically. This information is stored in the `gaianet/nodeid.json` file. Please keep the JSON file carefully. - The ETH address is your node ID. You will use this ETH address to join the Gaia network. - The keystore stores the private key associated with the ETH address encrypted by the password. :::tip The `nodeid.json` file is the only proof that your node belongs to you. In many protocol operations, you will need this private key to sign request messages to send to the protocol smart contracts. ::: :::note The device ID is only visible to you. ::: ## Join a Domain Once your node is successfully bound, you can proceed to join a Gaia Domain. There are two ways to join a domain: * Join a domain from your nodes management page * Join a domain from the [AI Agent Domains page](https://www.gaianet.ai/agents) ### Steps to Join a Domain from Your Node Management Page 1. **Locate Your Nodes** Navigate to **Profile → Nodes** to view the list of nodes you already registered. 2. **Initiate the Join Process** Click the `...` button next to your node and select **Join Domain**. This will guide you through the steps to join a Gaia Domain. 3. **Follow the Join Steps** - **Update Node Domain** Change your node's domain to `gaia.domains`, which is necessary. The following command line will change your node's domain to `gaia.domains`. If your node has already used `gaia.domains`, then you can skip this step. ``` gaianet stop gaianet config --domain gaia.domains gaianet init gaianet start ``` - **Select a Domain** Choose a domain from the available online domain list. You can review essential details for each domain, such as: - Required LLM - Join policy - Number of joined nodes - **Verify Node Status** Ensure your node meets the domain's requirements. If everything checks out, you can submit a join request to the domain. > Normally, your node needs to be online and meet the specific LLM requirement. ### Steps to Join a Domain from the AI Agent Domains page The AI agent Domains page featured lots of domains various from crypto knowledge to useful tools. Each has options for "Chat Now" or "Join Now". You can click on "Join Now" to join that domain. ### Important Notes - Some Gaia Domains require approval for new nodes to join. - Ensure you comply with the domain's rules before your node becomes publicly accessible under the domain's URL. Following these steps will seamlessly integrate your node into a Gaia Domain. --- ## System Requirements # System Requirements You can install the Gaia on a wide variety of devices and operating systems with or without GPUs. The node installing and operating instructions work on devices ranging from Raspberry Pi, MacBooks, Linux servers, Windows Desktop, to cloud-based Nvidia H100 clusters. For institutional operators, we recommend EITHER of the following for a Gaia node. * Mac desktop or server computers (i.e., iMac, Mini, Studio or Pro) with Apple Silicon (M1 to M4), and at least 16GB of RAM (32GB or more recommended). * Ubuntu Linux 22.04 server with NVIDIA CUDA 12 SDK installed. At least 8GB of VRAM on the GPU is required (24GB or more recommended). On AWS and Azure, that means GPU instances with at least the Nvidia T4 GPU installed. > Check out our [tutorial](../advanced-deployment-options/cuda) on how to install the NVIDIA driver and the CUDA toolkit on a Ubuntu 22.04 machine. If you are hosting the node in your home or office, it needs access to the public Internet to join the Gaia network. ## Supported on Gaia node software is designed to be cross-platform, allowing it to run on various CPU and GPU architectures. The Gaia installer automatically detects the presence of NVIDIA CUDA drivers and leverages the power of GPU accelerators on the device. More hardware support is on the way. ### GPU The Gaia node can run on all types of NVIDIA GPU products from H100 to NVIDIA Jetson series of hardware. It also runs on all Apple Silicon M-series GPUs. ### CPU * Arm-64 based on CPU chips * X86 based on CPU chips * Apple M1 chips * Apple M1 Pro chips * Apple M1 Max chips * Apple M1 Ultra chips * Apple M2 chips * Apple M2 Pro chips * Apple M2 Max chips * Apple M2 Ultra chips * Apple M3 chips * Apple M3 Pro chips * Apple M3 Max chips * Apple M3 Ultra chips * Apple M4 chips * Apple M4 Pro chips * Apple M4 Max chips ### OSs * macOS * Linux-like OS --- ## Troubleshooting # Troubleshooting ### The system cannot find CUDA libraries {#hidden-headings} The system cannot find CUDA libraries Sometimes, the CUDA toolkit is installed in a non-standard location. The error message here is often not able to find `libcu*12`. For example, you might have CUDA installed with your Python setup. The following command would install CUDA into Python's environment. ```bash sudo apt install python3-pip -y pip3 install --upgrade fschat accelerate autoawq vllm ``` The easiest way to fix is simply to link those non-standard CUDA libraries to the standard location, like this. ```bash ln -s /usr/local/lib/python3.10/dist-packages/nvidia/cublas/lib/libcublas.so.12 /usr/lib/libcublas.so.12 ln -s /usr/local/lib/python3.10/dist-packages/nvidia/cuda_runtime/lib/libcudart.so.12 /usr/lib/libcudart.so.12 ln -s /usr/local/lib/python3.10/dist-packages/nvidia/cublas/lib/libcublasLt.so.12 /usr/lib/libcublasLt.so.12 ``` ### Failed to recover from collection snapshot on Windows WSL {#hidden-headings} Failed to recover from collection snapshot on Windows WSL On Windows WSL, you could see this error while running `gaianet init`. ```bash * Import the Qdrant collection snapshot ... The process may take a few minutes. Please wait ... * [Error] Failed to recover from the collection snapshot. {"status":{"error":"Service internal error: Tokio task join error: task 1242 panicked"},"time":0.697784244} ``` When you look into the `~/gaianet/log/init-qdrant.log` file, you could see this line of error ```bash 2024-05-20T07:24:52.900895Z ERROR qdrant::startup: Panic occurred in file /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/cgroups-rs-0.3.4/src/memory.rs at line 587: called `Result::unwrap()` on an `Err` value: Error { kind: ReadFailed("/sys/fs/cgroup/memory.high"), cause: Some(Os { code: 2, kind: NotFound, message: "No such file or directory" }) } ``` The solution is to disable the `autoMemoryReclaim` feature in WSL. Step to turn on/off this feature: 1. Edit `C:\Users.wslconfig` 2. Remove or comment out `autoMemoryReclaim` in `[experimental]` section. ![](/img/docs/disable_autoMemoryReclaim_wsl.png) Thanks to [RoggeOhta](https://github.com/RoggeOhta) for discovering this. You can learn more about it [here](https://github.com/GaiaNet-AI/gaianet-node/issues/46). ### Failed to start the node with an error message `Port 8080 is in use. Exit ...` {#hidden-headings} Failed to start the node with an error message `Port 8080 is in use. Exit ...` You may see the following error when you run `gaianet start`. ```bash gaianet start [+] Checking the config.json file ... You already have a private key. [+] Starting LlamaEdge API Server ... Port 8080 is in use. Exit ... ``` The solution is to run `gaianet stop` first to kill all processes, and then run `gaianet start` to start the node. ### Load library failed: libgomp.so.1: cannot open shared object file: No such file or directory {#hidden-headings} Load library failed: libgomp.so.1: cannot open shared object file: No such file or directory On Windows WSL, you may see this error when running `curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash` ```bash * Generate node IS [2024-07-02 17:50:55.175] [error] loading failed: invalid path, Code: 0x20 [2024-07-02 17:50:55.175] [error] Load library failed: libgomp.so.1: cannot open shared object file: No such file or directory [2024-07-02 17:50:55.176] [error] loading failed: invalid path, Code: 0x20 [2024-07-02 17:50:55.176] [error] Load library failed: libgomp.so.1: cannot open shared object file: No such file or directory ``` The error is caused by the lack of `libgomp.so.1`, a library that should be automatically installed on Ubuntu by default. To solve this, you must install the `libgomp.so.1` library. ```bash sudo apt-get update sudo apt-get install libgomp1 ``` If you're using CentOS, you can use ```bash yum install libgomp ``` This issue was fixed in `version 0.2.2`. ### Failed to remove the default collection {#hidden-headings} Failed to remove the default collection ```bash Failed to remove the default collection. {"status":{"error":"Service internal error: No such file or directory (os error 2)"},"time":0.050924542} ``` It typically indicates that the Qdrant instance was not shut down properly before you try to init it again with a new snapshot. The solution is to stop the GaiaNet node first. ```bash gaianet stop ``` Alternatively, you could manually kill the processes from the terminal or in the OS's Activity Monitor. ```bash sudo pkill -9 qdrant sudo pkill -9 wasmedge sudo pkill -9 frpc ``` Then you can run `gaianet init` and then `gaianet start` again. ### File I/O error {#hidden-headings} File I/O error ```bash * Import the Qdrant collection snapshot ... The process may take a few minutes. Please wait ... * [Error] Failed to recover from the collection snapshot. An error occurred processing field `snapshot`: File I/O error: Operation not permitted (os error 1) ``` It typically indicates that the Qdrant instance was not shut down properly before you try to init it again with a new snapshot. The solution is to stop the GaiaNet node first. ```bash gaianet stop ``` Alternatively, you could manually kill the processes from the terminal or in the OS's Activity Monitor. ```bash sudo pkill -9 qdrant sudo pkill -9 wasmedge sudo pkill -9 frpc ``` Then you can run `gaianet init` and then `gaianet start` again. ### The "Failed to open the file" Error {#hidden-headings} The "Failed to open the file" Error ```bash Warning: Failed to open the file Warning: https://huggingface.co/datasets/max-id/gaianet-qdrant-snapshot/resolve Warning: /main/consensus/consensus.snapshot: No such file or directory curl: (23) Failure writing output to destination ``` The reason for this type of error is a misconfigured `config.json` file. The solution is to delete the comments in `config.json` and re-run the `gaianet init` command. ### The "Too many open files" Error on macOS {#hidden-headings} The "Too many open files" Error on macOS When running `gaianet init` to initialize a new node on macOS, you may encounter an error related to snapshot recovery if your snapshot contains a large amount of text. The error message may be the following: ```bash * [Error] Failed to recover from the collection snapshot. {"status":{"error":"Service internal error: Too many open files (os error 24)"},"time":1.574064833} * [Error] Failed to recover from the collection snapshot. {"status":{"error":"Service internal error: Too many open files (os error 24)"},"time":1.574064833} ``` This issue is caused by the default file descriptor (FD) limit on macOS, which is set to a relatively low value of 256. To resolve this issue, you can increase the default FD limit on your system. To do so, run the following command: ```bash ulimit -n 10000 ``` This will temporarily set the FD limit to 10,000. Next, use `gaianet init` and `gaianet start` commands in the SAME terminal. ### Permission denied when use the installer script to install WasmEdge {#hidden-headings} Permission denied when use the installer script to install WasmEdge When running `curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash` to install GaiaNet node software, you may meet the permission denied error especially installing the WasmEdge runtime. ![](/img/docs/troubleshooting-01.png) This error is caused by the lack of `/tmp` write permission. You can use `--tmpdir` to specify where you want to install the WasmEdge runtime. Please note, you will need to have written permission to the `` folder. ```bash curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash -s -- --tmpdir YOUR_PATH ``` This problem is fixed in `version 0.2.3`. ### gaia-nexus is not ready (exit status: 56) {#hidden-headings} gaia-nexus is not ready (exit status: 56) If you see the following error message, please use `gaianet start --wait 60` and try again. ![gaia-nexus is not ready (exit status: 56)](/img/docs/gaia-nexus-exit-56.png) Alternatively, you should try reducing the `chat_ctx_size` value in the `config.json` to a smaller number like 8192 or 4096 as the default value may be too large for your machine. ![context size too large](/img/docs/ctx-size-too-large.png) ### My node response is slow {#hidden-headings} My node response is slow You should try reducing the `chat_ctx_size` value in the `config.json` to a smaller number like 8192 or 4096 as the default value may be too large for your machine. ![context size too large](/img/docs/ctx-size-too-large.png) If you still have slow response, you should use a higher end machine. --- ## What is a Gaia node? # What is a Gaia node? A Gaia node is an open-source developer platform that lets anyone build, launch, scale and monetize AI agents. It's like having your own personal AI assistant that you can customize and share with others. Here's a breakdown of the key components inside a Gaia node: - **WasmEdge Runtime**: WasmEdge is a lightweight, high-performance, and extensible WebAssembly runtime for cloud native, edge, and decentralized applications. It powers serverless apps, embedded functions, microservices, smart contracts, and IoT devices. It is the easiest and the fastest way to run LLMs on your own devices. Check the Github of WasmEdge [here](https://github.com/WasmEdge/WasmEdge). - **LLM (Large Language Models)**: You can use any LLM from Huggingface, which is a platform hosting thousands of open-source models. Gaia has its own organization page on Huggingface where you can find optimized models for Gaia nodes. You can choose or fine-tune these models for specific tasks or knowledge areas. For example, you could select a model that's an expert in chemistry or one that mimics a particular writing style. Checkout Gaia on Huggingface [here](https://huggingface.co/gaianet). - **RAG (Retrieval-Augmented Generation)**: This component helps the AI access and use relevant information from a knowledge base. It's like giving the AI the ability to quickly look up facts in a specialized encyclopedia before answering questions. - **Vector Database**: This stores information in a format that the AI can easily search and understand. It's similar to how a library organizes books, but for AI-friendly data. - **Multimodal | Embedding Models**: These allow the node to understand and work with different types of data, like text, images, or even audio. The embedding part helps convert this information into a format the AI can process efficiently. - **API Server**: This is the interface that allows users or applications to interact with the Gaia node. It's like a reception desk that takes requests and returns answers. - **Tool Usage and Function Calling**: These components allow the AI to use external tools or perform specific actions. For instance, the AI could use a calculator tool to solve math problems or call a weather API to get current weather information. - **Prompt Selection and Management**: This helps guide the AI's responses by providing context or instructions. It's like giving the AI a script to follow for different scenarios. - **Node ID**: This is a unique identifier for each Gaia node, allowing it to be recognized and connected to the larger Gaia system. Each Gaia node provides a specialized API service that encapsulates a unique combination of - a specialized and fine-tuned LLM (e.g., an LLM that excels in answering questions about the Rust programming language) - a domain-specific knowledge base (e.g., knowledge about the WasmEdge project) - an inference app that manages the context and history of conversations (e.g., RAG and MemGPT prompt injection) compute resources required to run the LLM app as an API service (e.g., a Nvidia GPU or a Mac M3 device) The Gaia node API service is fully compatible with the OpenAI JSON spec, and hence each Gaia node can function as an alternative backend for OpenAI-based frontends or agents. ## Gaia Protocol The Gaia protocol connects and incentivizes Gaia nodes and domains to form a coherent network of web services for AI agents. It provides a mechanism to discover, connect to, and pay for Gaia node services through a decentralized marketplace. It also incentivizes domains to manage node agents through a staking program. Furthermore, the Gaia protocol connects model creators (i.e., people who have skills to finetune models) and knowledge providers to node operators through a marketplace. ![Gaia Architecture Details](/img/docs/gaia-protocol.png) --- ## 👋 Welcome to Gaia # 👋 Welcome to Gaia Gaia is a decentralized computing infrastructure that enables everyone to create, deploy, scale, and monetize their own AI agents that reflect their styles, values, knowledge, and expertise. It allows individuals and businesses to create AI agents. Each Gaia node provides: * a web-based chatbot UI [Chat with a Gaia node](https://rustcoder.gaia.domains/chatbot-ui/index.html) that is an expert on the Rust programming language. * an OpenAI compatible API. [See how](/agent-integrations/intro) to use a Gaia node as a drop-in OpenAI replacement in your favorite AI agent app. 100% of today's AI agents are applications in the OpenAI ecosystem. With our API approach, Gaia is an alternative to OpenAI. Each Gaia node has the ability to be customized with a fine-tuned model supplemented by domain knowledge which eliminates the generic responses many have come to expect. For example, a Gaia node for a financial analyst agent can write SQL code to query SEC 10K filings to respond to user questions. Similar Gaia nodes are organized into Gaia domains, to provide stable services by load balancing across the nodes. Gaia domains have public-facing URLs and promote agent services to their communities. When a user or an agent app sends an API request to the domain's API endpoint URL, the domain is responsible for directing the request to a node that is ready. ## Next steps: ### Users If you are an end user of AI agent applications, you can: * [Find a list of interesting Gaia nodes you can chat with on the web, or access via API](/nodes). * [Use a Gaia node as the backend AI engine for your favorite AI agent apps](/agent-integrations). ### Node operators If you are interested in running Gaia nodes, you can * [Get started with a Gaia node](/getting-started/quick-start). * [Customize the Gaia node with a finetuned model and custom knowledge base](/getting-started/customize). * [Join the Gaia Protocol](/getting-started/register) ### Domain operators If you are a Gaia Domain Name owner, you can * [Launch your domain](/domain-guide/quick-start) ### Creators If you are a creator or knowledge worker interested in creating your own AI agent service, you can: * [Create your own knowledge base](/knowledge-bases). * [Finetune a model to "speak" like you](/tutorial/llamacpp). --- ## Knowledge base from source / summary pairs # Knowledge base from source / summary pairs In this section, we will discuss how to create a vector collection snapshot for optimal retrieval of long-form text documents. The approach is to create two columns of text in a CSV file. * The first column is the long-form source text from the knowledge document, such as a book chapter or a markdown section. * The long-form source text is difficult to search. The second column is a "search-friendly" summary of the source text. It could contain a list of questions that can be answered by the first column source text. We will create a vector snapshot where each vector is computed from the summary text (second column), but the retrieved source text for that vector is from the first column. The snapshot file can then be [loaded by a Gaia node as its knowledge base](../../../getting-started/customize#select-a-knowledge-base). > We have a simple Python script to build properly formatted CSV files from a set of articles or chapters. [See how it works](https://github.com/GaiaNet-AI/embedding-tools/tree/main/csv_embed#create-a-csv-file). ## Prerequisites Install the WasmEdge Runtime, the cross-platform LLM runtime. ``` curl -sSf https://raw.githubusercontent.com/WasmEdge/WasmEdge/master/utils/install_v2.sh | bash -s ``` Download an embedding model. ``` curl -LO https://huggingface.co/gaianet/Nomic-embed-text-v1.5-Embedding-GGUF/resolve/main/nomic-embed-text-v1.5.f16.gguf ``` The embedding model is a special kind of LLM that turns sentences into vectors. The vectors can then be stored in a vector database and searched later. When the sentences are from a body of text that represents a knowledge domain, that vector database becomes our RAG knowledge base. ## Start a vector database By default, we use Qdrant as the vector database. You can start a Qdrant instance by [starting a Gaia node with a knowledge snapshot](../../../getting-started/quick-start). :::note Or, you can start a Qdrant server using Docker. The following command starts it in the background. ``` mkdir qdrant_storage mkdir qdrant_snapshots nohup docker run -d -p 6333:6333 -p 6334:6334 \ -v $(pwd)/qdrant_storage:/qdrant/storage:z \ -v $(pwd)/qdrant_snapshots:/qdrant/snapshots:z \ qdrant/qdrant ``` ::: ## Create the vector collection snapshot Delete the default collection if it exists. ``` curl -X DELETE 'http://localhost:6333/collections/default' ``` Create a new collection called default. Notice that it is 768 dimensions. That is the output vector size of the embedding model `nomic-embed-text-v1.5`. If you are using a different embedding model, you should use a dimension that fits the model. ``` curl -X PUT 'http://localhost:6333/collections/default' \ -H 'Content-Type: application/json' \ --data-raw '{ "vectors": { "size": 768, "distance": "Cosine", "on_disk": true } }' ``` Download a program to create embeddings from the CSV file. ``` curl -LO https://github.com/GaiaNet-AI/embedding-tools/raw/main/csv_embed/csv_embed.wasm ``` You can check out the [Rust source code](https://github.com/GaiaNet-AI/embedding-tools/tree/main/csv_embed) here and modify it if you need to use a different CSV layout. Next, you can run the program by passing a collection name, vector dimension, and the CSV document. The `--ctx_size` option matches the embedding model's context window size, which in this case is 8192 tokens allowing it to process long sections of text. Make sure that Qdrant is running on your local machine. The model is preloaded under the name embedding. The wasm app then uses the embedding model to create the 768-dimension vectors from `paris.csv` and saves them into the default collection. ``` curl -LO https://huggingface.co/datasets/gaianet/paris/resolve/main/paris.csv wasmedge --dir .:. \ --nn-preload embedding:GGML:AUTO:nomic-embed-text-v1.5.f16.gguf \ csv_embed.wasm embedding default 768 paris.csv --ctx_size 8192 ``` ### Options You can pass the following options to the program. * Using `-c` or `--ctx_size` to specify the context size of the input. This defaults to 512. * Using `-m` or `--maximum_context_length` to specify a context length in the CLI argument. That is to truncate and warn for each text segment that goes above the context length. * Using `-s` or `--start_vector_id` to specify the start vector ID in the CLI argument. This will allow us to run this app multiple times on multiple documents on the same vector collection. Example: the above example but to append the London guide to the end of an existing collection starting from index 42. ``` wasmedge --dir .:. \ --nn-preload embedding:GGML:AUTO:nomic-embed-text-v1.5.f16.gguf \ csv_embed.wasm embedding default 768 london.csv -c 8192 -l 1 -s 42 ``` ## Create a vector snapshot You can create a snapshot of the collection, which can be shared and loaded into a different Qdrant database. You can find the snapshot file in the `qdrant_snapshots` directory, or the `~/gaianet/qdrant/snapshots` directory in the Gaia node. ``` curl -X POST 'http://localhost:6333/collections/default/snapshots' ``` We also recommend you to compress the snapshot file. ``` tar czvf my.snapshot.tar.gz my.snapshot ``` Finally, upload the `my.snapshot.tar.gz` file to Huggingface so that the [Gaia node can download and use it](../../../getting-started/customize#select-a-knowledge-base). ## Next steps * [Start](../../../getting-started/quick-start) a new Gaia node * [Customize](../../../getting-started/customize) the Gaia node --- ## Knowledge base from a URL # Knowledge base from a URL In this section, we will discuss how to create a vector collection snapshot from a Web URL. First, we will parse the URL to a structured markdown file. Then, we will follow the steps from [Knowledge base from a markdown file](../markdown/markdown.md) to create embedding for your URL. ## Parse the URL content to a markdown file Firecrawl can crawl and convert any website into LLM-ready markdown or structured data. It also supports crawling a URL and all accessible subpages. > To use Firecrawl, you need to sign up on [Firecrawl](https://firecrawl.dev/) and get an API key. First, install the dependencies. We are assuming that you already have Node.JS 20+ installed. ``` git clone https://github.com/JYC0413/firecrawl-integration.git cd firecrawl-integration npm install ``` Then, export the API key in the terminal. ``` export FIRECRAWL_KEY="your_api_key_here" ``` next, we can use the following command line to run the service. ``` node crawlWebToMd.js ``` After the application is running successfully, you will see the prompt appear on the Terminal. ![](/img/docs/firecrawl-01.png) You can type your URL in the terminal right now. Here we have two choices. * Multiple pages: input your link with `/` at the end, the program will crawl and convert the page and its subpages to one single markdown file. This way will cost lots of API token usage. * One single page: input your link without `/` at the end. the program will crawl and convert the current page to one single markdown file. The output markdown file will be located in this folder named `output.md`. ## Create embeddings from the markdown files Please follow the tutorial [Knowledge base from a markdown file](../markdown/markdown.md) to convert your markdown file to a snapshot of embeddings that can be imported into a GaiaNet node. --- ## Knowledge base from a markdown file # Knowledge base from a markdown file In this section, we will discuss how to create a vector collection snapshot from a markdown file. The snapshot file can then be [loaded by a Gaia node as its knowledge base](../../../getting-started/customize#select-a-knowledge-base). The markdown file is segmented into multiple sections by headings. [See an example](https://huggingface.co/datasets/gaianet/paris/raw/main/paris.md). Each section is turned into a vector, and when retrieved, added to the prompt context for the LLM. ## Prerequisites Install the WasmEdge Runtime, the cross-platform LLM runtime. ``` curl -sSf https://raw.githubusercontent.com/WasmEdge/WasmEdge/master/utils/install_v2.sh | bash -s ``` Download an embedding model. ``` curl -LO https://huggingface.co/gaianet/Nomic-embed-text-v1.5-Embedding-GGUF/resolve/main/nomic-embed-text-v1.5.f16.gguf ``` The embedding model is a special kind of LLM that turns sentences into vectors. The vectors can then be stored in a vector database and searched later. When the sentences are from a body of text that represents a knowledge domain, that vector database becomes our RAG knowledge base. ## Start a vector database By default, we use Qdrant as the vector database. You can start a Qdrant instance by [starting a Gaia node with a knowledge snapshot](../../../getting-started/quick-start). :::note Or, you can start a Qdrant server using Docker. The following command starts it in the background. ``` mkdir qdrant_storage mkdir qdrant_snapshots nohup docker run -d -p 6333:6333 -p 6334:6334 \ -v $(pwd)/qdrant_storage:/qdrant/storage:z \ -v $(pwd)/qdrant_snapshots:/qdrant/snapshots:z \ qdrant/qdrant ``` ::: ## Create the vector collection snapshot Delete the default collection if it exists. ``` curl -X DELETE 'http://localhost:6333/collections/default' ``` Create a new collection called default. Notice that it is 768 dimensions. That is the output vector size of the embedding model `nomic-embed-text-v1.5`. If you are using a different embedding model, you should use a dimension that fits the model. ``` curl -X PUT 'http://localhost:6333/collections/default' \ -H 'Content-Type: application/json' \ --data-raw '{ "vectors": { "size": 768, "distance": "Cosine", "on_disk": true } }' ``` Download a program to segment the markdown document and create embeddings. ``` curl -LO https://github.com/GaiaNet-AI/embedding-tools/raw/main/markdown_embed/markdown_embed.wasm ``` It chunks the document based on markdown sections. You can check out the [Rust source code](https://github.com/GaiaNet-AI/embedding-tools/tree/main/markdown_embed) here and modify it if you need to use a different chunking strategy. Next, you can run the program by passing a collection name, vector dimension, and the source document. You can pass in the desired markdown heading level for chunking using the `--heading_level` option. The `--ctx_size` option matches the embedding model's context window size, which in this case is 8192 tokens allowing it to process long sections of text. Make sure that Qdrant is running on your local machine. The model is preloaded under the name embedding. The wasm app then uses the embedding model to create the 768-dimension vectors from `paris.md` and saves them into the default collection. ``` curl -LO https://huggingface.co/datasets/gaianet/paris/raw/main/paris.md wasmedge --dir .:. \ --nn-preload embedding:GGML:AUTO:nomic-embed-text-v1.5.f16.gguf \ markdown_embed.wasm embedding default 768 paris.md --heading_level 1 --ctx_size 8192 ``` ### Options You can pass the following options to the program. * Using `-c` or `--ctx_size` to specify the context size of the input. This defaults to 512. * Using `-l` or `--heading_level` to specify the markdown heading level for each vector. This defaults to 1. * Using `-m` or `--maximum_context_length` to specify a context length in the CLI argument. That is to truncate and warn for each text segment that goes above the context length. * Using `-s` or `--start_vector_id` to specify the start vector ID in the CLI argument. This will allow us to run this app multiple times on multiple documents on the same vector collection. Example: the above example but to append the London guide to the end of an existing collection starting from index 42. ``` wasmedge --dir .:. \ --nn-preload embedding:GGML:AUTO:nomic-embed-text-v1.5.f16.gguf \ markdown_embed.wasm embedding default 768 london.md -c 8192 -l 1 -s 42 ``` ## Create a vector snapshot You can create a snapshot of the collection, which can be shared and loaded into a different Qdrant database. You can find the snapshot file in the `qdrant_snapshots` directory, or the `~/gaianet/qdrant/snapshots` directory in the Gaia node. ``` curl -X POST 'http://localhost:6333/collections/default/snapshots' ``` We also recommend you to compress the snapshot file. ``` tar czvf my.snapshot.tar.gz my.snapshot ``` Finally, upload the `my.snapshot.tar.gz` file to Huggingface so that the [Gaia node can download and use it](../../../getting-started/customize#select-a-knowledge-base). ## Video Guide ## Next steps * [Start](../../../getting-started/quick-start) a new Gaia node * [Customize](../../../getting-started/customize) the Gaia node --- ## Knowledge base from a PDF file # Knowledge base from a PDF file In this section, we will discuss how to create a vector collection snapshot from a PDF file. First, we will parse the unstructured PDF file to a structured markdown file. Then, we will follow the steps from [Knowledge base from a markdown file](../markdown/markdown.md) to create embedding for your PDF files. ## Tools to convert a PDF file to a markdown file ### Tool #1: LlamaParse LlamaParse is a tool to parse files for optimal RAG. You will need a LlamaCloud key from https://cloud.llamaindex.ai. First, install the dependencies. we are assuming that you already have Node.JS 20+ installed. ``` git clone https://github.com/alabulei1/llamaparse-integration.git cd llamaparse-integration npm install llamaindex npm install dotenv ``` Then, edit the `.env` file to set up the PDF file path and LlamaCloud Key. In this case, you don't need to care about the LLM-related settings. After that, run the following command line to parse your pdf into a markdown file. ``` npx tsx transMd.ts ``` The output markdown file will be located in this folder named `output.md` by default. You can change the path in the `.env `file. ### Tool #2: GPTPDF GPTPDF is an open-source tool using GPT-4o to parse PDF into markdown. You will need an OpenAI key here. First, install the gptpdf software. ``` pip install gptpdf ``` Then, enter the Python environment. ``` python ``` Next, use the following command to parse your pdf. ``` from gptpdf import parse_pdf api_key = 'Your OpenAI API Key' content, image_paths = parse_pdf(Your_Pdf_Path, api_key=api_key) print(content) ``` The output markdown files called `output.md` will be located in your root directory. ## Create embeddings from the markdown files Please follow the tutorial [Knowledge base from a markdown file](../markdown/markdown.md) to convert your markdown file to a snapshot of embeddings that can be imported into a GaiaNet node. --- ## Knowledge base from a plain text file # Knowledge base from a plain text file In this section, we will discuss how to create a vector collection snapshot from a plain text file. The snapshot file can then be [loaded by a Gaia node as its knowledge base](../../../getting-started/customize#select-a-knowledge-base). The text file is segmented into multiple chunks by blank lines. [See an example](https://huggingface.co/datasets/gaianet/paris/raw/main/paris_chunks.txt). Each chunk is turned into a vector, and when retrieved, added to the prompt context for the LLM. ## Prerequisites Install the WasmEdge Runtime, the cross-platform LLM runtime. ``` curl -sSf https://raw.githubusercontent.com/WasmEdge/WasmEdge/master/utils/install_v2.sh | bash -s ``` Download an embedding model. ``` curl -LO https://huggingface.co/gaianet/Nomic-embed-text-v1.5-Embedding-GGUF/resolve/main/nomic-embed-text-v1.5.f16.gguf ``` The embedding model is a special kind of LLM that turns sentences into vectors. The vectors can then be stored in a vector database and searched later. When the sentences are from a body of text that represents a knowledge domain, that vector database becomes our RAG knowledge base. ## Start a vector database By default, we use Qdrant as the vector database. You can start a Qdrant instance by [starting a Gaia node with a knowledge snapshot](../../../getting-started/quick-start). :::note Or, you can start a Qdrant server using Docker. The following command starts it in the background. ``` mkdir qdrant_storage mkdir qdrant_snapshots nohup docker run -d -p 6333:6333 -p 6334:6334 \ -v $(pwd)/qdrant_storage:/qdrant/storage:z \ -v $(pwd)/qdrant_snapshots:/qdrant/snapshots:z \ qdrant/qdrant ``` ::: ## Create the vector collection snapshot Delete the default collection if it exists. ``` curl -X DELETE 'http://localhost:6333/collections/default' ``` Create a new collection called default. Notice that it is 768 dimensions. That is the output vector size of the embedding model `nomic-embed-text-v1.5`. If you are using a different embedding model, you should use a dimension that fits the model. ``` curl -X PUT 'http://localhost:6333/collections/default' \ -H 'Content-Type: application/json' \ --data-raw '{ "vectors": { "size": 768, "distance": "Cosine", "on_disk": true } }' ``` Download a program to chunk a document and create embeddings. ``` curl -LO https://github.com/GaiaNet-AI/embedding-tools/raw/main/paragraph_embed/paragraph_embed.wasm ``` It chunks the document based on empty lines. So, you MUST prepare your source document this way -- segment the document into sections of around 200 words with empty lines. You can check out the [Rust source code here](https://github.com/GaiaNet-AI/embedding-tools/tree/main/paragraph_embed) and modify it if you need to use a different chunking strategy. > The `paragraph_embed.wasm` program would NOT break up code listings even if there are empty lines with in the listing. Next, you can run the program by passing a collection name, vector dimension, and the source document. Make sure that Qdrant is running on your local machine. The model is preloaded under the name embedding. The wasm app then uses the embedding model to create the 768-dimension vectors from [paris_chunks.txt](https://huggingface.co/datasets/gaianet/paris/raw/main/paris_chunks.txt) and saves them into the default collection. ``` curl -LO https://huggingface.co/datasets/gaianet/paris/raw/main/paris_chunks.txt wasmedge --dir .:. \ --nn-preload embedding:GGML:AUTO:nomic-embed-text-v1.5.f16.gguf \ paragraph_embed.wasm embedding default 768 paris_chunks.txt -c 8192 ``` ### Options You can pass the following options to the program. * Using `-m` or `--maximum_context_length` to specify a context length in the CLI argument. That is to truncate and warn for each text segment that goes above the context length. * Using `-s` or `--start_vector_id` to specify the start vector ID in the CLI argument. This will allow us to run this app multiple times on multiple documents on the same vector collection. * Using `-c` or `--ctx_size` to specify the context size of the input. This defaults to 512. Example: the above example but to append the London guide to the end of an existing collection starting from index 42. ``` wasmedge --dir .:. \ --nn-preload embedding:GGML:AUTO:nomic-embed-text-v1.5.f16.gguf \ paragraph_embed.wasm embedding default 768 london.txt -c 8192 -s 42 ``` ## Create a vector snapshot You can create a snapshot of the collection, which can be shared and loaded into a different Qdrant database. You can find the snapshot file in the `qdrant_snapshots` directory, or the `~/gaianet/qdrant/snapshots` directory in the Gaia node. ``` curl -X POST 'http://localhost:6333/collections/default/snapshots' ``` We also recommend you to compress the snapshot file. ``` tar czvf my.snapshot.tar.gz my.snapshot ``` Finally, upload the `my.snapshot.tar.gz` file to Huggingface so that the [Gaia node can download and use it](../../../getting-started/customize#select-a-knowledge-base). ## Video Guide ## Next steps * [Start](../../../getting-started/quick-start) a new Gaia node * [Customize](../../../getting-started/customize) the Gaia node --- ## How to chunk and segment your knowledge base # How to chunk and segment your knowledge base First, copy unformatted text into a txt file. Then follow the two rules to chunk your content, i.e. putting similar content together. - Each title and related content are a chunk. There are no blank lines in one chunk. - Use a blank line to recognize different chunks. After that, save it as a txt file. For example, below is your source. ![The input knowledge in a text file](/img/docs/web_tool_input.png) After formatting, it will look like the following. ``` What is a blockchain? A blockchain is a distributed, cryptographically-secure database structure that allows network participants to establish a trusted and immutable record of transactional data without the need for intermediaries. A blockchain can execute a variety of functions beyond transaction settlement, such as smart contracts. Smart contracts are digital agreements that are embedded in code and can have limitless formats and conditions. Blockchains have proven themselves as superior solutions for securely coordinating data, but they are capable of much more, including tokenization, incentive design, attack-resistance, and reducing counterparty risk. The very first blockchain was the Bitcoin blockchain, which was itself a culmination of over a century of advancements in cryptography and database technology. What is blockchain software? Blockchain software is like any other software. The first of its kind was Bitcoin, which was released as open source software, making it available to anyone to use or change. There are a wide variety of efforts across the blockchain ecosystem to improve upon Bitcoin's original software. Ethereum has its own open source blockchain software. Some blockchain software is proprietary and not available to the public. ``` Once you have the chunked knowledge base in either `.txt` or `.md` formats, you can follow our tutorials to convert into a Qdrant vector database snapshot: - [Text to embeddings](../text/text.md) - [Markdown to embeddings](../markdown/markdown.md) --- ## Intro to knowledge bases # Intro to knowledge bases Knowledge bases are crucial components in Gaia's context for creating effective AI agents. They serve as structured repositories of information that agents can draw upon to answer questions, make decisions, and perform tasks. In Gaia's ecosystem, knowledge bases are important for several reasons: - **Customization:** They allow developers to tailor their AI agents with specific domain knowledge, making them more useful for particular applications or industries. - **Improved Accuracy:** By providing agents with curated, relevant information, knowledge bases help ensure more accurate and contextually appropriate responses. - **Efficiency:** Agents can quickly access pre-organized information rather than having to process large amounts of unstructured data in real-time. - **Scalability:** As the knowledge base grows, the agent's capabilities can expand without requiring retraining of the entire model. To leverage knowledge bases in creating AI agents using Gaia: - **Create a Knowledge Base:** Gaia supports multiple formats for creating knowledge bases, including plain text files, markdown files, PDF documents, CSV files with source/summary pairs, and even web URLs. This flexibility allows you to use the most convenient format for your data. - **Organize Information:** Structure your knowledge base logically, focusing on the key information your agent will need to access. - **Import to Gaia:** Use Gaia's tools to import your knowledge base into the system. The platform likely provides interfaces or APIs for this purpose. - **Configure Agent:** When setting up your AI agent in Gaia, specify which knowledge base(s) it should use as a reference. - **Test and Refine:** Interact with your agent, observe its performance, and iteratively refine your knowledge base to improve outcomes. - **Update Regularly:** Keep your knowledge base current by adding new information or updating existing entries as needed. ### Knowledge Base Format Gaia employs a specific format for knowledge bases to optimize performance and integration with its AI agents. While the exact details may vary, here are some key aspects: - **Structured Data:** Knowledge bases in Gaia are likely organized in a structured format that allows for efficient indexing and retrieval. - **Metadata:** Each entry in the knowledge base may include metadata such as timestamps, categories, or tags to enhance searchability. ### Vector Database Integration Gaia utilizes Qdrant as its default vector database to enhance the capabilities of its knowledge bases: - **Qdrant Integration:** [Qdrant](https://qdrant.tech/) is a high-performance, open-source vector database optimized for similarity search and machine learning applications. - **Embedding Storage:** Text from knowledge bases is converted into high-dimensional vectors (embeddings) and efficiently stored in Qdrant. - **Fast Similarity Search:** Qdrant enables rapid similarity searches, allowing AI agents to quickly find the most relevant information from the knowledge base. - **Scalability:** Qdrant is designed to handle large-scale vector search problems, supporting extensive knowledge bases with millions of entries. - **Flexible Filtering:** Qdrant allows for complex filtering during search operations, enabling more precise information retrieval based on metadata or other attributes. - **CRUD Operations:** Gaia can leverage Qdrant's support for real-time updates, allowing for dynamic modifications to the knowledge base without significant performance impact. - **Cloud-Native Architecture:** Qdrant's design aligns well with cloud environments, facilitating easy scaling and deployment of Gaia's knowledge base system. By using Qdrant, Gaia provides a robust and efficient vector search capability, enabling AI agents to quickly access and utilize relevant information from large and complex knowledge bases. ### Leveraging the System To make the most of Gaia's knowledge base and vector database system: - **Optimize Content:** Structure your knowledge base entries to align with Gaia's preferred format for best performance. - **Use Appropriate Queries:** When designing your AI agent, formulate queries that take advantage of the vector search capabilities. - **Regular Updates:** Keep your knowledge base current, as the vector database can be used to update embeddings for new or changed content. - **Performance Monitoring:** Pay attention to retrieval speed and accuracy, adjusting your knowledge base structure if needed. By understanding and effectively using Gaia's specific knowledge base format and vector database system, you can create more powerful and efficient AI agents that leverage information retrieval capabilities to their fullest extent. --- ## 📄 Litepaper # 📄 Litepaper # GaiaNet: GenAI Agent Network ## Abstract Specialized, finetuned and RAG-enhanced open-source Large Language Models are key elements in emerging AI agent applications. However, those agent apps also present unique challenges to the traditional cloud computing and SaaS infrastructure, including new requirements for application portability, virtualization, security isolation, costs, data privacy, and ownership. GaiaNet is a decentralized computing infrastructure that enables everyone to create, deploy, scale, and monetize their own AI agents that reflect their styles, values, knowledge, and expertise. A GaiaNet node consists of a high-performance and cross-platform application runtime, a finetuned LLM, a knowledge embedding model, a vector database, a prompt manager, an open API server, and a plugin system for calling external tools and functions using LLM outputs. It can be deployed by any knowledge worker as a digital twin and offered as a web API service. A new class of tradeable assets and a marketplace could be created from individualized knowledge bases and components. Similar GaiaNet nodes are organized into GaiaNet domains, which offer trusted and reliable AI agent services to the public. The GaiaNet node and domains are governed by the GaiaNet DAO (Decentralized Autonomous Organization). Through Purpose Bound Money smart contracts, the GaiaNet network is a decentralized marketplace for AI agent services. ## Introduction The emergence of ChatGPT and Large Language Model (LLM) has revolutionized how humans produce and consume knowledge. Within a year, AI-native applications have evolved from chatbots to copilots, to agents. > AI agents would increasingly evolve from supportive tools (akin to Copilots) to autonomous entities capable of completing tasks independently. — Dr. Andrew Ng at Sequoia Capital AI Ascent 2024 Summit Agents are software applications that can complete tasks on its own autonomously like a human. The agent can understand the task, plan the steps to complete the task, execute all the steps, handle errors and exceptions, and deliver the results. While a powerful LLM could act as the “brain” for the agent, we need to connect to external data sources (eyes and ears), domain-specific knowledge base and prompts (skills), context stores (memory), and external tools (hands). For agent tasks, we often need to customize the LLM itself * to reduce hallucinations in a specific domain. * to generate responses in a specific format (e.g., a JSON schema). * to answer “politically incorrect” questions (e.g., to analyze CVE exploits for an agent in the security domain). * and to answer requests in a specific style (e.g., to mimic a person). ![What is a GaiaNet agent](/img/docs/gaianet_agent.png) Agents are complex software that require significant amount of engineering and resources. Today, most agents are close-source and hosted on SaaS-based LLMs. Popular examples include GPTs and Microsoft/GitHub copilots on OpenAI LLMs, and Duet on Google’s Gemini LLMs. However, as we discussed, a key requirement for agents is to customize and adapt its underlying LLM and software stack for domain-specific tasks — an area where centralized SaaS perform very poorly. For example, with ChatGPT, every small task must be handled by a very large model. It is also enormously expensive to fine-tune or modify any ChatGPT models. The one-size-fits-all LLMs are detrimental to the agent use case in capabilities, alignment, and cost structure. Furthermore, the SaaS hosted LLMs lack privacy controls on how the agent’s private knowledge might be used and shared. Because of these shortcomings, it is difficult for individual knowledge workers to create and monetize agents for his or her own domain and tasks on SaaS platforms like OpenAI, Google, Anthropic, Microsoft and AWS. In this paper, we propose a decentralized software platform and protocol network for AI agents for everyone. Specifically, our goals are two-folds. **Goal #1:** Empower individuals to incorporate his/her private knowledge and expertise into personal LLM agent apps. Those apps aim to perform knowledge tasks and use tools just as the individual would, but also reflect the individual’s style and values. **Goal #2:** Enable individuals to provide and scale their LLM agents as services, and get compensated for their expertise and work. > GaiaNet is “YouTube for knowledge and skills.” ## Open-source and decentralization As of April 2024, there are over 6000 open-source LLMs published on Hugging face. Compared with close-source LLMs, such as GPT-4, open-source LLMs offer advantages in privacy, cost, and systematic bias. Even with general QA performance, open-source LLMs are closing the gap with close-source counterparties quickly. ![Open vs close source LLMs](/img/docs/closed_vs_open.jpg) For AI agent use cases, it has been demonstrated that smaller but task-specific LLMs often outperform larger general models. However, it is difficult for individuals and businesses to deploy and orchestrate multiple finetuned LLMs on their own heterogeneous GPU infrastructure. The complex software stack for agents, as well as the complex interaction with external tools, are fragile and error-prone. Furthermore, LLM agents have entirely different scaling characteristics than past application servers. LLM is extremely computationally intensive. A LLM agent server can typically only serve one user at a time, and it often blocks for seconds at a time. The scaling need is no longer to handle many async requests on a single server, but to load balance among many discrete servers on the internet scale. The GaiaNet project provides a cross-platform and highly efficient SDK and runtime for finetuned open-source LLMs with proprietary knowledge bases, customized prompts, structured responses, and external tools for function calling. A GaiaNet node can be started in minutes on any personal, cloud, or edge device. It can then offer services through an incentivized web3 network. ## GaiaNet node The basic operational unit in the GaiaNet network is a node. A GaiaNet node is a streamlined software stack that allows any technically competent person to run an AI agent of his own. The software stack on the GaiaNet node consists of the following 7 key components. ![GaiaNet node architecture](/img/docs/gaianet_node.png) **1 Application runtime.** GaiaNet applications run in a lightweight, secure and high-performance sandbox called WasmEdge. As an open-source project managed by the Linux Foundation and CNCF, WasmEdge runtime works seamlessly with leading cloud native tools such as Docker, containerd, CRI-O, Podman and Kubernetes. It is also the virtual machine of choice by leading public blockchains to securely and efficiently execute on-chain and off-chain smart contracts. WasmEdge is a high-performance and cross-platform runtime. It can run AI models on almost all CPUs, GPUs, and AI accelerators at native speed, making it an ideal runtime for decentralized AI agents. **2 Finetuned LLM.** The GaiaNet node supports almost all open-source LLMs, multimodal models (eg Large Vision Models or LVMs), text-to-image models (eg Stable Diffusion) and text-to-video models. That includes all finetuned models using personal or proprietary data. The node owner can finetune open-source models using a wide variety of tools. For example, the node owner can finetune an LLM using personal chat histories so that the finetuned LLM can mimic his own speaking style. He can also finetune an LLM to focus it on a specific knowledge domain to reduce hallucinations and improve answer quality for questions in that domain. A finetuned LLM can guarantee to output JSON text that matches a pre-determined schema for use with external tools. Besides LLMs, the node owner could finetune Stable Diffusion models with her own photos to generate images that look like her. **3 Embedding model.** The GaiaNet node needs to manage a body of public or proprietary knowledge for the AI agent. It is a key feature that enables the agent to specialize and outperform much larger models in a specific domain. The embedding models are specially trained LLMs that turns input sentences into a vector representation, instead of generating completions. Since the embedding models are trained from LLMs, they can “embed” the “meaning” of the sentences into the vectors so that similar sentences are located close together in the high dimensional space occupied by those vectors. With the embedding model, a GaiaNet node can ingest a body of text, images, PDFs, web links, audio and video files, and generate a collection of embedding vectors based on their contents. The embedding model also turns user questions and conversations into vectors, which allows the GaiaNet node to quickly identify contents in its knowledge base that are relevant to the current conversation. **4 Vector database.** The embedding vectors that form GaiaNet node’s knowledge base are stored on the node itself for optimal performance and maximum privacy. The GaiaNet node includes a Qdrant vector database. **5 Custom prompts.** Besides finetuning and knowledge arguments, the easiest way to customize an LLM for new applications is simply to prompt it. Like humans, LLMs are remarkable one-shot learners. You can simply give it an example of how to accomplish a task, and it will learn and do similar tasks on its own. Prompt engineering is a practical field to research and develop such prompts. Furthermore, effective prompts could be highly dependent on the model in use. A prompt that works well for a large model, such as Mixtral 8x22b, is probably not going to work well for a small model like Mistral 7b. The GaiaNet node can support several different prompts that are dynamically chosen and used in applications. For example, * The `system_prompt` is a general introduction to the agent task the node is supposed to perform. It often contains a persona to help the LLM respond with the right tone. For example, the `system_prompt` for a college teaching assistant could be: “You are a teaching assistant for UC Berkeley’s computer science 101 class. Please explain concepts and answer questions in detail. Do not answer any question that is not related to math or computer science.” * The `rag_prompt` is a prefix prompt to be dynamically inserted in front of knowledge base search results in an RAG chat. It could be something like this: “Please answer the question based on facts and opinions in the context below. Do not make anything that is not in the context. ---------” The LLM community has developed many useful prompts for different application use cases. GaiaNet node allows you to easily manage and experiment with them. Through the our developer SDK, GaiaNet owners and operators could customize the logic of dynamic prompt generation in their own way. For example, a GaiaNet node could perform a Google search for any user question, and add the search results into the prompt as context. **6 Function calls and tool use.** The LLM not only is great at generating human language, but also excels at generating machine instructions. Through finetuning and prompt engineering, we could get some LLMs to consistently generate structured JSON objects or computer code in many language tasks, such as summarizing and extracting key elements from a paragraph of text. The GaiaNet node allows you to specify the output format of the generated text. You can give it a grammar specification file to enforce that responses will always conform to a pre-defined JSON schema. Once the LLM returns a structured JSON response, the agent typically need to pass the JSON to a tool that performs the task and comes back with an answer. For example, the user question might be. ``` What is the weather like in Singapore? ``` The LLM generates the following JSON response. ``` {"tool":"get_current_weather", "location":"Singapore","unit":"celsius"} ``` The GaiaNet node must know what is the tool associated with get_current_weather and then invoke it. GaiaNet node owners and operators can configure any number of external tools by mapping a tool name with a web service endpoint. In the above example, the get_current_weather tool might be associated with a web service that takes this JSON data. The GaiaNet node sends the JSON to the web service endpoint via HTTPS POST and receives an answer. ``` 42 ``` It then optionally feeds the answer to the LLM to generate a human language answer. ``` The current weather in Singapore is 42C. ``` Through the GaiaNet node SDK, developers are not limited to using web services. They can write plugins to process LLM responses locally on the node. For example, the LLM might return Python code, which can be executed locally in a sandbox and for the GaiaNet node to perform a complex operation. **7 The API server.** All GaiaNet nodes must have the same API for questions and answers. That allows front-end applications to work with, and potentially be load-balanced to any GaiaNet node. We choose to support the OpenAI API specification, which enables GaiaNet nodes to become drop-in replacements for OpenAI API endpoints for a large ecosystem of applications. The API server runs securely and cross-platform on the WasmEdge runtime. It ties together all the other components in the GaiaNet node. It receives user requests, generates an embedding from the request, searches the vector database, adds search results to the prompt context, generates an LLM response, and then optionally uses the response to perform function calling. The API server also provides a web-based chatbot UI for users to chat with the RAG-enhanced finetuned LLM on the node. ## GaiaNet network While each GaiaNet node is already a powerful AI agent capable of answering complex questions and performing actions, individual nodes are not suitable for providing public services. There are several important reasons. * For the public consumers and users, it is very hard to judge the trustworthiness of individual GaiaNet nodes. Harmful misinformation could be spread by malicious node operators. * For GaiaNet node owners and operators, there is no economic incentive to provide such services to the public, which could be very costly to run. * The AI agent servers have very different scaling characteristics than traditional internet application servers. When the agent is processing a user request, it typically takes up all the computing resources on the hardware. Instead of using software to scale concurrent users on a single server, the challenge of GaiaNet is to scale to many different identical nodes for a large application. Those challenges have given rise to the GaiaNet domain, which forms the basis of the GaiaNet web3 network. A GaiaNet domain is a collection of GaiaNet nodes available under a single Internet domain name. The domain operator decides which GaiaNet nodes can be registered under the domain and makes the node services available to the public. For example, a GaiaNet domain might be a Computer Science teaching assistant for UC Berkeley. The domain could provide services through `https://cs101.gaianet.berkeley.edu`. The domain operator needs to do the following. * Verify and admit individual nodes to be registered under the domain. Those nodes must all meet requirements, such as the LLM, knowledge base, and prompts, set by the domain operator to ensure service quality. The node registration on a domain could be done via a whitelist or blacklist. It is up to the domain operator. * Monitor each node’s performance at real time and remove inactive ones. * Promotes the “teaching assistant” chatbot apps to the target audience. * Set the price for the API services. * Load balance between active nodes. * Getting paid by users. * Pay nodes for their services. ![GaiaNet network architecture](/img/docs/gaianet_eco.png) Each GaiaNet node has an unique node ID in the form of an ETH address. The private key associated with the ETH address is stored on the node. Once a node is successfully registered with a domain, it is entitled to receive payments from both service revenue and network awards from the domain. The domain could send payments directly to the node's ETH address. Or, the domain could provide a mechanism for a node operator to register multiple nodes under a single Metamask address, such as signing a challenge phrase using the node private keys. In that case, the node operator will receive aggregated payments in his Metamask account for all associated nodes. Each GaiaNet domain has an associated smart contract that is used for escrow payments. It is similar to OpenAI’s credit payment model, where users purchase credits first, and then consume them over time. When the user pays into the smart contract, an access token will be automatically issued to him. He uses this token to make API calls to the domain, which is then load-balanced to random nodes in the domain. As the user consumes those services, his fund in the contract depletes and the access token stops working if he no longer has any balance. The pricing and payment for the API service are determined by the domain operator. It is typically denominated in USD stable coins. The domain operator pays a share of the revenue to node operators who provided the services. The GaiaNet network is a decentralized marketplace of agent services. > The funds locked in GaiaNet domain contracts are for a single purpose of consuming API services. It is called Purpose Bound Money. A key aspect of the GaiaNet protocol is that the domain operators are “trust providers” in the ecosystem of decentralized nodes. The protocol network is designed to incentivize the trust of the operators through tokenomics designs such as mining and staking. GaiaNet nodes, domains, users, and developers form a DAO to grow the network and benefit all contributors. ## GaiaNet token The GaiaNet token is a utility token designed to facilitate transactions, support governance, and foster trust in the network. It serves three primary purposes. * As a DAO governance token, holders can participate in setting the rules of the network. * As a staking token, holders vouch for domain operators’ trustworthiness. Stakers get a cut from the domain operator’s service revenue. But they could also be slashed if the domain operator misbehave, such as spreading misinformation or providing unreliable services. * As a payment token, the GaiaNet token could be deposited into the domain’s escrow contract and be used to pay for services over time. The payment utility of the GaiaNet token is designed to balance the network supply and demand. The value of the GaiaNet token asset is determined at the time when it enters or leaves the escrow smart contract based on real-time exchange rates. Service consumers could lock in savings from the potential appreciation of the token. For example, if a user deposits $100 worth of GaiaNet tokens into the contract, and when the domain and nodes get paid, the token value has gone up to $110, he would have received $110 worth of agent services. Conversely, if the token price drops, the service providers (domains and nodes) now have an opportunity to “mine” the tokens on the cheap. If the $100 initial tokens is only worth $90 now, service providers will get more tokens for each unit of electricity and compute they provide. That incentivizes more nodes to join the network and speculate on a later rise in token value. > An exercise: OpenAI is projected to reach $5 billion in ARR in 2024. Assume that most enterprise customers pay quarterly, that is $1.25 billion of circulation market cap in addition to OpenAI’s current enterprise value if they were to issue a payment token. The overall AI services market size is projected to reach $2 trillion in a few years. That translates to $500 billion market cap for a payment utility token alone. ## Component marketplace for AI assets GaiaNet is a developer platform to create your agent services. We provide tools for you to do these. * Tools to generate finetuning datasets and perform finetuning on CPU and GPU machines. * Tools to ingest documents and create vector embeddings for the knowledge base. * Rust-based SDK to dynamically generate and manage prompts. * Rust-based SDK to extend the agent’s capability for invoking tools and software on the node. For developers who do not wish to operate nodes, we are building a marketplace for * finetuned models * knowledge bases and datasets * function-calling plugins All those components are blockchain-based assets represented by NFTs. A node operator could purchase NFTs for the components it wishes to use, and share service revenue with the component developers. That enables diverse and cashflow-generating assets to be issued from the GaiaNet ecosystem. ## Conclusion GaiaNet provides open-source tools for individuals and teams to create agent services using their proprietary knowledge and skills. Developers could create finetuned LLMs, knowledge collections, and plugins for the agent, and issue assets based on those components. The GaiaNet protocol makes those nodes discoverable and accessible through GaiaNet domains. --- ## 🛠️ For Node Operators # 🛠️ For Node Operators Node Operators focus on setting up, running, and maintaining Gaia nodes without requiring coding expertise. Running a Gaia node allows you to participate in the decentralized AI network and potentially earn rewards. The process is straightforward and can be completed in just a few minutes. Before you begin, ensure your system meets these [minimum requirements](./getting-started/system-requirements) Here you'll find everything you need to get started with running and managing your Gaia node. ## Getting Started Learn how to get started with your first GaiaNet node. [Learn More →](./getting-started/what-is-a-node.md) ## Node Setup & Authentication Step-by-step guide for node installation, setup, and security. - [Setting up your own node](./getting-started/quick-start) - [Authentication guide](./getting-started/authentication) ## Node Operations Learn about day-to-day operations and customization. - [Using your Gaia Node](./getting-started/mynode) - [Customizing Your Gaia Node](./getting-started/customize) ## Network & Protocol Join the network and start earning rewards. - [Gaia CLI options](./getting-started/cli-options) - [Joining the Gaia Protocol](./getting-started/register) ## System Requirements & Troubleshooting Make sure your system is ready and get help when needed. - [System requirements](./getting-started/system-requirements) - [Troubleshooting guide](./getting-started/troubleshooting) ## Advanced Deployment Options Learn about different deployment options. - [Install or Uninstall the CLI](./getting-started/install) - [Install multiple nodes](./getting-started/advanced-deployment-options/multiple) - [Run a local-only node](./getting-started/advanced-deployment-options/local) - [Protect the server process](./getting-started/advanced-deployment-options/protect) --- ## 🌐 Public Gaia Domain # 🌐 Public Gaia Domain Each Gaia node provides a web-based chatbot UI and an OpenAI compatible web service. Here are some popular nodes. Please refer to the [agent apps](../agent-integrations/intro) section to see how to use the Gaia API in your favorite agent frameworks or apps. Gaia nodes are organized into Gaia domains to provide public services. Each domain has a single API endpoint that load-balances across multiple nodes to ensure service availability. Below are some Gaia domains we offer developers for free. **However, you must apply for a developer account to access these domains**. ## Public Gaia domains ### Voice-to-text: Whisper v2 large This domain runs Whisper v2 Large agent nodes for voice to text transcription and translation. Replace OpenAI configuration in [your app](../agent-integrations/intro) with the following. |Config option | Value | |-----|--------| | API endpoint URL | https://whisper.gaia.domains/v1 | | Model Name | whisper | | API key | [Get your API Key here](../getting-started/authentication/authentication.md) | ### Text-to-image: Realistic vision Coming soon! ### Text-to-voice: GPT-SoVITS Coming soon! ## LLM domains ### Qwen 7b The Qwen3 0.6b LLM is great for non-coding tasks such as translation. [Chat with it](https://0x0c82e25e1f996fa3d227d23e83cef721ee42ff69.gaia.domains/chatbot-ui/index.html) or use it from another app. Replace OpenAI configuration in [your app](../agent-integrations/intro) with the following. |Config option | Value | |-----|--------| | API endpoint URL | https://0x0c82e25e1f996fa3d227d23e83cef721ee42ff69.gaia.domains/v1 | | Model Name (for LLM) | qwen3 0.6b | | Model Name (for Text embedding) | nomic-embed | | API key | [Get your API Key here](../getting-started/authentication/authentication.md) | ### GPT OSS 20b The GPT OSS 20b LLM is the first open sourced LLM by OpenAI. It is very capable with a thinking proces. [Chat with it](https://0xfa1fc68813d687215be75fba4fffb60f184590bc.gaia.domains/chatbot-ui/index.html) or use it from another app. Replace OpenAI configuration in [your app](../agent-integrations/intro) with the following. |Config option | Value | |-----|--------| | API endpoint URL | https://0xfa1fc68813d687215be75fba4fffb60f184590bc.gaia.domains/v1 | | Model Name (for LLM) | qwen72b | | Model Name (for Text embedding) | nomic-embed | | API key | [Get your API Key here](../getting-started/authentication/authentication.md) | ### Mistral The Mistral 3.1 24b LLM is a top open source LLM. It is very capable but could also be slow. It is capable of tool / function calling. [Learn more](../tutorial/tool-call/tool-call.md) how to use tool call models in your agent app. [Chat with it](https://0x3b70c030a2baaa866f6ba6c03fde87706812d920.gaia.domains/chatbot-ui/index.html) or use it from another app. Replace OpenAI configuration in [your app](../agent-integrations/intro) with the following. |Config option | Value | |-----|--------| | API endpoint URL | https://0x3b70c030a2baaa866f6ba6c03fde87706812d920.gaia.domains/v1 | | Model Name (for LLM) | qwen72b | | Model Name (for Text embedding) | nomic-embed | | API key | [Get your API Key here](../getting-started/authentication/authentication.md) | --- ## Token Generator on Base Chain # Token Generator on Base Chain Generate and deploy meme tokens automatically using Gaia's AI Agent for creative naming and tokenomics! Built for Base Sepolia testnet, but easily adaptable to other networks. :::info The complete source code and detailed instructions for setting up and running this example can be found in the [Gaia Cookbook repository](https://github.com/GaiaNet-AI/gaia-cookbook/tree/main/js/gaia-meme-coin-generator). ::: ## What does it do? This tool automates the entire process of creating a meme token: - Uses Gaia's AI node (https://llama3b.gaia.domains/v1) to generate creative token names and themes - Automatically creates tokenomics with safe defaults - Deploys a secure, limit-based token contract - Handles all the complex blockchain interactions ## Features - 🤖 AI-powered name generation - 📊 Automatic tokenomics calculation - 🔒 Built-in safety features: - Maximum transaction limits - Maximum wallet limits - Excludable addresses for CEX/DEX - 📝 Full deployment records - 🔍 Automatic contract verification links ## Quick Start 1. Clone the repo: ```bash git clone https://github.com/harishkotra/gaia-meme-coin-generator cd gaia-meme-coin-generator ``` 2. Install dependencies: ```bash npm install ``` 3. Create a `.env` file: ```env GAIA_URL=your_gaia_url GAIA_MODEL=your_model_name BASE_NODE_URL=your_base_sepolia_rpc_url PRIVATE_KEY=your_wallet_private_key ``` 4. Run the generator: ```bash npm start ``` ## Use Cases 1. **Token Launch Platforms** - Quickly generate and deploy tokens for new projects - Automate the token creation process - Create themed token collections 2. **Community Tokens** - Deploy tokens for online communities - Create fan tokens with custom themes - Launch social tokens quickly 3. **Educational Purposes** - Learn about token deployment - Understand tokenomics - Practice with testnet deployments 4. **Marketing Campaigns** - Create themed tokens for promotions - Launch event-specific tokens - Generate buzz with creative names ## Security Features Our generated tokens come with built-in protections: - Anti-whale mechanics - Transaction limits - Wallet limits - Owner controls for DEX/CEX exclusions ## Contract Details The deployed token includes: - ERC20 standard compliance - Ownership controls - Limit system for transactions - Wallet amount restrictions - Exclusion system for special addresses ## Disclaimer This is a tool for testnet experimentation. Always: - Test thoroughly before mainnet use - Review generated contracts - Verify tokenomics match your needs - Consider regulatory compliance ### Example Output ``` Initializing agent... Initializing MemeCoin Agent... Deployer address: 0xb.............................cD45 Successfully connected to Gaia node Creating meme coin... Generating coin details... Generated coin details: { "name": "AstroPup", "symbol": "APU", "description": "Pioneering a new era of canine space travel" } Generating tokenomics... Raw tokenomics response: {"total_supply": 500000000, "initial_liquidity_percent": 75, "transaction_limit_percent": 1, "max_wallet_percent": 2} Generated tokenomics: { "total_supply": 500000000, "initial_liquidity_percent": 75, "transaction_limit_percent": 1, "max_wallet_percent": 2 } Deploying contract... Compiling contract... Deploying contract... Waiting for deployment confirmation... Meme Coin Created Successfully! ================================================== Name: AstroPup Symbol: APU Description: Pioneering a new era of canine space travel Contract Address: 0x41B20e82DBFDe8557363Ca0B7C232C7288EA3Aae Transaction Hash: 0x86c0fd39683b8950d543d647294d8c0a2761cab8b7ab50f12cfd908a595e337e Block Number: 18332172 Tokenomics: total_supply: 500000000 initial_liquidity_percent: 75 transaction_limit_percent: 1 max_wallet_percent: 2 Deployment details saved to deployment_details.json Verify your contract on Base Sepolia Explorer: https://sepolia.basescan.org/address/0x41B20e82DBFDe8557363Ca0B7C232C7288EA3Aae ``` :::info The complete source code and detailed instructions for setting up and running this example can be found in the [Gaia Cookbook repository](https://github.com/GaiaNet-AI/gaia-cookbook/tree/main/js/gaia-meme-coin-generator). ::: --- ## Token Generator on Celo # Token Generator on Celo This project demonstrates how to auto create and deploy ERC20 tokens on the Celo blockchain using ContractKit. It includes an AI-powered name generator (using Gaia's Public Node running Llama 3.2 8B parameter model) and automatic deployment scripts. :::info The complete source code and detailed instructions for setting up and running this example can be found in the [Gaia Cookbook repository](https://github.com/GaiaNet-AI/gaia-cookbook/tree/main/js/celo-token-agent). ::: ## Prerequisites - Node.js v20.x + - A wallet with some test tokens (we'll help you get these!) ## Getting Started 1. Clone this repository ```bash git clone https://github.com/harishkotra/celo-token-agent cd celo-token-agent ``` 2. Install dependencies ```bash npm install ``` 3. Create a `.env` file: ```env PRIVATE_KEY=your_private_key GAIA_API_KEY=your_gaia_api_key ``` ## Getting Test Tokens Before deploying your token, you'll need some test tokens: 1. Visit the [Celo Faucet](https://faucet.celo.org/alfajores) 2. Connect your wallet or paste your account address 3. Click to receive: - A-CELO (for gas fees) - cUSD (optional) The faucet will send you test tokens that you can use for deployment. ## How It Works ### ContractKit Integration This project uses Celo's ContractKit to interact with the blockchain. Here's what each part does: - `tokenGenerator.js`: Creates unique token names using AI (using Gaia's Public Node running Llama 3.2 8B) or falls back to random generation - `tokenDeployer.js`: Handles the smart contract deployment using ContractKit - `MemeToken.sol`: The ERC20 token contract built with OpenZeppelin Key ContractKit features we use: ```javascript // Initialize ContractKit const web3 = new Web3(rpcUrl); const kit = newKitFromWeb3(web3); // Add your account kit.addAccount(privateKey); // Deploy using A-CELO for gas const tx = await deploy.send({ from: defaultAccount, gas }); ``` ### Smart Contract Our token is a standard ERC20 token with: - Custom name and symbol - Initial supply set at deployment - Standard transfer and approval functions ## Deployment 1. Compile the contract: ```bash npx hardhat compile ``` 2. Deploy your token: ```bash node deploy.js ``` The script will: 1. Generate a token name 2. Check your balance 3. Deploy the contract 4. Provide you with the contract address and transaction details ## Understanding the Code The project uses three main components: 1. **Token Generation** - Generates creative token names - Uses AI with fallback to random generation - Configures initial token supply 2. **Contract Deployment** - Uses ContractKit to interact with Celo - Handles gas estimation and transaction monitoring - Provides deployment status updates 3. **Smart Contract** - Standard ERC20 implementation - Built with OpenZeppelin for security - Deployable to Celo's Alfajores testnet ### Example Responses ``` AI generated token: { name: "Satoshi's Catnip", symbol: 'SCP' } Reading artifacts from: /Users/shk/experiments/onchainkit-gaia/artifacts/contracts/MemeToken.sol/MemeToken.json Deploying from account: 0xbDe71618Ef4Da437b0406DA72C16E80b08d6cD45 Account balance: A-CELO: 10.353296994614 A-CELO Sending deployment transaction... Transaction sent! Hash: 0xd5b17d8ce38ddf50ca7366cf658b3d24d6d9a1d0e3bce6e50b870bd50e961792 Deployment confirmed in block: 35794429 Token deployed successfully! { name: "Satoshi's Catnip", symbol: 'SCP', address: '0x0563109c80733Ea484F86b653262ecA50b8a06d6', transactionHash: '0xd5b17d8ce38ddf50ca7366cf658b3d24d6d9a1d0e3bce6e50b870bd50e961792', explorer: 'https://alfajores.celoscan.io/address/0x0563109c80733Ea484F86b653262ecA50b8a06d6' } ``` ``` AI generated token: { name: 'LolToken', symbol: 'LOL' } Reading artifacts from: /Users/shk/experiments/onchainkit-gaia/artifacts/contracts/MemeToken.sol/MemeToken.json Deploying from account: 0xbDe71618Ef4Da437b0406DA72C16E80b08d6cD45 Account balance: A-CELO: 10.337778442114 A-CELO Sending deployment transaction... Transaction sent! Hash: 0xfe83c066173362374b1c6a420c2fdc37f7fd4f923bd3d8a3b94e384988cbde13 Deployment confirmed in block: 35797227 Token deployed successfully! { name: 'LolToken', symbol: 'LOL', address: '0x47442330f26B58D7C1b7D13ed20fE1244aE58Dbe', transactionHash: '0xfe83c066173362374b1c6a420c2fdc37f7fd4f923bd3d8a3b94e384988cbde13', explorer: 'https://alfajores.celoscan.io/address/0x47442330f26B58D7C1b7D13ed20fE1244aE58Dbe' } ``` ## Need Help? - For Celo-specific questions: [Celo Docs](https://docs.celo.org/) - For ContractKit details: [ContractKit Documentation](https://docs.celo.org/developer/contractkit) - Read a detailed blog article about this agent [here](https://hackmd.io/@harishatgaia/celo-token-agent). - [Gaia's Public Nodes](https://docs.gaianet.ai/user-guide/nodes/) :::info The complete source code and detailed instructions for setting up and running this example can be found in the [Gaia Cookbook repository](https://github.com/GaiaNet-AI/gaia-cookbook/tree/main/js/celo-token-agent). ::: --- ## CDP's Agentkit Starter Template # CDP's Agentkit Starter Template ## Overview This template shows an onchain agent powered by Coinbase's AgentKit with the Next.js framework on the frontend and LangGraph for the agent's setup. The agent is designed for AI-driven on-chain capabilities. AgentKit handles these interactions by using a Gaia node for Large Language Model (LLM) inferencing. ![cdp-image](../cdp/cdp-image.png) ## Features - **AI-Driven on-chain interactions**: Leverages AgentKit to enable AI agents to perform actions on blockchain networks. - **Bootstrapped**: Built as a Next.js project with a LangGraph in the server, bootstrapped with `npm create onchain-agent@latest`. - **Configurable LLM**: Supports integration with LLMs hosted on Gaia nodes, specifically configured for tool use inferencing (e.g., [Llama-3-Groq-8B-Tool](https://github.com/GaiaNet-AI/node-configs/tree/main/llama-3-groq-8b-tool), [Llama-3.3-70B-Instruct-Q5_K_M](https://llama70b.gaia.domains/v1/info)). - **Wallet management**: Integrates with a `SmartWalletProvider` for blockchain interactions, with persistent wallet data management. - **Extensible actions**: Utilizes various Action Providers (e.g., WETH, Pyth, ERC20, CDP API, Wallet actions) to define the agent's capabilities. - **Chat interface**: Provides a user-friendly chat interface for interacting with the agent. - **Streamed responses**: Agent responses are streamed for a more interactive user experience. - **Memory**: Incorporates memory for conversations using `MemorySaver` from LangGraph. ## Getting started ### Prerequisites 1. Node.js 18 or later is installed 2. Confirm that npm 9 or later is installed Check your Node.js and npm versions: ```bash node --version # Should be 18+ npm --version # Should be 9+ ``` 3. You can use either a public Gaia node for example: [https://llama70b.gaia.domains/v1](https://llama70b.gaia.domains/v1) or [run the node locally](https://github.com/GaiaNet-AI/node-configs/tree/main/llama-3-groq-8b-tool). ### Creating a new project You can use the CLI to bootsrap a new Agenkit project with the command below: ```bash npm create onchain-agent@latest ``` Follow the instructions on the CLI to setup your project and choose the Smart wallet (default) option for the setup. You can also make a framework choice among: - LangChain - Vercel AI SDK There is also an Model Context Protocol (MCP) option, but, in this guide, we cover using the LangChain option. ### Configure secrets and values Rename the `.env.example` to `.env` and ensure that you have the below values: ```bash CDP_API_KEY_NAME= CDP_API_KEY_PRIVATE_KEY= # Optional NETWORK_ID=base-sepolia ``` To obtain the values for `CDP_API_KEY_NAME` and `CDP_API_KEY_PRIVATE_KEY` head over to [CDP portal](https://portal.cdp.coinbase.com/projects/api-keys) to create a new API key. Copy the API key name and private key values from the modal that appears. The `NETWORK_ID` can stay as `base-sepolia` and you can explore the possible [network options](https://docs.cdp.coinbase.com/api-reference/networks#network-identifiers) as well. ## Project structure ```bash └── onchain-agent/ ├── README.md ├── next-env.d.ts ├── next.config.js ├── package.json ├── postcss.config.mjs ├── tailwind.config.ts ├── tsconfig.json ├── wallet_data.txt ├── .eslintrc.json ├── .npmignore ├── .yarnrc.yml └── app/ ├── globals.css ├── layout.tsx ├── page.tsx ├── api/ │ └── agent/ │ ├── create-agent.ts │ ├── prepare-agentkit.ts │ └── route.ts ├── hooks/ │ └── useAgent.ts └── types/ └── api.ts ``` ## Gaia Integration :::info A local Gaia node does not require an API key. You will need a [Gaia API key](https://www.gaianet.ai/setting/gaia-api-keys) to use public nodes. ::: The LLM inferencing is offloaded to a Gaia node: - The LLM is configured in `app/api/agent/create-agent.ts`. - The project uses `ChatOpenAI` from `@langchain/openai` to connect to the Gaia node. - The specific model configured is "Llama-3-Groq-8B-Tool". - The Gaia node endpoint is set via the `baseURL` in the `ChatOpenAI` configuration: For example with a local node running on a machine: ```ts const llm = new ChatOpenAI({ model: "Llama-3-Groq-8B-Tool", configuration: { baseURL: "https://YOUR_NODE_ID.gaia.domains/v1", // Gaia node URL apiKey: "gaia", // API key for the Gaia node (if required) }, }); ``` ## Run template The below command runs the template: ```bash npm run dev ``` With the template running, there a few example prompts you can use to test the agent: - "What is your wallet address?" - "What is your wallet balance? Check and confirm." - "Share your wallet details including every relevant information." ## Documentation For further information and advanced topics, refer to the following official documentation: - **AgentKit Documentation**: - GitHub: https://github.com/coinbase/agentkit - CDP Docs: https://docs.cdp.coinbase.com/agentkit/docs/welcome - **GaiaNet Documentation**: - Gaia Node Setup (example Llama-3-Groq-8B-Tool): https://github.com/GaiaNet-AI/node-configs/tree/main/llama-3-groq-8b-tool - **Coinbase Developer Platform (CDP)**: https://docs.cdp.coinbase.com/ --- ## DynamicRAG # DynamicRAG DynamicRAG is a React-based web application that implements a dynamic Retrieval-Augmented Generation (RAG) system. It allows users to input text, process it into embeddings, and perform question-answering tasks using a combination of vector search and Large Language Models. :::info The complete source code and detailed instructions for setting up and running this example can be found in the [Gaia Cookbook repository](https://github.com/GaiaNet-AI/gaia-cookbook/tree/main/js/dynamic-rag-gaia). ::: ## Screenshots #### Input text based Dynamic RAG ![screencapture-localhost-3000-2024-12-24-00_55_11](https://github.com/user-attachments/assets/b9ccb3b8-590a-4080-9297-03ff79ea1d86) #### Github Repo to Dynamic RAG ## Features - Real-time text processing and chunking - Dynamic vector database creation using Qdrant - Batch processing of large text inputs - Integration with a local Gaia node - GitHub repository analysis using GitIngest - Progress tracking and error handling - Clean, responsive UI using Tailwind CSS and shadcn/ui components ## Prerequisites - Node.js (v16 or higher) - Python (for GitIngest installation) - A running Qdrant instance (local or remote) - A local LLM server through Gaia running on port 8080 (Tutorial: [https://docs.gaianet.ai/node-guide/quick-start](https://docs.gaianet.ai/node-guide/quick-start)) - The `nomic-embed` embedding model (auto-downloaded in `gaianet` folder when the Gaia CLI is installed) - The `llama 3.2` language model (auto-downloaded in `gaianet` folder when the Gaia CLI is installed. You can replace the model depending on your use-case) ## Setup 1. Clone the repository: ```bash git clone https://github.com/harishkotra/dynamic-rag-gaia cd dynamic-rag ``` 2. Install dependencies: ``` npm install ``` 3. Install GitIngest: ``` pip install gitingest ``` 4. Ensure your Qdrant server is running and accessible at http://localhost:6333 5. Ensure your Gaia node is running and accessible at http://localhost:8080 (Tutorial: [https://docs.gaianet.ai/node-guide/quick-start](https://docs.gaianet.ai/node-guide/quick-start)) 6. Start the development server: ``` npm run dev ``` ## Usage 1. Choose Input Mode: - Text Input: Paste your knowledge base text into the input textarea - GitHub Repository: Enter a GitHub repository URL to analyze its contents 2. Ask Questions: Enter your query in the question field. 3. Process: Click "Submit" to process your query. The system will: - For text input: Split the input text into manageable chunks - For GitHub repos: Fetch and process repository content using GitIngest - Create embeddings for each chunk - Store embeddings in a temporary Qdrant collection - Find relevant context using vector similarity search - Generate an answer using the local Gaia Node 4. View Results: The system will display the generated response based on the relevant context found. ## Technical Details ### Text Processing - Maximum chunk size: 2000 characters - Batch processing size: 3 chunks at a time - Chunks are created based on natural text boundaries (paragraphs and sentences) ### Vector Database - Uses Qdrant for vector storage and similarity search - Creates temporary collections for each query session - Automatically cleans up collections after use - Uses 768-dimensional vectors for embeddings ### API Integration - Compatible with OpenAI-style API endpoints - Supports both embeddings and chat completion endpoints - Uses the `nomic-embed` model for embeddings - Uses the `llama` model for text generation - Integrates with GitIngest for repository analysis - NextJS API routes for command-line tool integration ### Components - `DynamicRAG.js`: Main component implementing the RAG system - `app/api/gitingest/route.js`: API route for GitHub repository processing - Input handling and validation - Progress tracking and error display - Vector database management - LLM integration ### Error Handling The system includes comprehensive error handling for: - Text processing failures - GitHub repository fetch and analysis errors - Embedding creation errors - Vector database operations - LLM query failures - Network issues ### Dependencies - React - Tailwind CSS - shadcn/ui components - Lodash for utility functions - Various UI components (@/components/ui/*) :::info The complete source code and detailed instructions for setting up and running this example can be found in the [Gaia Cookbook repository](https://github.com/GaiaNet-AI/gaia-cookbook/tree/main/js/dynamic-rag-gaia). ::: --- ## Supavec Integration # Supavec Integration A powerful document question-answering system that combines Supavec's RAG capabilities with Gaia's language understanding. This system enables intelligent conversations with your documents through semantic search and natural language processing. ![image](https://github.com/user-attachments/assets/d3069ce5-a8eb-4cc5-90b0-1ec926ca9c55) ![image](https://github.com/user-attachments/assets/6c9075f2-784b-4da8-9938-15cbb792b81c) :::info The complete source code and detailed instructions for setting up and running this example can be found in the [Gaia Cookbook repository](https://github.com/GaiaNet-AI/gaia-cookbook/tree/main/js/gaia-supavec). ::: ## Project Structure The project is divided into two main parts: 1. **Backend**: Handles file uploads, text uploads, and communication with the Supavec and Gaia APIs. 2. **Frontend**: Provides a user interface for uploading files, listing uploaded files, and interacting with the chat interface. ## Key Components - Frontend Layer: React application with real-time updates and file management - Backend API: Express.js server handling request orchestration - Document Processing: Supavec API for document chunking and embedding - Language Model: Gaia API for contextual question answering - Data Flow: Bidirectional communication with optimized response streaming ### API Endpoints - `POST /api/upload`: Uploads a file to Supavec. - `POST /api/upload-text`: Uploads text content to Supavec. - `GET /api/files`: Retrieves a list of uploaded files. - `POST /api/search`: Searches embeddings based on a query and file IDs. - `POST /api/ask`: Asks a question about documents using Gaia. ## Getting Started ### Prerequisites - Node.js and npm installed on your machine. - Get your Supavec API key from the [dashboard](https://www.supavec.com/) - Run your own local node using [Gaia](https://docs.gaianet.ai/node-guide/quick-start/) ### Installation 1. Clone the repository: ```sh git clone https://github.com/your-username/gaia-supavec.git cd gaia-supavec ``` 2. Install backend dependencies: ```sh cd backend npm install ``` 3. Install frontend dependencies: ```sh cd ../frontend npm install ``` ### Running the Project 1. Start the backend server: ```sh cd backend npm start ```` 2. Start the frontend development server: ```sh cd ../frontend npm start ``` 3. Open your browser and navigate to `http://localhost:3000` to access the application. ### Usage 1. *Upload Documents:* Use the file upload interface to upload PDF or text files. 2. *List Files:* View the list of uploaded files. 3. *Ask Questions:* Select files and ask questions about their content using the chat interface. ## Resources - [Supavec](https://www.supavec.com/) team for the RAG infrastructure (@supavec on [github](https://github.com/taishikato/supavec)) - [Gaia](https://www.gaianet.ai/) for the simple infra to launch local LLMs or use Public nodes. :::info The complete source code and detailed instructions for setting up and running this example can be found in the [Gaia Cookbook repository](https://github.com/GaiaNet-AI/gaia-cookbook/tree/main/js/gaia-supavec). ::: --- ## MetaMask Delegation Toolkit Agent # MetaMask Delegation Toolkit Agent ## 🌟 Overview The MetaMask Gaia DTK Starter is a powerful Next.js template that seamlessly integrates Gaia's advanced capabilities with MetaMask's Develegation Toolkit (DTK). This project allows developers to build sophisticated blockchain applications with AI-enhanced functionalities while leveraging MetaMask's secure wallet infrastructure. ![Gaia Delegation Toolkit Gif](../gaia-supavec/gaia-dtk-2.gif) The starter includes a full ERC20 token creation system through an integrated factory contract, allowing users to deploy custom tokens directly through the application. Combined with AI-powered interactions, this creates a powerful platform for building next-generation decentralized applications. ## ✨ Features - **AI-Powered Interactions**: Utilize GaiaNet AI to create intelligent and responsive dApp experiences - **ERC20 Token Creation**: Create custom ERC20 tokens through the integrated factory contract - **Secure Blockchain Integration**: Connect with the Ethereum ecosystem through MetaMask's trusted wallet infrastructure - **Delegation Management**: Metamask's Delegation Toolkit for managing user-to-AI agent delegations - **Bundler Service Integration**: Pre-configured connection to bundler services for transaction handling - **Modern UI Components**: Ready-made UI components including chat interfaces, cards, and inputs - **Next.js App Router**: Built on Next.js 13+ with the new App Router architecture - **TypeScript Support**: Full TypeScript integration for type safety and better developer experience ## 🚀 Getting Started ### Prerequisites - Node.js (v16 or later) - pnpm package manager - MetaMask extension installed in your browser ### Installation 1. Clone the repository: ```bash git clone https://github.com/meowyx/metamask-gaia-starter.git ``` 2. Navigate to the project directory: ```bash cd metamask-gaia-starter ``` 3. Install dependencies using pnpm: ```bash pnpm install ``` 4. Create a `.env` file in the root directory with the following configuration: ``` # Factory contract configuration NEXT_PUBLIC_FACTORY_CONTRACT_ADDRESS=0x... NEXT_PUBLIC_CREATE_TOKEN_SELECTOR=0x... # Bundler service configuration NEXT_PUBLIC_BUNDLER_URL=https://api.pimlico.io/v2/137/rpc?apikey=YOUR_API_KEY NEXT_PUBLIC_CHAIN_ID=59141 # Infura and private key configuration INFURA_PROJECT_ID=your_infura_project_id PRIVATE_KEY=your_private_key # Delegation storage configuration NEXT_PUBLIC_DELEGATION_STORAGE_API_KEY=your_delegation_api_key NEXT_PUBLIC_DELEGATION_STORAGE_API_KEY_ID=your_delegation_api_key_id NEXT_PUBLIC_DELEGATION_STORAGE_ENVIRONMENT=development # Gaia AI configuration GAIA_MODEL_BASE_URL=your_gaia_model_url GAIA_API_KEY=your_gaia_api_key // not needed if you run your own node ``` 5. Start the development server: ```bash pnpm dev ``` 6. Open [http://localhost:3000](http://localhost:3000) in your browser to see the application running. ## 📖 Project Structure ``` ├── .next/ # Next.js build output ├── ai/ # AI-related utilities │ └── tools.ts # AI tools implementation ├── app/ # Next.js App Router │ ├── api/ # API routes │ ├── globals.css # Global styles │ ├── layout.tsx # Root layout component │ └── page.tsx # Home page component ├── components/ # Reusable UI components │ ├── ui/ # Basic UI components │ │ ├── badge.tsx # Badge component │ │ ├── button.tsx # Button component │ │ ├── card.tsx # Card component │ │ └── input.tsx # Input component │ ├── Chat.tsx # Chat interface component │ ├── DelegationManager.tsx # Delegation management component │ └── Message.tsx # Message component ├── lib/ # Utility functions and libraries ├── services/ # API services │ ├── account.ts # Account-related services │ ├── bundler.ts # Bundler service implementation │ └── utils.ts # Service utilities ├── public/ # Static assets │ ├── file.svg # File icon │ ├── globe.svg # Globe icon │ ├── next.svg # Next.js logo │ └── vercel.svg # Vercel logo ├── node_modules/ # Dependencies ├── .env # Environment variables ├── package.json # Project dependencies └── pnpm-lock.yaml # pnpm lock file ``` ## 🔧 Configuration ### ERC20 Factory Contract Setup This project integrates with the [ERC20 Factory Contract](https://github.com/meowyx/erc20-factory) to enable token creation capabilities. Follow these steps to set up the integration: 1. Clone and deploy the ERC20 Factory contract: ```bash git clone https://github.com/meowyx/erc20-factory cd erc20-factory npm install npx hardhat compile npx hardhat ignition deploy ignition/modules/tokenFactory.ts --network linea-testnet ``` 2. After deployment, update your `.env` file with the deployed contract address: ``` NEXT_PUBLIC_FACTORY_CONTRACT_ADDRESS=0x... # The deployed factory contract address NEXT_PUBLIC_CREATE_TOKEN_SELECTOR=0x... # The function selector for createToken ``` 3. Update the `constants.ts` file with the ERC20 Factory ABI: ```typescript // Add the ERC20 Factory ABI to your constants.ts file export const FACTORY_ABI = [ // ... ABI contents from the compiled contract { "inputs": [ {"internalType": "string", "name": "name", "type": "string"}, {"internalType": "string", "name": "symbol", "type": "string"}, {"internalType": "uint8", "name": "decimals", "type": "uint8"}, {"internalType": "uint256", "name": "initialSupply", "type": "uint256"} ], "name": "createToken", "outputs": [{"internalType": "address", "name": "", "type": "address"}], "stateMutability": "nonpayable", "type": "function" } // ... other ABI entries ]; ``` 4. The factory contract allows you to create new ERC20 tokens with custom parameters such as name, symbol, decimals, and initial supply. The ERC20 Factory project structure: ``` erc20-factory/ ├── contracts/ │ ├── BaseERC20Token.sol # Base token implementation │ └── ERC20Factory.sol # Factory for deploying tokens ├── test/ │ └── ERC20Factory.test.js # Test scripts ├── ignition/ │ └── modules/ │ └── tokenFactory.js # Deployment configuration ├── hardhat.config.js # Hardhat configuration └── package.json # Project dependencies ``` ### MetaMask Setup 1. Install the [MetaMask extension](https://metamask.io/) in your browser 2. Create or import a wallet 3. Connect your dApp using the provided hooks in the starter ### Gaia Integration 1. Sign up for an API key over [here](https://gaianet.ai) 2. Add your API key to the `.env` file under `GAIA_API_KEY` 3. Set the model base URL in the `.env` file under `GAIA_MODEL_BASE_URL` 4. Use the pre-configured AI tools in `ai/tools.ts` to interact with GaiaNet features ### Bundler Service Configuration 1. Get an API key from [Pimlico](https://pimlico.io/) or your preferred bundler service 2. Add the bundler URL to the `.env` file under `NEXT_PUBLIC_BUNDLER_URL` 3. Set the correct chain ID in the `.env` file under `NEXT_PUBLIC_CHAIN_ID` ### Delegation System Setup 1. Configure the delegation storage API keys in the `.env` file 2. Use the `DelegationManager.tsx` component to manage delegations between users and AI agents ## 📚 Documentation For more detailed information about the technologies used in this starter: - [Next.js Documentation](https://nextjs.org/docs) - [MetaMask Documentation](https://docs.metamask.io/) - [Gaia Documentation](https://docs.gaianet.ai/) - [pnpm Documentation](https://pnpm.io/documentation) ## 🙏 Acknowledgements - [MetaMask Delegation Toolkit](https://docs.gator.metamask.io/) for their Delegation Toolkit. - [Gaia](https://gaianet.ai/) for their AI platform - [Next.js](https://nextjs.org/) for the React framework - [Vercel AI SDK](https://sdk.vercel.ai/) for The AI Toolkit for TypeScript --- ## Natural Language Weather App # Natural Language Weather App This simple web application allows you to check the current weather or forecast for a location using natural language queries (e.g., "Forecast for Tokyo", "Is it raining in Seattle?"). It leverages the **Gaia Language Model** (via an OpenAI compatible endpoint) to understand your request and the **Nubila Weather API** to fetch the actual weather data. ![image](https://github.com/user-attachments/assets/ec83b1fc-5749-43de-ba9d-093bd315d54e) ![image](https://github.com/user-attachments/assets/246d6358-1d48-4c70-a7ed-1c051ea767b4) :::info The complete source code and detailed instructions for setting up and running this example can be found in the [Gaia Cookbook repository](https://github.com/GaiaNet-AI/gaia-cookbook/tree/main/js/gaia-nubila). ::: ## Features * Get current weather or forecasts using plain English. * Uses Gaia LLM to interpret the location and desired information (current/forecast) from your query. * Fetches detailed weather data from the Nubila API. * Displays the LLM's interpretation (location, coordinates, request type). * **Illustrates** the concept of an LLM "tool call" by showing which function and arguments the LLM conceptually decided to use. * Provides clickable example prompts for quick use. * Simple, clean UI built with vanilla JavaScript, Node.js, and CSS. ## How it Works 1. **User Query:** You enter a query like "What's the weather like in Berlin?" into the web interface. 2. **Backend Request:** The query is sent from your browser to the Node.js backend server. 3. **LLM Analysis (Gaia):** The backend sends your query to the Gaia API. Gaia analyzes it to: * Identify the location (e.g., "Berlin"). * Determine approximate geographical coordinates (latitude and longitude). * Understand if you want the 'current' weather or a 'forecast'. 4. **API Call (Nubila):** The backend uses the coordinates and the request type ('current' or 'forecast') identified by Gaia to make a request to the appropriate Nubila Weather API endpoint. 5. **Weather Data:** The Nubila API responds with the requested weather data. 6. **Frontend Display:** The backend sends both Gaia's analysis and Nubila's weather data back to your browser. The frontend then displays: * How Gaia interpreted your request. * A simulation of the "tool call" Gaia would make. * The formatted current weather or forecast details. ## Prerequisites * **Node.js and npm:** Download and install from [nodejs.org](https://nodejs.org/). * **Nubila API Key:** You need an API key from Nubila Weather API. [Sign up](https://nubila.ai/) or [log in](https://nubila.ai/) here to obtain one. ## Setup & Installation 1. **Clone the repository (or download the files):** ```bash git clone cd ``` (Replace `` and `` accordingly) 2. **Install dependencies:** ```bash npm install ``` This installs Express, node-fetch, dotenv, and cors. ## Configuration 1. **Create `.env` file:** In the root directory of the project, create a file named `.env`. 2. **Add API Key and Settings:** Open the `.env` file and add the following lines, replacing `YOUR_NUBILA_API_KEY_HERE` with your actual Nubila API key: ```dotenv NUBILA_API_KEY=YOUR_NUBILA_API_KEY_HERE GAIA_API_ENDPOINT=https://llama70b.gaia.domains/v1/chat/completions GAIA_API_KEY=your-gaia-api-key PORT=3000 ``` * `NUBILA_API_KEY`: Your secret key for the Nubila API. * `GAIA_API_ENDPOINT`: The endpoint for the Gaia LLM. * `GAIA_API_KEY`: Get your Gaia API key by following this tutorial: https://docs.gaianet.ai/getting-started/authentication * `PORT`: The port the local server will run on (default is 3000). 3. **Important:** The `.env` file contains sensitive information (your API key). Ensure it is listed in your `.gitignore` file (it should be by default if you cloned) so you don't accidentally commit it to version control. ## Running the Application 1. **Start the server:** ```bash node server.js ``` 2. **Open the app:** Open your web browser and navigate to: `http://localhost:3000` (or `http://localhost:YOUR_PORT` if you changed the `PORT` in `.env`). 3. Enter your weather query or click one of the suggestion buttons! ## Technology Stack * **Backend:** Node.js, Express.js * **Frontend:** Vanilla JavaScript (ES6+), HTML5, CSS3 * **APIs:** * Gaia API (via OpenAI compatible endpoint) for Natural Language Understanding * [Nubila Weather API](https://nubila.ai/) for weather data :::info The complete source code and detailed instructions for setting up and running this example can be found in the [Gaia Cookbook repository](https://github.com/GaiaNet-AI/gaia-cookbook/tree/main/js/gaia-nubila). ::: --- ## PDF Question-Answering with Gaia and Qdrant # PDF Question-Answering with Gaia and Qdrant Gaia PDF RAG is a Retrieval-Augmented Generation (RAG) application that allows users to ask questions about PDF documents using a local Gaia node and Qdrant vector database. It combines the power of local LLMs with efficient vector search to provide accurate, context-aware answers. ![pdf-rag-image-1](./pdf-rag-1.png) ![pdf-rag-image-2](./pdf-rag-2.png) ![pdf-rag-image-3](./pdf-rag-3.png) ![pdf-rag-image-4](./pdf-rag-4.png) ![pdf-rag-image-5](./pdf-rag-5.png) :::info The complete source code and detailed instructions for setting up and running this example can be found in the [Gaia Cookbook repository](https://github.com/GaiaNet-AI/gaia-cookbook/tree/main/python/gaia-pdf-rag). ::: **What You Can Learn and Build:** By exploring and adapting this example, you can learn how to: * **Process and Chunk PDF Documents:** Understand the steps involved in breaking down PDF files into manageable segments for effective retrieval. * **Implement Semantic Search with Qdrant:** See how to leverage a vector database for efficient and context-aware searching of your document chunks. * **Integrate Local LLMs via Gaia:** Learn how to connect to your local Gaia node to utilize the power of locally hosted language models for question answering. * **Enhance Relevance with Cross-Encoder Reranking:** Discover techniques to refine search results and improve the accuracy of retrieved information. * **Provide a Seamless User Experience with Streaming Responses:** Implement real-time feedback for users as the answer is generated. * **Ensure Trustworthiness with Smart Source Citation:** Learn how to provide clear references to the source documents for each answer. * **Mitigate Hallucinations with Relevance Filtering:** Explore strategies to filter out irrelevant information and reduce the likelihood of inaccurate responses. * **Build a Customizable PDF Q&A Bot:** This example provides a solid foundation for you to tailor and deploy your own question-answering system for specific PDF documents or collections. **Key Features Demonstrated:** * 📑 **PDF Document Processing and Chunking:** Efficiently handles the extraction and segmentation of content from PDF files. * 🔍 **Semantic Search using Qdrant:** Leverages vector embeddings for intelligent retrieval of relevant document parts. * 🤖 **Local LLM Integration through Gaia node:** Connects to your locally running LLM for generating answers based on retrieved context. * ↗️ **Cross-encoder reranking for improved relevance:** Optimizes search results by applying a more sophisticated ranking model. * 💨 **Streaming responses for better UX:** Provides a more interactive and responsive user experience. * 🎯 **Smart source citation:** Clearly indicates the source documents used to generate each answer. * ⚡ **Relevance filtering to prevent hallucinations:** Enhances the reliability of answers by filtering out less relevant information. **Getting Started:** ## Prerequisites Before running Gaia RAG, ensure you have: 1. A local Gaia node running (Check this link to learn how to run your own local LLM: [https://docs.gaianet.ai/node-guide/quick-start](https://docs.gaianet.ai/node-guide/quick-starthttps://docs.gaianet.ai/node-guide/quick-start)) 2. Qdrant server running 3. Python 3.8+ 4. Required system libraries for PDF processing ## Installation 1. Clone the repository: ```bash git clone https://github.com/harishkotra/gaia-pdf-rag.git cd gaia-pdf-rag ``` 2. Create a virtual environment: ```bash python -m venv venv source venv/bin/activate # On Windows use: venv\Scripts\activate ``` 3. Install dependencies: ```bash pip install -r requirements.txt ``` ## Setting Up Components ### 1. Gaia Node Start your local Gaia node: ```bash gaianet init gaianet start ``` ### 2. Qdrant Server Start Qdrant using Docker: ```bash docker run -d -p 6333:6333 -p 6334:6334 \ -v $(pwd)/qdrant_storage:/qdrant/storage \ qdrant/qdrant ``` ## Running the Application 1. Make sure both Gaia node and Qdrant are running 2. Start the Streamlit app: ```bash streamlit run app.py ``` 3. Open your browser at `http://localhost:8501` ## Usage 1. Upload a PDF document using the sidebar 2. Click "Process Document" to index it 3. Ask questions in the main input field 4. View answers and relevant source documents ## Configuration You can modify the following parameters in `app.py`: - `GAIA_NODE_URL`: URL of your local Gaia node - `QDRANT_HOST`: Qdrant server host - `QDRANT_PORT`: Qdrant server port - `VECTOR_SIZE`: Embedding dimension size - `COLLECTION_NAME`: Name for vector database collection ## Project Structure ``` gaia-pdf-rag/ ├── app.py # Main Streamlit application ├── requirements.txt # Python dependencies ├── .gitignore # Gitignore file ├── README.md ``` :::info The complete source code and detailed instructions for setting up and running this example can be found in the [Gaia Cookbook repository](https://github.com/GaiaNet-AI/gaia-cookbook/tree/main/python/gaia-pdf-rag). ::: --- ## StoryWeaver AI # StoryWeaver AI StoryWeaver AI is your creative co-pilot, helping you transform fledgling ideas into fully-formed stories. Generate unique characters, sculpt immersive worlds, and weave compelling plots with our intelligent storycrafting assistant. Built with Next.js, Tailwind CSS, shadcn/ui, and powered by Gaia's LLM API. ![image](https://github.com/user-attachments/assets/7a32a9ad-9e5a-496e-a2e0-e8676995feac) :::info The complete source code and detailed instructions for setting up and running this example can be found in the [Gaia Cookbook repository](https://github.com/GaiaNet-AI/gaia-cookbook/tree/main/js/story-weaver). ::: ## Features * **Idea to Story:** Input your core story idea, genre, desired length, and optional details like protagonist, conflict, world vibe, and tone. * **AI-Powered Generation:** Leverages a Large Language Model (via Gaia's OpenAI-compatible API) to craft unique stories. * **Markdown Support:** Displays generated stories with rich text formatting. * **Download as Image:** Save your favorite stories as PNG images. * **Social Sharing:** Quickly share links to your app (users can attach their downloaded story image) on X (Twitter), LinkedIn, and Facebook. * **Sleek UI:** Modern and responsive interface built with shadcn/ui and Tailwind CSS. ## Tech Stack * **Framework:** [Next.js](https://nextjs.org/) (App Router) * **Language:** [TypeScript](https://www.typescriptlang.org/) * **Styling:** [Tailwind CSS](https://tailwindcss.com/) * **UI Components:** [shadcn/ui](https://ui.shadcn.com/) * **Form Management:** [React Hook Form](https://react-hook-form.com/) & [Zod](https://zod.dev/) * **LLM Integration:** `openai` npm package configured for [Gaia's API](https://docs.gaianet.ai) * **Markdown Rendering:** `react-markdown` * **HTML to Image:** `dom-to-image-more` * **Icons:** `lucide-react` ## Getting Started These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. ### Prerequisites * [Node.js](https://nodejs.org/) (v18.x or later recommended) * [npm](https://www.npmjs.com/) or [yarn](https://yarnpkg.com/) * A Gaia API Key ([Get one here](https://docs.gaianet.ai/getting-started/authentication)) ### Setup & Installation 1. **Clone the repository:** ```bash git clone https://github.com/harishkotra/story-weaver-ai.git cd story-weaver-ai ``` 2. **Install dependencies:** ```bash npm install # or yarn install ``` 3. **Set up Environment Variables:** Create a `.env.local` file in the root of the project. This file is ignored by Git and is used for local environment configuration. ```bash cp .env.example .env.local ``` Now, open `.env.local` and add your API key: ```env # .env.local # Required: Your API Key for the Gaia LLM service (or other OpenAI-compatible API) GAIA_API_KEY="your_actual_gaia_api_key_here" # Optional: If Gaia changes their endpoint or you use a different compatible service # Default is 'https://llama70b.gaia.domains/v1' if not set GAIA_API_ENDPOINT="https://your-custom-openai-compatible-endpoint/v1" GAIA_API_MODEL="llama70b" ``` **Important:** Replace `"your_actual_gaia_api_key_here"` with your real API key. ([Get one here](https://docs.gaianet.ai/getting-started/authentication)) 4. **Initialize shadcn/ui (if you need to add more components):** While the project is set up, if you intend to add more shadcn/ui components later, you might need to run init (though it should be configured already): ```bash npx shadcn@latest init ``` Follow the prompts, accepting the defaults or aligning with existing project configuration (`src/app/globals.css`, `tailwind.config.ts`, aliases `@/components` and `@/lib/utils`). ### Running the Application Locally 1. **Start the development server:** ```bash npm run dev # or yarn dev ``` 2. **Open your browser:** Navigate to `http://localhost:3000`. You should now see the StoryWeaver AI application running! :::info The complete source code and detailed instructions for setting up and running this example can be found in the [Gaia Cookbook repository](https://github.com/GaiaNet-AI/gaia-cookbook/tree/main/js/story-weaver). ::: --- ## Zerion Integration # Zerion Integration A powerful AI-driven crypto assistant powered by [Gaia AI](https://docs.gaianet.ai/intro) and [Zerion API](https://developers.zerion.io/reference/intro/getting-started). This project showcases how AI can interact with Web3 APIs to provide real-time portfolio insights. :::info The complete source code and detailed instructions for setting up and running this example can be found in the [Gaia Cookbook repository](https://github.com/GaiaNet-AI/gaia-cookbook/tree/main/js/gaia-zerion-demo). ::: ## 🌟 Features - 🤖 **AI-Powered Crypto Assistant** – Uses **Gaia AI** to interpret queries and fetch wallet balances. - 💰 **Real-Time Portfolio Data** – Retrieves live wallet data via **Zerion API**. - ⚡ **Smart Function Calling** – AI decides when to fetch data instead of blindly calling an API. - 🚀 **Extensible Template** – Can be expanded to track transactions, DeFi holdings, or AI-powered investment insights. ## 📌 Why Not Just Call the API Directly? Unlike traditional API calls, this project leverages **AI-driven agent behavior**: 1. **Natural Language Understanding** – No need to manually enter API parameters. Just chat with the AI. 2. **Smart Data Retrieval** – AI determines if and when to fetch portfolio data, preventing unnecessary API calls. 3. **Scalability** – This approach can be extended to support multi-agent interactions, consensus mechanisms, or even blockchain-verified queries. ## 🛠️ Setup & Installation ### **1️⃣ Clone/Fork the Repo** ```sh git clone https://github.com/harishkotra/gaia-zerion-demo cd gaia-zerion-demo ``` ### **2️⃣ Install Dependencies** ```sh npm install ``` ### **3️⃣ Add Your API Keys** Create a .env.local file and add: ```sh NEXT_PUBLIC_GAIA_NODE_URL=your_gaia_node_URL NEXT_PUBLIC_GAIA_API_KEY=your_gaia_api_key NEXT_PUBLIC_ZERION_API_KEY=your_zerion_api_key NEXT_PUBLIC_ZERION_BASE_URL==zerion_base_URL ``` ### **4️⃣ Run the App** ```sh npm run dev ``` Visit `http://localhost:3000` to chat with the AI assistant. ## 🚀 What Can You Build With This? This is just a starting point! You can expand it into: 🔍 On-Chain AI Analysis – Let AI analyze wallet transactions and detect patterns. 📊 DeFi Portfolio Tracker – Monitor DeFi positions, LP tokens, and staking rewards. 🤝 Multi-Agent AI Trading Bot – Implement multi-agent consensus for AI-driven trade recommendations. 🔐 Blockchain-Verified Queries – Use smart contracts to verify AI responses before showing them. ## 📚 Resources [Gaia AI](https://docs.gaianet.ai/intro) – Gaia is a decentralized computing infrastructure that enables everyone to create, deploy, scale, and monetize their own AI agents that reflect their styles, values, knowledge, and expertise. [Zerion API](https://developers.zerion.io/reference/intro/getting-started) - The Zerion API can be used to build feature-rich web3 apps, wallets, and protocols with ease. Across all major blockchains, you can access wallets, assets, and chain data for web3 portfolios. Zerion's infrastructure supports all major chains! :::info The complete source code and detailed instructions for setting up and running this example can be found in the [Gaia Cookbook repository](https://github.com/GaiaNet-AI/gaia-cookbook/tree/main/js/gaia-zerion-demo). ::: --- ## Agent Zero # Agent Zero [Agent Zero](https://github.com/frdel/agent-zero) is a general purpose AI agent application. You can simply ask it to accomplish tasks on the command line. It is designed to be dynamic, organically growing, and learning as users use it. It leverages your computer as a tool to accomplish your tasks. ## Prerequisites You will need a Gaia node to provide LLM services to the agent app. You can * [run your own node](../../../getting-started/quick-start) * [use a public node](../../../nodes) In this tutorial, we will use the public [Llama-3.1-8B node](https://github.com/GaiaNet-AI/node-configs/tree/main/llama-3.1-8b-instruct) to power the Agent Zero. | Model type | API base URL | Model name | |-----|--------|-----| | Chat | https://llama8b.gaia.domains/v1/ | llama | | Embedding | https://llama8b.gaia.domains/v1/ | nomic-embed | **You will also need to make sure your Docker engine is running.** Because the Agent Zero framework will leverage Docker to execute the generated code. > You can start a local LLM service using [Gaia](https://github.com/GaiaNet-AI/node-configs/tree/main/llama-3.1-8b-instruct) or [LlamaEdge](https://llamaedge.com/docs/user-guide/quick-start-command) or [Moxin](https://github.com/moxin-org/moxin), and then use `http://localhost:8080/v1/` as the LLM API service endpoint URL. ## Configure the agent First, we will need to get the source code of a Gaia-compatible version of Agent Zero. ``` git clone https://github.com/JYC0413/agent-zero-gaia.git cd agent-zero-gaia ``` Then, let's install the required dependencies. ``` pip install -r requirements.txt ``` Next, let's configure the gaia node and other parameters. ``` cp example.env .env ``` You will need to configure the following items: * `CHAT_MODEL_BASE_URL`: URL for the LLM API base URL. E.g., `https://llama8b.gaia.domains/v1/` * `CHAT_MODEL_NAME`: Name of the chat model to be used. E.g., `llama` * `CHAT_API_KEY`: An API key to access the LLM services. You can enter several random characters here. E.g., `GAIA` * `EMBEDDING_MODEL_BASE_URL`: URL for the embedding model API base URL. E.g., `https://llama8b.gaia.domains/v1/` * `EMBEDDING_MODEL_NAME`: Name of the embedding model name. E.g., `nomic-embed` * `EMBEDDING_API_KEY`: An API key to access the embedding services. You can enter several random characters here. E.g., `GAIA` ## Run the agent Finally, let's run the Agent Zero application backed by the [Llama 3.1 8b](https://github.com/GaiaNet-AI/node-configs/tree/main/llama-3.1-8b-instruct) Gaia node. ``` python main.py ``` You can interact with Agent Zero via the command line. You will see the Agent Zero framework will call the related tools and execute some code to complete your task. ### Example 1 Get the time in berlin and seattle ![](/img/docs/agent-zero-01.png) ![](/img/docs/agent-zero-02.png) Save the above result in a file using node.js ![](/img/docs/agent-zero-03.png) ### Example 2 Install [mc](https://midnight-commander.org/). do not use `-y` flag ![](/img/docs/agent-zero-04.png) ### Example 3 Run `top`, show the system load ![](/img/docs/agent-zero-05.png) ### Example 4 Memorize my openai api key - 563434643464 ![](/img/docs/agent-zero-06.png) ### Example 5 Extract an mp3 audio track from a mp4 video file using ffmpeg. You will need to put an mp4 file (e.g., `video.mp4`) into the `work_dir` and tell the agent about it. ![](/img/docs/agent-zero-07.png) ![](/img/docs/agent-zero-08.png) ![](/img/docs/agent-zero-09.png) --- ## Gaia Chatbot Widget # Gaia Chatbot Widget ## Overview This gaia chatbot widget is a lightweight, customizable solution for embedding an AI-powered chat interface into any web page. It provides an interactive way for users to engage with your AI assistant, enhancing user experience and providing instant support or information. ![image](https://github.com/user-attachments/assets/f4b8ef6f-1ba0-4671-8f50-af7b7235cb2a) ![image](https://github.com/user-attachments/assets/5aff9649-43d8-4d85-ab42-72ae9229ff70) ![image](https://github.com/user-attachments/assets/80fad06f-c95b-492e-bcf7-130a9f09a06d) ![image](https://github.com/user-attachments/assets/e47d21ff-df33-4b13-b4f1-491041a71972) :::info The complete source code and detailed instructions for setting up and running this example can be found in the [Gaia Cookbook repository](https://github.com/GaiaNet-AI/gaia-cookbook/tree/main/js/embeddadble-chatbot-ui). ::: ## Features - Easy to embed in any HTML page - Customizable appearance to match your website's design - Supports markdown and code formatting in responses - Responsive design for desktop and mobile devices - Configurable chatbot behavior and appearance ## Installation 1. Include the chatbot script in your HTML file: ```html ``` 2. Add the configuration object to your page: ```html window.CHATBOT_CONFIG = { apiKey: "YOUR_API_KEY", apiUrl: "https://YOUR_NODE_ID.gaia.domains/v1/chat/completions", botTitle: "AI Assistant", welcomeMessage: "Hello! How can I assist you today?", placeholderText: "Type your message here...", brandColor: "#000000", chatHeaderBackground: "#ffffff", chatHeaderTextColor: "#000000", chatBubbleBackgroundUser: "#000000", chatBubbleTextColorUser: "#ffffff", chatBubbleBackgroundBot: "#f2f2f2", chatBubbleTextColorBot: "#000000", systemMessage: "You are a helpful assistant.", maxResponseTokens: 1000, temperatureValue: 0.7, suggestedQuestions: [ "What services do you offer?", "How can I contact support?", "Tell me about your company" ] }; ``` 3. The chatbot widget will automatically initialize and appear on your page. ## Configuration Options | Option | Type | Description | Default | |--------|------|-------------|---------| | `apiKey` | String | Your API key for authentication | Required | | `apiUrl` | String | The endpoint URL for the chat completions API | Required | | `botTitle` | String | The title displayed in the chat header | "AI Assistant" | | `welcomeMessage` | String | The initial message displayed by the bot | "Hello! How can I assist you today?" | | `placeholderText` | String | Placeholder text for the input field | "Type your message here..." | | `brandColor` | String | Primary color for the chatbot (HEX) | "#000000" | | `chatHeaderBackground` | String | Background color of the chat header (HEX) | "#ffffff" | | `chatHeaderTextColor` | String | Text color of the chat header (HEX) | "#000000" | | `chatBubbleBackgroundUser` | String | Background color of user message bubbles (HEX) | "#000000" | | `chatBubbleTextColorUser` | String | Text color of user message bubbles (HEX) | "#ffffff" | | `chatBubbleBackgroundBot` | String | Background color of bot message bubbles (HEX) | "#f2f2f2" | | `chatBubbleTextColorBot` | String | Text color of bot message bubbles (HEX) | "#000000" | | `systemMessage` | String | Initial system message to set the bot's behavior | "You are a helpful assistant." | | `maxResponseTokens` | Number | Maximum number of tokens in the bot's response | 1000 | | `temperatureValue` | Number | Randomness of the bot's responses (0-1) | 0.7 | | `suggestedQuestions` | Array | List of suggested questions to display | [] | ## Browser Compatibility This widget is compatible with modern browsers including Chrome, Firefox, Safari, and Edge. Internet Explorer is not supported. :::info The complete source code and detailed instructions for setting up and running this example can be found in the [Gaia Cookbook repository](https://github.com/GaiaNet-AI/gaia-cookbook/tree/main/js/embeddadble-chatbot-ui). ::: --- ## Build an Agent for your Developer Docs # Build an Agent for your Developer Docs Normally AI Agents are considered as some bots that can do stuff and a bit intelligent. But in practice, most useful developer agents are super-focused specialized agents with private and highly curated knowledge. For example: the official docs of a programming language or maybe a protocol/company’s internal API reference. Gaia provides the three pieces which lets anyone to create such a specialist agent without much complexity. A pluggable knowledge base where you can drop chunks, a RAG-aware chat stack that speaks the OPENAI style v1/chat/completions and modular system prompts. --- ## **A Vyper Smart Contract Language Documentation Agent** Let’s go ahead and create an agent for **Vyper smart-contract language** (a Python-flavored alternative to Solidity).  Everything you see here—directory layout, embedding commands, node config flags—translates 1-to-1 to any other doc set: Rust book chapters, Django REST API docs, an RFC archive or just any docs. Make sure you’ve your docker running and Qdrant vector database installed by running `docker pull qdrant/qdrant` ### **Prerequisites: Tooling You Install Once** --- **WasmEdge Runtime** – Gaia’s embedding utility ships as a lightweight WebAssembly module; WasmEdge is the execution engine that runs it natively on Linux/macOS/Windows: ``` curl -sSf https://raw.githubusercontent.com/WasmEdge/WasmEdge/master/utils/install_v2.sh | bash -s ``` *Why Wasm?* Because you get a build artifact (a single .wasm file) that never needs pip install. Security teams love the deterministic binary; DevOps loves the zero-dependency story. **Embedding Model Weights** – The model that produces your 768-dimensional sentence vectors lives on Hugging Face: ``` curl -LO https://huggingface.co/gaianet/Nomic-embed-text-v1.5-Embedding-GGUF/resolve/main/nomic-embed-text-v1.5.f16.gguf ``` These weights are in the **GGUF** format, the modern successor to GGML, which streams efficiently from disk and supports quantized variants for CPU-only boxes. Let’s create two folders as `qdrant_storage` and `qdrant_snapshots` . We’ll find our snapshotted knowledge base inside `qdrant_snapshots`. ``` mkdir qdrant_storage mkdir qdrant_snapshots nohup docker run -d -p 6333:6333 -p 6334:6334 \ -v $(pwd)/qdrant_storage:/qdrant/storage:z \ -v $(pwd)/qdrant_snapshots:/qdrant/snapshots:z \ qdrant/qdrant ``` A couple of best-practice notes: - Map 6333 (REST/JSON) and 6334 (gRPC) if you think you’ll ever script bulk uploads or run benchmarks with the Rust/Go Qdrant clients. - Persisting qdrant_storage on a mounted volume means your vectors survive a container restart *and* can be tar-gz’d for snapshots later. ### **Create the Collection** We start fresh because the default Gaia sample collection uses a different dimension size. Let’s remove if we’ve any data: ```jsx # Clear any stale data curl -X DELETE 'http://localhost:6333/collections/default' ``` And then run the following. ``` # Provision a 768-dimensional, cosine-distance space curl -X PUT 'http://localhost:6333/collections/default' \ -H 'Content-Type: application/json' \ --data-raw '{ "vectors": { "size": 768, "distance": "Cosine", "on_disk": true } }' ``` The on_disk flag is your friend when the corpus grows into the hundreds of thousands of chunks; Qdrant memory-maps the vectors and leaves RAM for your LLM. ### **Producing the Embeddings** Lets Grab the Embedding CLI ``` curl -LO https://github.com/GaiaNet-AI/embedding-tools/raw/main/csv_embed/csv_embed.wasm ``` ### **Preparing the Documentation** **Why CSV instead of Markdown?**  It's quite easier to use CSV than markdown and it’s better to paste your each chunk into different columns and one cell per column. Fell free to use `llm_info.txt` file, if that’s what you prefer. You can use [gitingest](https://gitingest.com/) to turn the vyper docs git repository into simple text digest of it’s codebase which is truly helpful for feeding a codebase/docs into LLM. ### **Generate the Vectors** Let’s make sure the csv file/the llm_info.txt file is in the same directory and lets generate the vectors. ``` wasmedge --dir .:. \ --nn-preload embedding:GGML:AUTO:nomic-embed-text-v1.5.f16.gguf \ csv_embed.wasm embedding default 768 your_docs.csv --ctx_size 8192 ``` - Breakdown of flags: - -dir .:. gives the wasm sandbox read/write in the working directory, so it can stream in the CSV and write vectors back to Qdrant. - -nn-preload embedding:GGML:AUTO:... pre-loads the model once, saving ~2 s per 1,000 chunks. - ctx_size is the secondary token window used *inside the embedding model*, not the chat model; crank it up if your doc paragraphs are long and you see truncations in logs. You can run the command multiple times with --start_vector_id to append new shards without collisions; Qdrant IDs are integers, so pick an offset like 100_000 if you plan quarterly doc updates. ### **Packaging and Distributing Your Knowledge Base** A snapshot is nothing more than a **directory of JSON and Parquet files** that Qdrant can restore atomically.  Compressing it usually gives a 3-to-1 size win because vectors are highly repetitive. ``` curl -X POST 'http://localhost:6333/collections/default/snapshots' ``` Now it’s time to compress the snapshots, let’s head over to `qdrant_snapshots/default` and then compress it by running. ```jsx tar czvf my.snapshot.tar.gz my.snapshot ``` Instead of `my.snapshot` just use the file name. Upload my.snapshot.tar.gz to huggingface, you’ll need to have an account there. Now click on new dataset and upload your knowledge base there. --- ## **Some Best Practices** 1. **Chunking Strategy** *Aim for 200–500 tokens per vector.*  Too short and you flood the index with thousands of nearly identical embeddings (noise).  Too long and the retrieval step returns sprawling blocks the model never fully reads. 2. **System Prompt Engineering** Phrase capabilities *and* guardrails.  Example: > “You are a Vyper language assistant. If asked non-Vyper questions, politely refuse with a single-sentence apology. Cite code line numbers in every answer.” > 3. **Continuous Evaluation Loop** Pipe real user queries into a spreadsheet, hand-label them “Helpful/Unhelpful,” and look for patterns: are people asking for *security best practices* more than *syntax*?  That tells you what doc sections need richer examples. --- After uploading knowledge base into huggingface, it’s time to change knowledge base for our llm. If you haven’t had the change to run your own node yet you can head over [here](https://docs.gaianet.ai/getting-started/quick-start) and start. If we want want to sun an specific node, so let's will head over [here](https://github.com/GaiaNet-AI/node-configs) and pick a LLM that works for me. After installing we will need to run the following: ```jsx gaianet config \ --snapshot https://huggingface.co/datasets/meowy-ai/vyper-lang/resolve/main/default-845259036638694-2025-04-22-09-28-18.snapshot.tar.gz \ --system-prompt "You are a helpful vyper lang instructor, please answer the questions" ``` Over here `--snapshot https://huggingface.co/...` is the down load link of my knowledge base which I’ve uploaded to huggingface. You’ll se something like the following on your terminal: ```jsx [+] Updating the system prompt of chat model ... * Old system prompt: You are a helpful vyper lang instructor, please answer the questions * New system prompt: You are a helpful web3 instructor, please answer the questions [+] Updating the url of snapshot ... * Old url: https://huggingface.co/datasets/meowy-ai/web3-knowledge-base/resolve/main/default-8461598741381726-2025-04-29-07-50-41.snapshot.tar.gz * New url: https://huggingface.co/datasets/meowy-ai/vyper-lang/resolve/main/default-845259036638694-2025-04-22-09-28-18.snapshot.tar.gz ✅ COMPLETED! The config.json is updated successfully. ``` Now you should go ahead and start the gaia node by using `gaianet start`. Now the node will use the vyper knowledge base that we’ve just updated. We’ve successfully created our documentation agent. --- ## Gaia Web3 Voting Starter # Gaia Web3 Voting Starter ![Gaia AI Voting](../ai-voting//ai-voting.gif) A decentralized voting application built with Next.js, Hardhat, and [Gaia](https://docs.gaianet.ai/intro/). You can create voting situation and select the choices you want to vote and vote it with the AI Agent. ## About Gaia [Gaia](https://docs.gaianet.ai/intro/) is a decentralized computing infrastructure that enables everyone to create, deploy, scale, and monetize their own AI agents. This project uses Gaia's AI capabilities to provide an intelligent agent for blockchain voting interactions. ## Project Structure ``` gaia-web3-voting-starter/ ├── packages/ │ ├── blockchain/ # Smart contracts and blockchain code │ │ ├── contracts/ # Solidity smart contracts │ │ ├── ignition/ # Hardhat Ignition deployment scripts │ │ ├── scripts/ # Hardhat scripts │ │ ├── test/ # Contract tests │ │ └── hardhat.config.ts # Hardhat configuration │ │ │ └── site/ # Next.js frontend application │ ├── app/ # Next.js app router │ │ ├── api/ # API routes │ │ ├── chat/ # Chat page │ │ └── page.tsx # Home page │ ├── components/ # React components │ │ ├── chat/ # Chat-related components │ │ └── ui/ # UI components (button, card, etc.) │ ├── ai/ # AI integration │ │ └── tools.ts # AI tools for blockchain interaction │ └── public/ # Static assets ``` ## Features - **AI-Agent Powered Voting**: Interact with the blockchain through natural language commands - **Smart Contract Factory**: Create new voting instances with customizable options and durations - **On-Chain Voting**: Cast votes securely on the blockchain with transparent results - **Voting Management**: View active and historical votings with detailed status information - **Cross-Chain Compatibility**: Deploy on Linea Sepolia or any other EVM-compatible chain ## Setting Up Your Gaia Node To use your own Gaia node with this application, follow these steps: ### Option 1: Run Your Own Node 1. **Install Gaia Node**: ```bash curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash ``` 2. **Initialize with a Model**: ```bash # For Llama-3-Groq-8B model (recommended for this project) gaianet init --config https://raw.githubusercontent.com/GaiaNet-AI/node-configs/main/llama-3-groq-8b-tool/config.json # Or choose another model from the available configurations # Browse options at: https://github.com/GaiaNet-AI/node-configs ``` 3. **Start the Node**: ```bash gaianet start ``` 4. **Update Your Application**: - Modify `packages/site/app/api/chat/route.ts` to point to your local node: ```typescript const GAIA_API_ENDPOINT = 'http://gaiaURL/v1'; const GAIA_MODEL = 'Llama-3-Groq-8B-Tool'; ``` ### Option 2: Get an API Key 1. **Create an Account**: - Go to [https://gaianet.ai](https://gaianet.ai) and click on **Launch App** - Connect your MetaMask wallet 2. **Generate an API Key**: - Click on your profile dropdown and select **Settings** - Navigate to **Gaia API Keys** and click **Create API Key** - Give your key a name and save it securely 3. **Update Your Application**: - Modify `packages/site/app/api/chat/route.ts` to use your API key: ```typescript const GAIA_API_ENDPOINT = 'https://api.gaianet.ai/v1'; const GAIA_MODEL = 'Llama'; // Add your API key here const openai = createOpenAI({ baseURL: GAIA_API_ENDPOINT, apiKey: "" // API key Here }); ``` 1. **Add to Environment Variables**: - Create or update `.env.local` in the `packages/site` directory: ``` GAIA_API_KEY=your_api_key_here ``` ### System Requirements If running your own node, ensure your system meets these requirements or start with a small model of LLM: | System | Minimum Requirements | |--------|---------------------| | OSX with Apple Silicon (M1-M4 chip) | 16GB RAM (32GB recommended) | | Ubuntu Linux 20.04 with Nvidia CUDA 12 SDK | 8GB VRAM on GPU | | Azure/AWS | Nvidia T4 GPU Instance | ## AI Agent Commands The application includes an AI agent that helps users interact with the voting system through natural language commands: ### 1. Create a New Voting ``` create voting "Your voting description" options: option1, option2, option3 duration: ``` Duration options: - 1 - 1 Hour - 2 - 1 Day - 3 - 1 Week ### 2. View Votings Show all votings (including ended): ``` show all votings get all votings list votings ``` Show only active votings: ``` show active votings list active votings get active votings ``` ### 3. Cast a Vote ``` vote for [contract-address] option [number] ``` Example: ``` vote for 0x1234...5678 option 2 ``` ### 4. Additional Commands - `help` or `commands` - Show all available commands - `voting status` - Get a summary of active and ended votings - `my votes` - View your voting history ### Enhancing the AI Agent with Knowledge Bases To make your voting application's AI agent more powerful and context-aware, you can integrate Gaia's knowledge base system: 1. **Create a Voting-Specific Knowledge Base**: - Document common voting patterns and use cases - Include explanations of blockchain voting concepts - Add frequently asked questions about the voting process - Include examples of successful voting campaigns 2. **Structure Your Knowledge Base**: - Organize information by categories (e.g., "Creating Votes", "Casting Votes", "Viewing Results") - Include metadata like timestamps and tags for better searchability - Use markdown or plain text files for easy maintenance 3. **Import to Gaia**: - Use Gaia's tools to import your knowledge base - The system will automatically convert your content into embeddings using Qdrant - Configure your AI agent to reference this knowledge base 4. **Benefits of Knowledge Base Integration**: - **Improved Accuracy**: The agent will provide more accurate and contextually relevant responses - **Enhanced User Experience**: Users will receive more helpful guidance on voting processes - **Scalability**: As your knowledge base grows, the agent's capabilities expand without retraining - **Efficiency**: The agent can quickly access pre-organized information rather than processing data in real-time ### Pre-configured Knowledge Base We've created a pre-configured knowledge base specifically for Web3 voting DApps. You can use it with your Gaia node: 1. **Knowledge Base Files**: - Snapshot file: [https://huggingface.co/datasets/meowy-ai/web3-knowledge-base/resolve/main/default-6695366476678026-2025-04-29-13-58-26.snapshot.tar.gz](https://huggingface.co/datasets/meowy-ai/web3-knowledge-base/resolve/main/default-6695366476678026-2025-04-29-13-58-26.snapshot.tar.gz) 2. **Configure Your Gaia Node**: Let's the following command to configure your Gaia node with the web3 knowledge base: ```bash gaianet config \ --snapshot https://huggingface.co/datasets/meowy-ai/web3-knowledge-base/resolve/main/default-8461598741381726-2025-04-29-07-50-41.snapshot.tar.gz \ --system-prompt "You are a helpful web3 instructor, please answer the questions" ``` For more information on knowledge bases in Gaia, visit the [Gaia Knowledge Bases Documentation](https://docs.gaianet.ai/knowledge-bases/intro). ## Getting Started ### Prerequisites - Node.js installed - A wallet with some test ETH on Sepolia (You can use any L1/L2 you want) - [Alchemy](https://www.alchemy.com/) or Infura API key ### Setup 1. Clone the repository: ```bash git clone git@github.com:meowyx/gaia-web3-voting-starter.git cd gaia-web3-voting-starter ``` 2. Install dependencies: ```bash pnpm install ``` 3. Configure environment variables: ```bash # In packages/blockchain cp .env.example .env # Add your Alchemy API key and wallet private key ``` 4. Compile and deploy contracts: ```bash cd packages/blockchain npx hardhat compile npx hardhat ignition deploy ignition/modules/votingFactory.ts --network linea-testnet ``` 5. Start the frontend: ```bash cd packages/site pnpm dev ``` ## Architecture - **Smart Contracts**: Factory pattern for deploying voting instances - **Frontend**: Next.js with AI-powered chat interface - **Blockchain**: Linea Sepolia testnet for deployment - **AI Integration**: Custom tools for blockchain interaction --- ## A planning agent # A planning agent The [gpt planner](https://github.com/mshumer/gpt-prompt-engineer/blob/main/gpt_planner.ipynb) is a Python application that demonstrate the planning capabilities of LLMs. When you run it, it will ask the LLM to generate multiple action plans for a goal or a query. It will then ask the LLM to compare and select the best plan, and then rewrite it to answer the user query. Since the program uses the official OpenAI Python library, we can [easily change it to use a Gaia node](intro.md). ## Prerequisites You will need a Gaia node ready to provide LLM services through a public URL. You can * [run your own node](../../../getting-started/quick-start/quick-start.md) * [use a public node](../../../nodes/nodes.md) In this tutorial, we will use a public node. | Attribute | Value | |-----|--------| | API endpoint URL | https://llama8b.gaia.domains/v1 | | Model Name | llama | ## Run the agent First, [load the notebook in colab](https://colab.research.google.com/github/mshumer/gpt-prompt-engineer/blob/main/gpt_planner.ipynb). Edit the code to create an OpenAI client. We will pass in the `base_url` here. ``` client = openai.OpenAI(base_url="https://llama8b.gaia.domains/v1", api_key=OPENAI_API_KEY) ``` Next, replace all the `gpt-4o-mini` model name with the `llama` model name in the code. Here is an example. ``` response = client.chat.completions.create( model="llama", messages=[ {"role": "system", "content": "You are a helpful assistant. Use the given plan to create a detailed and high-quality response to the user's query."}, {"role": "user", "content": f"User Query: {user_query}\n\nPlan: {best_plan}\n\nGenerate a detailed response based on this plan."} ], temperature=0.5, max_tokens=2000 ) ``` Change the query to your own. ``` user_query = "How do I debug a TLS connection timeout?" ``` Finally, run the notebook to see the results! --- ## LlamaCoder # LlamaCoder LlamaCoder is an open-source tool designed to generate small apps with a single prompt. It leverages LLM to help you quickly create and enhance React applications. ## Prerequisites You will need a Gaia node ready to provide LLM services through a public URL. You can * [run your own node](../../../getting-started/quick-start/quick-start.md) * [use a public node](../../../nodes/nodes.md) In this tutorial, we will use a public Llama3 node. | Attribute | Value | |-----|--------| | API endpoint URL | https://llama8b.gaia.domains/v1 | | Model Name | llama | | API KEY | gaia | ## Run the agent First, we will need to get the source code of the forked LlamaTutor ``` git clone https://github.com/second-state/llamacoder.git cd llamacoder ``` Next, configure the `.env` file. ``` cp .example.env .env ``` You will need to configure three parameters here. * LLAMAEDGE_BASE_URL: URL for the LLM API base URL. * LLAMAEDGE_MODEL_NAME: Name of the model to be used. * LLAMAEDGE_API_KEY: API key for accessing the LLM services. For example, you can use the following `.env` setting. ``` LLAMAEDGE_BASE_URL=https://llama8b.gaia.domains/v1 LLAMAEDGE_MODEL_NAME=llama LLAMAEDGE_API_KEY=GaiaNet ``` Then, we will need to install the required dependencies. ``` npm install ``` Next, let's run the application locally. ``` npm run dev ``` Finally, open http://localhost:3000 in your browser and start building your React app. ![](/img/docs/llamacoder.png) --- ## LlamaEdgeBook # LlamaEdgeBook LlamaEdgeBook, forked from GroqBook, is an open-source tool that scaffolds the creation of books from a one-line prompt using open-source LLMs. You can configure the LlamaEdgeBook framework using any Gaia node as the backend LLM API. ## Steps First, get the source code of the LlamaEdgeBook. Open your terminal and enter the following command line. ``` git clone https://github.com/second-state/LlamaEdgeBook cd LlamaEdgeBook ``` Ensure you have Python 3.11 or later installed. Then, install the necessary packages: ``` pip install -r requirements.txt ``` Next, let's configure the Gaia node as the LLM backend. ``` export OPENAI_BASE_URL="https://llama8b.gaia.domains/v1" export OPENAI_MODEL_NAME="llama" export OPENAI_API_KEY="GAIANET" ``` **Hint:** if you don't know the model name of the Gaia node, you can retrieve the model information using: ``` curl -X POST https://0x57b00e4f3d040e28dc8aabdbe201212e5fb60ebc.gaia.domains/v1/models ``` Then, use the following command line to run the app. ``` streamlit run main.py ``` When the app runs successfully, the terminal will output the following information. ``` Collecting usage statistics. To deactivate, set browser.gatherUsageStats to false. You can now view your Streamlit app in your browser. Local URL: http://localhost:8501 Network URL: http://10.128.0.8:8501 External URL: http://35.222.115.181:8501 ``` Finally, you can open the `http://localhost:8501` in your browser to generate a book. ![](/img/docs/book-01.png) The LlamaEdgeBook will first generate an outline based on your prompt, and then create the chapter content based on the outline. ![](/img/docs/book-02.png) You can also download the book after the generation is complete. ![](/img/docs/book-03.png) --- ## LlamaTutor # LlamaTutor The [LlamaTutor](https://github.com/Nutlope/llamatutor) is a TypeScript application using Llama 3.1 to act as an open-source AI personal tutor. When you run it, it will ask the LLM to search on the web based on your inquiry topic and then generate content for you to learn. The program didn't use the official OpenAI library, we can use the forked one that supports [using a Gaia node](../../../intro.md) as the LLM backend. ## Prerequisites You will need a Gaia node ready to provide LLM services through a public URL. You can * [Run your own node](../../../getting-started/quick-start/quick-start.md) * [Use a public node](../../../nodes/nodes.md) In this tutorial, we will use a public Llama3 node. | Attribute | Value | |-----|--------| | API endpoint URL | https://llama8b.gaia.domains/v1 | | Model Name | llama | | API KEY | gaia | ## Run the agent First, we will need to get the source code of the forked LlamaTutor ``` git clone https://github.com/JYC0413/llamatutor.git cd llamatutor ``` Next, configure the `.env` file. ``` cp .example.env .env ``` You will need to configure four parameters here. * SERPER_API_KEY: The [serper API key](https://serper.dev/) for searching content online. You can also use BING_API_KEY here. * LLAMAEDGE_BASE_URL: URL for the LLM API base URL. * LLAMAEDGE_MODEL_NAME: Name of the model to be used. * LLAMAEDGE_API_KEY: API key for accessing the LLM services. Then, we will need to install the required dependencies. ``` npm install ``` Next, let's run the application locally. ``` npm run dev ``` Finally, open http://localhost:3000 in your browser and start to learn. ![](/img/docs/llamatutor-01.png) --- ## Notion Gaia Assistant # Notion Gaia Assistant 👋 This is a Chrome extension that brings the power of Gaia AI nodes right into your Notion pages. Think of it as your personal AI assistant that sits quietly in the corner of your Notion workspace, ready to help whenever you need it. ![image](https://github.com/user-attachments/assets/5b60a516-46e2-4a22-a54f-f66bad3fbeaa) ![image](https://github.com/user-attachments/assets/c3e63a49-85dd-4186-9d88-1ce7a53a89c3) ![image](https://github.com/user-attachments/assets/7e55ef86-6c4e-49cd-a043-b360d6fc8bf8) ![image](https://github.com/user-attachments/assets/2e77def1-2a39-4729-a005-9c82cba9ef32) ![image](https://github.com/user-attachments/assets/ae1d2161-07fc-4613-8edc-91a984adb834) :::info The complete source code and detailed instructions for setting up and running this example can be found in the [Gaia Cookbook repository](https://github.com/GaiaNet-AI/gaia-cookbook/tree/main/js/gaia-notion-chrome-extension). ::: ### For Node Builders If you build and run nodes of Gaia, then this is a fantastic way to showcase your work! Your node can now directly help people with their Notion documents. It's almost like giving your AI a job as a personal assistant in Notion. Because the extension works with any OpenAI-compatible endpoint, your node can plug right in. ### For Notion Users Imagine having a smart assistant that can: - Summarize your long Notion pages with one click - Suggest improvements to your documents - Help you research related topics - Answer specific questions about your page content And the best part? You can choose which Gaia node to connect to. Maybe you want one that's great at academic writing, or another that's fantastic at creative suggestions - it's totally up to you! ## Getting Started 1. Install the extension from Chrome Web Store (coming soon!) 2. Click the extension icon in your browser 3. Add your preferred Gaia node URL - it should look something like `https://YOUR_NODE_ID.gaia.domains/v1` 4. Go to any Notion page 5. Locate the tiny floating button at the bottom right - that's your new AI assistant! ## Features - 🎯 **Page Summaries**: Get quick summaries of any Notion page - 💡 **Smart Suggestions**: Get ideas on how to improve your content - 🔍 **Research Helper**: Discover related topics to explore - 💬 **Ask Questions**: Chat with the AI about anything on your page - 🎨 **Clean Interface**: Everything fits nicely into Notion's design ## Setting Up Your Own Node If you're running a Gaia node and want to use it with this extension: 1. Make sure your node is publicly accessible 2. Copy your node's URL (the `/v1` endpoint) 3. Paste it into the extension's settings 4. That's it! Your node is now powering the Notion assistant ## Prerequisites You'll need: - Google Chrome browser - A code editor (like Visual Studio Code) - Basic familiarity with Chrome's developer mode ## Steps to Install 1. **Get the Code** ``` # Clone this repository git clone https://github.com/harishkotra/gaia-notion-chrome-extension cd gaia-notion-chrome-extension ``` 2. Open Chrome Extensions Page - Open Google Chrome - Type `chrome://extensions` in the address bar - Or navigate through: Menu (⋮) > More Tools > Extensions 3. Enable Developer Mode - Look for the "Developer mode" toggle in the top-right corner - Switch it on - you'll see new options appear at the top 4. Load the Extension - Click the "Load unpacked" button that appears after enabling Developer mode - Navigate to the folder where you cloned this repository - Select the folder and click "Open" - You should see the extension appear in your list of installed extensions! ### Testing the Extension 1. Check Installation - Look for the extension icon in Chrome's toolbar - If you don't see it, click the puzzle piece icon to find it in the extensions menu 2. Set Up a Gaia Node - Click the extension icon - Add your Gaia node URL in the settings - The URL should end with `/v1` (like `https://YOUR_NODE_ID.gaia.domains/v1`) 3. Try It Out - Go to any Notion page - You should see a small floating button in the bottom-right corner - Click it to start using the assistant! :::info The complete source code and detailed instructions for setting up and running this example can be found in the [Gaia Cookbook repository](https://github.com/GaiaNet-AI/gaia-cookbook/tree/main/js/gaia-notion-chrome-extension). ::: --- ## Stockbot # Stockbot Stockbot is a lightning fast AI Chatbot that responds with live interactive stock charts, financials, news, screeners, and more. You can configure a Gaia node as the LLM backend. > Please note, the Stockbot leverages function call to call the external API. You will need to use the model which supports function call, like [llama-3.1-8b](https://github.com/GaiaNet-AI/node-configs/tree/main/llama-3-groq-8b-tool). ## Prerequisites You will need a Gaia node ready to provide LLM services through a public URL. You can * [run your own node](../../../getting-started/quick-start/quick-start.md) * [use a public node](../../../nodes/nodes.md) In this tutorial, we will use a public Llama3 node with the function call support. | Attribute | Value | |-----|--------| | API endpoint URL | https://llama8b.gaia.domains/v1 | | Model Name | llama | | API KEY | gaia | ## Run the agent First, we will need to get the source code of the forked Stockbot. ``` git clone https://github.com/JYC0413/stockbot-on-groq.git cd stockbot-on-groq ``` Next, configure the `.env` file. ``` cp .env.example .env.local ``` You will need to configure four parameters here. * LLAMAEDGE_BASE_URL: URL for the LLM API base URL. * LLAMAEDGE_MODEL_NAME: Name of the model to be used. * LLAMAEDGE_API_KEY: API key for accessing the LLM services. Then, we will need to install the required dependencies. ``` npm install ``` Next, let's run the application locally. ``` npm run dev ``` Finally, you can open http://localhost:3000 and ask the stock related questions. ![](/img/docs/stockbot-01.png) --- ## Translation Agent + Gaia # Translation Agent + Gaia The Translation Agent originally built by Prof. Andrew Ng, designed to facilitate accurate and efficient translation across multiple languages. It employs open source LLMs (Large Language Models) to provide high-quality translations. You can use any Gaia node as the LLM backend. >For commands on starting and running this agent, refer to [GitHub - Second State/translation-agent](https://github.com/second-state/translation-agent/blob/use_llamaedge/step-by-step-use-LocalAI.md). You can run the Translation Agent on top of a public Gaia Node as a backend and then translate the content in your target language. If you’d like to learn more about the Translation Agent and how open source LLMs perform, check out the article [Agentic translation on Gaia](https://docs.gaianet.ai/tutorial/translator-agent). ## Prepare the environment Here, we will use the public Gaia node with Llama 3.1 8b model. `https://llama8b.gaia.domains/`. >As an alternative, you can also start a Gaia node locally on your device. Refer to [this guide](https://github.com/GaiaNet-AI/node-configs/tree/main/llama-3.1-8b-instruct). To get started, clone the Translation Agent that supports open source LLMs. ``` git clone https://github.com/second-state/translation-agent.git cd translation-agent git checkout use_llamaedge ``` Set environment variables and install necessary Python packages if needed. Replace the OPENAI_BASE_URL with `https://llama8b.gaia.domains/` ``` export OPENAI_BASE_URL="https://llama8b.gaia.domains/v1" export PYTHONPATH=${PWD}/src export OPENAI_API_KEY="GET YOUR OWN API KEY" pip install python-dotenv pip install openai tiktoken icecream langchain_text_splitters ``` ## Prepare your translation task Find the `examples/sample-texts` folder in your cloned repo. Put the file you want to translate in this folder and get its path. Here because we named our [source text](https://hackmd.io/tdLiVR3TSc-8eVg_E-j9QA?view#Source-text-Intro-of-Forbidden-City) in Chinese `forbiddencity.txt` since it is an introduction on this Chinese royal palace, then note down its document path, `sample-texts/forbiddencity.txt`. This will be the `relative path` in our `example_script.py` file. Find the `examples/example_script.py` file in your cloned agent repo and review its code. It tells the agent where to find your document and how to translate it. Change the relative path to the above. Change the model name to the one you are using; here, we're using the `gemma` model; also change the source and target languages you want (here we put `Chinese` as the source language and `English` as the target language). ``` import os import translation_agent as ta if __name__ == "__main__": source_lang, target_lang, country = "Chinese", "English", "Britain" relative_path = "sample-texts/forbiddencity.txt" script_dir = os.path.dirname(os.path.abspath(__file__)) full_path = os.path.join(script_dir, relative_path) with open(full_path, encoding="utf-8") as file: source_text = file.read() print(f"Source text:\n\n{source_text}\n------------\n") translation = ta.translate( source_lang=source_lang, target_lang=target_lang, source_text=source_text, country=country, model="gemma", ) print(f"Translation:\n\n{translation}") ``` ## Translate Run the python translation script as follows. ``` cd examples python example_script.py ``` Wait a few minutes and [the English translation](https://hackmd.io/tdLiVR3TSc-8eVg_E-j9QA?view#English-Translation-by-gemma-2-27b) will appear on your terminal screen. --- ## Working with Coinbase AgentKit # Working with Coinbase AgentKit You can use a Gaia node to power the [Coinbase AgentKit](https://github.com/coinbase/agentkit). The Gaia node must run an LLM that is optimized for [tool calling](../tool-call/tool-call.md). Or, you could simply use our public Gaia domain as follows. | Attribute | Value | |-----|--------| | API endpoint URL | https://llama70b.gaia.domains/v1 | | Model Name | llama70b | | API KEY | gaia | > You can [start your own Gaia node](https://github.com/GaiaNet-AI/node-configs/tree/main/llama-3.3-70b-instruct), and point the endpoint URL to `http://localhost:8080/v1` ## Quickstart First, you need a [Coinbase Developer Platform account](https://www.coinbase.com/developer-platform) and then [create an API key](https://portal.cdp.coinbase.com/projects/api-keys). Next, check out the AgentKit example code. ``` git clone https://github.com/coinbase/agentkit cd agentkit/python/examples/langchain-cdp-chatbot/ uv sync ``` Set the environment variables for your API key. ``` export CDP_API_KEY_NAME='organizations/.../apiKeys/...' export CDP_API_KEY_PRIVATE_KEY='-----BEGIN EC...END EC PRIVATE KEY-----\n' ``` Edit the `chatbot.py` file to configure the agent to use the Gaia node above. ``` llm = ChatOpenAI(model="llama", api_key="your-gaia-key", base_url="https://llama8b.gaia.domains/v1") ``` > You will need to get an API key from Gaia. Finally, run the agent using Python. ``` python chatbot.py ``` You can see a [video demo here](https://x.com/juntao/status/1858634152599224828). ## A Telegram bot for AgentKit We have also built a Telegram bot that allows you to interact with your own wallet through text and voice messages. You will need to bring your own Coinbase credentials and wallet for your bot to operate on. Go to chat the with bot on Telegram. https://t.me/agentkit_bot It asks you to use slash commands to set your wallet credentials for your bot. It is like this. ``` /name organizations/.../apiKeys/... /pk -----BEGIN EC...END EC PRIVATE KEY-----\n /wallet {"wallet_id": "... ``` Then, you can just tell the Telegram bot what you want to do on-chain, and the agent will do it for you. --- ## Gaia nodes with long-term knowledge # Gaia nodes with long-term knowledge The LLM app requires both long-term and short-term memories. Long-term memory includes factual knowledge, historical facts, background stories etc. They are best added to the context as complete chapters instead of small chunks of text to maintain the internal consistency of the knowledge. [RAG](https://blogs.nvidia.com/blog/what-is-retrieval-augmented-generation/) is an important technique to inject contextual knowledge into an LLM application. It improves accuracy and reduces the hallucination of LLMs. An effective RAG application combines real-time and user-specific short-term memory (chunks) with stable long-term memory (chapters) in the prompt context. Since the application's long-term memory is stable (even immutable), we package it in a vector database tightly coupled with the LLM. The client app assembles the short-term memory in the prompt and is supplemented with the long-term memory on the LLM server. We call the approach "server-side RAG". > The long context length supported by modern LLMs are especially well-suited for long-term knowledge that are best represented by chapters of text. A Gaia node is an OpenAI compatible LLM service that is grounded by long-term knowledge on the server side. The client application can simply chat with it or provide realtime / short-term memory since the LLM is already aware of the domain or background. > For example, if you ask ChatGPT the question What is Layer 2, the answer is that Layer 2 is a concept from the computer network. However, if you ask a blockchain person, they answer that Layer 2 is a way to scale the original Ethereum network. That's the difference between a generic LLM and knowledge-supplemented LLMs. We will cover the external knowledge preparation and how a knowledge-supplemented LLM completes a conversation. If you have learned how a RAG application works, go to [Build a RAG application with Gaia](../../knowledge-bases/how-to/web-tool) to start building one. 1. Create embeddings for your own knowledge as the long-term memory. 2. Lifecycle of a user query on a knowledge-supplemented LLM. For this solution, we will use * a chat model like Llama-3-8B for generating responses to the user. * a text embedding model like [nomic-embed-text](https://huggingface.co/second-state/Nomic-embed-text-v1.5-Embedding-GGUF) for creating and retrieving embeddings. * a Vector DB like Qdrant for storing embeddings. ## Workflow for creating knowledge embeddings The first step is to create embeddings for our knowledge base and store the embeddings in a vector DB. ![create-embedding](https://github.com/GaiaNet-AI/docs/assets/45785633/2ff40178-64f4-4e2e-bbd9-f12ce35186b7) First of all, we split the long text into sections (i.e, chunks). All LLMs have a maximum context length. The model can't read the context if the text is too long. The most used rule for a Gaia node is to put the content in one chapter together. Remember, insert a blank line between two chunks. You can also use other algorithms to chunk your text. After chunking the document, we can convert these chunks into embeddings leveraging the embedding model. The embedding model is trained to create embeddings based on text and search for similar embeddings. We will use the latter function in the process of user query. Additionally, we will need a vector DB to store the embeddings so that we can retrieve these embeddings quickly at any time. On a Gaia node, we will get a database snapshot with the embeddings to use at last. Check out how to create your embeddings [from a plain text file](../../knowledge-bases/how-to/text), and [from a markdown file](../../knowledge-bases/how-to/markdown). ## Lifecycle of a user query on a knowledge-supplemented LLM Next, let's learn the lifecycle of a user query on a knowledge-supplemented LLM. We will take [a Gaia Node with Gaia knowledge](https://gaia.gaia.domains/chatbot-ui/index.html) as an example. ![user-query-rag](https://github.com/GaiaNet-AI/docs/assets/45785633/c64b85ea-65f0-43d2-8ab3-78889d21c248) ### Ask a question when you send a question in human language to the node, the embedding model will first convert your question to embedding. ### Retrieve similar embeddings Then, the embedding model will search all the embeddings stored in the Qdrant vector DB and retrieve the embeddings that are similar to the question embeddings. ### Response to the user query The embedding node will return the retrieved embeddings to the chat model. The chat model will use the retrieved embeddings plus your input questions as context to answer your queries finally. --- ## Integrating DeepSeek R1 with Cursor Editor # Integrating DeepSeek R1 with Cursor Editor ## Overview This guide walks through setting up a private coding assistant by integrating the **DeepSeek R1 Distilled Llama-8B** model with Cursor editor. This setup provides efficient code assistance while keeping your code private and secure. ## Prerequisites ### Hardware Requirements **Recommended Configuration:** - Mac with 16GB RAM - NVIDIA GPU or Huawei Ascend NPU **Minimum Requirements:** - Machine with 16GB RAM ## Installation ### Step 1: Install Gaia Software Run the following command to install Gaia: ```bash curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash ``` ### Step 2: Initialize DeepSeek R1 Model Download and initialize the **DeepSeek R1 Distilled Llama-8B** model: ```bash gaianet init --config https://raw.githubusercontent.com/GaiaNet-AI/node-configs/main/deepseek-r1-distill-llama-8b/config.json ``` ### Step 3: Start the Model Launch Gaia and run the model: ```bash gaianet start ``` Upon successful startup, you'll receive an HTTPS URL (e.g., `https://NODE-ID.gaia.domains`). :::note We start the DeepSeek R1 model with an 8k context window by default. If your machine has larger GPU memory (e.g., 64GB), you can increase the context size to 128k. A larger context window is particularly useful in coding tasks, as we need to compress large source code files into prompts to complete complex tasks. ::: ## Cursor Configuration 1. Open Cursor settings ![Cursor Settings](../cursor/cursor-settings.png) 2. Locate the LLM Backend configuration ![LLM Backend configuration](../model-config/model-config.png) 3. Configure the following: - Base API URL: Your Gaia node HTTPS URL - Model Name: `DeepSeek-R1-Distill-Llama-8B` - API Key: Make sure to replace `YOUR_API_KEY_GOES_HERE` with your **own API key**. To get your own API key, follow [this](../../getting-started/authentication) tutorial. ![DeepSeek-R1 Setup](../deepseek-cursor/deepseek-setup.png) ## Technical Details ### WasmEdge Runtime Features The implementation uses WasmEdge, a WebAssembly-based runtime hosted by CNCF under the Linux Foundation, offering: - Lightweight deployment (30MB) - No dependencies required - Root-free operation - Cross-platform compatibility - Sandbox isolation - Multimodal model support - Cloud-native integration ### Context Window - Default: 8k context window - Expandable to 128k with 64GB GPU memory - Larger context windows enable processing of bigger source code files ## Usage Tips - Use for code generation tasks - Get code explanations - Build complete applications - Perfect for maintaining code privacy - Suitable for both personal and enterprise use :::note [Other DeepSeek large models](https://huggingface.co/collections/gaianet/deepseek-r1-and-distills-67954070e0c6002f119c9bb5) on this page are equally applicable, so give them a try in your Cursor! If you find it interesting or encounter any issues, please star our [GitHub repo](https://github.com/LlamaEdge/LlamaEdge) or raise an issue. ::: ## Troubleshooting If you encounter issues: - Verify hardware requirements are met - Ensure Gaia is properly installed - Check Cursor configuration settings - Confirm HTTPS URL is correctly entered ## Video Guide --- ## Working with eliza # Working with eliza Eliza is a simple, fast, and lightweight AI agent framework. Recently, eliza has integrated Gaia as one of the [model service provider](https://github.com/elizaOS/eliza/pull/762). This means you can now use Gaia as the LLM service backend for the Eliza framework. ### Build a Trump agent with eliza and Gaia This guide demonstrates how to create an agent with Trump-like characteristics using [the Get Started guide from eliza](https://elizaos.github.io/eliza/docs/quickstart/). #### Set up the environment > Note: Ensure your Node.js version is above 0.23.0 before proceeding. Firstly, clone the Eliza repository: ``` git clone https://github.com/elizaos/eliza.git cd eliza git checkout v0.1.7-alpha.1 ``` Next, install the required dependencies. ``` pnpm install ``` Then, build the local libraries. ``` pnpm build ``` #### Choose Gaia as the model service provider After that, we will need to configure the environment and use a Gaia node as model service provider. ``` cp .env.example .env ``` Then, edit the `.env` file to include Gaia-related configuration values: ``` # Gaianet Configuration GAIANET_MODEL=llama3b GAIANET_SERVER_URL=https://llama3b.gaia.domains/v1 SMALL_GAIANET_MODEL= # Default: llama3b SMALL_GAIANET_SERVER_URL= # Default: https://llama3b.gaia.domains/v1 MEDIUM_GAIANET_MODEL= # Default: llama MEDIUM_GAIANET_SERVER_URL= # Default: https://llama8b.gaia.domains/v1 LARGE_GAIANET_MODEL= # Default: qwen72b LARGE_GAIANET_SERVER_URL= # Default: https://qwen72b.gaia.domains/v1 GAIANET_EMBEDDING_MODEL=nomic-embed USE_GAIANET_EMBEDDING=TRUE # Set to TRUE for GAIANET/768, leave blank for local ``` By using this configuration, the system will utilize the Llama 3b Gaia domain as the LLM backend. You can replace `GAIANET_SERVER_URL` with a URL for your custom node or domain. #### Create the Trump agent The default character templates are located in the `characters` folder. Update the `modelProvider` for the desired character. For Gaia, the provider name is `gaianet`. ``` "name": "trump", "clients": [], "modelProvider": "gaianet", "settings": { "secrets": {}, "voice": { "model": "en_US-male-medium" } }, "plugins": [], ``` Then, we can use the following command line to start running the agent. ``` pnpm run dev --character="characters/trump.character.json" ``` After the service runs successfully, we can launch the client UI to interact with the agent: ``` pnpm start:client ``` Finally, open `http://localhost:5174/` on your browser to start chatting with the agent. ### Advanced use case For more inspiration, refer to [Nader Dabit's example on building a Twitter AI bot](https://x.com/dabit3/status/1863772029565981144). --- ## Fine-tune an open-source LLM with llama.cpp # Fine-tune an open-source LLM with llama.cpp # Fine-tune LLMs You could fine-tune an open-source LLM to * Teach it to follow conversations. * Teach it to respect and follow instructions. * Make it refuse to answer certain questions. * Give it a specific "speaking" style. * Make it response in certain formats (e.g., JSON). * Give it focus on a specific domain area. * Teach it certain knowledge. To do that, you need to create a set of question and answer pairs to show the model the prompt and the expected response. Then, you can use a fine-tuning tool to perform the training and make the model respond the expected answer for each question. # How to fine-tune an open-source LLM with llama.cpp The popular llama.cpp tool comes with a `finetune` utility. It works well on CPUs! This fine-tune guide is reproduced with permission from Tony Yuan's [Finetune an open-source LLM for the chemistry subject](https://github.com/YuanTony/chemistry-assistant/tree/main/fine-tune-model) project. ## Build the fine-tune utility from llama.cpp The `finetune` utility in llama.cpp can work with quantized GGUF files on CPUs, and hence dramatically reducing the hardware requirements and expenses for fine-tuning LLMs. Check out and download the llama.cpp source code. ``` git clone https://github.com/ggerganov/llama.cpp cd llama.cpp ``` Build the llama.cpp binary. ``` mkdir build cd build cmake .. cmake --build . --config Release ``` If you have NVIDIA GPU and CUDA toolkit installed, you should build llama.cpp with CUDA support. ``` mkdir build cd build cmake .. -DLLAMA_CUBLAS=ON -DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc cmake --build . --config Release ``` ## Get the base model We are going to use Meta's Llama2 chat 13B model as the base model. Note that we are using a Q5 quantized GGUF model file directly to save computing resources. You can use any of the Llama2 compatible GGUF models on Hugging Face. ``` cd .. # change to the llama.cpp directory cd models/ curl -LO https://huggingface.co/gaianet/Llama-2-13B-Chat-GGUF/resolve/main/llama-2-13b-chat.Q5_K_M.gguf ``` ## Create a question and answer set for fine-tuning Next we came up with 1700+ pairs of QAs for the chemistry subject. It is like the following in a [CSV file](https://raw.githubusercontent.com/YuanTony/chemistry-assistant/main/fine-tune-model/train.csv). Question | Answer ----- | ------- What is unique about hydrogen? | It's the most abundant element in the universe, making up over 75% of all matter. What is the main component of Jupiter? | Hydrogen is the main component of Jupiter and the other gas giant planets. Can hydrogen be used as fuel? | Yes, hydrogen is used as rocket fuel. It can also power fuel cells to generate electricity. What is mercury's atomic number? | The atomic number of mercury is 80 What is Mercury? | Mercury is a silver colored metal that is liquid at room temperature. It has an atomic number of 80 on the periodic table. It is toxic to humans. > We used GPT-4 to help me come up many of these QAs. Then, we wrote a [Python script](https://raw.githubusercontent.com/YuanTony/chemistry-assistant/main/fine-tune-model/convert.py) to convert each row in the CSV file into a sample QA in the Llama2 chat template format. Notice that each QA pair starts with `` as an indicator for the fine-tune program to start a sample. The result [train.txt](https://raw.githubusercontent.com/YuanTony/chemistry-assistant/main/fine-tune-model/train.txt) file can now be used in fine-tuning. Put the [train.txt](https://raw.githubusercontent.com/YuanTony/chemistry-assistant/main/fine-tune-model/train.txt) file in the `llama.cpp/models` directory with the GGUF base model. ## Finetune! Use the following command to start the fine-tuning process on your CPUs. I am putting it in the background so that it can run continuously now. It could take several days or even a couple of weeks depending on how many CPUs you have. ``` nohup ../build/bin/finetune --model-base llama-2-13b-chat.Q5_K_M.gguf --lora-out lora.bin --train-data train.txt --sample-start '' --adam-iter 1024 & ``` You can check the process every few hours in the `nohup.out` file. It will report the `loss` for each iteration. You can stop the process when the `loss` goes consistently under `0.1`. **Note 1** If you have multiple CPUs (or CPU cores), you can speed up the fine-tuning process by adding a `-t` parameter to the above command to use more threads. For example, if you have 60 CPU cores, you could do `-t 60` to use all of them. **Note 2** If your fine-tuning process is interrupted, you can restart it from `checkpoint-250.gguf`. The next file it outputs is `checkpoint-260.gguf`. ``` nohup ../build/bin/finetune --model-base llama-2-13b-chat.Q5_K_M.gguf --checkpoint-in checkpoint-250.gguf --lora-out lora.bin --train-data train.txt --sample-start '' --adam-iter 1024 & ``` ## Merge The fine-tuning process updates several layers of the LLM's neural network. Those updated layers are saved in a file called `lora.bin` and you can now merge them back to the base LLM to create the new fine-tuned LLM. ``` ../build/bin/export-lora --model-base llama-2-13b-chat.Q5_K_M.gguf --lora lora.bin --model-out chemistry-assistant-13b-q5_k_m.gguf ``` The result is this file. ``` curl -LO https://huggingface.co/juntaoyuan/chemistry-assistant-13b/resolve/main/chemistry-assistant-13b-q5_k_m.gguf ``` **Note 3** If you want to use a checkpoint to generate a `lora.bin` file, use the following command. This is needed when you believe the final `lora.bin` is an overfit. ``` ../build/bin/finetune --model-base llama-2-13b-chat.Q5_K_M.gguf --checkpoint-in checkpoint-250.gguf --only-write-lora --lora-out lora.bin ``` --- ## LlamaParse # LlamaParse LlamaParse is an API created by LlamaIndex to efficiently parse and represent files for efficient retrieval and context augmentation using LlamaIndex frameworks. LlamaParse can support different kinds of files, like pdf, doc, .ppt, and other formats. You can configure LlamaParse to use the Gaia node as the LLM backend, hence you can create a RAG application based on your PDF files locally. ## Prerequisites You will need a Gaia node ready to provide LLM services through a public URL. You can * [run your own node](../../getting-started/quick-start/quick-start.md) * [use a public node](../../nodes/nodes.md) In this tutorial, we will use public nodes to power the Continue plugin. | Model type | API base URL | Model name | |-----|--------|-----| | Chat | https://llama8b.gaia.domains/v1 | llama | | Embedding | https://llama8b.gaia.domains/v1 | nomic-embed | ## Steps We will use an open-sourced GitHub repo, called `llamaparse-integration`, to make LlamaPase easy to use. The `llamaparse-integration` application supports * Multiple file formats, like `.pdf` and `.doc`, * Multiple files We will need to get the source code in your terminal first. ``` git clone https://github.com/alabulei1/llamaparse-integration.git cd llamaparse-integration ``` Next, install the required mode packages. ``` npm install llamaindex npm install dotenv ``` Start a Qdrant instance. The Qdrant instance is to store the embeddings. ``` mkdir qdrant_storage mkdir qdrant_snapshots nohup docker run -d -p 6333:6333 -p 6334:6334 \ -v $(pwd)/qdrant_storage:/qdrant/storage:z \ -v $(pwd)/qdrant_snapshots:/qdrant/snapshots:z \ qdrant/qdrant ``` Then, we will need to set up the LLM model settings. We can configure the model setting in the `.env` file. ``` OPENAI_BASE_URL=https://llama8b.gaia.domains/v1/ OPENAI_API_KEY=gaianet LLAMAEDGE_CHAT_MODEL=llama LLAMAEDGE_EMBEDDING_MODEL=nomic LLAMA_CLOUD_API_KEY=Your_Own_KEY FILE_PATH= FILE_DIR=./pdf_dir COLLECTION_NAME=default QDRANT_URL=http://127.0.0.1:6333 SAVE_MARKDOWN_PATH=output.md ``` Here are some notes about the `.env` setting: * You can get the LlamaCloud key from https://cloud.llamaindex.ai * You may need to make changes according to your model setting and file path. * If you put your file name in the `FILE_PATH=`, the program will build a RAG application with this single pdf file. * If the `FILE_PATH=` is empty, the program will build a RAG application with the files under the `FILE_DIR=./pdf_dir`. You can include multiple files in the folder. Next, we can run the program to build an RAG application based on the PDF file ``` npx tsx pdfRender.ts ``` After it runs successfully, you can send a query via the command line. ![](/img/docs/llamaparse-01.png) --- ## Prompt Engineering Tool # Prompt Engineering Tool ## Video Guide ### Test your AI Agents with multiple LLMs using Gaia's Prompt Engineering Tool Are you working with AI language models? This tool is a game-changer! It lets you: - ✨ Test Prompts on Multiple AI Models: Quickly compare results side-by-side to see which model performs best. - 🔄 Save and Reuse Prompts: Store your favorite prompts for easy access anytime. - 🔧 Use Variables for Flexibility: Customize your prompts with variables to suit any use case. Whether you're fine-tuning, experimenting, or just exploring the capabilities of different models, this tool will make your workflow smoother and more efficient. Try it out now and see how easy it is to get the most out of your AI language models! 🔗 [https://prompt-engineering-toolkit-rho.vercel.app/](https://prompt-engineering-toolkit-rho.vercel.app/) --- ## Calling external tools # Calling external tools Tool calling is one of the truly "LLM native" interaction modes that has never existed before. It gives the "thinking" LLMs the ability to "act" -- both in acquiring new knowledge and in performing real world actions. It is a crucial part of any agentic application. Open source LLMs are increasingly good at using tools. The Llama 3 models have now made it possible to have reliable tool calling performance on 8b class of LLMs running on your own laptop! In this tutorial, we will show you a simple Python program that allows a local LLM to run code and manipulate data on the local computer! ## Prerequisites You will need a Gaia node ready to provide LLM services through a public URL. You can * [run your own node](../../getting-started/quick-start). You will need to start a Gaia node for the [Llama 3.3 70B model](https://github.com/GaiaNet-AI/node-configs/tree/main/llama-3.3-70b-instruct) or the [Llama 3.1 8B model](https://github.com/GaiaNet-AI/node-configs/tree/main/llama-3.1-8b-instruct) or the [Llama 3 Groq 8B model](https://github.com/GaiaNet-AI/node-configs/tree/main/llama-3-groq-8b-tool). You can then use the node's API URL endpoint and model name in your tool call apps. * [use a public node](../../nodes) In this tutorial, we will use a public Llama 3.3 node with the function call support. | Attribute | Value | |-----|--------| | API endpoint URL | https://llama70b.gaia.domains/v1 | | Model Name | llama70b | | API KEY | gaia | ## Run the demo agent The [agent app](https://github.com/second-state/llm_todo) is written in Python. It demonstrates how the LLM could use tools to operate a SQL database. In this case, it starts and operates an in-memory SQLite database. The database stores a list of todo items. Download the code and install the Python dependencies as follows. ``` git clone https://github.com/second-state/llm_todo cd llm_todo pip install -r requirements.txt ``` Set the environment variables for the API server and model name we just set up. ``` export OPENAI_MODEL_NAME="llama" export OPENAI_BASE_URL= "https://llama8b.gaia.domains/v1" ``` Run the `main.py` application and bring up the command line chat interface. ``` python main.py ``` ## Use the agent Now, you can ask the LLM to perform tasks. For example, you can say ``` User: Help me to write down it I'm going to have a meeting with the marketing team. ``` The LLM understands that you need to insert a record into the database and returns a tool call response in JSON. ``` Assistant: {"id": 0, "name": "create_task", "arguments": {"task": "have a meeting with the marketing team"}} ``` The agent app (i.e., `main.py`) executes the tool call `create_task` in the JSON response, and sends back the results as role `Tool`. You do not need to do anything here as it happens automatically in `main.py`. The SQLite database is updated when the agent app executes the tool call. ``` Tool: [{'result': 'ok'}] ``` The LLM receives the execution result and then answers you. ``` Assistant: I've added "have a meeting with the marketing team" to your task list. Is there anything else you'd like to do? ``` You can continue the conversation. To learn more about how tool calling works, see [this article](https://github.com/LlamaEdge/LlamaEdge/blob/main/api-server/ToolUse.md). ## Make it robust One of the major challenges for LLM applications is the frequent unreliability of their responses. For example: *If the LLM generates an incorrect tool call that fails to address the user’s query,* you can refine and optimize the descriptions for each tool call function. The LLM chooses its tools based on these descriptions, so it's vital to craft them in a way that aligns with typical user queries. *If the LLM hallucinates and produces tool calls with non-existent function names or incorrect parameters,* the agent app should identify this issue and prompt the LLM to create a new response. Tool calling is a fundamental feature in the evolving field of agentic LLM applications. We’re eager to see the innovative ideas you bring forward! --- ## Agentic translation on Gaia # Agentic translation on Gaia Prof. Andrew Ng's [agentic translation](https://www.linkedin.com/posts/andrewyng_github-andrewyngtranslation-agent-activity-7206347897938866176-5tDJ/) is a great demonstration on how to coordinate multiple LLM "agents" to work on a single task. It allows multiple smaller LLMs (like Llama-3 or Gemma-2) to work together and produce better results than a single large LLM (like ChatGPT). [Gaia](https://www.gaianet.ai/), with 2000+ nodes running all kinds of finetuned LLms and knowledge bases, provides a huge opportunity for agentic apps to choose and use their own LLM backends. ## Introduction to the LLM Translation Agent This LLM Translation Agent is designed to facilitate accurate and efficient translation across multiple languages. It employs open source LLMs (Large Language Models) to provide high-quality translations. You can use your own fine-tuned models or any LLMs on Hugging Face like Meta's Llama 3. > For detailed commands on starting and running this agent, please visit [GitHub - Second State/translation-agent](https://github.com/second-state/translation-agent/blob/use_llamaedge/step-by-step-use-LocalAI.md). To get started, clone the Translation Agent. ``` git clone https://github.com/second-state/translation-agent.git cd translation-agent git checkout use_llamaedge ``` Next, we will install a local Gaia node, which provides the backend API services required by the agent. You can, of course, use [Gaia nodes from the community](../../nodes) if you do not want to start your own. ``` curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash ``` You will also need the following configurations and prerequisites to run the agent app. If you are using a Gaia node instead of your local node, replace the `http://localhost:8080` with `https://node_id.gaia.domains`. ``` export OPENAI_BASE_URL="http://localhost:8080/v1" export PYTHONPATH=${PWD}/src export OPENAI_API_KEY="GAIANET" pip install python-dotenv pip install openai tiktoken icecream langchain_text_splitters ``` > If you're using a Domain service, you will [need to get an API key from Gaia](../../getting-started/authentication). ## Demo 1: Running Translation Agents with Llama-3-8B First, let's run the translation agent with Meta AI's popular Llama-3 model. We select the smallest Llama-3 model (the 8b model) for this demo. The translation task is from Chinese to English. Our [source text](https://hackmd.io/tdLiVR3TSc-8eVg_E-j9QA?view#Source-text) is in Chinese, a brief intro to the ancient Chinese royal palace, the Forbidden City. ### Step 1.1: Run a Llama-3-8B Gaia node Configure and download the model. Since the size of the model is 5.73 GB. It can take a while to download. ``` gaianet init --config https://raw.githubusercontent.com/GaiaNet-AI/node-configs/main/llama-3-8b-instruct/config.json ``` Next, use the following command to start the Gaia node. ``` gaianet start ``` ### Step 1.2 Run the Translation Agent on top of Llama-3-8B Find the `examples/example_script.py` file in your cloned agent repo and review its code. It tells the agent where to find your document and how to translate it. Change the model name to the one you are using, here we’re using `Meta-Llama-3-8B-Instruct-Q5_K_M` model; also change the source and target languages you want (here we put `Chinese` as the source language and `English` as the target language). ``` import os import translation_agent as ta if __name__ == "__main__": source_lang, target_lang, country = "Chinese", "English", "Britain" relative_path = "sample-texts/forbiddencity.txt" script_dir = os.path.dirname(os.path.abspath(__file__)) full_path = os.path.join(script_dir, relative_path) with open(full_path, encoding="utf-8") as file: source_text = file.read() print(f"Source text:\n\n{source_text}\n------------\n") translation = ta.translate( source_lang=source_lang, target_lang=target_lang, source_text=source_text, country=country, model="Meta-Llama-3-8B-Instruct-Q5_K_M", ) print(f"Translation:\n\n{translation}") ``` Then, you can find an `examples/sample-texts` folder in your cloned repo. Put the file you want to translate in this folder and get its path. Here, because we named our source text `forbiddencity.txt`, the relative path to the document would be `sample-texts/forbiddencity.txt`. Run the below commands to have your text file translated into English. ```bash cd examples python example_script.py ``` Wait for several minutes and you will have [a fully translated version](https://hackmd.io/tdLiVR3TSc-8eVg_E-j9QA?view#English-Translation-by-Llama-3-8B) appear on your terminal screen. ## Demo 2: Running Translation Agents with gemma-2-27b The benefit of running the Translation Agent with Gaia is the ability for users to choose and embed different LLMs for different agentic tasks. To demonstrate this point, we will now change the translation agent LLM from Llama-3-8b to Google's gemma-2-27b, which is of similar size but scores higher on many language-related benchmarks. The translation task is the same as before. Our [source text](https://hackmd.io/tdLiVR3TSc-8eVg_E-j9QA?view#Source-text) is in Chinese, a brief intro to the ancient Chinese royal palace, the Forbidden City. The translation target is English. ### Step 2.1 Run a gemma-2-27b Gaia node Configure and download the model. Since the size of the model is 6.40G, it could take a while to download. ``` gaianet init --config https://raw.githubusercontent.com/GaiaNet-AI/node-configs/main/gemma-2-27b-it/config.json ``` Next, use the following command to start the Gaia node. ``` gaianet start ``` ### Step 2.2 Run the Translation Agent to run on top of gemma-2-27b Find the `examples/example_script.py` file in your cloned agent repo and review its code. It tells the agent where to find your document and how to translate it. Change the model name to the one you are using, here we’re using `gemma-2-27b-it-Q5_K_M` model; also change the source and target languages you want (here we put `Chinese` as the source language and `English` as the target language). ``` import os import translation_agent as ta if __name__ == "__main__": source_lang, target_lang, country = "Chinese", "English", "Britain" relative_path = "sample-texts/forbiddencity.txt" script_dir = os.path.dirname(os.path.abspath(__file__)) full_path = os.path.join(script_dir, relative_path) with open(full_path, encoding="utf-8") as file: source_text = file.read() print(f"Source text:\n\n{source_text}\n------------\n") translation = ta.translate( source_lang=source_lang, target_lang=target_lang, source_text=source_text, country=country, model="gemma-2-27b-it-Q5_K_M", ) print(f"Translation:\n\n{translation}") ``` Then, you can find an `examples/sample-texts` folder in your cloned repo. Put the file you want to translate in this folder and get its path. Here,because we named our source text `forbiddencity.txt`, the relative path to the document would be `sample-texts/forbiddencity.txt`. Run the below commands to have your text file translated into English. ``` cd examples python example_script.py ``` You can find the translated result in English [here](https://hackmd.io/tdLiVR3TSc-8eVg_E-j9QA?view#English-Translation-by-gemma-2-27b). ## Demo 3: Running Translation Agents with Phi-3-Medium long context model The Llama-3 and Gemma-2 models are great LLMs, but they have relatively small context windows. The agent requires all text to fit into the LLM context window, and that limits the size of articles they can translate. To fix this problem, we could select an open source LLM with a large context window. For this demo, we choose Microsoft's Phi-3-medium-128k model, which has a massive 128k (over 100 thousand words or the length of several books) context window. We run [a lengthy Chinese article on Forbidden City's collaboration with the Varsaille Palace](https://hackmd.io/vuFYZTVsQZyKmkeQ3ThZQw?view#Source-text) through our Translation Agent powered by a Phi-3-medium-128k model we start locally. ### Step 3.1: Run a Phi-3-medium-128k Gaia node Configure and download the model. ``` gaianet init --config https://raw.githubusercontent.com/GaiaNet-AI/node-configs/main/phi-3-medium-instruct-128k/config_full.json ``` Next, use the following command to start the Gaia node with a 128k context window. ``` gaianet start ``` ### Step 3.2 Clone and run the Translation Agent on top of Phi-3-medium-128k Find the `examples/example_script.py` file in your cloned agent repo and review its code. It tells the agent where to find your document and how to translate it. Change the model name to the one you are using, here we’re using `Phi-3-medium-128k-instruct-Q5_K_M` model; also change the source and target languages you want (here we put `Chinese` as the source language and `English` as the target language). ``` import os import translation_agent as ta if __name__ == "__main__": source_lang, target_lang, country = "Chinese", "English", "Britain" relative_path = "sample-texts/long_article.txt" script_dir = os.path.dirname(os.path.abspath(__file__)) full_path = os.path.join(script_dir, relative_path) with open(full_path, encoding="utf-8") as file: source_text = file.read() print(f"Source text:\n\n{source_text}\n------------\n") translation = ta.translate( source_lang=source_lang, target_lang=target_lang, source_text=source_text, country=country, model="Phi-3-medium-128k-instruct-Q5_K_M", ) print(f"Translation:\n\n{translation}") ``` Then, you can find an `examples/sample-texts` folder in your cloned repo. Put the file you want to translate in this folder and get its path. Here, because we named our source text `long_article.txt`, the relative path to the document would be `sample-texts/long_article.txt`. ``` cd examples python example_script.py ``` [The translated results were impressive,](https://hackmd.io/vuFYZTVsQZyKmkeQ3ThZQw?view#Source-text) with the translation capturing the nuances and context of the original text with high fidelity. ## Evaluation of Translation Quality The three models, Llama-3-8B, gemma-2-27b, and Phi-3-medium, have exhibited varying levels of performance in translating complex historical and cultural content from Chinese to English. Llama-3-8B provides a translation that effectively captures the factual content but shows occasional stiffness in language, possibly indicating a direct translation approach that doesn't fully adapt idiomatic expressions. It does not keep section title and the format of the original text and left certain part untranslated. In contrast, The translation by gemma-2-27b is quite accurate and retains the original meaning of the short intro article of Forbidden city. gemma-2-27b's translation exhibits a smooth and natural English flow, suggesting a sophisticated understanding of both the source language and the target language’s grammatical structures. The choice of words and sentence structures in gemma-2-27b's output demonstrates a high degree of linguistic finesse, suggesting it might be well-suited for translating formal and historically nuanced texts. The Phi-3-medium-128k model can translate book-length text from Chinese to English. It demonstrates robust capabilities in handling large volumes of complex content, suggesting advanced memory handling and contextual awareness. The quality of translation remains consistent even with increased text length, indicating Phi's utility in projects requiring extensive, detailed translations. But you can see it makes certain mistakes like mistaken "Wenhua Hall" as "also known as Forbidden City" in the first paragraph. Overall, each model has its strengths, with gemma-2-27b standing out for linguistic finesse and Phi-3-medium-128k for handling lengthy texts. ## Conclusion [Gaia](https://github.com/GaiaNet-AI) provides an easy way to select and use different open-source LLMs in your agentic applications to fully take advantage of their finetuned capabilities for specific tasks. Once you have a local Gaia node up and running, you could share it with others and make $$$ by joining the [Gaia network](https://www.gaianet.ai/)! --- ## Redirecting to Whitepaper... import Head from '@docusaurus/Head'; You are being redirected to the [Gaia Whitepaper](https://whitepaper.gaianet.ai/?ref=docs). Click the link if you are not redirected.