SpeedyBot

Appearance

🔗 Integrate with Azure ​

This guide will show you how to set up SpeedyBot on your local machine and connect it to a large language model managed by Azure. SpeedyBot streamlines the process of designing, deploying, and securing sophisticated conversation systems, tailored for enterprises and large teams with complex needs.

Step 0: Get your Azure details ​

To authenticate with Azure, you'll need to obtain some key data. Your organization might be able to provide this information.

sh
BOT_TOKEN=__REPLACE__ME__
OAUTH_ENDPOINT=https://yourDomain.com/oauth2/default/v1/token
CLIENT_ID=aabbccddeeffgghhiijjkk
CLIENT_SECRET=abcd1234567890987654321

BASE_URL_LLM=https://{your-resource-name}.openai.azure.com/openai
MODEL=gpt-35-turbo
APP_KEY=app-name-here
API_VERSION=2023-12-01-preview

Further reading:

Step I: Grab Your Access Token ​

The flow to get a token will look roughly like this:

Validate Token ​

Once you've got your token, pop it into the box below to validate it & review your bot's details

Is this safe??

Your bot token is a highly sensitive credential and should be protected with encryption and proper secrets management.

SpeedyBot does NOT log/persist or do anything (except what you tell it to do) with your bot token.

REMEMBER: If your agent's access token is ever compromised/exposed, you can always invalidate it + get a new one by tapping "Regenerate Access Token" under your agent's settings page

Step II: Get your System Ready ​

To keep things simple at the start, you'll run the bot from your own machine (meaning when your computer is off, your bot is "off" too). Later, if needed, you can deploy your bot to virtually any standard server or scalable serverless cloud infrastructure you prefer

Use Bun (easy)

Getting Started with Bun on Your Computer ​

Bun is a tool that helps make working with JavaScript faster and easier. It's like a special tool that helps you run JavaScript code really quickly. You can find helpful guides and explanations about Bun here.

Setting up Bun is simple. It ships as a single executable that you can easily install on your computer. If you're not interested in using Bun, that's okay too. SpeedyBot works with many runtimes and infrastructure-- see the NodeJS instructions by tapping the toggle above

Enter the commands below in your terminal to get up and running with Bun:

sh
curl -fsSL https://bun.sh/install | bash
sh
powershell -c "irm bun.sh/install.ps1 | iex"

Verify that Bun is installed correctly and available from your terminal-- if you can run bun --version in your terminal and you see a version number you're good to go!

💡 Tips for Windows Users + Bun Questions

Bun is an experimental high-performance JavaScript runtime and toolkit that combines a package manager, bundler, and task runner into one tool.

Note: Install scripts are available for inspection below

SpeedyBot can run on any Javascript/Typescript runtime or infrastructue-- Bun just happens to be a speedy and easy way to get up and running

Note for Mac Users ​

If you're running an older version of Mac OS and you encounter errors when installing Bun try upgrading your operating system

Note for Windows Users ​

On PC if you're having trouble getting Bun working on Windows/Powershell try the following method instead:

Step 1: Download and run Git Bash https://git-scm.com/download/win

Note: Step-by-step instructions available here: https://www.git-tower.com/blog/git-bash/

Step 2: Run the following command inside Git Bash to setup Bun:

curl -fsSL https://bun.sh/install | bash

Step 3: Completely close out from Git Bash

Step 4: Re-open Git Bash again and try Step III

Step III: Boot it up! ​

Copy the command below into your terminal to turn on your bot

sh
bunx --bun create-speedybot@2.0.9 setup -e OAUTH_ENDPOINT -e BASE_URL_LLM -e MODEL -e CLIENT_ID -e CLIENT_SECRET -e APP_KEY -e API_VERSION --project azure --boot --bun --install
sh
git clone --depth 1 https://github.com/valgaze/speedybot
cd speedybot
cd examples/azure
bun install
bun util/cli.ts setup __ACCESS__TOKEN__HERE__
cp .env.example .env # Fill out .env with real values
bun --watch util/launch.ts

You can turn off your bot by holding down CTRL-C on your keyboard or exiting the terminal. To turn your bot back "on", open your terminal to your project directory and enter bun run dev

Note: when you press CTRL-C the terminal will display the location of your bot

Step IV: Customize your agent ​

Once your bot is loaded, you can customize the bot.ts file to choose which experience you want to deliver to your users. If you've never edited a bot before see here for the basics

Under the first step in bot.ts you need to pick one of three LLM strategies to configure your bot's behavior with Azure (ie simple, threaded conversation threads, or streaming) hit save and the bot will reload:

Ex. If you selected "streamExample" response tokens will "stream" in to the chat client as they arrive (rather than forcing your users to wait for the full completion):

Things to try ​

  • Additionally, you can take a look at the system prompts and tune this starter sample to suit your requirements

  • Work with file uploads to see how to extract file data and get meta-data (file-name, file-type, size, etc), try uploading.

Upload a file and use the example code to extract file data like name, type, and size. Test with the "andy.txt" file to see how you can inject file content into a conversation context

Note: To customize your bot's behavior or its responses, you'll need an editor-- one popular choice is Visual Studio Code.

For installation details, visit https://code.visualstudio.com/download

It works great with SpeedyBot and even provides helpful hints as you build ex.

When you're ready to deploy it to a server, serverless function or virtually any infrastructure/device, check out the examples