SlackBolt is Underrated: Building an AI Knowledge Bot for Your Team
I usually start my blog posts with something that dropped recently, but today I want to highlight a library that’s been around for a while and still provides massive value while being surprisingly underrated. I’m talking about SlackBolt.(opens in a new tab)
Picture this: You’re at a company with a giant, difficult-to-manage knowledge base spread across multiple platforms — Confluence,(opens in a new tab) Notion,(opens in a new tab) internal wikis, and scattered documentation. You’ve been there before — frantically searching for answers while a potential client waits (im)patiently on the line. We’ve all made it through situations like that as a team, but there has to be a better way.
SlackBolt has been sitting in my GitHub stars for MONTHS because I kept seeing it in random documentation and thinking “oh cool, another Slack thing” — completely missing the point. Then I actually read the docs during a late-night rabbit hole session and realized this thing is basically a Swiss Army knife disguised as a butter knife.
The SDK is genuinely beautiful — like, whoever designed the Python interface actually cares about developer experience. Compare this to wrestling with webhooks and parsing Slack’s Event API manually, or trying to make Zapier do something it wasn’t designed for. SlackBolt feels like someone finally said “what if developers could just… write code?”
So I threw together a prototype using SlackBolt, Atlassian’s MCP Server, and an MCP Agent to show how easy (and fun!) it can be to home roll useful Q&A bots. MCP (Model Context Protocol)(opens in a new tab) basically standardizes how AI agents talk to external systems — the Atlassian server means I can search Confluence without writing API integration code. SlackBolt + MCP + some routing logic = internal knowledge assistant that actually works.
I’m actually going to walk you through this ~
Our Slackbot will:
Accept questions via a /ai command
Route queries intelligently based on content
Search through Confluence (or other knowledge bases)
Provide contextual answers with conversation memory
Scale to handle multiple knowledge sources
Architecture Overview
I’m a novice developer and wanted to make sure that my architecture was scalable and easily extendable due to the varying number of requests I get for these bots.
There’s definitely more advanced hybrid strategies to get better results than the Confluence search and get page tools provide, this was a POC so please extend and modify then share what you do!
Slackbolts framework made this super easy to build out once I got my slackbot connected locally — deployment was a little more tricky but will easily run in a Docker ;-)
Main Components:
Slack Front-End — Accepts /ai queries and forwards them to the processing pipeline
Router LLM — Classifies messages using a lightweight model
Context Builder — Dynamically assembles the correct prompt and knowledge base
MCP Agent — Runs with various tools (Confluence, etc.) to fetch information
Memory System — Maintains conversation context across threads
Tutorial: Building Your Own AI Slack Bot
Step 1: Setting Up SlackBolt
READ THE DOCS — [https://tools.slack.dev/bolt-python/(opens in a new tab)]
There’s SOOO much you can do — have fun. Below is a super basic outline of what I did.
First, let’s set up the basic Slack application: Here you make sure the Slack app can listen and respond to Slack messages. You can also control interactivity here ~ I made my bot start a new thread and continue the conversation with a new UUID to keep track over multiple threads of conversation history.
# slack-bolt.py
import os
import logging
from slack_bolt import App
from slack_bolt.adapter.socket_mode import SocketModeHandler
# Initialize your app with your bot token and socket mode handler
app = App(token=os.environ.get("SLACK_BOT_TOKEN"))
@app.command("/ai")
def handle_ai_command(ack, respond, command):
ack()
user_query = command['text']
thread_ts = command.get('thread_ts', command['ts'])
# Process the query (we'll implement this next)
response = generate_response(user_query, thread_ts)
respond(response, thread_ts=thread_ts)
if __name__ == "__main__":
handler = SocketModeHandler(app, os.environ["SLACK_APP_TOKEN"])
handler.start()Step 2: Implementing the Router System
Create a routing system that classifies queries (use a lighter-weight model for this, I picked 4.1-mini):
# bot.py
import openai
from typing import Dict, Any
# Define your categories
CATEGORIES = {
"general": {
"prompt_file": "prompts/general.txt",
"kb_file": None,
"description": "For general company questions and policies",
"aliases": ["general", "company", "policy", "process"],
"context_prefix": "**Answering a general company question.**"
},
"technical": {
"prompt_file": "prompts/technical.txt",
"kb_file": "knowledge_bases/tech_kb.txt",
"description": "For technical documentation and development questions",
"aliases": ["tech", "development", "api", "code", "deployment"],
"context_prefix": "**Answering a technical question. Here is relevant context:**"
}
}
async def get_query_category(user_message: str) -> str:
"""Route the user query to the appropriate category"""
# Build router prompt dynamically
router_prompt = """
You are a query router. Classify the user's question into one of the following categories and respond with ONLY the category name:
"""
for category, config in CATEGORIES.items():
router_prompt += f"\n- '{category}': {config['description']}"
response = await openai.AsyncOpenAI().chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "system", "content": router_prompt},
{"role": "user", "content": user_message}
],
temperature=0,
)
category = response.choices[0].message.content.strip().lower()
return category if category in CATEGORIES else "general"Step 3: Setting Up MCP Integration
Install the MCP agent and configure it for your knowledge sources:
pip install mcp-agent# mcp_integration.py
from mcp_agent import MCPApp, Agent
from mcp_agent.models import Settings, MCPSettings, MCPServerSettings, OpenAISettings
def setup_mcp_agent():
"""Configure MCP agent with Confluence tools"""
# Configure Atlassian/Confluence server
atlassian_args = [
"mcp-atlassian",
f"--confluence-url={os.environ.get('CONFLUENCE_URL')}",
f"--confluence-username={os.environ.get('CONFLUENCE_USER')}",
f"--confluence-api-token={os.environ.get('CONFLUENCE_TOKEN')}"
]
settings = Settings(
execution_engine="asyncio",
mcp=MCPSettings(
servers={
"atlassian": MCPServerSettings(
command="uvx",
args=atlassian_args
)
}
),
openai=OpenAISettings(
api_key=os.environ.get("OPENAI_API_KEY"),
default_model="gpt-4o"
),
)
return MCPApp(name="knowledge_bot", settings=settings)Step 4: Adding Conversation Memory
Implement a memory system to maintain context across conversations:
# memory.py
from typing import Dict, List, NamedTuple
from datetime import datetime, timedelta
import threading
class ConversationMessage(NamedTuple):
role: str
content: str
timestamp: datetime
class ConversationMemory:
def __init__(self, max_messages: int = 10, ttl_hours: int = 24):
self.conversations: Dict[str, List[ConversationMessage]] = {}
self.max_messages = max_messages
self.ttl_hours = ttl_hours
self.lock = threading.Lock()
def add_message(self, thread_id: str, role: str, content: str):
"""Add a message to the conversation history"""
with self.lock:
if thread_id not in self.conversations:
self.conversations[thread_id] = []
message = ConversationMessage(
role=role,
content=content,
timestamp=datetime.now()
)
self.conversations[thread_id].append(message)
# Keep only the most recent messages
if len(self.conversations[thread_id]) > self.max_messages:
self.conversations[thread_id] = self.conversations[thread_id][-self.max_messages:]
def get_conversation_history(self, thread_id: str) -> str:
"""Get formatted conversation history"""
if thread_id not in self.conversations:
return ""
history = []
for msg in self.conversations[thread_id]:
history.append(f"{msg.role}: {msg.content}")
return "\n".join(history)
# Global memory instance
memory = ConversationMemory()Step 5: Putting It All Together
Now let's create the main response generation function:
# response_generator.py
async def generate_response(message: str, thread_id: str) -> str:
"""Generate AI response to user query"""
# Get conversation history
conversation_history = memory.get_conversation_history(thread_id)
# Route the query
category = await get_query_category(message)
config = CATEGORIES[category]
# Load prompt and knowledge base
prompt = load_prompt(config["prompt_file"])
kb_content = load_knowledge_base(config["kb_file"]) if config["kb_file"] else ""
# Build final message with context
final_message = f"""
{config['context_prefix']}
{kb_content}
Conversation History:
{conversation_history}
Current Question: {message}
"""
# Execute with MCP agent
mcp_app = setup_mcp_agent()
async with mcp_app:
finder_agent = Agent(
name="knowledge_finder",
instruction=prompt,
server_names=["atlassian"]
)
async with finder_agent:
from mcp_agent.models import OpenAIAugmentedLLM
llm = await finder_agent.attach_llm(OpenAIAugmentedLLM)
result = await llm.generate_str(message=final_message)
# Store the interaction in memory
memory.add_message(thread_id, "User", message)
memory.add_message(thread_id, "Assistant", result)
return result
def load_prompt(filename: str) -> str:
"""Load prompt from file"""
try:
with open(filename, 'r') as f:
return f.read()
except FileNotFoundError:
return "You are a helpful assistant that answers questions based on available knowledge."
def load_knowledge_base(filename: str) -> str:
"""Load knowledge base content"""
if not filename:
return ""
try:
with open(filename, 'r') as f:
return f.read()
except FileNotFoundError:
return ""Extending Your Bot
This is where the bot truly shines. Implement custom prompts and knowledge bases and pass the responses to those inferences to your main MCP Agent (or not) to extend the knowledge base to artifacts that are not in your company's main knowledge platform.
To add new capabilities to your bot:
Create new prompt files in a prompts/ directory
Add knowledge base files in a knowledge_bases/ directory
Register new categories in the CATEGORIES dictionary
Define routing keywords in the aliases list
Example of adding a new category:
"sales": {
"prompt_file": "prompts/sales.txt",
"kb_file": "knowledge_bases/sales_kb.txt",
"description": "For sales process and customer questions",
"aliases": ["sales", "customer", "pricing", "demo"],
"context_prefix": "**Answering a sales question. Here is relevant context:**"
}Production Considerations
Plug into LangSmith (or whatever you use for LLM Traces) and trace all your runs.
Evaluate your prompts for best results.
Test against real questions people ask at your company.
All of this can run in a Docker container —
This code won’t fix a lackluster knowledge base for your company; you still need quality content.
Store your secrets safely!!!
Resources to Get Started
People should start building now to take advantage of the vacuum in the space. There are so many opportunities to level up your career and get un-stuck leveraging AI-Coding assistants to help you through this stuff.
Quick start: Copy and paste this blog post into Cursor. 😉
Happy Coding!
The combination of SlackBolt and MCP creates a powerful foundation for building intelligent workplace assistants. With this architecture, you can easily extend your bot to work with any system that has an MCP server implementation, making it a scalable solution for growing teams and evolving knowledge bases.
Do you want to build code that helps entrepreneurs and small businesses grow with confidence? We’re hiring across our Technology teams, come build with us! Check out our Careers page.





