How to Build an AI Agent: Building PerieAI Without a Framework
An engineering walkthrough of building a custom Agentic AI system using Python, FastAPI and LLM APIs.
Introduction
When people talk about building AI agents, the conversation often starts with frameworks such as LangChain, LangGraph, CrewAI or other agent orchestration tools. These frameworks provide useful abstractions and make it easier to build sophisticated systems.
But I wanted to explore a different approach.
How do you build an AI agent without depending on a heavyweight agent framework?
Could we build something practical using Python, an LLM API and a few well-defined components? Something that can collect information, make decisions, evaluate its own output and interact with external services?
That question led to the development of PerieAI.
PerieAI is a custom-built technical research and intelligence agent. It collects technology developments from configured sources, identifies meaningful topics, generates technical briefings and delivers approved results to Slack.
If you’ve been exploring how to build an AI agent, PerieAI is a practical example of designing one from scratch using Python and an LLM API.
The objective was not to build another chatbot or demonstrate a simple LLM integration. I wanted to build a focused agent with a defined responsibility and a controlled execution flow.
In this article, I’ll walk through the architecture, technology choices and how the different components work together.
GitHub: https://github.com/azeemuddinn/perie-ai
1. How to Build an AI Agent: Understanding the Basics
Before getting into the architecture, it is worth understanding what makes this different from a traditional LLM-powered application.
A typical LLM application takes an input, sends it to a model and returns the response. The application might produce a useful summary or answer, but the workflow generally depends on someone initiating each interaction.
An agent is designed around a responsibility that it can execute through a sequence of operations.
For PerieAI, that responsibility is technical research.
It needs to collect information, identify relevant developments, use an LLM to evaluate and generate insights, validate the resulting briefing and deliver it through an external service.
The LLM provides the intelligence, but the surrounding application controls how the work gets done.
This distinction is important. We are not handing the entire application over to an LLM. We are designing a system in which the model performs specific reasoning tasks while Python manages execution, data and integrations.
2. How to Build an AI Agent Without a Framework
There are several frameworks available for agent development, and they are useful when the application requires complex orchestration, multiple agents or sophisticated state management.
However, not every use case needs that level of abstraction.
PerieAI has a clearly defined responsibility and a relatively straightforward execution pipeline. Introducing a large framework at the beginning would add concepts and dependencies that we didn’t necessarily need.
I wanted direct control over the important parts of the system: how information is collected, how the LLM is called, how responses are validated, how quality is evaluated and when an external action is performed.
Building these components directly in Python also makes the architecture easier to understand and debug.
This doesn’t mean frameworks are unnecessary. It simply means that the choice of framework should follow the requirements of the application rather than becoming the starting point.
For PerieAI, a custom implementation was a reasonable starting point.
3. Technology Stack
The technology choices were intentionally simple. Each component has a specific responsibility, and the goal was to avoid introducing infrastructure that the application didn’t need.
| Component | Technology |
|---|---|
| Backend | Python, FastAPI |
| Data validation | Pydantic |
| Database | SQLite |
| LLM providers | Groq / Google Gemini |
| Structured responses | JSON |
| Agent instructions | Custom SKILL.md |
| Quality evaluation | LLM-as-a-Judge |
| Slack integration | Slack Web API |
| HTTP communication | httpx |
FastAPI provides the API layer through which the application can be triggered and tested. SQLite stores collected articles and application data.
Groq and Google Gemini provide the language model capabilities. Keeping the LLM interaction behind an application-level client also makes it easier to work with different providers.
Pydantic helps validate structured responses, while httpx handles communication with external services.
The important design decision here is separation of responsibilities. The application logic should not be tightly coupled to a particular LLM provider or external service.
When considering how to build an AI agent, one of the most important decisions is how to separate reasoning, execution and data management.
4. AI Agent Architecture: How PerieAI Works
The architecture follows a controlled pipeline.
Information enters the system through configured technology feeds. The collector retrieves articles and stores them in SQLite. The application then selects suitable topics for further processing.
The LLM performs the research and generates a structured technical briefing. This briefing is passed through an independent editorial evaluation stage.
If the briefing passes the quality gate, it is delivered to Slack. If it fails, it is rejected rather than being sent automatically.
The high-level architecture looks like this:
Technology Feeds → Collection → SQLite → Topic Selection → LLM Research → Brief Generation → Editorial Evaluation → Slack Delivery
The important thing to notice is that the LLM doesn’t directly control every part of this process.
The Python application orchestrates the workflow, manages the data and decides what happens next.
This gives us a predictable execution path while still using an LLM for tasks that require language understanding and judgement.

5. How to Build an AI Agent: The Execution Workflow
Let’s walk through what happens when PerieAI executes.
Step 1: Collect technology updates
The process starts with collecting articles and developments from configured technology feeds.
The collector retrieves the available information and stores it in SQLite. Maintaining a local record allows the application to track collected content and avoid repeatedly processing the same articles.
This also separates information collection from the actual research and evaluation process.
Step 2: Identify topics worth researching
Once the articles are collected, the application selects topics for further processing.
The goal is not to generate a summary of every article. Instead, we want to identify developments that have technical or engineering significance.
This is where the LLM becomes useful. It can interpret the content and help identify why a particular development matters beyond its headline.
Step 3: Generate the technical briefing
The selected development is passed to the LLM for research and briefing generation.
The output is structured rather than being an unrestricted block of text.
A typical briefing includes the topic, suggested title, explanation of its significance and a suggested outline.
Structured output is important because the next stages need predictable fields to work with.
Step 4: Evaluate the generated briefing
Generating a briefing doesn’t automatically make it suitable for delivery.
An LLM can produce fluent content that lacks technical depth or meaningful engineering insight.
To address this, PerieAI introduces a separate evaluation stage.
The generated briefing is passed to an editorial judge that assesses it against predefined criteria. These include technical depth, accuracy, engineering significance, practical relevance and overall quality.
The judge returns a structured evaluation with scores, feedback and a PASS or FAIL decision.
Step 5: Deliver the approved result
Only briefings that pass the quality gate proceed to delivery.
The application prepares a Slack message containing the technical briefing and sends it using the Slack Web API.
This completes the execution pipeline.
Instead of simply generating text and returning it to an API caller, PerieAI produces a useful result and delivers it to an external service.
6. Separating Agent Instructions from Application Code
One of the design decisions worth highlighting is the use of a custom SKILL.md file.
In many simple LLM applications, instructions are embedded directly inside Python code. As the application grows, these instructions can become difficult to manage and maintain.
For PerieAI, the editorial responsibilities and evaluation criteria are maintained separately from the application logic.
The SKILL.md file provides the instructions and standards that guide the agent’s research and editorial behaviour.
This separation gives us flexibility. We can refine the agent’s expectations without having to restructure the Python application every time an instruction changes.
The application remains responsible for execution, validation, storage and integrations.
The instructions define what the agent is expected to achieve.
This is a small but useful step towards making the system easier to maintain.
7. Moving Beyond a Simple LLM Integration
Looking at the complete implementation, there are several engineering decisions that make PerieAI more than a basic LLM wrapper.
First, the system has a defined responsibility. It is not designed to answer arbitrary questions but to perform a specific research and intelligence task.
Second, the workflow is controlled by application code. The LLM is used for research, interpretation and evaluation, while Python manages the overall execution.
Third, the system separates content generation from content validation. This introduces an additional quality gate before the application performs an external action.
Finally, the application has persistence and external integration. It can maintain collected information and deliver approved results beyond the immediate API response.
These decisions create a foundation that can be extended without making the initial architecture unnecessarily complicated.
8. What Would It Take to Move This Towards Production?
A working agent is a starting point, not the end of the engineering process.
Moving an AI agent towards production requires attention to areas beyond the LLM itself.
For PerieAI, some of the important considerations include reliable execution, retry handling, API rate limits, provider failures, structured logging, duplicate prevention and secure management of credentials.
The evaluation process also needs to be tested against a representative set of topics to understand how consistently it makes good decisions.
Scheduling and background execution would allow the application to run without requiring a manual API trigger.
These are the kinds of considerations that become increasingly important as the system evolves from a working implementation into a dependable service.
I deliberately started with a smaller architecture so that these capabilities can be introduced as actual requirements emerge.
Conclusion
Building PerieAI gave me a practical understanding of how to build an AI agent without depending on a heavyweight framework.
Building PerieAI was an opportunity to explore how to build an AI agent from an engineering perspective, without starting with an agent framework.
The interesting part wasn’t simply connecting Python to an LLM API. It was designing the surrounding system: how information enters, how the model evaluates it, how the output is validated and how the application performs an external action.
The experience reinforced something I believe is important when building AI-powered software: the LLM is only one component of the system. The surrounding architecture determines how useful, controlled and maintainable the application becomes.
You don’t always need a complex framework to get started. A focused use case, clear responsibilities, structured execution and appropriate quality controls can take you a long way.
PerieAI is a working exploration of that approach, and there is more to build.
The next step is to look deeper into the engineering behind it: how to control agent execution, manage state, evaluate quality and connect external services through standardised interfaces.
That’s what I’ll cover in Part 2.
Explore the source code: https://github.com/azeemuddinn/perie-ai
PerieAI — Less noise. More engineering intelligence.
Want to go deeper? Explore Part 2 — Agent Harnesses, Editorial Judges and MCP Integration.


