What We Learned Building a Chatbot for La Silla Vacía’s Newsroom Archive

The logic of the steps was correct, but implementing it with many agents that only passed information to each other through inputs and outputs created problems. The agent that identified the type of question didn’t pass enough context to the agent that searched — only the type and the original text — so at each step the next agent lost information. It was a leaky process.

We decided to make the move to semantic search just as autonomous agents were starting to gain ground. When we started, reasoning models were barely in beta; six months later they were largely accepted technology. That leap gave us the occasion to redesign the logic.

We kept the same underlying logic (safety, predetermined answers, type identification, iterative search with ranking, and answer), but we changed how the work is distributed. Instead of separate models, each with its own context, we set up most of those steps as tasks for a single autonomous agent, which calls sub-agents in specific cases. Both versions were multi-agent, but we went from a linear scheme, where each agent did one step and handed it to the next, to a hierarchical one, where one large agent runs most of the flow and asks sub-agents for help with specific tasks.

For the runtime, we used Mastra, a LangChain-style agent framework, built entirely in TypeScript and very lightweight. We chose it because the Python runtime didn’t work well for us on Vercel. Mastra is model-agnostic, so it allowed us to test many models and change them fast. We started testing in Opus, and then launched in GPT 5.1. Right now, we are using a mix of GPT 5.4 with high thinking and effort for the main agent and GPT 5.4 nano with the same settings for specific tasks in the process. We are constantly monitoring and changing the models, and the next step is evaluating a change to GPT frontier models 5.5 Sol and Luna.

The system has three components: tools for running searches, a main agent that answers and orchestrates, and sub-agents for specialized searches.

Before the main agent starts operating, the question passes through a separate filter: the one for predetermined answers. Resolving them outside the agent process makes them much faster and cheaper: the correct answer is already written and doesn’t have to be produced again.

If there’s no match, the main agent takes over. It receives the question, classifies it, and, according to its type, decides which database to consult. The routing rule is simple: the agent directly queries La Silla Vacía’s semantic database hosted in Elasticsearch, which can answer about 90% of the questions.

Searches that require more complex queries in a database like MongoDB or SQL are delegated to a sub-agent, almost always with a smaller model. That’s where you’ll find, for example, historical election results, La Silla Vacía’s directory of journalists, and the database of congressional candidates. After receiving the information from a tool or a sub-agent, the main agent verifies that it has everything it needs and drafts the answer.

This contextual analysis allows steps that used to be separate to work together in an integrated way. For example, the safety filter is no longer an independent step but part of the question-type verification carried out by the same agent. This allows the agent, using the context provided by the tools, to determine whether a query is safe or not.

Image: Courtesy of Generative AI in the Newsroom Medium channel

The Obsession with Measuring

A project like this rests on layers. At the base, a leadership that decided to bet on it and keep it running. On top of that, the infrastructure that makes it possible. And above the two, resting on both, the routine that has given it direction this first year: measuring, nonstop, from day one.

We track how many interactions we get, how much they cost us and, most revealingly, what people ask us. In its first twelve months, SillaIA answered 26,011 questions from about 20,000 unique users: an average of about 2,167 questions a month. Our busiest day was June 21, 2026, the second round of the presidential election, with 1,020 questions in 24 hours, 14 times a normal day.

Those numbers describe the traffic. The type of question describes the reader. It works like a compass: it shows us how the audience is understanding the product and guides decisions about its direction. Every week we classify the questions with AI and validate the patterns manually. The project lead reviews the spreadsheet, defines the actions to follow and, through a color code, each team member can identify at a glance which tasks require their attention.

That listening changes the product. With the legislative elections upon us, for example, we noticed that users were looking for candidates, list numbers, and parties, but in our editorial calendar the profiles of the congressional candidates were still weeks away from publication. So that no one would leave empty-handed, we prepared a basic database listing who was running for the Senate and the House, which party they represented, and their ballot number. We integrated it as a temporary tool, the kind we could retire once the news moment had passed.

Election night for the legislative races brought something we hadn’t anticipated: users asked SillaIA for real-time results. We weren’t ready. But the question, repeated, flagged a gap we could fill for the presidential elections, and in the first and second rounds we offered real-time vote-count data, at the national and departmental level, with another temporary tool set up for those days.

Not everything we added was temporary. That same listening led us to integrate sources that stayed: the Detector de Mentiras (“Lie Detector,” our fact-checking section) and Silla Amazonía (our Amazon-region section).

The listening also changes what we report. Recurring questions have turned into journalistic pieces like these:

• Respuestas a las preguntas más importantes sobre las elecciones hechas a la SillaIA. (Answers to the Most Important Election Questions Asked to SillaIA)
• Respuestas a las preguntas más frecuentes sobre los escrutinios hechas a la SillaIA. (Answers to the Most Frequently Asked Questions About Vote Counts Asked to SillaIA)
• Elecciones 2026: guía completa para votar este 8 de marzo. (Colombia 2026 Elections: Complete Guide to Voting This March 8)

We record some of these changes in a public changelog, which we set up when the team itself pointed out that it was hard to stay on top of everything we kept modifying.

• Bitácora de SillaIA: novedades y cambios, mes a mes. (SillaIA Log: Updates and Changes, Month by Month)

How We Contained Costs

When we started, we didn’t know how to estimate the costs of a project like this. We talked with other organizations that had similar projects to understand how they’d contained theirs, and we made our own projections: how much it would cost if a portion of readers, or of the Superamigos (“Super-friends,” the members of our membership program), used up all their questions each month, or what would happen if someone with bad intentions wanted to drain our tokens in the middle of the night.

That’s where the limits came from: 10 questions a month for any reader, 50 for Superamigos, and a separate quota for journalists, editors, and the team that tests the system. Also, a monthly spending cap on the LLMs, with alerts at certain thresholds, and specific tagging of generation and querying that has let us track usage and cost for each step of the process and understand the bottlenecks.

The tiers are an invitation to support us, yes, but above all the way to move within a limited budget without being exposed to an impossible bill.

The Product That Never Ends

There’s a silent expectation in almost all software: it gets built, launched, and then enters a phase of light support, low-cost maintenance, occasional tweaks. With SillaIA that phase never arrived. And we’ve now understood that it never will.

Model updates hit us in the face, almost always when we least expect it. A provider changes a version, deprecates another, modifies a price, or adjusts the way the system responds. The worst part isn’t the fix itself. It’s that those changes don’t fit into any schedule. We can’t plan which week of the quarter we’re going to have to drop what we were doing to deal with the ground shifting. It arrives, and we have to.

The practical consequence is that SillaIA goes back to the workbench every time a piece we don’t control changes. It’s not a product we maintain. It’s a product we partly remake, over and over. That changes the math of what it costs to operate AI in a newsroom: the expense isn’t only in building or in the monthly bill, but in sustaining a team capable of noticing in time that something broke and fixing it before a reader gets a wrong answer about an election.

Acknowledgements

This project would not have been possible without the support of the Konrad Adenauer Foundation (KAS) or the active involvement of several colleagues. Iván Montes, La Silla Vacía’s tech lead, built the front end. Laura Torrado, an editor on our team, contributed her judgment throughout the process. And Juan Romero from the technology team supported multiple rounds of testing and set up the predetermined questions.

Editor’s Note: This story was originally published on the Generative AI in the Newsroom Medium channel and is reprinted here with permission. 

Karen De la Hoz is the AI lead at La Silla Vacía and teaches new technologies at the University of Cartagena, Colombia. She also runs the Generative AI in the Newsroom Medium channel and the blog Noches de Media, which explores the intersection between journalism, technology, and sociology.

Pablo Manrique is an economist and data scientist with experience in startups and consultancies, specializing in data analysis and artificial intelligence development. He uses artificial intelligence and data mining for writing and analysis, with a focus on economic issues. He is also the back-end developer of La Silla Vacia AI Lab.