How to build a multi-stage AI pipeline for news monitoring and source verification

A multi-stage AI pipeline for news monitoring combines procedures in which a language model finds events, verifies sources, and prepares topics for an editor. Unlike a single large prompt, this approach distributes tasks across stages: each stage receives a limited area of responsibility, returns a structured result, and allows a repeated check.

In the author’s experiment, the news aggregator’s search funnel began with a general query and a selected period. The system then clarified the research direction, created a thematic map, identified event families, generated search queries, checked pages, and passed confirmed events on for editorial selection. The author considers the project a complex MVP for testing an idea, rather than a finished standalone product.

The main engineering conclusion: news research is best built as a chain of verifiable steps rather than as a single query to an LLM.

Why one prompt is not enough

A general prompt can ask the model to find news, determine its significance, check dates, evaluate sources, and write a final list all at once. In such a task, the model mixes search, interpretation, and editing. An error at an early stage then turns into a confident but unreliable final conclusion.

A multi-stage pipeline separates the roles. One step formulates the search task, another broadens coverage, a third discards irrelevant pages, a fourth verifies events, and the last selects topics for publication. The result of each step can be reviewed, saved, repeated, or passed to a human for assessment.

Initial query: subject, period, and boundaries

Research begins with a description of the subject, audience, and time interval. The editor should define the geography, event types, acceptable sources, relevance criteria, and stop topics in advance. A broad initial formulation increases the risk that the system will spend resources on materials that do not match the task.

Geography is especially important for news monitoring. A query about a market, regulation, or technology without specifying countries and regions allows too many interpretations. In the author’s experiment, the clarifying question for the general query was mainly about geography.

The task can be structured as follows:

• Subject: what phenomena and events are being researched.
• Audience: who the selection is being prepared for.
• Period: which dates are permitted.
• Geography: which countries, cities, or markets are included in the coverage.
• Relevance: what makes an event significant for the specific project.
• Stop topics: which areas should be excluded.

Thematic map controls coverage

A thematic map describes the areas within which the system conducts its search. It is needed not only for the interface: the map turns a vague editorial task into a set of manageable areas and makes it possible to compare the results of different runs.

In the author’s prototype, thematic maps could be uploaded from files, selected automatically based on the initial query, or assembled manually. The user could enable all available topics or limit the research to specific areas. In one scenario, the map contained 97 topics, but the author emphasizes that the actual coverage depended on the selected settings.

For an editorial team, the map should store not only the topic name but also a brief description, examples of relevant events, exclusions, and priority. Otherwise, the model will interpret the same section differently in repeated runs.

Event families before search queries

An event family groups reports of the same type—for example, a product launch, a regulatory change, a deal, a study, or a public conflict. The system first describes exactly what needs to be found and only then turns that description into specific search formulations.

This intermediate level reduces dependence on a single set of keywords. If queries are generated immediately, the model may focus on the most obvious formulations and miss reports using different terminology. An event family defines the semantic framework, while search queries provide its coverage.

In the author’s experiment, one run produced eight event families. In the prototype settings, their number could range from 3 to 12. The author also states that before the next stage, the interface displayed an estimate of processing costs, but such an estimate cannot be considered universal: it depends on the model, the volume of text, the number of sources, and the conditions of the specific service.

Source search and preliminary filtering

For each family, the pipeline generates specific web-search queries. The resulting pages undergo preliminary filtering by topic and period, after which the system extracts event candidates and links them to the original URLs.

Preliminary filtering does not replace verification. It reduces the volume of material sent to the next stage. A page may contain the necessary keywords but be recounting an old story, quoting an unverified rumor, or discussing the topic in a context unrelated to the task.

In the experiment described, the system received 30 sources, of which 28 were passed on for substantive verification. This observation applies to the author’s specific run and is not a guaranteed measure of the effectiveness of any news pipeline.

Event verification: date, topic, and evidence

Event verification should answer several different questions. Does the source really discuss this event? Does the publication fall within the specified period? Does the material confirm a fact or merely repeat someone else’s report? Are there enough independent, high-quality confirmations for editorial use?

In the author’s experiment, the system checked the exact date, whether the event matched the period and topic, the quality of the source, and the sufficiency of the confirmations. In one run, 15 events passed verification. Such a stage requires a separate data structure that stores not only the final conclusion but also its grounds: the date, quotation, source, detected contradictions, and the decision of the verification step.

The question “Is one publication enough?” has no universal answer. For an official announcement from a primary source, one item may sometimes be sufficient, but a controversial statement, market assessment, or report citing an anonymous source requires additional verification. The pipeline must distinguish between the fact that something was published and the substantiation of the fact being described.

Verification statuses and duplicate merging

A status turns an editorial decision into an explicit rule. The author’s prototype used four statuses:

• Confirmed: the event matches the period and topic, and the sources provide sufficient grounds.
• Requires additional evidence: the event appears relevant, but the evidence is insufficient or contradictory.
• Rejected: the material does not meet the topic, period, or quality criteria.
• Merged: the event matches another candidate and should not be counted as a separate news item.

Merging is important for news monitoring. One event can appear in dozens of publications, while automatically counting pages creates a false impression of the number of news items. When merging, all related sources should be preserved, one primary description of the event should be assigned, and the materials that duplicate one another should be identified.

Editorial selection after mechanical verification

Verification answers the question of whether an event can be considered a suitable candidate. Editorial selection solves a different task: which of the confirmed events actually deserve a place in the final selection.

After the mechanical checks, the LLM in the author’s experiment selected events and transformed them into five compact topics. This does not mean that the language model replaced the editor. It performed a grouping and compression operation, while the criteria for a topic’s value must be defined in advance: novelty, impact on the audience, practical significance, contentiousness, the presence of a primary source, and distinction from topics already selected.

The editor should review not only the final list but also the path from topics to events and from events to sources. If the final wording becomes broader than the original evidence, the material must not be published without manual editing.

Interface, reruns, and data storage

The user interface should display runs, topics found, and the step-by-step processing flow. In the author’s experiment, a search task could be copied and repeated with a different date, while unnecessary research could be deleted from the list and database.

Reproducibility matters more than an attractive demonstration. Each run should store the initial query, period, selected topics, coverage settings, prompt versions, source list, event statuses, and the final editorial selection. If this data is not saved, the editor will not be able to understand why yesterday’s result differs from today’s.

Deleting outdated runs should take the history of decisions into account. In the operational interface, old results can be hidden, but confirmations, editorial edits, and the error log are best stored separately if they are needed for quality control.

Where a human is needed

The author of the described experiment critically evaluated the results of nearly every step and repeated the processing whenever the result did not satisfy him. He also kept the ability to change the scale and coverage of the search, although in the specific test he used the settings suggested by the model.

A human is needed where the criterion depends on editorial policy, the risk of error, or context. The editor checks disputed sources, decides whether wording is acceptable, changes topic priorities, and conducts acceptance testing on examples prepared in advance.

What happens if the processing log is not completed? The team will be unable to distinguish a search error from a verification error and will not know which stage to fix. A step-by-step log is needed for diagnosis.

Prototype limitations and technical debt

In the author’s assessment, the prototype created is useful for testing an idea and demonstrating the capabilities of a development environment with a language model, but it is not mature enough to be an independent product or service. This conclusion is related to technical debt, the large number of iterations, and the need for repeated analysis, development, and testing.

The author reports that the project included a graphical interface, a logic layer, a database, model calls, and recursive verification through a critique mechanism. The project contained more than 20 000 lines of code. This information relates to the specific experiment and does not make it possible to estimate the effort required for any analogous solution.

The expenses reported by the author also should not be treated as a current rate. In his test scenario, using a cloud model through an API cost approximately 25 dollars, while he estimated the total expenses at approximately 121 dollars, including the subscription and purchased credits. Without some of the purchases, the author estimated the expenses at approximately 49 dollars. The cost depends on the provider, model, limits, text volume, and number of repeated runs.

The main risk of this approach is false confirmation. The model may correctly find a page but misunderstand the date, confuse a primary source with a retelling, or merge different events. Multiple stages reduce the risk, but on their own they do not turn a probabilistic conclusion into an established fact.

Checklist for independent testing

1. Record the initial query, audience, period, and geography.
2. Describe the relevance criteria and stop topics before the first run.
3. Create a thematic map with clear section boundaries.
4. Divide the task into event families rather than relying only on keywords.
5. Save every search query and all pages found.
6. Check the exact date, topic, source quality, and sufficiency of evidence separately.
7. Use statuses that explain the next action to take regarding the event.
8. Merge duplicates while preserving the associated sources.
9. Perform editorial selection only after fact-checking.
10. Add manual review for controversial and socially significant materials.
11. Calculate the cost of repeated model calls, not just the initial run.
12. Test reproducibility on another date and with a preassembled news set.

Conclusion

A multistage AI pipeline transforms news discovery from a single unmanageable query into a sequence of operations with clear inputs and outputs. The topical map ensures coverage, event families define the meaning of the search, verification separates candidates from confirmed materials, and the editorial stage compiles a concise list of topics.

The author's experience demonstrates the value of such a prototype as a hypothesis-testing tool. A production-ready product would require separate engineering work: robust data storage, cost control, handling web-search errors, protection against false confirmations, reproducible tests, and the editor's ongoing involvement in disputed cases.