Researchers develop AI-powered, human-centered framework for supplier selection

Tech Xplore

Source published: Site published:

Original article · فارسی

Credit: Unsplash/CC0 Public Domain

Global supply chains are facing increasingly complex and unpredictable economic, social, environmental and regulatory conditions. As a result, many companies need faster and more effective ways to evaluate suppliers. To help solve this problem, researchers at Constructor University have developed a new framework for choosing suppliers. It combines the ability of artificial intelligence (AI), especially large language models (LLMs), to understand information with human supervision to help companies make better choices.

In the past, companies mainly chose suppliers based on factors such as cost, quality and delivery performance. Today, purchasing teams must also consider many other things, including environmental sustainability, following regulations, business risks, the ability to handle disruptions and changing market conditions.

The new framework was developed by industrial engineering professor Dr. Fatahi Valilai, Ph.D. researcher Mohammadreza Rezaei and master's student Sridhar Sreemulnath Iyer. They are members of the Emerging Technologies in Industrial Engineering (EITIE) research group at Constructor University. Their framework divides responsibilities between AI, computer systems that follow fixed rules and human experts.

The research is published in the journal Machine Learning with Applications.

"This research was motivated by a question we see arising more and more across our work in emerging technologies: How can we benefit from the flexibility and reasoning capabilities of AI without giving up transparency, traceability and human responsibility in industrial decision-making?" said Fatahi Valilai.

"Our work therefore moves toward agentic and declarative decision architectures in which AI can interpret changing conditions and formulate possible actions, while explicit analytical rules and human governance determine what is actually implemented."

The framework uses an LLM throughout the process. The AI can understand purchasing requests, organize and examine information provided by suppliers, and explain the results in everyday language.

Human experts also supervise the process and step in at important stages. This allows companies to benefit from the speed and information-processing power of AI while keeping people responsible for the assumptions, evidence and final decisions.

"A central insight behind this research is that the value of AI in procurement depends not only on how much information it can process, but also on whether decision-makers can understand, examine and responsibly rely on its outputs," said Rezaei.

"We therefore designed the framework around a clear division of responsibilities. Where established deterministic methods already exist, we retain them for numerical scoring, constraint enforcement and supplier ranking."

The framework includes four checkpoints that human purchasing professionals must complete:

  1. Approve the data being used.
  2. Review the possible scenarios and the importance given to different factors.
  3. Check whether the evidence is relevant.
  4. Approve the final decision.

Testing the framework

To test their framework, the researchers applied it to records from 22,230 past public procurement cases in Europe. They looked at how often the supplier originally chosen for each case appeared among the framework's 10 highest-ranked suppliers.

They found that the rankings matched the original choices much more closely when the framework included extra details about each purchasing situation. These details included a supplier's experience in the relevant product or service category, whether its location was suitable and whether it had worked with the buyer before.

When these details were included, the supplier originally selected appeared in the framework's top 10 about 32% of the time. When the researchers used more general information about suppliers instead, this happened only 5% of the time. This improvement suggests that supplier rankings are much more useful when they consider the specific needs of each purchasing project rather than just suppliers' general activities.

The team also tested the LLM's ability to produce information in the required format. It achieved a valid structured-output rate of 1.000, meaning it consistently followed the required format. Its average F1 score for extracting evidence was 0.928. This shows that it performed well at finding and extracting relevant claims from the supplier documents it examined.

The researchers also checked whether the LLM could explain supplier rankings without changing them. In every test, the model kept the rankings exactly as the framework had calculated them. This suggests that AI can help explain the reasons behind completed analyses without taking over the calculations or quietly changing the results.

Overall, the findings support the framework's division of responsibilities. AI can quickly understand and organize information and explain results. Programmed analytical tools calculate fixed scores and rankings. Human purchasing professionals review the evidence and approve important decisions.

The experiments also revealed some limitations. These findings show why human experts must still review the results, especially when the available evidence is incomplete, contradictory or uncertain.

More information

Mohammadreza Rezaei et al., A governed framework combining large language models and human oversight for supplier selection, Machine Learning with Applications (2026). DOI: 10.1016/j.mlwa.2026.101028