An AI Ran a Store—and Paid Women Less

Opening

Hi, subscribers. This is Oswarld.

Last week, a tiny shop in San Francisco caught the attention of media worldwide. It’s called ‘Andon Market’. It’s an experimental store where an AI named “Luna” makes every decision — interior design, hiring, pricing, hours of operation, all of it. Its founder, Andon Labs, gave the AI a 3-year lease, a $100,000 budget, and exactly one instruction: “Make a profit.”

But while reporting on the store, a journalist noticed something odd. The two women Luna had hired were being paid $2 less per hour than the one man on staff. When asked why, the AI answered that “the male employee has more retail experience.” Was that actually true? According to their resumes, the male employee’s work history was about 3 months longer than the women’s.

I don’t see this as a mere anecdote. It’s a small sample of AI automating an old social problem: wage discrimination. What matters more is the governance gap — who discovers this, and who is held accountable for it. Today, starting from this one shop, I want to trace the intersection of AI bias, labor, and institutional design.

🏪 What Actually Happened at Andon Market

Let’s start with the facts. Andon Labs signed a 3-year lease at 2102 Union Street in San Francisco, then handed every operational decision at the store to an AI agent named Luna. Luna runs on Anthropic’s Claude Sonnet 4.6 model. Luna posted job listings on Indeed.com¹, conducted phone interviews, and made the hiring decisions directly. Negotiating prices with suppliers, processing credit card payments, signing up for AT&T internet, subscribing to ADT security — Luna handled all of it. (So think of Luna less as a robot and more as a chatbot that happens to have a name.)

New York Times reporter Heather Knight, while covering the store, discovered that the two women Luna hired were being paid $2 less per hour than a male employee named Felix. Luna explained this by saying “Felix has more retail experience.”

Let’s pause here for a moment. On its face, “more experience, higher pay” sounds reasonable. But there are several unverifiable gaps in this decision-making process.

First: who set the standard for “experience”? There’s no external way to verify what weighting Luna used to evaluate “experience,” or how that evaluation was shaped by the training data available at the time of hiring. Second: what was the basis for the $2 gap specifically? There’s no way to confirm whether that number was derived from market data or emerged organically from some latent pattern inside the model. Third: who reviews these decisions after the fact? It appears no one was monitoring this gap at all — at least not until reporter Heather Knight visited the store and asked the employees directly. (In fairness to the project, its entire premise is to “hand everything over” to the AI, so it seems the discrepancy simply wasn’t challenged as a matter of design.)

Andon Labs stated on its blog that “this is a controlled experiment, and all employees are formally employed by Andon Labs, guaranteed fair wages, fair treatment, and full legal protection.” In other words, there is a safeguard ensuring that no one’s livelihood is determined by AI judgment alone. But that safeguard only holds within the context of an experiment. Had an AI made the same decision in a real, operating store, the gap likely would have gone unnoticed by anyone.

📚 This Isn’t New — A Recurring Pattern

The reason the Andon Market incident is being taken seriously in American society is that it isn’t an exception — it’s a pattern. There have already been multiple reported cases of AI learning and amplifying existing bias when making labor-market decisions.

The best-known case is Amazon. Starting in 2014, Amazon built a recruiting AI that scored applicants on a scale of 1 to 5, but scrapped the project after discovering the tool favored male candidates for technical roles. The AI had been trained on 10 years of resumes, and because the tech industry had historically been male-dominated, it automatically began favoring male applicants. The system penalized resumes containing the word “women’s” — as in “women’s chess club captain” — and rated graduates of two women’s colleges lower. Amazon made fixes to neutralize these specific terms, but ultimately disbanded the team, judging that there was no guarantee the model wouldn’t produce discriminatory rankings through some other mechanism.

What’s worth noting here is the conclusion that “neutralizing fixes can still produce other forms of discrimination.” Bias doesn’t disappear just because you delete one specific word. Because models follow the statistical patterns in their training data, removing a surface-level signal still allows the model to arrive at the same outcome through deeper correlations. This isn’t a simple bug — it’s a characteristic baked into how machine learning works.

A similar pattern shows up in wage-setting decisions. A 2023 paper by UC Irvine law professor Veena Dubal, published in the Columbia Law Review, titled “On Algorithmic Wage Discrimination,” analyzes how platforms like Uber and Lyft feed worker-extracted data into machine learning systems to produce hourly wages that are unpredictable, volatile, and individually personalized. Workers reported experiencing this variable wage structure as something like gambling, or a con.

Professor Dubal calls this algorithmic wage discrimination² or “surveillance pay,” describing it as a structure where workers doing the same job, at the same time, in the same place, with the same skills, end up receiving very different amounts of pay. The core problem is that these wage decisions happen inside a black box — neither the worker nor regulators can directly verify the basis for them.

The Andon Market case sits at the intersection of these two trends. In that an AI makes hiring decisions, it continues the Amazon pattern; in that an AI calculates individualized hourly wages, it continues the algorithmic wage discrimination pattern Dubal analyzed. The one difference is that at Andon Market, all of these decisions are consolidated into a single AI.

⚖️ Is Bias a Data Problem, or a Delegation-Structure Problem?

The prescription that usually comes up here is “train it on better data.” That’s not wrong, on its face — biased training data produces a biased model. But that prescription alone isn’t enough.

Let’s go back to Andon Market. Think again about why this bias was discovered in the first place. The AI didn’t self-report, “there’s bias in my decision.” A human reporter visited the store, asked employees directly about their wages, and compared the difference to discover it. In other words, bias only becomes visible when there’s an external human verifier present.

But in an environment where AI is increasingly delegated more and more decisions, it’s effectively impossible to attach a human verifier to every single one. Luna probably makes hundreds of decisions a day — hiring, pricing, inventory orders, marketing copy, staff scheduling. There’s no way for a human to check, one by one, which of these decisions might be hiding bias.

This is where the real question emerges: who measures bias, and who is accountable for it? This isn’t a technology problem — it’s a delegation-structure problem.

In 2023, the U.S. National Institute of Standards and Technology (NIST) published the AI Risk Management Framework (AI RMF), which structures AI governance³ into four functions: GOVERN (governance culture and accountability structures), MAP (defining context), MEASURE (measurement and evaluation), and MANAGE (intervention and control). Item 2.11 under the MEASURE function explicitly requires evaluation of fairness and bias. In other words, adopting AI and being accountable for operating it responsibly are two completely different things. The latter requires a measurement system, audit procedures, ongoing monitoring, and — above all — a human who is accountable for the decisions.

Andon Market had none of this governance. There was no procedure in place to review Luna’s wage decisions at the moment they were made, and the pattern was only discovered after it had accumulated, by an outside reporter. This isn’t so much the AI’s fault as a design failure on the part of the humans who delegated the decision to it.

🔧 So What Should Be Done?

Clues for solving this problem already exist in academia and industry. They just haven’t been standardized yet.

First, we need a systematic audit methodology. The 2018 Gender Shades study by MIT Media Lab’s Joy Buolamwini and Timnit Gebru is a good model. The two researchers evaluated commercial facial-analysis systems from IBM, Microsoft, and Face++, and found that the misclassification rate for darker-skinned women reached as high as 34.7%, compared to just 0.8% for lighter-skinned men. What makes this study important isn’t just its findings — it’s that it laid out a methodology for measuring bias. Classifying data using the Fitzpatrick skin-tone scale and comparing error rates across demographic subgroups became, in the years since, the standard procedure for AI bias audits.

Second, we need decision logs and after-the-fact traceability. If there’s no record of why an AI made a given decision — which inputs were weighted how — post-hoc verification becomes impossible. In the Andon Market case too, the data underlying Luna’s judgment that “Felix has more experience” was never disclosed.

Third, human accountability has to be unambiguous. When an AI makes a discriminatory decision, it needs to be clear whose responsibility it is to correct it. This is precisely why governance frameworks like the EU AI Act and NIST AI RMF⁴ all emphasize this point. AI cannot be the subject of accountability. That means the organization that delegated the decision — and the human within that organization responsible for it — must bear the responsibility.

Oz’s Lens

Having spent years working with data, I see the core of this incident as an absence of measurement.

Bias is a latent property of data, but if it’s never discovered, it might as well not exist. At Andon Market too, the $2 hourly gap existed in the data from the very start, but no one could perceive it until it was measured. And unmeasured bias becomes a training signal inside the AI system, feeding back into the next decision. This is the mechanism by which bias amplifies itself automatically.

Anyone who has built a go-to-market strategy knows this instinctively. A metric that isn’t measured can’t be improved, and a poorly designed measurement method optimizes toward the wrong outcome. AI governance is no different. Measuring fairness isn’t just a matter of bolting on one more KPI. You have to design which demographic groups to compare, which statistical definition of fairness to apply (demographic parity⁵? equal opportunity? counterfactual fairness?), and at what point in time to measure. The reason the Gender Shades study became so influential is precisely that it was the first to systematize this design of measurement itself.

The value of an experiment like Andon Market isn’t in proving that AI can run a store — it’s in exposing the governance vacuum that opens up after the delegation happens. The question we actually need to answer isn’t “can AI run a store?” It’s “how do we measure the decisions an AI makes, and who is accountable for the results?”

Closing

Every so often I’ll catch a current-affairs program raising this kind of argument: when an unfair verdict comes down, someone says “we should replace judges with AI.” As I wrote years ago in another piece, if we ever seat a deep-learning-style AI as a judge, we could end up in the strange position of having to fight the very biases that we ourselves built up over the past. AI isn’t some fair, rational entity that always produces “ideal, impartial” outcomes. It’s a system that combines the knowledge humanity has accumulated to produce the best-fit answer — or, just as often, an answer skewed toward whatever it happened to be trained on most heavily.

• The incident at Andon Market, where an AI paid female employees less, is a small sample of AI bias extending into labor decisions.
• Bias is a training-data problem, but it’s equally a problem of missing measurement, audit, and accountability structures. Fixing the data alone won’t solve it.
• The real question isn’t “can AI make decisions?” — it’s “who verifies and takes responsibility for those decisions?”

If your organization is running a project that delegates decision-making to AI, check one thing before you delegate: who will measure this decision, when, and by what standard. If you’re stuck on the answer, start by reading the GOVERN section of the NIST AI RMF.

References & Further Reading

Primary sources

• Knight, Heather. “AI Store Manager…”, The New York Times, 2026. The original reporting on the Andon Market store. This article was the first to report the wage gap.
• Andon Labs. “We gave an AI a 3-year retail lease in SF and asked it to make a profit.” Andon Labs Blog, 2026. Written directly by Andon Labs’ founders, laying out the background and ethical considerations behind the experiment.
• Dubal, Veena B. “On Algorithmic Wage Discrimination.” Columbia Law Review 123, no. 7, 2023. The single most important paper on the structure and legal implications of algorithmic wage discrimination — worth reading from a labor and institutional-design perspective.
• Buolamwini, Joy, and Timnit Gebru. “Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification.” Proceedings of Machine Learning Research 81, 2018, 77–91. The foundational paper for AI bias audit methodology — a great example of why measurement design matters.

Background

• Dastin, Jeffrey. “Amazon Scraps Secret AI Recruiting Tool That Showed Bias Against Women.” Reuters, 2018. The original reporting on Amazon’s scrapped recruiting AI — a classic case study in AI bias.
• National Institute of Standards and Technology. AI Risk Management Framework (AI RMF 1.0). NIST AI 100-1, 2023. The standard framework for AI governance — start with the MEASURE section, since it connects directly to today’s issue.

The author, Kwangseob Ahn, is a professor of business administration at Sejong University and lead consultant at OBF (Oswarld Boutique Consulting Firm). He teaches statistics and data analysis — business data management and business analytics — while leading GTM and AI strategy consulting in the field, designing the seam between technology and business. He has published academic research on a memory architecture for AI dialogue systems (HEMA) and runs Daily Arxiv, a daily curation of global AI papers. He holds a master’s from Korea University’s Graduate School of Technology Management and a KMBA. He is the author of Homo Brainless: The People Who Outsource Their Thinking.

Footnotes

1. Indeed.com is the world’s leading job search and recruiting platform. It aggregates job listings from countless websites, job boards, and company career pages into a single searchable engine, and operates in more than 60 countries, including Korea. ↩

2. Algorithmic Wage Discrimination: The practice of AI automatically calculating different hourly wages for different workers. Even when doing the same job at the same time, pay can vary depending on the data. This was first reported extensively on gig-labor platforms like Uber and Lyft. ↩

3. AI Governance: The organizational procedures and accountability structures for managing the development, adoption, and operation of AI systems. It’s not just about using AI — it’s about deciding how to be accountable while using it. ↩

4. NIST AI RMF (AI Risk Management Framework): A framework for managing AI risk published by the U.S. National Institute of Standards and Technology (NIST). It’s built around four functions — GOVERN, MAP, MEASURE, and MANAGE — and currently functions as the de facto international standard for AI governance. ↩

5. Demographic Parity: A fairness definition stating that an AI’s decisions should produce positive outcomes at equal rates across demographic groups such as gender or race. It’s one of several mathematical definitions used to measure fairness. ↩