Oosto

Why recognition in live video is much harder than in a photo

Comparing two photos of a face is a relatively solved problem. A passport photo against a photo taken at a checkpoint, in reasonable light and facing the camera: here the algorithms have been good for years, and are usually better than people.

Moving to live video turns this into a completely different problem. Dozens of people moving at the same time, random angles, light that changes from one corner to the next, faces partly hidden by another person or by a hat. And all of this has to happen at the pace the video streams, not after it. A system that lags one second behind the stream is no longer a real-time system.

This is the familiar gap between a demo and reality. A model that shows high accuracy on a tidy photo database can collapse in a crowded train station, and the difference is not the quality of the model but the conditions. This is the engineering problem the company was built around. It is also what made it one of the most prominent in the world in its field, and one of the most controversial, for exactly the same reason.

From camera to alert: what happens in each frame

The process behind "the system recognized a person" is made of several separate stages that run in sequence, without stopping. First, detection: in every frame, the system has to find where there are faces at all. Then tracking: linking the faces found in this frame to the same person from the previous frame, even if they turned around or were blocked for a moment.

Only then comes the stage the public thinks about: choosing the best-quality frame from the sequence, and comparing it against a database. This choice matters because a comparison on a bad image produces an error, and a sequence of a hundred frames usually has a few good ones.

The constraint that shapes the whole architecture is where the processing runs. Sending raw video from every camera to the cloud is expensive in bandwidth, complicated in terms of privacy, and sensitive to network failures. So a large part of the work in companies like this goes into running the process on cheap hardware at the edge of the network. In other words, into making the computing cheaper, not into improving the accuracy, which is what research papers measure.

2019–2020: the report, the audit, and the investor that left

In October 2019, NBC News, along with Forbes and TheMarker, reported that AnyVision's technology ran a classified military surveillance project tracking Palestinians in the West Bank. Microsoft's investment arm was then among the company's investors, after taking part in a $74 million funding round in June of that year.

In response, Microsoft commissioned an outside audit, led by former US Attorney General Eric Holder and his team at the law firm Covington & Burling. A summary of the findings, published in March 2020, stated that the technology is used at border crossings between Israel and the West Bank, but that the available evidence shows it does not power, and has not powered, the mass surveillance program attributed to it in the reports. The summary also noted that the audit was limited in some respects because of legal restrictions on disclosing sensitive information. In other words, it did not include access to classified material.

Despite this finding, Microsoft sold its holdings, and announced it would stop making minority investments in face recognition companies. The explanation it gave was structural: as a small investor, it has no ability to oversee how a sensitive technology is used.

$235 million, a new name, and then what

In business terms, 2021 was the peak year. The company raised a $235 million round led by SoftBank's Vision Fund and Eldridge, and in the same year changed its name from AnyVision to Oosto. The move is commonly read as an attempt to distance itself from the public controversy attached to the old name. In total it raised about $352 million, and its investors also included Qualcomm, Lightspeed and Bosch.

Already at the end of 2020, Avi Golan, who came from a senior role at SoftBank, was appointed CEO. About eight months later, SoftBank led the big round and became the main investor. At its peak, the company employed more than 200 people.

The years that followed did not deliver the growth this funding assumed. Selling to the security sector and to public authorities is slow by nature: long procurement cycles, approvals, and often local public opposition that delays or cancels a project after it has already been signed. On top of that came a regulatory environment in which various cities and states banned police use of face recognition entirely.

From $352 million raised to $125 million in a sale

In January 2025, the company was sold for $125 million to the American company Metropolis. The important detail in the deal is not the amount but its form: most of the payment was made in Metropolis shares rather than cash. That means the shareholders received paper in another private company, not money in hand.

According to Calcalist, the company's annual revenue was then about $10 million, and its headcount had dropped to only a few dozen. The same report says that SoftBank's Vision Fund, which invested $25 million, was expected to get its investment back. That necessarily means other investors did not.

It is also interesting who brought the two sides together: Eldridge was a long-time investor in AnyVision and also a shareholder in Metropolis. The gap between $352 million raised and $125 million received is one of the clearest cases in the Israeli industry where recognized technological ability did not translate into a business result.

The technology's second life: a parking lot instead of a station

Metropolis is not a security company. It is a large American parking operator, with about 20,000 employees and about 4,000 facilities. It uses computer vision to identify cars and charge the driver automatically, with no barrier and no ticket.

So the technology keeps working, but inside a completely different product from the one it was built for. Instead of matching faces against a watch list, it identifies vehicles for payment. The engineering parts are quite similar: detection in a stream, tracking across frames, a fast decision on local hardware. What changed is the context.

This is part of the broader lesson the case offers. In a market where every customer is subject to changes in law, public criticism and government procurement cycles, the quality of the technology is not the variable that decides the outcome. The very same ability, when pointed at a use nobody objects to, can be a completely stable business. In this case it did become one, only under different owners.