Phase 2 (2012-2013): let the local vendor do my product matching
Soon after price monitoring solutions appeared, such companies came to conclusion that product matching is essential for onboarding new clients – and that they should offer it as an add-on service, which is essential for setting up new client’s account.
So, they have formed local teams who were forwarded product matching tasks – at the cost which is lower than what category manager pay was. (As you can see, the primary motive was cost reduction)
One obstacle that needed to be overcome was quality – as these dedicated matching teams did not have specific industry experience that was needed for each client. Today you may be working on product matching task for a client from cosmetics industry, tomorrow it can be tires, and the day after tomorrow garments. As no man can be expert in all of these fields, the probability of wrong matches was getting higher (we have covered the topic of dangers of wrong product matches in a separate post). The solution to this went by trial-and-error (which made quite a few clients unhappy in the early days), and at later stage, by introducing QA as part of the process. However, introducing QA meant extra costs, and soon enough the costs tipped over the benefits. In other words, a different kind of solution was needed.
Phase 3 (2014-2015): can my products be matched automatically?
Let us suppose you have hundreds of thousands of products, that need to be matched across dozen competitor sites. How big does matching team need to be to fulfill this task? And how much time do they need for it? The answers weren’t looking good – as we’re looking at teams of 50+ agents, with project delivery time in 6-12 months. Unacceptable for majority of clients, who needed the competitor pricing data as soon as possible.
That is when the idea of automated product matching (Automatch as it’s called today) was born – we’re talking about years 2014 and 2015, when companies like Competera, Price2Spyand Pricevastarted offering this kind of service.
In essence, the idea is simple: if client’s products do come with a unique identifier (usually a UPC, EAN, ASIN or another type of product code), and if this code can be found on competitor’s site – voila, we have a solution for product matching automation! As whole solution was fully automated, the time needed to establish those 100 000’s of product matches drastically went down – usually such tasks were completed within one week.
As for the costs – automated product matching did involve some engineering effort which of course comes with higher costs – but the scale of the project usually justified such costs. QA was needed as well (comes with costs which are usually not that high), but compared to months and months of work for 50 low-paid staff, the idea made a lot of sense.
Another problem that came with automated product matching was that it caused a lot of strange-looking traffic on websites were product matching needed to be done. Website with diligent system administrators would notice such odd-looking traffic, and they would try to stop it. This was resolved by various anti-detection mechanisms (either spreading the traffic across more and more IPs (proxies) or by ever-changing browser’s user-agent.
The only trouble was – what if client’s products did not come with any kind of identifier? Or if the competitor site did not work with these same identifiers?
This problem has significantly reduced the feasibility of automated product matching, so nowadays Automatch is applicable in no more than 30% of product matching cases. As you can imagine, the clients were not happy with this – so the vendors had to come up with something better, yet again.