Interview: Exclusive interview with ‘BMX’ – the seller that hacked BestMatch

April 14, 2008

Marketplaces ChannelAdvisor By ChannelAdvisor

Last week we reported that sellers had found a bug/hack in BestMatch that gave them premium placement on eBay and of course corresponding traffic/bids/conversions.  Since that report we were contacted by a seller purporting to be the originator of the hack (turns out he reads eBay Strategies!).  He gave compelling evidence that he was the original exploiter of the hack (note: we didn’t get CSI on this, but are pretty sure this is the originator.)  He agreed to an interview for eBay Strategies under the condition that his identity remain anonymous as he fears retribution from eBay because of the BM hack.  We’ve dubbed him BMX (BestMatch eXpert).

Like many eBay sellers, BMX turns out to be a very clever, entrepreneurial, business person with a passion for eBay and lots of interesting insights into the recent changes.  His knowledge of BestMatch is very deep and you can tell he’s spent lots of time digging into the algorithm to figure out how it ticks inside the black box.  I learned a ton from this interview and I’m sure you will to.

Here is our interview with BMX – he will be reading comments and replying as BMX – I will verify his identify before publishing the comments to make sure we don’t get anyone pretending to be BMX.

Hello Scot,

Thank you for taking the time to allow me to answer a few questions.  I would just take a moment to say that I’ve read a lot of your blogs and am deeply honoured with the opportunity to address your readers and the eBay community.

Q: BMX, you have found a clever hack for the BM algorithm, how did you go about doing that?

As any committed eBay business owner does, I always am looking for a way to stay ahead of the competition.  As you know the eBay marketplace is quite competitive and sellers are constantly clamoring for high visibility in the search as that equals higher sell-through and conversion rates.  One day you are on top then next day you are in the middle of the pack. I find that it is very easy to get lost in the shuffle.  With the volume of items being listed on eBay transparency can become a real issue.

As eBay neared its launch of ‘Best Match’ as the default sort, I played with the sort on and on in order to see where I stood before the official rollout took place.  Using the eBay labs BayEstimator tool, I optimised my keywords to improve my visibility in the new sort.  To my delight I was able to always place myself at the top of the sorts by optimising my keywords, so I felt that I was ahead of the curve and when best match rolled out, I would be near the top.

To my dismay, when bestmatch actually did roll out, my listings were severely disadvantaged.  The reason why still is not readily apparent to me but I have a feeling it is because a customer closed an INR dispute against me as unresolved.  I’ve only had probably 2-3 of these types of closures against me in my 3 years selling on eBay so I’m speculating that INR disputes closed with no resolution weigh very heavily in the new sort.  If this is the case, as I suspect it is, I think eBay places too much weight on these INR disputes and I think large sellers who move large amounts of merchandise will be severely disadvantaged in the Best Match sort because of these rare INR disputes.  They are rare but they do happen from time to time and I don’t believe they are representative of a seller’s business (1-2 transactions never is in my opinion).  Sellers, who sell antique, used, and refurbished merchandise are going to be severely disadvantaged by this because of the higher failure rates of used and refurbished products.  What’s worse is eBay launched the Best Match sort without launching the full seller dashboard tool to give sellers information as to how these new factors are affecting their business, which is a direct contradiction of the comments that I’ve heard the directors speaking of regarding making information available to sellers.

I continued to do research as far as my DSR ratings which were all 4.4 and above, with my S&H star being the lowest, and when the dashboard light was released, the stars were 4.6 or higher in all categories.  I continued to optimise my titles to improve my position in the best match search.  I was optimising my titles in the BayEstimator tool when by chance I had the word NEW twice in the title.  I took the extra new out so I could use other relevant keywords and to my surprise the tool’s ‘BayEstimate’ dropped significantly.  So I added a few more NEW’s and the relevance jumped to 100%.  Ah ha!  The trick was born.  I was a bit dumbfounded at the simplicity because, being a former programmer myself, I thought it would have been easy to parse the title string and discount (or ignore) and words which were duplicated, but apparently that was not the case.

To understand how the trick works, you have to understand how keyword relevance works with best match.  EBay tracks what keyword searches users are typing into the search box when they perform searches.  We’ll call these ‘base keywords’.  EBay keeps data on which listings have the highest conversion rates based on those keywords, eBay tracks what I’ll call ‘Relevant keywords’, keywords other than the base search which are likely to produce a sale (i.e.: buyer data has shown that buyers are looking for this by completing a purchase).EBay then tries to assign a relevance score to the relevant keywords and then give you an overall ‘BayEstimate’ which is one of the major components to Best Match.  For example if you search for ‘Wii’ the word NEW and NINTENDO will have a high relevance score because almost all listings that are sold have those keywords in them.

I was sure that few sellers knew about this trick, so I tried it in a few different categories that I sell in and it worked wonderfully for the listings.  I was consistently near the top of the sort and my conversion rates and ASP’s rose dramatically.

I ran the listings from the time Best Match rolled out until early April I tried it on a few Wii listings.  As I figured, my listings trumped almost everyone else’s; my ASP’s and conversion rates were phenomenal.  The problem was other sellers caught onto my trick and a few sellers figured out the keyword relevance using the BayEstimator tool.  Quickly they found other keywords which to Best Match were highly relevant.  Hence the reason you saw 5 5 5 and 2 2 2 in a lot of the listings.

What surprised me even more was how quickly other sellers caught on and furthermore how some sellers blindly copied and started putting keyword combinations which did nothing to help them such as 7 7 7 and Wii Wii Wii.  What these sellers didn’t know is that 7’s weren’t relevant to Best Match and keywords that have very low conversion rates are actually assigned negative numbers so they go to hurt your BayEstimate score.  Words such as ‘Wii’ are part of the base keyword search and would have no effect on your BayEstimate score.

One final word about the Best Match system:  I think based on keyword relevance alone, it is a wonderful system.  It would help buyers find what they are looking for and reward sellers who accurately describe the titles of their items.  The problem is that eBay puts in a lot of other factors into the sort such as DSR ratings, Ship Cost, Distance of Buyer to seller, and many other factors.  No one really knows how it works exactly and frankly neither does EBay I believe.  I think maybe over time the adjustments will be made and the actual sort will be more relevant than it is today but right now to me it looks like nothing more than a jumbled mess at times which doesn’t help the buyer or the seller.  EBay could potentially have a lasting scar from the Best Match move, much as they did with the Stores in Search experiment in 2006.  I believe the removal of SIS severely crippled the market and drove many good sellers away from the site.

Q: Did you use the BayEstimator tool to figure out the ‘new new new’ hack?

Yes, without the BayEstimator tool it would have been nearly impossible to figure this type of thing out.  What I think is inherently unfair about best match is that without the BayEstimator tool you really are shooting around in the dark with your listing titles.  For example, adding PS3 as compared to Playstation 3 when listing a game can have a dramatic effect on your BayEstimate score because most sellers put PS3 and not Playstation 3.  A simple rewording of your title can have dramatic results.  The casual seller will never know this and this puts them at a significant disadvantage.It’s like having a daily pop quiz on something you’ve never studied or hitting a piñata blindfolded at a birthday party.

Q: eBay seems to be wise to the hack, what’s next?

The problem with Best Match is that it is constantly changing, no one REALLY knows how it works, so it will be inherently impossible to forever game the system.  As soon as someone figures out a new trick, eBay will change their policy and shut it down.  Best Match acts as a punishment of sorts for seller behaviour, but I would compare it to punishing a child for doing something wrong but not telling them what they did so that they can correct their behaviour.  The grossly unfair part of this is that there are financial implications to the sellers are they try to figure out the system.

Q: How do you feel about DSRs? Any insights into how to game that system?

I believe the intention of DSR’s are good inherently, but I believe that Ebay’s implementation of the DSR ratings are ultimately unfair.  As has been stated in many industry blogs and conversations around the eBay circles, I think eBay has misinterpreted their data.  I would be curious to see how the DSR data has changed over time and how it would look if you parsed out the DSR’s on a category-by-category level.  My guess is that buyers have unrealistic expectations as to the cost of properly packing and shipping an item and the time it takes for proper delivery.  If you have any doubt about the unrealistic expectations of buyers, go to Kinkos and have them pack a few boxes for you I think you will be in for some major sticker shock.  Sellers are providing a service and should not be required to give it away for free.  EBay doesn’t work for free and neither should sellers.

Inherently the DSR system is unfair because you are dealing with constraints which are out of the control of the seller such as cost of shipping and packing, actual shipping costs, and accuracy and timely delivery by third party carriers.  Most sellers have very high DSR scores for Description and Communication.

EBay has placed great weight on the S&H star more than any other, but the fact is they created that monster themselves.  I doubt many of the execs at eBay have run a large eBay business themselves and they just don’t see the constraints they have placed sellers under.

Ultimately buyers come to eBay to get a good deal.  There is only so much of a good deal that can be given by small to medium sellers before they have to resort to small cost cutting measures to recover their costs.  When those measures are removed, the good deals go away and so do the buyers and sellers.  A good example is the media category.  My guess is that it’s nearly impossible to sell 100% brand new legitimately acquired media items to sell on the eBay site.  With eBay’s ever increasing fees and confusing free structure it has become more and more difficult to compete and find profitable items.  EBay is forcing good sellers off the site rather than going after the few sellers who truly do abuse the system, not to mention that very low shipping recovered in the selling price results in a higher percentage of the sale

As far as gaming the DSR system I don’t think you really can.  What I would recommend to most sellers is that you communicate as best you can with your buyers, as you recommended in your webinar.  DSR’s are a buyer’s subjective opinion on how they feel your service to them was.  The way eBay words their DSR instructions, and the hideous balloon that eBay placed on the DSR page encourages lower DSR ratings.  Ultimately what a buyer thinks of the transaction, how eBay grades you on that transaction, and what is actually feasible with the constraints you are given, are what I believe a recipe for a disaster.  You will catch far more good sellers in the DSR net than is actually necessary and reduce their visibility to the point it kills their business.  EBay is trying to come up with a blanket system to cover all transactions on eBay and I really believe that is impossible.  One size fits all doesn’t really work the way they’ve designed it.

Real word studies have shown that customer satisfaction rates for the world’s best and brightest companies such as Amazon run in the high 80%’s as far as customer satisfaction ranks.  EBay itself as a company ranked 81 according to the Univ of Michigan customer satisfaction report.  Now take into account the massive buying power and volume discounts that an e-tailer like receives, yet eBay expects its sellers to maintain a 98% satisfaction rate while providing cheaper and faster service on a small business budget?  Changes such as no feedback for buyers will likely lower everyone’s score all around and give new buyers (and old ones) the perception that eBay is now ‘less safe’ because they will see more real world customer satisfaction numbers.  If you saw a seller with a75% feedback percentage would you buy from them?  What about the average buyer who doesn’t really pay attention to all of the changes?  Let me share some data with you about some major companies and their customer satisfaction scores in 2006. Macy’s scored a 75 on a broad list of criteria ranked by shoppers, while Wal-Mart received a 68.  Nordstrom led the department store sector at 80, followed by Kohl’s at 79.  Kroger Co. results were off at a score of 75, down from a score of 76 in 2006.  With all these numbers in mind lets translate that to how it translates to eBay.  On a raw number basis, 4.6 is 92% on a 5-point scale.  If you were looking at the same number on the feedback scale a 92% would get you suspended off of the eBay site.  I think eBay has teetered the scales too far and I think it will result in the exit of many vibrant sellers from the eBay marketplace which will in turn be bad for the site overall.  What adds even more insult to injury is eBay’s own customer service is quite lacking and is lower than all the etailers surveyed other than

Moreover, the constant site glitches and failure of eBay and paypal to take responsibility for those glitches with the site directly affect buyers and ultimately leaves the seller to clean up.  Many times the mess caused by eBay’s failure to correct these issues right away affects the sellers DSR ratings which is extremely unfair.

Q: Any message you want to deliver to eBay or to other sellers?

Honestly I can say I used to love eBay, unfortunately eBay sellers are victims of what is known as contract of adhesion.  For those who don’t know, a contract of adhesion is a contract so imbalanced in favor of one party over the other party such that there is a strong implication that it was not freely bargained for.  Sure you could argue that sellers have the right to take their business elsewhere when changes are announced, but eBay makes constant changes that turn the seller’s businesses upside down and this is inherently unfair.  Once you have made the monetary commitment to run an eBay business, it’s hard to back out easily with no negative financial implications.  What’s a seller to do with all of their inventory and staff if they are at that level?  I find the situation really saddening.  When I was at eBay live, I saw many handicapped and disabled who were proud sellers on eBay.  EBay revolutionized their life and gave them a second chance at survival.  I am fearful of their plight in light of all of the new changes, which are taking place on the site.

I personally fell in love with the idea of selling on eBaya few years ago and have built a rather successful business with eBay as my main venue for selling.  I can say I truly do enjoy it.  However, lately, as most sellers I think would agree, I feel abused by eBay’s constant and sometimes erratic and drastic policy changes and frequent and steep fee increases.  However more importantly, I believe eBay’s tragic flaw is that eBay is consistently reactionary with their policies and procedural changes.  All great business leaders will agree, change is necessary to promote and foster growth inspire innovation.  With that said I believe that eBay’s changes are all in hindsight and not in line with the visionary type leadership that are what eBay as great as it is today.  Pierre Oydimar was a true visionary when he saw a way to connect individual small buyers directly with sellers through the Internet.  EBay needs to revisit its roots and reinvent the marketplace with the core principals that make buying and selling fun.  Constant fear of whether the next policy change will be the one that makes you go under is a pressure and stress that many people would rather live without.

One last comment, as with all opinions, mine is what it is, just that, an opinion. (a somewhat educated one I would hope everyone agrees!)I welcome you and others who read your publications to comment on their experiences and whether or not they agree with some of my arguments about the current state of eBay.

Thanks for having me.