Introducing the eBay Defect Rate – eBay tweaks the feedback system again in Seller Release 14.1 this Spring

March 11, 2014

ChannelAdvisor Scot Wingo By Scot Wingo

Today, March 11, 2014 eBay announced their first seller release of 2014 (we shorten this to SR14.1 to save time).  The changes are substantial and we think warrant a pretty in-depth look so we have provided this two part blog post:

  • Part I – (you are here) in the first part we will introduce the changes coming in SR14.1 (written by Scot Wingo)
  • Part II –  In part 2 (now available here), we will recommend strategies for both getting the most out of SR14.1 and avoiding any bad consequences.  We do believe that most sellers will want to take some action(s) and the changes go into effect pretty quickly. (written by Jenny Hock)

In this post we focus on the US details, but it’s important to note that SR14.1 applies to the UK, DE and global standards programs as well.

Introducing SR14.1 – Why is eBay making these changes?

eBay has done a great job providing tons of details about SR14.1 including:

  • Detailed micro-site about the changes is here
  • Customer care support is available
  • Webinars and videos are coming soon
  • town hall (detailed in this post)
  • in-person meetups

Also, if you are a ChannelAdvisor customer, your account manager has been briefed on all of these changes for quite a while and is prepared to talk about them generally and with you in detail.  In addition we’ll be hosting our own Webinar – keep an eye on our twitter feed and webinar page for details.

Therefore in this blog post we are going to quickly go through the highlights of the changes and then dig into analysis of what this means to you.  Then in post II we’ll go through remediation strategies.


Since the eBay Donahoe era (March ’08 – happy 6yr anniversary JD!), eBay has increasingly been focused on collecting information from buyers and using that data to improve the buyer experience on eBay.  Donahoe is a big fan of the Net Promotor Score (NPS) system and eBay surveys most transactions in this way.  While customer satisfaction is a noble and smart pursuit (most eBay bonuses are determined by NPS scores for example).  It’s important to realise that eBay doesn’t operate in a vacuum.  E-commerce is increasingly competitive with Amazon setting the Bar.  In fact Amazon is the gold standard (ranked number 1 for 9yrs in a row by the ForeSee experience index) but all of the brick-and-mortar players are waking up to the importance of the buyer experience as are other ‘pure plays’.  In a world where the same product is available in a click or a finger-swipe on a mobile device, a great customer experience is even more important.

The good news is with $76.5b in GMV last year with say a $75 ASP, that’s 1b transactions where eBay has detailed data from NPS, money-back claims, PayPal claims, DSRs and feedback.  In fact, I imagine the data created from this is quite massive and complex to analyze because of some of the subjective datapoints (opinions that are hard to measure) and find the objective (measurable and reliable) datapoints.

Well, it seems like eBay has been doing just that – mining their buyer data because 100% of the changes in SR14.1 are oriented toward improving the buyer experience, which translates to a higher set of standards for eBay sellers.  Long-time eBay sellers are familiar with the constant bar raising, but before we get into the details, we wanted to go through a brief history of eBay and ‘sellers standards’.

Note: if this is a topic that is of interest to you, we strongly recommend downloading the December 2013 Foresee online experience report.

A Brief History of Feedback/Seller Standards

If we don’t learn from the past, we are doomed to repeat things in the future, so before digging into the new changes, we wanted to remind everyone of the five major periods of time for the eBay feedback/seller standard system:

  • 1996 – 2007 – Feedback is born – Pierre recently told Inc magazine about the birth of the eBay feedback system and why he chose ‘pos/neg/neutral’. It was added to the site 6 months after launch in February of 1996 so this system has been around for 18 years now.  The problem with feedback is it doesn’t give eBay enough data, why did the person leave a negative?  Was the neutral a bad experience, or was the buyer having a bad day?  Also, by the mid-2000’s, most sellers had 99% positive feedback ratings, so it was hard to separate the wheat from the chaff.  Finally, unfortunately bad actors came up with numerous ways to game/manipulate the feedback system.
  • 2007 – The age of DSRs – In 2007, eBay rocked the seller community by introducing a major overhaul in the feedback system.  What they added were four additional criteria called Detailed Seller Ratings (DSRs) that ask the buyer to rank a seller from 1 to 5 ‘stars’ on communication, item description, shipping speed and shipping cost.  eBay lore has it that the ruckus created by the community over this change led to eBay forever canceling the eBay Live tradeshow they used to have.
  • 2009 – Top Rated Seller Era – In July of 2009, eBay introduced the Top Rated Seller (TRS) program.  The changes from the DSR program were largely viewed as ‘sticks without carrots’ – all a seller received for having great DSRs was to stay in the eBay ecosystem and the punishments (sticks) ranged from search disadvantaging and suspension.  After reviewing the buyer implications from the DSRs, eBay decided to add a carrot called the TRS program.  If a seller maintained stellar DSRs, they were rewarded with a TRS badge, discounts on fees and (most importantly) search elevation.
  • 2010-2013 – Seller Standards bar raising Period – Every year since DSRs and TRS were introduced, eBay has tweaked the programs.  For example in 2010, they used the DSRs to drive free shipping by promising a 5 star on shipping costs.
  • 2010-2014 – BBE and NPS the time of a ‘shadow feedback system’ – As early as 2010, sellers started to be contacted about something called a BBE – Bad Buyer Experience.  Usually your account would be suspended or limited due to this metric being too high.  The challenge for sellers is 1) there’s no way to know what your BBE is on a regular basis (e.g. in myeBay) 2) There was widely different information on what a BBE is and 3) it was confusing to have feedback, DSRs and BBEs.  Also, while eBay started using NPS to understand how customers view feedback as early as 2009, in the 2012 timeframe many sellers were warned, limited and suspended because of poor NPS scores.  Even if you had great feedback, DSRs and BBEs, you were at risk.  Many sellers called BBE and NPS ‘shadow feedback systems’ because they were out there, but not provided to the seller so the seller could improve their business.

With that background in the history of all things feedback and how sellers have reacted to different programs, SR14.1 brings a new era in feedback.There’s a new sheriff (metric)  in town – the eBay Defect Rate (we’ll call it eDR for short) to avoid confusion with Amazon’s existing ODR – Order Defect Rate).


What are the big news items in SR1?

Last year’s SR13.1 contained a bunch of protections for sellers and large fee changes as you see here, so it was a very mixed release – fees, standards, some free stuff, etc.

This year’s SR14.1 is more focused and  has these major new components:

  • The introduction of eDR – one measure to rule them all.
  • TRS changes – TRS is being changed to now be based on the eDR.
  • Return policy changes – To qualify for the Top Rated Seller Plus seal and the 20% final value fee discount between November 1 and December 31, TRS sellers must offer extended holiday returns, with returns accepted until January 31st.
  • Returns policy – There will be improved messaging about hassle-free returns to consumers.
  • Category changes – There are some minor category changes coming April 7, 2014.

Analyzing the SR14.1 release, we believe the introduction of eDR and the ties to TRS are the most impactful part of the release and focus on them going forward.  We encourage you to read the eBay details on the holiday returns and improved return processing pieces.


How is eDR calculated? What is the eDR? What’s in and what’s out? Where do I need to be?

We STRONGLY recommend sellers take time to understand this one because it will have serious ramifications for your business for the foreseeable future.

Now that we have your attention, eDR is a percentage that looks like this formula:

Number of defective transactions / number of transactions * 100 (to get a percentage)

So for example, if you have 5000 transactions a month on eBay,  if 10 of them had an eDR ‘defect’ you would have: 10/5000*100 = .2% eDR.

This calculation is done over a 90 day period for most large sellers.  The eDR is a meta-metric, meaning it is derived from several other metrics.  In a way this makes it easier to understand because it is based of concepts like feedback and DSRs that everyone is familiar with.  It’s also very interesting to see what eBay has included and NOT included.

The new eDR includes (drumroll please):

  • Item as described DSR – If you get a 1,2 or 3, that transaction will hit your eDR
  • Ship time DSR – if you get a 1, that transaction will hit your eDR
  • Feedback – negative or neutral – If you are left a non-positive feedback, that transaction will hit your eDR.
  • Returns – If an item is returned and the reason is ‘not as described’ (SNAD in eBay-speak – significantly not as described), that transaction hits your eDR.
  • eBay Money back guarantee / Paypal purchase protection cases opened for INR (Item not received) or SNAD.
  • Seller-cancelled transactions – e.g. if you say you have 10 of something, sell them and then cancel 5 unique transactions, that is 5 eDR ‘hits’.

Some caveats –

  • Only sales with a US buyer count towards eDR – so if you have 1000 sales to the UK, they are excluded from the eDR calculation.
  • You have to have at least 8 unique buyers for this to kick in (doesn’t matter for larger sellers) for TRS sellers (5 for Above Standard Sellers)
  • Global Shipping Program (GSP) – For US sellers these transactions are not included (see first bullet).  FYI we are getting a lot of mixed messages from sellers about GSP suggesting that it is negatively impacting their TRS, hopefully this will clear that up.
  • Fast and Free – When buyers choose fast free shipping as calculated and offered by eBay, the seller is still automatically given 5 stars for both the ship cost DSR and shipping time.  Since shipping time is in eDR, FnF can help your eDSR.  However, remember that you still have the item as described DSR and other components that aren’t ‘helped’ by FnF.
  • Only one of these can ding you per transaction – for example if a buyer leaves you all 1 DSRS, leaves a neutral, hits you with a SNAD return and a money back guarantee, that’s only 1 strike against your eDR.

Analyzing what eBay has included here it is clear they are really focusing in on SNAD, INR and canceled orders.

 What’s NOT in eDR:

  • Shipping cost DSR
  • Communication DSR
  • BBE (seems to be replaced by eDR)
  • NPS (also replaced by eDR)

The logical question from these items missing is – “why would I ever charge free shipping again or answer customer’s questions.”  That’s a good question and one I’m sure these metrics will still matter in some way, especially if they get egregious, but for now eBay wants you to focus on eDR which does not include those items, so they must be less important now to the buyer experience.

Implications of your eDR

As you can imagine, your eDR determines which of three ‘buckets’ of seller you are:

  1. Top Rated Seller – You must have an eDR that is < 2% or 2 / 100 or 20/1000 or 200/1,000 transactions to be TRS.
  2. Above Standard Seller – If you have > 2% and < 5%, you are still a good seller, but you lose TRS benefits.  For this bucket there are no ‘consequences’ other than being disadvantaged in search.
  3. Below Standard Seller – If you have > 5% eDR, then you are a below standard seller and will face increasingly severe consequences from limits to all-out account suspension.  That’s 5/100, 50,1000, 500,10,000

To have your DR applied there is a minimum of 5 unique buyers ( for below standard) and 8 for top rated.  Also, unique buyers are applied at these tiers, but they are filtered out over the 90 day period.  For example, maybe you have a persistent bad actor that buys 90 items from you over a 90 day period and files a negative each time (of course you would block them before this but bear with me).  Those 90 DRs would be calculated as 90, but tallied as 1 over that period.  You will see the total number in your dashboards, but not the unique-buyer number.

We created this handy graphic to help you remember where you want to be on the eDR scale and where you start to get into trouble:


One other thing that is very interesting about eDR, is the better your eDR, the more you will be ‘search advantaged’  for example, if you have exactly the same item sold by two TRS, and one is a 1% eDR and the other 2%, the 1% eDR seller will show up higher in search results.  It will be interesting to see how sellers go through the calculus of the costs of this type of eDR vs. other levers such as lowering the product price, or offering free shipping. More on that in post II.  Before eDR, it was more of a ‘you are advantaged’ or ‘you are not advantaged’ type system (binary vs. linear).


eDR vs. Amazon ODR

As mentioned in the ‘Why?’ section, we highlighted that Amazon is the top online customer service brand.  Therefore we thought it would be interesting compare these two metrics:

Note Amazon looks at three metrics that determine your seller health:

  1. ODR < 1%
  2. cancellation rate < 2.5%
  3. Late shipment rate < 4%


For this comparison we’ll look just at ODR:



Looking at this table, you see there’s a key difference in how eBay and Amazon handle these things:

First, Amazon focuses on objective, measurable metrics for the bulk of their ‘inputs’.  Second, Amazon acknowledges that returns are part of the online shopping experience and does not hold them over a seller’s head.  This would create a misalignment and sellers would stop offering generous return policies.  In categories like shoes you have return rates as high as 30% and you would essentially cut all the supply out of this category if you focused on return rates (even if it is just SNAD/INR).  In fact A-Z claims are there as a safety net to help the buyer in the case where the seller has tried all the normal courses of action.

Sellers we’ve talked to find the Amazon requirements more logical and aligned with online selling whereas the eBay metrics (historically) are subjective and buyers learn quickly to manipulate the items to cause the maximum pain – a common design flaw in an objective system.  Conversely when one of these metrics is at risk, sellers can quickly and summarily be dismissed from the platform with zero human interaction, whereas to their credit, eBay does do a lot via customer service reps and what-not to educate sellers and keep them from driving off the cliff.

The conclusion is both eBay and Amazon have very different requirements and if you sell on both, you need to be very aware of them or face permanent suspension.

Sr14.1 Timing and impact

The new eDR goes into effect on the August 20 monthly seller evaluation, but that is a 90 day look back from the first of the month so it effectively goes into effect May 1st, so as of today you have ~50 days to get ready for the eDR.

We’ve heard from eBay that 2/3 of the current TRS folks are in good shape, while 1/3 have some work to do.  With some proactive work, I’m sure all of those folks can get back in the TRS saddle and we may have some new additions as people look at this program and decide it is more valuable/less expensive than the four-DSR based TRS system.

Note that there are some seller grace periods detailed in the eBay announcement that give TRS folks time to get changes implemented and counteract the new eDR if needed.

Also, eBay plans to send out letters with details of what your eDR would look like TODAY along with a comprehensive dashboard in mid-April.

Conclusion and What should I do?

Reviewing the SR14.1 release, it looks like eBay is taking a big step to address what they believe drives customer satisfaction (and thus increased activity/buyer).

By now, you may be starting to break a bit of a sweat and have started worrying about what this means for your business – that’s good.  We have detailed recommendations coming soon in post II, stay tuned.

We look forward to hearing your opinions (in comments or through other ChannelAdvisor channels) and helping you navigate the ever changing world of selling on eBay.

This blog post was written by Scot Wingo, CEO, ChannelAdvisor Corp.