IntelliShop Report Integrity

Our clients rely on the evaluations we do for them to help make very important decisions about their businesses: whether someone receives a reward for exceptional performance or a reprimand when they didn’t perform to standards; whether they believe their customers are being taken care of, and what to do about that; whether their training programs are effective, or if additional training is needed; in some cases, they impact whether an employee or manager gets a raise, gets promoted or possibly is asked to leave.  So we know that every report must be 100% accurate.

We strongly believe that we have the most robust systems in our industry to assure the highest level of quality assurance, to identify any breakdowns that occur, and to prevent any such reports from making their way to our clients.  Our processes for this are broken into three categories: preventing mystery shoppers from submitting falsified reports; identifying falsified reports if someone attempts to submit one; and, assuring that the information submitted matches our client’s requirements, is truthful, and complies with our high standards.

When we originally designed our reporting and quality assurance processes, we started with the issue of preventing such occurrences; we also frequently review and test our systems and processes, and update them as we learn more about them to create new ways to make them even more robust.  It’s similar to retail loss prevention systems: as new techniques are developed to prevent such things, a few bad apples will always try to find ways around them, so we must always be vigilant about awareness, communication, and tweaking our approach as necessary.

One important step in this process is educating our clients that one reason we provide such a high level of detail in each report is so they can partner with us to be the “last line of defense”, and can help assure maximum integrity and quality by reading the detailed reports and bringing such things to our attention immediately.  In some cases, there is information that we wouldn’t know to be true or false that only the client can verify.  Without our requirement for such detail in the reports, this final step would not be possible.

Here’s some more detail on specific measures we follow to assure maximum quality and integrity in every report:

Our President/Owner, Ron Welty, has been on the board of our international trade association for more than six years.  He frequently networks with competitors to stay current on fraud prevention and detection issues, and use that information every day in our company.  We also have a group of friendly competitors, and whenever a fraudulent mystery shopper is identified, we share their information with the group so that we can all deactivate them in our databases, thus protecting our entire industry.

We verify every mystery shopper’s Social Security Number with the IRS, automatically.  If the number they submit cannot be verified, they are not allowed to shop for us.

We insert “integrity questions” into the evaluations, viewable only to the mystery shopper and our quality assurance team.  These questions are intended to “trip them up” if submitting false or inaccurate information.  If an mystery shopper answers the question in a way they shouldn’t, that triggers us to investigate further, and to question everything that mystery shopper submitted in that report.

New mystery shoppers cannot enter our database without first taking a simple screening test.  This has two benefits: if someone is trying to get into our database simply to be malicious, making them expend this additional time and effort should stop them from doing so.  It also is so simple, that if someone cannot “pass” it, we will not allow them to receive any shop offers.

We grade mystery shoppers following each report submitted on a 10-point quality scale, 10 being literally, “perfect”.  Our Schedulers then view the average rating for each mystery shopper, and can view each evaluation grade they submit, when deciding whom to offer additional work to.

Our database requires specific demographic information to be entered by the mystery shopper.  Various demographics are often used in the mystery shopper screening process.

We were a key driver within our trade association of developing a mystery evaluator certification program several years ago, at Gold and Silver levels.  We now have almost 100,000 certified mystery shoppers nationwide, and they are given strong preference and first choice of our assignments.

When the mystery shoppers apply for the assignments, our scheduler hand-picks them based on what their profiles show in terms of past experience, average grades, demographic matching, and certification level.

Mystery shoppers are assigned a maximum number of locations for any one client program, as well as overall for our entire company.  This is to try to prevent someone from accepting a high number of assignments, then trying to get them all completed in one or two days, which leads to an increased potential for inaccurate, inadequate, “thin” or just falsified information.

We alert our mystery shoppers that many of our clients now utilize digital security video systems, and the recordings can be viewed to match some of their report content.

Another preventive measure is that we also let them know that we review their reports word-for-word and score-for-score, and match them against past reports to assure that all detail is fresh and accurate.

We rotate the mystery shoppers so that they are not allowed to visit the same location more than “x” times, or must wait “x” months before going back to (or calling) the same location.

We require a tremendous amount of detailed, story-like comments from the mystery shopper.  We were the first in the industry to do this, 11 years ago.  Our goal then was that, by requiring this, not only would we provide more value to our clients to act upon, but it would also be a major tool to prevent anyone from submitting falsified reports, because of the effort and accuracy required, and the verification that such detail can bring.  This has proven true, and we believe it is a major reason we provide such high quality and integrity to our clients.  As mentioned before, another benefit of this is that it allows our clients to read the reports and let us know if they suspect incorrect or falsified information

If purchases are made, and receipts can be obtained, the mystery shopper must upload those into our reports digitally, or fax them to us and we upload them.  Those are verified by our quality assurance team. 

We are developing a new “Photo DNA” software module that will allow us to scan receipts, photos, and other digitized images, and will flag us (based on that file’s digital signature) if there are any potential duplicates across our entire system.  We are also partnering with several friendly competitors on this, to allow cross-system scanning for further fraud prevention measures.

We require business cards, brochures, and similar items when purchases are not actually made.  This is a fine line to walk, because we want these as a verification of the visit, but if your customers don’t routinely request such things, we don’t want it to become a “giveaway” that someone is a mystery shopper.

We have a team of dedicated Proofreaders who are assigned to specific client programs.  Once a report is submitted, one of them reviews it word-for-word and score-for-score, to assure compliance with client requirements, our high standards, and to detect any potentially inaccurate information.   

Unlike many of our competitors these days, we have not “automated” the comments process in which they require mystery shoppers to use pull-down menus of pre-scripted comments and don’t allow verbatims, in order to save time and money in their QA processes.  We will never do this, and we have a major professional difference of opinion with such a tactic, as we strongly believe it opens up the potential for falsified reports even more.

Following the Proofreader’s review, they award the mystery shopper a score on our 10-point quality scale, and provide them with constructive feedback.  This creates better-educated mystery shoppers, and allows us to have a record of their performance that is then used in determining to whom future assignments should be offered.

The following are just some of the “red flags” our Proofreaders are trained on to watch for in reports from mystery shoppers:

• Uploading a receipt/business card/deposit slip from a different location, making it appear as though it is the correct receipt/business card/deposit slip.
• Uploading photos from a prior report into a new report.
• Copying/pasting comments from one report to another after being assigned multiple reports.
• Copying/pasting comments and descriptions from a previous month’s report to the new report.
• Gave all generic comments and descriptions, and, typically, wrote “N/A” or “no name tag” for the associates’ names. Additionally, the mystery shopper probably scored the location 100% so s/he didn’t have to make specific comments.
• Did 2+ evaluations in a very short amount of time.
• Did 2+ evaluations on one day, and the times listed on the reports conflict.
• Attempted to “create” a receipt/business card/deposit slip, either by writing it by hand (and claiming that the register wasn’t working), or by making it on a computer.
• Wrote that s/he visited a certain area (i.e.: bathroom, department, gas pumps, etc.) that doesn’t exist at the assigned location.
• Did a gender-specific report incorrectly (i.e.: male performing female-specific evaluation).
• When we suspect falsified information in a report, we immediately attempt to phone the mystery shopper.  If we reach them, we walk them through a series of questions and techniques we’ve developed over the years that put the onus on them to prove to us that the report is not falsified.  By the end of that conversation, we know one way or the other.
• If we don’t reach them, we leave them a voice mail and email (often several of each over a 24-hour period), basically letting them know we suspect their report is falsified.  That results in mystery shoppers who did falsify them to not contact us, leading to a conclusion.  Or, the ones who did actually submit a “good” report contact us immediately, to rebut our accusations in order to get their work accepted and get paid.
• If a report doesn’t meet our standards and/or we suspect that it’s been submitted with inaccurate or falsified information, we place it into Excluded status in our system, find another qualified mystery shopper, go through the education and coaching process with them again, and in the end submit only a (hopefully) “great” report to you; you never know about those reports which don’t meet these standards. 
• If a fraudulent mystery shopper is identified, they are immediately deactivated in our database, and that information is communicated among our friendly competitors in order to protect our industry.
• We also have created a “Evaluator Trapping Tool” in our database, that allows us to secretly deactivate a mystery shopper, and then our system continually checks new applications to make sure they aren’t attempting to get back in; it looks for similar names, phone numbers, birth dates, passwords, IP Addresses, etc.