It has come to our attention that representatives of GDS Publishing, Inc. which publishes magazines such as “CXO”, “Financial Services Technology” and “Corporate Finance Management”, and conducts trade events in the IT industry, have told a number of its potential customers that Gartner or Gartner analysts contribute to a GDS publication or is participating in a GDS event. These statements are false. Gartner has instructed its analysts not to work with GDS Publishing in any capacity.

Submit a Comment | Permalink

The February 6 issue of InformationWeek had an article entitled, Credibility of Analysts. While I was pleased that this blog was quoted a couple of times, there were several issues that bear comment.

One was the insinuation that Gartner writes reports that are funded directly by tech vendors. Knowing how unethical that can be, Gartner simply doesn’t do white papers. Period.

InformationWeek also questioned the transparency of Gartner's research methodologies. In fact, Gartner has documented and published its research methodologies for all to see and has a team dedicated to maintaining and improving these every day.

Another comment may have left the impression that Gartner’s research is somehow influenced by Gartner’s investment in the SI Ventures II fund and by outside investors such as Silver Lake Partners. Gartner sold its investment in SI Ventures II last year. And a distinct line is maintained between the “research” side of Gartner and the “business” side (I’d include Silver Lake Partners here, but also Gartner’s relationships with vendor clients): Gartner’s analysts are trained very specifically how to steer well clear of that line when they join the firm. If someone inside or outside Gartner wants to raise a question about our independence, they are always encouraged to contact the office of the ombudsman.

Finally, the article questions why end users would pay for independent research when so much information is available free. This isn’t so much an objectivity issue as one of value. So, wearing my Gartner hat instead of my ombudsman’s, here’s my response: You get what you pay for. Gartner has 700 analysts studying all aspects of IT, using a rigorous, quality-controlled research process. Wouldn’t you rather have 700 full-time experts there to help you make mission-critical decisions worth millions of dollars that could make or break your business and career than depend on a document you got for free on the web?

Here are some links to previous ombudsman blog posts that address some of the article’s other issues:

"Pay to Play" – in addition to the fact that half of the vendors in our most recent "Cool Vendors" report are not clients, check out these posts:

Vendors can't "opt out" of a magic quadrant or pay to get in

In Response to Rob's 14 Sept. Comment - More on Pay-to-Play

Specific vendor client relationships:

Why Gartner Won't Release the Names of Vendor Clients

Transparency in evaluation models:

New Magic Quadrant processes: Have we done enough? You tell us.

I’ve also included below a couple of links that might be helpful for those with further questions:

Code of conduct that all Gartner associates follow.

To schedule a Vendor Briefing

14 Comments | Submit a Comment | Permalink

In an advertisement run in The Wall Street Journal on November 28, 2005, Oracle made the following claims:

"94% of customers run up-to-date Oracle Applications (Easy to upgrade at no additional cost)

4% of customers run up-to-date SAP Applications (So expensive and difficult to upgrade 96% of SAP customers didn’t do it)"

The advertisement included a citation stating: "Statistics from Gartner Report March 2005"

The Gartner Report titled "ERP Upgrades Will Reflect the Uncertainties of the Overall Market" of March 2005 appears to be Oracle’s source of the data quoted. The relevant data is listed below.

To compare 94% with 4%, Oracle has contrasted numbers for two different time points and aggregated all the Oracle applications shipped as Release 11i (which comprise five distinct versions, shipped over a five-year period) and compared that with one version of SAP product (which became available to customers in March 2005).

The advertisement egregiously misrepresents the data.

It should also be noted the products referenced by Oracle are only those shipped as Oracle E-Business Suite. Oracle currently has a number of other Application products (including PeopleSoft's Enterprise, JD Edwards' EnterpriseOne, and JD Edwards' World). A higher percentage of Oracle customers using those products are on versions which would not be considered "up-to-date."

2 Comments | Submit a Comment | Permalink

Gartner analysts have started using a new research note format called, "Findings". This note type is short, focused on an important tidbit of information, and timely -- that is, a research meeting on a Friday could yield a "Findings" note on a Monday.

The Findings notes are not intended to usurp the standard research note -- a best practice is not to make decisions based on a Findings alone since there's probably not enough detail -- but rather to release research to clients while its on the tip of the analyst's tongue, so to speak. The benefit of this is that it helps draw you closer to the collaborative process and whets your appetite for more detailed research to come.

Findings joins podcasts and blogs as one of the new ways Gartner analysts have to disseminate research, but they differ slightly in that they go through the same processes as standard research and are representative of the opinion of Gartner research, not necessarily of a single analyst, and they are available only to clients.

What do you think of Findings? We'd love to hear from you.

4 Comments | Submit a Comment | Permalink

This entry is long overdue but I won't burden you with all of the excuse ... suffice it to say it's been a busy week catching up from a busy Symposium. What a treat last week receiving so much reinforcement from all of Gartner's constituents (end users, vendors, press, investors, and consultants), that reputation and trustworthiness are cornerstones of the operation (and success) of the IT industry. And, that Gartner plays a pivotal role in assessing and validating those characteristics, particularly with respect to vendors. As such, Gartner's own reputation and trustworthiness must remain above reproach to lend credibility to the marketplace.

We take this responsibility to heart. And we think, largely, the IT industry is alive and well from reputation and trust perspective. We'd love to hear what you think!

Submit a Comment | Permalink

It's that time of year again ... the annual trek to Orlando for Gartner Symposium/ITexpo. This marks my eighth year attending this event, and my first one as an Ombudsman. Symposium is an extraordinary event, and I don't say that simply as a Gartner employee. The sheer amount of information communicated, the sheer number of interactions between attendees and analysts, or even better, attendees and attendees is just overwhelming. The activity level is impossible to describe ... you need to be there in person to experience it.

For me, and for Nancy, Symposium is an especially important opportunity to spend quality time with attendees, exhibitors and analysts. We get to watch the presentations, talk to audiences, and gauge their reactions to Gartner research quality and objectivity. All of our constituencies are here, and we are able to observe them at work and at play, in their natural habitat. These observations yield poignant insights about the character of Gartner, the intentions of the technology providers, and the aspirations of the end-users.

There's nothing sinister about this, on the contrary, we try to do it an open, accessible, and visible way. That said, if you're at Symposium, and have something to say or just want to meet, feel free to track us down - you're sure to find us somewhere around the conference sites, or send us an email at ombudsman@gartner.com and we might be able to arrange a specific appointment. And we'll give you an update later this week with some more commentary from Symposium.

Submit a Comment | Permalink

Nancy and I would love to put the perception (and if there is one, the reality) of pay-to-play to bed forever, but I suspect that it will be impossible to convince all the people, all the time. While much of the time what we hear is just noise, we take all allegations very seriously - substantial time and money (including our compensation) is devoted to ensuring that Gartner is operating in an ethical, objective and independent manner. Be assured, we will investigate suspicious activities within any part of Gartner and infractions of Gartner's Code of Ethical Conduct could subject the associate to termination of employment.

Frankly, aside from the occasional comment on a blog or backroom chatter, we have not had any vendors complain to the Office of the Ombudsman directly since the Office opened in September 2004. So my message is, if anyone believes that they know of a legitimate issue, please step forward regardless of your client status. You can come to me directly, and I will personally investigate the situation. And I want to better understand what the issues are; does pay-to-play mean the more you pay the better your position in the MQ, or that without being a client you can't get any coverage or schedule a briefing with an analyst, or all of the above?

In addition, I will do some internal investigation looking at our client and non-client lists to see what conclusions we can draw about research coverage, and I will share what I can about those conclusions with you within the limits of our privacy policy. I probably won't name names, but I can tell you what patterns I find. But beware this may take some time, and, since a large percentage of vendors are already Gartner clients, this analysis may not reveal anything meaningful.

5 Comments | Submit a Comment | Permalink

A recent road trip to visit a number of IT vendors and service providers on the West Coast of the United States yielded lots of great comments and suggestions. Why just vendors, you might ask? Vendors can be both clients and the objects of our analysis; their perspectives on both sides of this equation are important for us to understand.

Sally Pinkerton, VP Vendor Relations, accompanied me on this trip. To meet the specific needs of vendors, the Office of the Ombudsman includes a vendor relations function that helps vendors understand how best to interact with Gartner and acts as a clearinghouse for issues with Gartner research findings, processes and policies. Sally's team reports to me because so often queries that come in as ombudsman issues require that both departments get involved - indeed, often that we work together, to address them.

It was fantastic for us to meet vendors face to face, on their turf, without specific "issues" to deal with. We asked for their feedback, and wow did we get it. We've since come back and made some pretty detailed recommendations to Gartner senior management about addressing this feedback.

Here are a couple of things raised by more than one company:
- We haven't articulated a clear product strategy that addresses vendor client needs. Response: The fact is, we do have a cohesive strategy, and to describe it we've scheduled a call for Analyst Relations contacts (and anyone else who's interested - contact ombudsman@gartner.com). The call will take place on 14 September at 11:30 AM U.S. Eastern Time.
- Ordering reprints seems to take longer than it should. What can be done to simplify the process? While we already offer an option to place a "rush" reprints order, we're now working with the Reprints department to create a simple, step-by-step "How to Order Reprints" guide.

We make the same offer to other vendors, as well as end users - clients and non-clients. If you would like us to visit you at your site, please let your Gartner Account Executive know, or if you don't have one, contact us at ombudsman@gartner.com. We'll make a point of stopping by next time we're in your area.

Submit a Comment | Permalink

We are asked regularly, "Why won't Gartner release the names of information technology vendors or service providers that are Gartner clients?" The questioner's intent is to assess how client status affects favorability of Gartner's research positions. This makes an assumption that we favor clients over non-clients in our positions. This assumption is dead wrong.

Regardless, we can't provide the names of any Gartner clients -- vendors or users. We're contractually bound to treat all client information as confidential, including the client name. This is important to protect the sanctity of the advisory relationship, which is why companies become Gartner clients in the first place.

All Gartner research goes through rigorous peer review and management approval processes that ensure all vendors are being treated equally. Client status doesn't enter into any of these discussions. And frankly, Gartner analysts would be the first ones to raise a stink if something was awry: they value their independence and objectivity that strongly.

Of course, if you believe that a vendor or service provider is getting favorable treatment due to their status as a Gartner client, please bring that issue to the Ombudsman. We will thoroughly investigate and report on any perceived bias.

2 Comments | Submit a Comment | Permalink

We occasionally get calls from vendors asking to be removed from a Magic Quadrant because they disagree with our market definition, or their positioning, or that they even compete in the specific market. On the flip side, we've gotten questions from vendors about how much they need to pay to get on the Magic Quadrant. We sternly remind both sets of callers that they can't elect to be omitted or pay to be included in a Magic Quadrant.

As you've seen, the recent updates to Magic Quadrant processes expose the metrics we use to evaluate every vendor, as well as the weights we give to each of those metrics. Gartner analysts create these metrics and weights with one thing in mind: what end users need to know to make informed decisions. You can see the metrics yourself. Not one of them identifies how much revenue a vendor pays Gartner. Not one of them gives a vendor the ability to opt in or opt out: If a product meets the criteria for inclusion in a Magic Quadrant, that vendor is included; if it doesn't, it is not included. Client status has nothing to do with it. If we were not to follow these policies, obviously this would compromise the integrity of the research.

This doesn't mean that an analyst occasionally will not include a vendor that does meet the market's inclusion criteria because that analyst simply wasn't aware of that vendor. We're not omniscient.

Don't agree? Call or email us. We'd love to hear from you.

4 Comments | Submit a Comment | Permalink

We received a comment posted by Greg Coticchia of LogicLibrary (a vendor) asking about Gartner analyst consulting activities and incentives. Since this is probably a question on the minds of many, we've elected to respond to it here (rather than in the comments).

Greg's questions were:
"How are the Gartner analysts incented? Is there a divide between consulting and publishing (especially when covering vendors?) How are potential 'conflicts' like these handled or disclosed to readers and customers? (These seem inevitable and similar to the Big 5 conflicts between consulting and accounting that we witnessed in the last several years)."

Our response:
Gartner analyst compensation is a combination of fixed salary and bonus. The bonus is determined by a weighted combination of quantitative and qualitative measures for written research, inquiries and client-facing activities. It also depends on overall corporate performance. There is no direct relationship between Gartner revenue received from vendors and analyst compensation or research agendas.

In addition, Gartner analysts may receive a one-time incentive payment for consulting business they've identified that is successfully closed by Gartner Consulting (regardless of the analyst who works on the project, if any). This amount, when awarded, is a very small percentage of analysts' overall compensation. Gartner consultants do not receive any incentive payments for the closure of Gartner Research business. There are no other forms of incentives for "leads" shared between consultants and analysts.

It is possible, and likely, that an analyst will do consulting for both vendors and end-users in their area of coverage. There are numerous safeguards to prevent analysts from manipulating this situation, such as:

• A limit on the number paid days an analyst can spend with any one vendor


• Rigorous peer review processes to identify potential conflicts of interest


• Management approval required for all paid engagements


• Analysts are restricted from using content derived from specific Consulting engagements in their research


• Continuous random client satisfaction surveys by a 3rd party for both vendors and end-users


• Scrutiny from both vendors and end-users pointing out any perceived irregularities.

Submit a Comment | Permalink

Gartner Research has made changes to the Magic Quadrant (MQ) processes - partly based on feedback from clients, and partly based on observations we have made about how MQs are used. We're excited about these changes, because they bring more transparency into the process, which frankly is important to all of the Ombudsman's constituencies: vendors, end users and Gartner associates. As you might imagine, we get more feedback on MQs than any other evaluation models we use - by far. Many of the changes we've made have been based on your -- shall we say "candid" -- feedback. Keep it coming.

You've told us that you want to understand more about what's actually behind the "dots" we place on MQs. Some typical questions:

• What criteria do you use, and how do you weight it?


• Why do some markets get MQs and others Market Scopes (MS), or nothing at all?


• Why do some MQ documents actually look different from others - are they truly different?


• Why do I have to toggle back and forth between MQ criteria and vendor documents; why can"t I just view it all together?


• Why don't you add the best features of the META Spectrum model to the MQ (for starters, providing relative weights to metrics)?


• Why can't you tell us even remotely when you'll be publishing MQs for the markets I care about?

We will answer these questions and more with the announcement.

We've also noted that some end-user clients have made limited - and often inappropriate - use of MQs by evaluating only the vendors in the Leaders' quadrant. You'll see in the updated MQ document that we explain why it's essential that end users not make such limited use of the MQ.

We think the changes are broad and substantive, but will they be enough to allow end users to derive greater value from our research? It's too soon to tell. Please let us know what you think about these changes as you see them unrolling. Did we convince you to look at vendors other than Leaders in a market? Have we provided enough consistency and transparency? If not, what should we do to change that? The more you tell us, and each other, what you think, the more we can continue to improve this process.

5 Comments | Submit a Comment | Permalink

So now that you know what an ombudsman is, I'll introduce the people behind the titles. Larry Perlstein and I are co-ombudsmen at Gartner. Larry is an eight-year Gartner veteran, much of that time as a research analyst and later as a research manager. He has 25 years of IT industry experience - with IBM, General Electric and others. Me? I've been at Gartner for more than 20 years and have been a research analyst, created our rigorous "analyst boot camp" training program and ran our publishing operation, among other things. I've been here for so long that if I don't know the answer to a question, you can bet I'll know where to find it.

There are two of us because so many issues are not "black and white," and it's often valuable to talk through an issue with someone who also deals with them every day. You'll hear from us about once a week in this blog. Stay tuned.

3 Comments | Submit a Comment | Permalink

Since the formation of the Office of the Ombudsman, Larry and I have fielded many questions from vendors, users and Gartner associates. The first one we hear is, "What's an ombudsman?"

In journalism, the ombudsman is the "voice of the reader." In government, he or she serves as the process and policy police . In both cases, the focus is on protecting integrity, expecting ethical behavior and having an objective eye. At Gartner, the ombudsman is all of these.

We established the Office of the Ombudsman in the fall of 2004 so you would have a place to share your concerns with us about the independence, objectivity or accuracy of our research. Since then, we've gotten hundreds of inquiries and feedback. A few typical questions that cross our desks: Why is my "dot" so low on the Magic Quadrant? Why did your analysts not accept my Vendor Briefing invitation? How come my competitor "gets away" with posting out-of-context material on their Web site?

All good questions. We'll tackle these -- and more -- right here. We encourage your comments and we're looking forward to some lively discussions.

3 Comments | Submit a Comment | Permalink

Welcome to the Ombudsman’s Blog. We're excited to have this forum to share some of what’s been crossing our desks. In this blog, we'll talk about issues relating to Gartner's standards of objectivity and integrity.

Think of the blog as a place to go with questions about our standards and how they relate to our research or any aspect of the Gartner business. You can help us set the agenda by sharing what’s on your mind. We promise to have an open and frank discussion about things like why our policies are the way they are (the reasons make sense – at least to us), which issues keep coming up, and even on occasion admitting when we don’t quite get it right.

Submit a Comment | Permalink