Another pseudo-survey on corporate blogging hits the Web

Try to say this with a straight face:

The vast majority of companies (89%) are either blogging now or planning to blog soon.

Are you LOL? ROTFL?

Anyway, the staggering, unbelievable, hard to believe “statistic” is one of the results of a Guidewire Group “survey” called Blogging in the Enterprise, sponsored by iUpload.

To its credit, the Guidewire Group is transparent about the survey’s methodology (links added):

The “Blogging in the Enterprise” survey was fielded for two weeks in September 2005. The online survey was open to public participation, encouraged by direct email to a random sample of 5000 CMO Magazine readers, a press release announcing the survey, and unsolicited postings in various blogs and blog search engines. 140 individuals responded to the survey.

(Source: PR Newswire press release)

(Missing: the fact that those completing the survey were eligible to enter a draw to win an iPod Nano or a complimentary registration to BlogOn 2005.)

A couple of questions about the survey:

  1. Why would anyone use a “random sample of 5000 CMO Magazine readers“, then give up any control of the survey instrument’s distribution by making public the link to the questionnaire, and by asking bloggers to post about it? The result is that the survey has used a nonprobabilistic sample, and that no generalizations can be made about the data outside the small number of survey respondents.
  2. What’s the guarantee that those who took the survey were in the position of knowing the information they provided?
  3. If Guidewire was aware of the survey’s limitations –as the press release shows– why is ignoring them by talking about the results as if they were relevant for the corporate world? E.g.: Staggering Stats on Blog Adoption (Demo Letter), BlogOn 2005 Opening Remarks (BlogOn 2005 Blog).
  4. Finally, I wonder what was the reaction of participants to BlogOn 2005 when these staggering “stats??? were announced. Unfortunately, I couldn’t find anything about it in the entries tagged with BlogOn and BlogOn2005 on Technorati; and there’s no reaction so far from the fellow PR bloggers who participated (most of them as speakers) to the event: Elizabeth Albrycht, Shel Israel, Mike Manuel, Laurie Mayers, Jeremy Pepper, and Lisa Poulson. Steve Rubel has a link (Survey: Corporate Blogging Takes Off) –but no comment– to an article from internetnews.com.

Side note: there’s no difference, essentially, between Guidewire’s survey and the Edelman/Technorati recent survey. In fact, Guidewire was more upfront about the study’s limitations then Edelman (we’re still waiting for Edelman’s official report about the methodology, two weeks after the results were released). But the effects of self-selection are more visible in Guidewire’s survey.

Resources

Reactions from the blogosphere

  • Mike Gotta: “The survey has some potential and initial statistics are interesting but the survey unfortunately does not have enough data points to warrant serious attention. […] My first concern is the sample size. 140 respondents is small. The respondent group was also self-selecting which can skew results (even though the survey was publicized, it is not the most balanced way of gathering statistically relevant data). Only 19% of the people who responded were from companies with greater than 1000 employees (that’s about 26 people). Around 32 people responded from companies with revenue of over $100 million. It’s unfortunate since the media has picked these statistics up as trends across “corporate america” and clearly that inference is a leap of faith unless you want to extrapolate that based on 26-32 people. You can of course. I would prefer to still classify corporate blogging as embryonic and in an exploratory/pilot phase within the vast majority of companies.”
  • Shel Holtz: “Guidewire Group’s BlogOn 2005 Social Media Adoption Survey suggests 91.4% of these corporations are using blogs internally and 96.6% externally. If you believe these results, I have the deed to a nice bridge we should talk about.”
  • Anu Gupta: “I don’t understand why the general reaction [to the survey] hasn’t been more disdainful – I wonder what the reaction would have been if, with the same sample size, the headline was ‘2% of companies are using blogs’?”

(More to come.)

Media coverage

7 Comments

  1. Hi Constantin,

    Thanks for the critiques. Our goal as a research firm is to provide information that will explain emerging technology markets, so your feedback will help us develop more robust products in the future. I’m going to post a couple of general comments in response to your post and others as an update to the original post, but I wanted to answer your questions directly:

    1) The primary goal of the survey was to explore why and how corporations were adopting blogs. That’s why we promoted the survey via press release and to the blogosphere…because we wanted folks who were blogging to answer it. The fact that such a large percentage of the sample were either already blogging or had plans to blog (and frankly the fact that a huge percentage of the adopters started in 2005) is what lead us to conclude that corporate blogging has already turned the corner into hypergrowth.

    2) No guarantee. That’s why we were explicit about the sample and methodology. What surprised us was the distribution of titles/roles that responded to the survey.

    3) It’s relevant to the corporate world because there are actual examples of ‘real’ companies that are using blogs and other social media tools for significant business benefit. There were terrific examples at BlogOn. Given that most coverage of blogging outside the blogosphere has been extremely light on corporate case studies, benefits, etc., we wanted those folks trying to learn something about social media to know that SOMEONE is looking into it. Hopefully it will cause MORE people to look into it.

    4) The reaction at BlogOn to our annoucement (we only showed the adoption curve…the Executive Summary was put on everyone’s seats so they could read it) seemed pretty positive. In my at-the-show conversations with attendees and speakers, they we’re all pretty enthusiastic that we were drawing attention (both via the show and the survey) to corporate adoption and benefits, rather than the same old debates about journalists vs. bloggers, media transformation, etc. We’ll be posting the podcasts and webcasts from the show in the near future, so between that and the folks you mentioned getting back to their offices and deciding they want to post about the survey…I’m sure you’ll be able to come to your own conclusions.

    Again, thanks for the feedback. And if you have suggestions about what we should investigate and/or how to improve our methodology…we welcome them!

    Mike Sigal
    Co-Founder and CEO, Guidewire Group

  2. Pingback: PR meets the WWW » Guidewire Group CEO responds to criticism

  3. Pingback: BusinessBlogWire.com

  4. As the author of the internetnews.com article, I am rightly reprimanded for not mentioning the small sample. I did discuss this with Mike during our interview; I should have included his statement that, because the various results were well distributed, he felt the results were indicative of the market.

    As for being more skeptical, unlike bloggers, I’m not allowed to inject my own opinions. I could, however, have called some other corp. blogging consultants to ask if the results mirrored their experience. Mea culpa!

  5. Pingback: infOpinions? :: Public Relations » Blog Archive » PR Blog Surveys Abound :: As of yet, only one makes the grade

  6. I do not believe anyone has to be reprimanded. What I think should have been more prominently acknowledged was the small sample size and the fact that people self-selected in terms of answering the survey. There are interesting nuggets in the results but people need to take them more as a signpost and not a current-state trend of adoption. The problem occurs when you try to extrapolate from such a small data set to make claims universally across large or small enterprises. A weak statistical foundation can sometimes be acceptable if you have a audience that is unique in some way (say, 300 data points from CXOs that collectively represent a larger population). When media reports pick up on the percentages, it sounds phenominal. In reality, it is not. Blogging is still emergent behind the firewall. I do observe more and more companies looking into blogging and applying the tools to improve process or community-based activities (e.g., competitive intelligence, project management and journal/diary related tasks). So the future is bright. We just need to avoid over-hyping the technology and misrepresenting current state adoption.

Comments are closed.