Our Survey Says ... Work Out Loud!

(This is Part One of a two-part post. You can find Part Two here.)

As you have probably surmised from my other blog posts here, I really like the idea of Working Out Loud. I believe it can have truly transformative effects at every level of an organization -- individual, team, and enterprise. John Stepper's book and web site have done a great job detailing the individual benefits for those who develop that mindset and behavior.

My interest tends to lean more toward understanding the organizational benefits. But a big hurdle toward studying this aspect of WOL when I started looking at it a few years ago was the lack of ability to actually measure it in some way. There are lots of good stories about how WOL made a difference in a company. I even have a few of my own. But there's an old saying that "data is not the plural of anecdote," and I felt there needed to be some way to measure WOL numerically so that organizations could track progress and correlate it with other metrics.

As luck would have it, I happened at the time to also be searching around for a topic for my PhD thesis, and decided this might be just the thing. I wanted to create a survey that had the following characteristics:

• a close relationship to WOL as commonly discussed
• something that measures actual behavior, not attitude
• ability to create a numerical score that could be tracked over time
• the smallest number of questions possible that would still constitute a statistically valid survey, so that it could easily be given repeatedly or added as an extension to other surveys

I took as a model the definition of WOL that Bryce Williams of Eli Lilly had proposed back in 2010:

Working Out Loud = Observable Work + Narrating Your Work

In that seminal blog post, he wrote:

Narrating Your Work implies the act of journaling (blogging, micro-blogging, etc.)what you are doing in an open way for those interested to find and follow…however, by terminology doesn’t necessarily describe creating the work outputs / deliverables themselves in a manner available for others to consume. It also brings with it a “feel” of an additive activity to already-existing workload, which in my experience, some folks can be reluctant to accept . . .

Whereas Observable Work to me implies creating / modifying / storing your work in places that others can see it, follow it and contribute to it IN PROCESS. The key being that items are available during the course of being worked on, and not waiting until a “final” deliverable to publish to a broader audience.

But those two concepts combined, however, bring it all together. Social-based software platforms can aid in this process, with capabilities that automatically “narrate” your Observable Work activities by publishing notices to the activity streams of your followers or the followers of communities in which you are conducting Observable Work. But the art we develop as socially proficient knowledge workers is where and how to best complement the activity-triggered auto-narrative with our own meta-narrative.

When I looked at the ideas of narrating work vs. observable work I immediately thought of individual vs group. Many teams already work in an online form through email or enterprise social networks. The WOL mindset says that you just keep doing what you're doing, only in the most open way possible. Individual work is often harder to do in the open, which is why it's important to narrate that work so that others can keep up with what you're working on.

So I targeted development of a 6-question survey, with three each for Individual and Group WOL. I collected a set of possible questions from my colleagues at Change Agents Worldwide and others, then ran a global pilot survey within my own company (Lexmark) to determine which questions (if any) were the best fit. You can read all the gory details in my dissertation, but the end result was a set of these six survey questions (responses to each on a scale of 1 to 7):

Group WOL:

1. "When I work on a team, we share the team's goals in a way that those in other parts of the organization can see."
2. "When I work on a team, we communicate with each other in ways that those in other parts of the organization can see."
3. "When I work on a team, we make our work visible to the larger organization before it is complete."

Individual WOL:

1. "I share my thoughts and ideas on <name of your ESN> with others beyond my immediate co-workers."
2. "I share difficult problems on <name of your ESN> with others beyond my immediate co-workers."
3. "I participate on <name of your ESN> by starting discussions, making comments, and creating status updates and blog posts."

OK, so let's assume you like this survey and decide to give it to the members of your organization. How might you use the results?

• It's been statistically validated such that you can average or sum each set of three questions to get a "score" for Individual WOL and Group WOL. The statistics didn't allow for averaging all six questions together into a single score, which makes sense since although the two types of WOL are related they are truly different kinds of behavior from each other and often operate at different levels of the organization. But once you have these numerical scores you can track them over time to see if they move up or down, and to see if any WOL initiatives you implement have the desired affect.
• Since the score is a number, it can also be correlated with other enterprise metrics (employee engagement, financial metrics, etc.) to uncover areas where WOL might be making a significant difference. And because there are so few questions, you could add those questions to other surveys such as employee engagement, making it easy to correlate the effect of WOL on engagement at an individual level.
• I deliberately chose a response scale of 1 to 7. I have always felt that survey with scales of 1 to 5 don't have enough power to help you really understand what is going on. And something I learned from a previous role working with the Net Promoter Score is that, while an average score itself can help you track changes, it's the responses on the tails that really tell you how to improve. So if you follow up the survey with interviews of the respondents who answered with a "1" or "7" and ask them why, I can guarantee that you will get insights into your organization that will help you create improvement initiatives. NPS uses an 11 point scale (0 to 10) which seems like overkill to me, so a 1-to-7 scale looks like a happy medium.

In my next post I will share some of the organizational benefits I think enterprises will begin to see when they create widespread of acceptance of WOL. If you have an interest in using this survey, feel free to do so and let me know how it goes. I'd also be interested in gathering and aggregating any data or conclusions you might be willing to share in order to hopefully identify some larger effects of WOL that might span multiple organizations across different industries.