Cindy Curling is the Electronic Resources Librarian at Fried Frank Harris Shriver & Jacobson in Washington, D.C., a web committee member for the Law Librarian’s Society of Washington, D.C. , and organizer of its Legal Research Training Focus Group.
Survey software has been around for some time, and Web-based surveys are beginning to become integral to online activity. You see them in all kinds of contexts from frivolous “Poll of The Day” questions to customer feedback forms on many retail Web sites. You may have even participated in a survey or two, but have you thought about how you could be using them to make your work life easier or more productive? You may think that opinion based research isn’t something useful to you, or is something you need so infrequently that it’s not worth spending time on, but think again. Surveys are now so easy to produce, deliver and evaluate that you’ll find many possible uses for them in your business once you become familiar with your options.
Using Surveys in Your Business
As a librarian, my internal customers or users are lawyers, legal assistants, administrators, secretaries and other non-legal staff as well as interning law students. I work with them mainly in delivering answers to their business research questions, and in teaching them how to efficiently use the firm’s information resources. I also work in tandem with librarians outside my firm in an effort to encourage resource sharing and to improve our ability as a community to teach research skills. General user surveys give the library staff feedback on what our customers find useful, and what they don’t, telling us where we could use improvement. While surveys can generate feedback and opinion, they can also give you concrete data about your customers. Surveys let me learn not only what my trainees think of how I teach, but also help me determine what our employees are learning from my classes and what they are not. By sending surveys to our incoming fall and summer associates, I can gauge the skill level of each individual or the groups as a whole, and that helps me tailor the training we offer to the skills they need. Don’t forget, too, that a survey is simply a series of questions and answers, a give and take can be designed to give as much information as it elicits. During library week, we use a fun interactive quiz whose questions and answers highlight library services and staff expertise.
While those are all library-based applications for surveys, their usefulness carries over to the broader range of law-related business or academic activities. Think for a moment about the work you could save if you could focus on the issues your customers — either internal or external — really care about, if you could capture their perceptions of what works and what doesn’t about your business processes, if you better understood what skills new employees possess and which they’ll need to be taught, what exiting employees think of your business individually or as a group. Surveys are useful to marketing, to human resources, to information systems, to accounting and for direct feedback between a firm and its clients. I work in the firm environment, so the options above share that focus, but the possible uses in academia are probably even broader, and courts and agencies would find many uses for survey information as well.
Web-Based Benefits
Until a few years ago, all my survey activities were paper-based. Each project required significant forethought and planning because the surveys were expensive to reproduce and distribute, and because it was impractical to make changes to the printed product, so we had to have them absolutely right the first time. Colorful, eye-catching graphics were pretty much out of the question for our budget, and response rates were low. What responses we did get had to be hand tabulated, an extremely time consuming process.
Then, as our firm began to develop its intranet, I worked with our information systems department (IS) to create some library-related internal surveys for our employees. This was a huge improvement for several reasons. First, it was much easier to distribute the surveys. We had only to e-mail our survey targets with a notification that the survey was available, then point them to an interactive document or Internet page, or simply include a poll or short survey within our e-mails to bring them to their question and answer sessions. Second, each of these document formats allowed for the use of colorful graphics, and our response rates improved. Also, these surveys were easy to adapt from one project to another. Once we created a question format, it was a fairly simple matter to reorganize our content or adjust it according to the group being surveyed.
However, there were still some drawbacks to this system. For one thing, the surveys took time for our IS people to develop because we didn’t really have any software specifically made for survey creation. Survey software options range widely by price and feature. Though there were some free Web-based survey products on the market, they were definitely a case of “you get what you pay for”, and their really capable competitors came with huge price tags and generally required that complex software be housed on our system, something the IS folks weren’t entirely pleased about. For the few surveys we did, there didn’t seem to be much out there worth the price. Also, there was a little lag involved because we had to go through another department. If information systems was dealing with a network crisis or working on a billable project, the library job obviously had to come second. Usually, that wasn’t a big problem because we planned things out well in advance, but it meant that we couldn’t produce surveys very quickly. Another drawback was that we still had to hand tabulate our results, an issue even more challenging than before because we were getting better response rates.
Zoomerang to the Rescue
Then, about a year ago, I started to see mentions of a site called Zoomerang in some articles and on sites around the Internet that were doing polls and surveys. Zoomerang is an entirely Web-based product for survey creation. It allows users to create their surveys on site, send invitations to participate in the survey from Zoomerang (there are other delivery options available, but more on that later), and point to a page on its site where the survey is housed. About that same time, the Law Librarians’ Society of Washington, D.C. (LLSDC) wanted to touch base with its membership and find out how it could better serve us. So I, along with other area law librarians, received an invitation to participate in a Zoomerang survey. Our membership isn’t gigantic, but it is large enough to make a paper survey an expensive and daunting task. Zoomerang, according to the LLSDC Webmaster, handled the job with relative ease. The key elements behind the association’s choice of Zoomerang to deliver its survey: it’s inexpensive, easy to use, and most important, tracks the results of your surveys for you.
I was impressed as a participant and could easily see its usefulness for external surveys, to determine, for example, the skill levels of our incoming fall associates. We already had our IS folks working on an exit survey for our summers, and while that could be delivered via our intranet, the falls were still outside the firm. Firewall policies and marketing strictures being what they are, it seemed impractical to house a library survey on our firm’s main Web site, and Zoomerang turned out to be just the ticket. As time passed, it also became obvious that our library needed better, faster creation options for less formal internal surveys like class evaluations, but the number we’d need to do made it impractical that we use our intranet and go through the IS department. Again, Zoomerang is great for the job because it’s easily to quickly turn around new surveys on each topic we teach, or even class by class. Feedback is immediate, and we don’t have to tabulate the results ourselves, or go through any other departments.
Using Zoomerang
Zoomerang has two levels of membership: a free basic membership, and a paid one called a zPro membership. For information comparing the two, Zoomerang offers a nice chart, but I’ve detailed the limits and advantages of each membership type below and set out the pros and cons as I see them. The main differences between the two are that the free basic membership is intended for smaller surveys, both in length and number of respondents, and of course the price. A zPro membership currently costs $599/year, a pretty stiff increase from previous years rates and from the introductory offers available when the site first became available. For most of the surveys we do as a department, the basic membership is all we need, but there are exceptions. The zPro membership is meant for larger survey projects and allows for long-term storage of your results. Depending on the number of surveys you send and the amount of work the service may save you, that may actually be a relatively reasonable amount considering the features it offers. Other survey software packages you load and/or maintain in-house offer very similar features and can cost much more – some are in the $10,000 range.
The Process
I think it’s easier to understand the differences between the two membership levels when you see how they compare over the process of the survey. Essentially, each survey has three creative elements: production, delivery and evaluation of results, and a management aspect.
Production
In the initial part of the process you have to come up with the questions for your survey and choose you format.
With a free membership, you can create an unlimited number of surveys modeled on over 100 existing templates available at Zoomerang, or from a survey you’ve already created there. Zoomerang offers a broad variety of templates covering everything from class evaluations to customer feedback, and new templates are added each month. The templates will fill most needs for simple surveys, and modifying individual questions to suit your projects is very easy. The templates can also be edited to add extra questions (up to a maximum of 20 with the basic membership), or delete questions if there are too many for your project. I found it helpful to work with the templates, at least at first, because it let me become familiar with the question formats. If you are totally new to creating surveys, they also serve as nice models for the kinds of questions you might want to ask in various situations, help you decide how to order your questions and more.
The zPro membership allows a little more flexibility in that you can create your survey from scratch, saving the step of modifying a template (especially handy if you have a survey from a different source that you simply want to transfer to Zoomerang), and you can include up to 30 questions. It also gives you more opportunities for branding your survey, with your logo for example, which at this stage means allowing you to add an image to the title area of the survey itself.
Zoomerang also offers about 10 design themes. My favorite is Cartographic Grey, the default. It seems easy to read, and it’s one of a very few that have any complex graphical elements. Most of the themes are simply color based, or if they have integrated graphics, are too topical (scientific instruments as an example) for my purposes, though I could definitely see them being handy in other contexts like schools or medical practices.Once you’ve picked your design theme, it’s time to choose what question formats will work for you. Currently, Zoomerang offers 12 different formats that are surprisingly flexible. Detailed descriptions of the formats along with examples are available at Zoomerang, but here’s a short overview of your options:
- Headings are used to define sections of the survey or for introductory or closing text. As far as I can tell, they do not count toward your question total.
- Multiple choice questions come in three formats. If you only want the respondents to pick one answer from a list of possible options, they can either click on their choice in a bulleted list or from a drop down menu. The drop down menu makes the survey look a little shorter, a nice option when you have loads of questions for your respondents to answer. The last multiple-choice format allows the respondent to click on one or more options, again from a bulleted list. This set of bullets looks different from the single bullet option, so you (and the user) can tell the question type is different.
- Zoomerang offers two types of rating scales. There is a one-answer format where you ask the respondents to rate their response to a single question along a range that runs horizontally, and there is a matrix format where respondents rate a range of items in a vertical list where each item in the list has its own horizontal rating range. This last is the only format at Zoomerang that I’m not entirely happy with. Compared to a simple radial choice with the range of options arrayed across the top, this is very crowded because the range choices are repeated across the page for every single line item. It would look less busy, and still be easy to use I think, if the rating range appeared only at the top, with simple bullets next to each line item.
- There are three formats for open ended questions, those for which the respondent doesn’t pick from a choice but fills in a blank. Those options are a single line for short answers, a long comment box, and one or more short lines with prompts to remind the respondent about the information needed like “E-Mail Address:” ________________ and “Telephone Number” _____________.
- The rest of the question formats are more strictly designed for specific, commonly needed answers. They are the “Yes or No”, “Name and Address,” and “Date and Time” formats.
Delivery
There are several options for notifying potential survey recipients, and here there are not really any differences between the zPro and basic memberships. Three options are discussed on the Zoomerang site. First, you can use an e-mail invitation you write at Zoomerang and send it to a list of recipients you build. Second, you can write your invitation and have it sent to an e-mail list provided by Zoomerang. This wasn’t useful for my department, but would be handy for marketing or if you needed a random survey response. In either case, when a survey is launched, recipients will see a plain text format e-mail with a message you create inviting them to participate in the survey, followed by a disclaimer from Zoomerang.
The third Zoomerang option is to link to the survey from your own site using the Zoomerang address. What they don’t emphasize here is that you could also send an e-mail from your own system and point recipients to that URL. Depending on your recipients and your mail program, that may let you customize your invitation with HTML formatting, an opportunity for branding that isn’t available at Zoomerang even with the zPro option. Unfortunately, they can’t predict what kinds of e-mail your recipients will be able to handle and some e-mail systems still don’t cope well with anything beyond plain text. In the case of the LLSDC survey, users were notified from a group e-mail list and directed to the LLSDC site where there was a link to the Zoomerang survey. For in-class evaluations, I don’t send any notification. Instead, I just direct the class to the appropriate Web address and have them fill in the form before they leave.
It may be to your advantage not to use the e-mail launch feature at Zoomerang for another reason. I noticed that for my surveys there was always some delay before e-mails sent from Zoomerang were received. I tested the delivery time by sending a short survey to some of my colleagues who not only have office e-mail accounts, but who also have Web-based personal accounts. In most cases, the delivery times were about the same — around an hour after I launched my survey. That’s not horrible, but it’s something to be aware of. Also, in order to use the e-mail feature at Zoomerang to send the survey invitations to specific individuals, you need to have their e-mail addresses in the Zoomerang system. It’s not difficult to add them and it’s very easy to assign addresses to particular groups, but if they are already part of your own e-mail contacts, you may want to consider whether it’s really useful to enter them again. It’s a convenience for people who don’t have an internal contact list or who are contacting individuals outside those lists, but in some instances it doesn’t make sense.
Evaluating Results
Results are kept on the Zoomerang site and are compiled in real time as respondents submit their answers. This is a great feature, and is available to both basic and zPro members. However, I think this is where having the paid membership can really be worthwhile.
First, under the free basic membership, results of your surveys are only kept for 10 days from each survey’s launch. For anyone who is surveying a target group over time, it’s essential to have your results for longer than 10 days. Also, Zoomerang will only report on 50 responses to any given survey at the basic membership level. While my group doesn’t often need to survey more than 50 people, and even more rarely gets more than 50 responses back, this is another important limit to be aware of. One could work around it by surveying smaller sub-groups of target participants (just our tax attorneys instead of all attorneys), but it may mean more work than is worthwhile.
Another important limitation is that you get slightly more flexible analysis with the paid membership. Take the tax attorney example above, for instance. With a zPro membership, I could survey all the attorneys in my office, include a question pertaining to their practice areas, and use a special cross-tabulation feature to cross-reference answers to the practice area question with any one other response. The paid membership also allows you to receive your results via e-mail in spreadsheet format, which would allow you to produce charts and graphs using your survey data. We have not used this feature yet, but over time it could make comparisons of one survey group to another much easier, and it would be invaluable if the aim of your surveys were to compile statistics to use in any sort of presentation.
Last, the paid membership allows you to share your results with others. Basic membership means results are always private and disclosed only to you. With zPro, you have that option or you can restrict view of the results to one or more other individuals, post the overall or individual results publicly or notify participants that results are available by e-mail. You’ve probably seen simple polls where you could see the results so far, like “Which do you prefer: A) customer service phone trees OR B) speaking to a customer service person immediately. Current Results: A) 15%, B) 85%. That’s just one possible option, but participants are often interested in how their responses compare to others’. A nice additional feature is the option to password shared results for added security.
Survey Management
Once you’ve submitted a survey, even in draft format, you can manipulate it in a range of ways. There is a “My Surveys” page from which you can review results, modify or launch your survey, or you can access “Survey Management.”
Survey management gives you access to some of the independent elements on the “My Surveys” page, but also allows some additional options. First, you can preview your survey. This basically lets you take the survey yourself without having your answers count in the results. Personally, after having worked on any project for a while, I’m no longer the best person to proofread it. I prefer to draft my survey as it will go out but start the title with the word “TEST,” and then launch it to several co-workers for their evaluation and comment. Their eyes usually catch mistakes I’ve missed, and because it’s very easy to base a new survey on one you’ve already created (either when you add a new survey or in survey management), it takes only a very short time to revamp it for its real audience.
You can also use the management options to edit the survey at any point before it is launched, and to create or modify an optional Web greeting page if you want people to see any introductory text before they actually get to the survey. Management is also the place to go to choose which delivery or deployment option (discussed above) suits you best. You can launch a survey here as well, or delete old surveys you no longer need.
One option in management only available to zPro users is ability to add an image and Web link to a thank you page that users see after they have completed a survey. This allows you to brand the page with your logo and send users from Zoomerang back to your own Web site or to another site related to your survey topic.
Additional Benefits, Drawbacks and Alternatives
The Zoomerang site is very user friendly, and it does give you some extra benefits aside from its actual functions. As I mentioned above, the templates are a great education on survey writing in themselves, but the site also offers some tips at the survey creation point. They also show you the latest ideas for surveys and the most commonly used surveys to help you get a better idea of the options available. Overall, their help is good and their descriptions are informative and useful. On the down side, I sometimes found the site a little difficult to navigate. Parts of it, especially the survey text itself, are easy to get to and modify, but I occasionally had trouble getting back to some of the peripheral elements. Also, while the branding options help reduce confusion when you send a survey participant to Zoomerang, a site outside your own, there is still no question that it isn’t your site because the Zoomerang brand is always there as well. I didn’t find that to be a huge problem, but can imagine that for some people that would make or break their decision to use the service.
I didn’t find any other survey software packages that were this sophisticated and still entirely Web-based. Frankly, that seems so unlikely that I’m sure I must have missed them, so if you know of any, I’d love to hear about it. Some that were not so sophisticated were Votations and Create A Poll. I’ve even used evite for some informal personal polls, but in most instances these services can’t touch Zoomerang for professional look and feel. I did find a few services that were partly Web-based, and though I wasn’t much impressed with their Web sites, they may suit as alternative. Survey-Web, for instance, allows you to use Survey Tools for Windows over the Web or asks that you download a free survey creation package, but they do host your survey online. They offer much more sophisticated survey analysis, but according to the press release on their site, “Prices start as low as $149 for a 100 respondent survey hosted for three months.” The press release was dated January of 2000, though, so whether or not that’s still accurate is open to question. As I said, the Web sites of some of these services left something to be desired. Another alternative was Survey Said Survey Software. They offer survey software that lets you host surveys on your own site and which lets you do a bit more with your data and for much larger survey groups (up to 1 billion respondents at the highest price point), but their pricing starts at $695 for one concurrent administrator and goes up to $995 for three.
For my money, and my firm’s, Zoomerang offers a nice balance of features that suit most needs. The free features can’t be beat for flexibility and ease of use by anything else I’ve seen where there’ s no charge. At the higher price point, Zoomerang is not for everyone. However, while branding isn’t very important for most of my surveys, the storage issue is, and we may choose paid membership. Also, considering that without survey software it took our IS Web expert a couple of days of work, on and off, to adapt a twenty question paper survey for our intranet and that it took me two hours to do the same on Zoomerang, for us the time saved will be worthwhile. Our survey needs are not so complex that we would really use the full analysis features of one of the more expensive software packages, and I love that the entire system is Web-based so we’ll never need to update or accommodate software in house.
In short, try Zoomerang; I think you’ll like it. If you do, I’d be curious, as always, to hear your comments.