T. R. Halvorson is a lawyer in sole practice in Sidney, MT, President of Pastel Programming Co. , and author of How to Avoid Liability: The Information Professional’s Guide to Negligence and Warranty Risks and Legal Liability Problems in Cyberspace: Craters in the Information Highway.
Searchers have responsibility for the quality of information in the Web world. We should publish quality evaluations of selected Web resources to commend information providers when they do well and promote improvement where quality problems exist. The Southern California Online Users Group (SCOUG) Rating Scale is an apt instrument for measuring quality.1 Previously I presented a SCOUG-inspired quality evaluation of the LOIS Law Library.2 In this article I apply the same Rating Scale to V.
First Look
V. is a subscription legal research service on the Web covering state and federal appellate court opinions provided by VersusLaw, Inc. V. provides databases for all 50 states and the District of Columbia, the federal courts of appeals for all circuits including the D.C. Circuit and the Federal Circuit, and the United States Supreme Court. The databases cover only court decisions; there is no coverage of statutes, administrative codes, court rules, jury instructions, attorney general opinions, or other material.
Among Web legal resources, V. is immediately interesting because:
- it provides one-stop shopping for multiple jurisdictions
- it supports multi-file searching
- the backfiles are much deeper than on any free Web site
- the search engine is reasonably capable and the search syntax is fairly obvious for professional searchers
- pricing is extremely attractive, especially for smaller firms
The SCOUG Rating Scale
The earliest full-orbed view of quality and value of information in the electronic age I can find is the work of the 1990 annual SCOUG retreat. The retreat produced the SCOUG Rating Scale, a framework for judging performance in ten broad categories: consistency, coverage and scope, timeliness, accuracy/error rate, accessibility/ease of use, integration, output, documentation, customer support and training, and value-to-cost ratio.3 The SCOUG Rating Scale has proven sturdy as we moved into the Web world and has inspired a number of rating scales adapted to Web resources. My proposal to publish SCOUG-inspired evaluations of selected Web resources would be ridiculous for many legal research sites: they aren’t worth rating. While viewing V. through SCOUG lenses reveals its defects, as with LOIS, at least this is a resource worth looking at that closely.
Value-to-Cost Ratio
In the case of V., the price point is a story in itself, so I address the value-to-cost ratio first. V. offers three subscription plans:
Subscription Plans
Plan Per Attorney
(not per user)24-Hour Plan 14.95 per 24-hour period
Monthly Plan 6.95 per month
Annual Plan 83.40 per year
All of these are flat-rate prices. The single price includes everything. There are no connect time, printing, or downloading charges. Though the backfiles are not as deep as on Westlaw and Lexis and the kinds of materials are not as broad as on LOIS, at 6.95 per month V. presents an extremely favorable value-to-cost ratio. It is especially attractive for many solo practitioners, smaller law firms, and non-lawyers. VersusLaw uses PLWeb Turbo, a well known and credible search engine. Though viewing the service through the lenses of the SCOUG Rating Scale does reveal defects in the service, at these prices, the value-to-cost ratio holds up.
Consistency
Searching V. is thoroughly consistent from file to file. The individual jurisdictions do not have their own search pages. State court opinions are searched on cross-file searching pages. Check boxes for state court databases appear on the state opinions searching page. The search page for each federal circuit includes check boxes for state court opinions from states with each respective federal circuit. V. obtains court opinions directly from the courts and reformats them very neatly and consistently. They are much more attractive than they way they look as typed by the courts and they have a consistent look and feel from file to file. In each opinion, V. provides a medium-neutral citation. In most instances this is a value-added feature since the courts themselves only began giving medium-neutral citations in recent years. The documentation pages at the site quote from the Bluebook telling how to use these citations. V.’s opinions provide internal paragraph numbering for pinpoint citation. This again is a value-added feature included in all opinions however old they may be. V. often provides the official citation and the parallel citation to West’s reporters. Footnotes within opinions are hyperlinked bi-directionally.
Coverage and Scope
You can determine the depth of the backfile for any of the included jurisdictions easily in either of two ways. On the opening page of V. (not the home page of VersusLaw), click on the Contents navigational button on the left side of the page. That takes you to a page documenting the depth of all files. You can call the same page by clicking the “Click here for jurisdiction descriptions” hyperlink near the top of any search page.
Federal Coverage
U.S. Reports 1900
Seventh Circuit
1930
First Circuit 1930
Eighth Circuit
1930
Second Circuit 1930
Ninth Circuit
1941
Third Circuit 1930
Tenth Circuit
1930
Fourth Circuit 1930
Eleventh Circuit
1981
Fifth Circuit 1961
D.C. Circuit
1950
Sixth Circuit 1930
Federal Circuit
1982
State Coverage
Alabama
1955
Montana
1993
Alaska
1960
Nebraska
1965
Arizona
1930
Nevada
1996
Arkansas
1957
New Hampshire
1930
California
1930
New Jersey
1950
Colorado
1930
New Mexico
1930
Connecticut
1950
New York
1955
Delaware
1950
North Carolina
1945
District of Columbia
1945
North Dakota
1930
Florida
1950
Ohio (Supreme Court)
1992
Georgia
1940
Oklahoma
1994
Hawaii
1996
Oregon
1950
Idaho
1965
Pennsylvania
1950
Illinois
1985
Rhode Island
1950
Indiana
1935
South Carolina
1996
Iowa
1995
South Dakota
1965
Kansas
1982
Tennessee
1950
Kentucky
1945
Texas (except 10th – 12th)
1950
Louisiana
1980
Utah
1950
Maine
1996
Vermont
1930
Maryland
1950
Virginia
1930
Massachusetts
1930
Washington
1940
Minnesota
1930
West Virginia
1991
Michigan
1930
Wisconsin
1945
Mississippi
1994
Wyoming
1993
Missouri
1960
Timeliness
As of the date of this review, V. did not document the currency of its databases anywhere at its site. Early on March 8, 1999 I sent email to the company’s president, Joe Action, inquiring about this and several other points. By the middle of the morning I received telephone call from Jim Corbett at VersusLaw’s offices one time zone behind mine. You have to call that responsive. He described the frequency of loading new cases onto the system. U. S. Supreme Court cases are loaded three to four times each day the Court is in session and issuing opinions. In the files for many other jurisdictions, new cases are load daily, usually the same day they are issued by the courts. I located an independent pilot study of case law currency conducted by Paul Axel-Lute at Rutgers University one year ago. He studied the currency of a sampling of cases using decisions of the New Jersey Supreme Court and the New Jersey Appellate Division. His findings for those courts support what Corbett told me. Axel-Lute’s study compared V. to Westlaw and Lexis and found V. more current than either of those services.
Accuracy/Error Rate
VersusLaw says it has established relationships with the courts to obtain the cases electronically from them. That means the accuracy rate should be at least as high as what the courts themselves produce. I saw cases where initial letters of words were missing, for example Weng v. United States, 1998.C02.79:
[21] “otice [must be] reasonably calculated, under all the circumstances, to apprise interested parties of the pendency of the action and afford them an opportunity to present their objections . . . . hen notice is a person’s due . . . he means employed must be such as one desirous of actually informing the absentee might reasonably adopt . . . . ” Mullane v. Central Hanover Bank & Trust Co., 339 U.S. 306, 314-15, 70 S. Ct. 652, 657 (1950) (citations omitted).
and Sipes v. Oklahoma, 1997.OK.138
[28] ¶14 Original jurisdiction is the “urisdiction to consider a case in the first instance. Jurisdiction of court to take cognizance of a cause at its inception, try it, and pass judgment upon the law and facts. Distinguished from appellate jurisdiction.” Blacks Law Dictionary 1099 (6th ed. 1990).
* * *
[31] ¶17 Before 1988, 47 O.S. 1981, Section 6-211(a), vested the district courts with original jurisdiction over petitions filed by “ny person denied a license, or whose license has been canceled, suspended, or revoked” by DPS, except in enumerated instances.
Accessibility/Ease of Use
Connecting is simple. The service is on the Web. A user navigates to the access page and clicks on an obvious Search navigational graphic. Somewhat unexpectedly, the system does not yet ask for authorization. Instead, the system displays the jurisdiction selection page. Eliminating graphics and simplifying a bit, it looks something like this:
The page is simple and loads rapidly. The whole page fits in one browser screen. Suppose you choose the Ninth Circuit, the system presents the search page. Eliminating graphics and simplifying a bit, the page looks something like this.
This page also is simple and loads rapidly. The sequence of pages is simple. There are only two: the jurisdiction selection page and the corresponding search page.
Notice that the search page for the example of the Ninth Circuit includes the state appellate court decisions for the states comprising that circuit and the U. S. Supreme Court. This is a sensible layout that works well for the smaller law firm. To search other states, you need to choose “State Appellate” at the jurisdiction selection page. To search other federal circuits, you need to choose “Federal Circuit.” There are are no pages for individual jurisdictions. If you want to search an individual jurisdiction, click the check box for that jurisdiction only. If you want to search multiple jurisdictions, click as many check boxes as you like. If you click the last check box labeled “All the above jurisdictions,” be sure not to check any others.
V. includes a hyperlink on every search page, Search Query Operators, letting you call a page summarizing how its operators work. That’s accessibility. The summary page is nicely formatted in a table. That facilitates its rapid-reminder purpose making for ease of use. At the bottom of that page, you can click on another link that calls a Search Operator Comparison Chart page. The comparison chart allows you to see quickly how V.’s operators compare with the ones on Lexis and Westlaw.
The service suffers from my pet peeve: you can’t just press Enter to submit a search. You have to click the Submit command button. On some search pages with many jurisdictions, the Submit command button sometimes scrolls out of view. A better design would be a vertical button that can be clicked from anywhere on the page. Any solution that would make it possible to submit a search from anywhere on the page would be an improvement.
V. supports AND, OR, NOT, NEAR, proximity, wildcards, stemming, and nesting. The default operator is ADJ. That means you will get phrase searching if you use no operator. For example, a search for “actual physical control” looks for that literal string of characters just as if you had typed “actual adj physical adj control.” V. supports two proximity operators. NEAR is bi-directional. A search for “actual physical control near/10 jury instructions” will hit documents where those two literal phrases both appear, occur within ten words of each other, and regardless which phrase appears before or after the other. On the other hand, a search for “reckless w/5 misrepresentation” will hit documents where those two words appear within five words of each other but only if “misrepresentation” follows “reckless.”
The stemming operator, +, works at least as a bi-directional truncation operator. That is, it truncates on both the right-hand side and the left-hand side of a word. True stemming involves more: It hits on words with the same semantic root. Truncation considers only similarity of form while stemming considers relatedness of meaning. Implementing true semantic stemming is not simple.4 PLWeb Turbo by default uses a modified version of the Porter Algorithm for suffix stripping,5 but stemmers can be replaced by the administrator via dynamically loadable components. I cannot tell for sure whether V.’s stemming operator provides semantic stemming or only bi-directional truncation.
The single-character wildcard operator is the question mark, ?, and the multiple-character wildcard operator is the asterisk, *. Wildcard operators and the stemming operator can be used at the end, middle, and even the beginning of a word. Nesting, alternatively denominated scope limiting, is accomplished using parentheses, ( and ). These can be used in the familiar way to state an explicit order of operation of the other operators. When they are not used,
The implicit order of operator precedence is:
Proximity, Adjacent, Near (processed in order from left to right)
NOT
AND
ORTo expressly state an order of operation, you can use parentheses, ( and ), in the familiar manner. V. supports nesting with parentheses.
The search form makes it obvious how you can restrict a search by date. You can perform some additional field-restricted searching, which V. calls shortcut searching. This terminology would confuse many expert searchers who consider “shortcut” searching to be command stacking. Although you can combine shortcut search expressions on V., since it is the only way to do field-restricted searching, the choice of terminology could be better. The supported fields include: court, parties, counsel, docket number (expressed as “docket”), citation (expressed as “cite”), date (expressed as “dated”), panel, and author. The form of the expression is the field and the term separated by a colon, but in the opposite order from what might be expected: You must type the term first, then the colon followed by the field name. To search by citation, for example, the form would be “868 p.2d 772:cite” rather than “cite:868 p.2d 772.” There is no data normalization on these fields: You have to know the way the court expressed it, which could vary not only from court to court but from opinion to opinion of the same court, so you are advised to combine alternatives with the OR operator.
V. says it offers natural language searching that is invoked whenever no Boolean operators appear in a search statement. It’s true that PLWeb Turbo offers a sort of natural language searching. Its query processor parses and filters out noise words including those on the stop list of non-searchable words and some others that convey little or not meaningful content in a query statement, and then it inserts a default Boolean operator between the remaining words of the query. V. sets the default operator used in the natural language mode to ADJ. The manual at V.’s site gives as an example the search statement “negligent infliction of emotional distress” and explains that this will retrieve documents where those words are adjacent to each other. The ADJ operator provides the same results as if a w/1 operator were used. The result really is phrase searching. There is another example in another section of the manual, an unfortunate one: “cases of civil forfeiture under 21 U.S.C. resulting from the sale of drugs or marijuana.” That example contains a Boolean operator. The second-to-last word is “or.” The query processor picks up on the OR operator, so the query resolves into this: [a bunch of stuff] or [marijuana]. Since marijuana is such a common word in cases, running that query in all federal files results in 10,271 hits as of March 6, 1999. A search for the single word “marijuana” in the same files yields exactly the same number of hits. Removing the two trailing words “or marijuana” and submitting the rest of the statement yields zero hits. The non-noise words in that query never appear in that order as a phrase. The natural language searching that results from this setting is not what one might expect, but one must temper any disappointment by remembering that VersusLaw only charges 6.95 a month per lawyer. As a Volkswagen or a Ben and Jerry’s, V. really is not touting natural language searching. Probably the manual mentions natural language because something has to be said given that the service uses PLWeb Turbo. If a searcher submits a query with no Boolean operators, instead of displaying a syntax error message, PLWeb Turbo will return a hit set and users need to know how that hit set was generated.
Integration
V. is reasonably well integrated. You can search multiple files. Selecting them is easy: Click check boxes for the files you want to search. Then enter your search statement.
V. fails to take advantage of hypertext technology. It does not cross-link citations the way, for example, LOIS does allowing you to immediately call a document from another database by simply clicking on the citation.
Output
When you submit a search query, V. processes the query and presents the hit list. For each hit, three items of information are shown: an abbreviation indicating which court issued the opinion, the date of the document, and the name of the document (which is an unofficial abbreviation of the names of the parties). V. supports relevancy ranking and lists the hits in relevancy-ranked order. There are many approaches to relevancy ranking and V.’s research manual does a good job of explaining the approach it takes.
Here’s the chief omission: there is no KWIC-like display format.
Search terms appear either in bold or red. V. is busy re-indexing its entire collection of over three million cases. The older format was red, and in the cases already re-indexed the newer, bold format appears.
V. displays documents in a very nice format. Internal paragraph numbers are added. Where the court issued the opinion with medium-neutral paragraph numbers, V. retains them and also provides its own pinpoint paragraph numbers. In the cases I viewed, there was no indication of pagination from the official report nor from West’s regional reporters. Sometimes the cases retain the syllabi by the court and sometimes not. Graphics are omitted and only a general notice is given at the bottom of the document that “any” graphics in the original are not included. It would be helpful if V. were to insert a notation in the text where illustrations existed in the original so that a researcher would know when it might be necessary to consult a hardcopy reporter.
When you chose File Save As in your browser, the proposed file name always is fastweb.exe. If you just press Enter, your browser saves the currently displayed case to a file on your hard drive giving it that name. Be careful. Unless you think of your own file names and type them, you’ll overwrite the first case you save with the second. LOIS also provides a default file name with a .exe extension, but nothing happens when you press enter. You must type a different file name. Now that I have seen what happens on V., what looked like an annoyance on LOIS now appears as a handy safeguard.
When you save a document, it becomes nicely formatted ASCII text.
Documentation
Generally, documentation on V. is good. There is a “V. Research Manual” that is well organized, not too long, and yet fairly complete. There is an operator chart and an operator comparison chart to compare V.’s operators with those on Westlaw and Lexis. When you have read the manual once, probably the operator chart will be sufficient to remind you of anything you might forget. The manual discloses the list of stop words which are not searchable.
The documentation of currency is absent.
Customer Support and Training
Subscribers receive both customer service and technical support. Live support is available by telephone Monday through Friday, 8 a.m. to 5 p.m. Pacific Time. The site offers a frequently asked questions page and a contacts page with email links to customer service, technical support, and departments.
************
Footnotes
- T. R. Halvorson, “Searcher Responsibility for Quality in the Web World,” Searcher, vol. 6 no. 9, October 1998, pp. 12-20. < back to text>
- T. R. Halvorson, “The LOIS Law Library: A View through the Southern California Online Users Group Rating Scale Lenses,” Law Library Resource Xchange (LLRX), March 1, 1999. < back to text>
- Reva Basch, “Measuring the Quality of the Data: Report on the Fourth Annual SCOUG Retreat,” Database Searcher, vol. 6, no. 8, October 1990, pp. 18-24. < back to text>
- Charles T. Meadow, Text Information Retrieval Systems, (Academic Press, Inc., 1992), pp. 55-60, 200-201. For example, right-truncating “half” to “hal” will hit on half, halve, halved, halves, and halving, all of which do convey a commonality of meaning, but it also will hit on halberd, halcyon, hale, and hall which do not convey any commonality of meaning. < back to text>
- For an explanation of the Porter Algorithm, see William B. Frakes, “Stemming Algorithms” in William B. Frakes and Ricardo Baeza-Yates, eds., Information Retrieval: Data Structures and Algorithms, (P T R Prentice Hall, Inc. 1992), pp. 139-142, 145, 147. < back to text>