Today’s Search in Reveal 11 post takes a first look at Reveal Query Language (RQL). New with Reveal 11, RQL enables users to construct powerful and complex queries that can search across both metadata and text. With all the RQL offers, this post is part 1.
In earlier posts in this series, I wrote about combining keyword and concept searches (Search in Reveal 11: Keyword, Concept, or Both), searching with just a few clicks (Search in Reveal 11: What a Click Can Do), and using the new Term List Search function in Reveal 11 (Search in Reveal 11: Term List Search), and filtering on metadata (Search in Reveal 11: Metadata Filtering). Here, I look at RQL.
For Reveal 11, we needed a query language that could support four types of searches: complex Boolean, metadata, faceted, and concept. The available options did not meet our needs. Hence, RQL.
Our top three objectives with RQL were that it be (1) clear and unambiguous, (2) able to support all Reveal 11’s use cases, and (3) easy to use for those familiar with common document review platforms.
In this first post about RQL, I look at three fundamental aspects of RQL: searching with terms, phrases, and special characters; moving past stop words; and searching metadata.
Search with Terms, Phrases, and Special Characters
With RQL, you search for terms and phrases. Terms are words and similar strings of characters such as dogs and cats.
Phrases are words entered together, such as dogs and cats, as well as strings of characters containing special characters, such as AT&T. To search for a phrase, enter it between quotation marks, for example “dogs and cats” or “AT&T”.
If I search for documents containing the terms dog or cat, I get 3,970 documents containing dog, 2,791 containing cat, and 6,435 containing either dog or cat:
If I search for dog and cat, the platform looks for any document containing both dog and cat anywhere in the document. With this search, I get 326 documents:
If I search instead for the phrase “dog and cat”, with the phrase in quotation marks so that the platform looks for that exact phrase, I get 30 documents:
Here is one of the documents returned by that search:
For special characters, enclose the word or phrase in quotation marks. If I search for AT&T without quotation marks.
Search “Stop Words”
When electronically stored information (ESI) is processed, text is identified, extracted, and added to a searchable index. Systems commonly are set up ignore “stop words” such as “to”, “be”, “or”, and “not”. This is done to improve performance but when it is done a search for a phrase such as “to be or not to be” will not return any documents containing that phrase.
A system using dtSearch, to use one example, might be set up to skip over this list of words, where I have highlighted the stop words “to”, “be”, “or” and “not”:
With RQL, we avoid this problem by using the Elasticsearch CommonGramsFilter. With this approach, there is not a need to develop and deploy a set of stop words. Here, I have composed a search for “to be or not to be”:
When I ran the search, I got back 28 documents. At the top of the list was a document containing the phrase “to be or not to be” in the subject line:
Search Metadata
To search metadata, enter the appropriate database field name, two colons, and the term or phrase you want to search for. Here, I have two searches. The first, subject::help, returned 1,003 documents with “help” in the subject line. The second, subject::”happy holidays”, returned 80 documents with “happy holidays” in the subject line:
When using RQL to search metadata, you need to make sure you use Reveal database field names rather than the display name shown in the Grid.
Here are the first few entries in a list of 152 display names and their corresponding database field names:
If you have administrative access to Reveal 11, (1) go to Project Admin, (2) select Fields, (3) search in Field for the display name shown in the Grid, and (4) use the database field name in the Name box:
Using RQL to search metadata can be useful in many ways. This approach lets you be quite precise. Let’s say you want to find all documents where “Mark” was the author. Here are four ways you could do that with RQL, going from the largest number of hits to the smallest.
Metadata Search with Front and Back Wildcards
For a broad search, I can ask the platform to look in the AUTHOR field for any string of characters containing “mark”. I do this by putting asterisks before and after “mark”. This search could return documents containing not only “mark” but also “remark”, “market” or “remarkable”. The search, author::*mark*, returned 1,073 documents:
There were 72 authors whose names contain the characters “mark”. If I look at the list, I see many names with “mark” as a separate word. I also see three authors where “mark” is part of a larger word: “linda b. hallmark”, “occidental energy marketing,” and “tobin bismark carlson”:
Metadata Search with Front or Back Wildcard
I could narrow my search, placing an asterisk either at the front of “mark” or at the end of the word. Here, I put the asterisk after “mark” – author::mark* – and got back 1,061 documents with 69 separate authors:
This time there were 54 names, all with “mark” as a separate set of characters.
Metadata Search for Word
To further narrow the scope of author names, my next step was to search for any document where “mark” is in the AUTHOR field as a standalone word. Using author::mark, I got back 867 documents:
This time there were 54 names, all with “mark” as a separate set of characters.
Exact Metadata Search
With RQL, another level of precision is available. If I search for exact author::mark, the search will return only those documents where the entire author name is “mark”:
This time, the search returned 18 documents. If I look at the list of authors, I see only one name, “mark”:
And There’s More
What I showed today is just an introduction to the new Reveal Query Language (RQL) and how it enables you to better search your data, helping you enjoy the highest quality speed to insight in the industry.
In posts to come, I’ll continue exploring RQL and other ways you can use Reveal 11 and its greatly enhanced search capabilities.
For more information about how Reveal can empower your organization, contact us for a demo.