Installation

If this is your first time installing, configuring, or developing with the Semanteer Search Layer, please take the time to go through this section and learn how to perform a standard installation, go through the initial configuration steps, and get started with a predefined schema, with or without a demo data set.

Sections

  • initial configuration, running — Learn how to deploy the Search Platform bundle, perform required initial configuration steps, and run the platform.

  • First steps after deployment — Once you have your Search Platform up and running, you can start with the tailoring of the installation to your needs, starting from predefined index schemata, and making use of sample search configurations and demo content.

Unpacking, initial configuration, running

Learn how to deploy the Semanteer Search Platform bundle, perform required initial configuration steps, and run the platform.

Installing the bundle

Bundle contents

The Semanteer Search Platform comes as a bundle that is made up by the following components:

  • Apache Tomcat application server 9.x

  • Semanteer Search Layer

  • Apache Solr 7.5

  • Semanteer Monitoring and Search Analysis

Initial configuration

The directory <install-dir>/conf/ contains all necessary files to configure startup parameters and search properties. As a first step, you will want to modify the basic execution parameters, such as the ports at which the main components will be running, logging, etc.

semanteer.properties

In semanteer.properties the startup ports for Solr and the Semanteer Search-Layer, the activated spring profile and the memory reserved for Solr must be specified, e.g.:

Code example

controller.port=8036
searchlayer.port=8030
searchlayer.shutdownport=8032
solr.port=8035
solr.mem=2g
springprofile=dev

search-layer-<profile>.conf

For your activated profile you can add configuration files that typically specify the server port and urls for the solr cores, as well as login information for the admin area of the Semanteer Search-Layer, e.g.

Code example

server.port=8030

# Solr URLs
solr.url=http://localhost:8035/solr/winedemo
solr.monitoring.url=http://localhost:8035/solr/sem_monitoring_core
solr.analytics.url=http://localhost:8035/solr/sem_monitoring_core

Syntax highlighting is enabled by setting the source-highlighter attribute in the document header or passed as an argument.

Start / Stop Semanteer Search

The Semanteer Search and all components can be started and stopped with the <install-dir>/bin/semanteer.sh script.

Status/(Re)Start/Stop all components
bin/semanteer.sh status|start|stop|restart
Status/(Re)Start/Stop the Search-Layer (tomcat)
bin/semanteer.sh tomcat status|start|stop|restart
Status/(Re)Start/Stop Solr
bin/semanteer.sh solr status|start|stop|restart

First steps after deployment

Once you have your Semanteer Search Platform up and running, you can start with the tailoring of the installation to your needs, starting from predefined index schemata, and making use of sample search configurations and demo content.

Schema

This chapter introduces you to the predefined schemata that come bundled with the Semanteer Search Layer, the available dynamic and static fields and their capabilities and intended use.

Predefined field types

Topics: basic-, text-, categorization- and special- field types in the default Semanteer index schema.

More information about the field types supported by Solr "out of the box" can be found in the section Solr Field Types of the Solr Reference Guide.

Basic field types

Field type Description

binary

Binary data.

boolean

Contains either true or false. Values of 1, t, or T in the first character are interpreted as true. Any other values in the first character are interpreted as false.

string

String (UTF-8 encoded string or Unicode).

date or tdate

Represents a point in time with millisecond precision. The format used is a restricted form of the canonical representation of dateTime in the XML Schema specification: YYYY-MM-DDThh:mm:ssZ

double or tdouble

Double field (64-bit IEEE floating point). double (precisionStep="0") enables efficient numeric sorting and minimizes index size; tdouble (precisionStep="8") enables efficient range queries.

float or tfloat

Floating point field (32-bit IEEE floating point). float (precisionStep="0") enables efficient numeric sorting and minimizes index size; tfloat (precisionStep="8") enables efficient range queries.

int or tint

Integer field (32-bit signed integer). int (precisionStep="0") enables efficient numeric sorting and minimizes index size; tint (precisionStep="8") enables efficient range queries.

long or tlong

Long field (64-bit signed integer). long (precisionStep="0") enables efficient numeric sorting and minimizes index size; tlong (precisionStep="8") enables efficient range queries.

 

Specialized entity field types

Field type Description

bbox

Used for "bounding boxes"

email

Field type that "understands" email addresses and maintains them as full tokens.

location

A latitude, longitude coordinate pair.

mime

A filed type that correctly tokenizes MIME-encoded content types

occurrence

A pair of start- and end- time numbers, used to mark the start and end time of event occurrences. The temporal space they refer to needs to be configured appropriately.

point

An arbitrary n-dimensional point.

tel_number

A field type that normalizes telephone numbers.

url

Field type that "understands" URL addresses and retains them as full tokens.

uuid

Universally Unique Identifier (UUID). Passing in a value of "NEW" causes the creation of a new UUID.

descendent_path

Field type used for FacetHierarchyComponent (see Hierarchical facets) specifying a character ":" used as divider.

ancestor_path

Field type used for FacetHierarchyComponent (see Hierarchical facets) specifying a character ":" used as divider.

cat_minimal

Specialized analysis for categories

cat_root

Specialized analysis for root categories

cat_leaf

Specialized analysis for leaf categories

location_rpt

A location field type able to store lat,lng locations and geometry (WKT strings). See also Spatial Search

location_rpt_approximate

Similar to location_rpt but less precise (and thus faster)

group_acls

Simple version of ACL checking, expects values separated by comma or semicolon

ean

International Article Numbers

 

Text field types

Field type Description

text_ws

Uses a white space tokenizer to split text into tokens.

text_general

Language-specific analysis, including stop-word and synonym treatment, as well as stemming.

text_precise

Minimal analysis for "precise" matching

text_minimal

An field type with tokenization aggressiveness "in the middle" between text_general and text_precise.

text_alpha

Used for alphabetic sorting.

text_spell

Used for the "sink" fields from which spelling suggestions are derived.

text_suggest

Used for the "sink" fields from which auto-complete suggestions are derived.

text_suggest_prefix

Alternative type definition (for suggestions) that uses shingles

text_suggest_phrase

Similar to text_suggest_prefix

lowercase

Lowercases the entire field value, keeping it as a single token.

text_geo

Text field used for geographic fields (names of places): Different synonyms (at query time) and no stemming.

Predefined fields

An overview of the available static and dynamic fields in the default schema.

Main record fields

Base fields that are typically used independently of the type of indexed document.

R = required, I = indexed, S = stored, M = multi-valued

Field Type R I S M Description

id

boolean

✓

✓

✓

Must be globally unique within a core.

sem_lang

string

 

✓

✓

Two-letter language code.

sem_source_id

string

✓

✓

 

The id of the indexing source.

sem_source_type

string

✓

✓

 

The type of indexing source.

sem_record_type

string

✓

✓

✓

 

The main type of the document.

sem_record_subtype

string

✓

✓

 

Optional sub-type of the document.

sem_record_info

string

✓

 

Opaque field where presentation frontend-related data can be stored.

sem_asset_info

string

✓

✓

sem_parent_record

string

✓

✓

 

Reference to the identifier of the parent document, where a parent-child relation exists.

sem_grouping

string

✓

✓

 

The main grouping field for documents; contents may vary per installation.

sem_active

boolean

✓

✓

 

sem_title

text_general

✓

✓

 

The document title.

sem_title_alpha

text_alpha

✓

✓

 

Same as sem_title but tokenized for alphabetic sorting.

sem_subtitle

text_general

✓

✓

 

The document sub-title.

sem_subtitle_alpha

text_alpha

✓

✓

 

Same as sem_subtitle but tokenized for alphabetic sorting.

sem_abstract

text_general

✓

✓

 

A summary of the document.

sem_content

text_general

✓

✓

 

The actual textual content of the document.

sem_url

url

✓

✓

 

The document’s URL.

sem_keywords

text_general

✓

✓

✓

Comma-separated list of document keywords.

sem_tags

string

✓

✓

✓

Comma-separated list of document tags.

sem_author

text_minimal

✓

✓

✓

The document’s author(s).

sem_author_facet

string

 

✓

✓

Automatically generated field to support faceting on authors.

sem_author_info

string

 

 

✓

An opaque field to store frontend-related information about the document author(s)

sem_creation_date

date

✓

✓

 

The document’s creation date.

sem_modification_date

date

✓

✓

 

The date at which the document was last modified.

sem_publication_date

date

✓

✓

 

The date at which the document was published (to be used if there is a publication workflow in place)

sem_popularity

float

✓

✓

 

Optional field that can be used for boosting or selecting the most popular documents in an index.

sem_sort_order

int

✓

✓

 

Field used for sorting

*_mfacet

string

✓

 

Dynamic field that may be used for faceting

Location-related fields

Predefined fields related to the "address style" location associated with an indexed document.

R = required, I = indexed, S = stored, M = multi-value

Field Type R I S M Description

sem_location

location_rpt_approximate

 

✓

✓

Geographic location - can be any geo shape, from a single point to a multi-polygon. Level of geographic precision can be adjusted in the schema to fit different use cases and performance goals.

sem_location_latlng

location

 

✓

✓

Geographic point expressed as latitude and longitude coordinates. Faster than the fields that support complex geo objects, ideal for distance sorting.

sem_geo_object

location_rpt_approximate

 

✓

✓

Geographic location - additional field with the same capabilities as sem_location.

sem_address_info

string

 

✓

Opaque field for frontend-related address information.

sem_address_street

text_minimal

 

✓

✓

Street name and number

sem_address_city

text_minimal

 

✓

✓

Town or city.

sem_address_city_facet

string

 

✓

Automatically generated field to support faceting on city names.

sem_address_postal_code

text_minimal

 

✓

✓

Postal code.

sem_address_postal_code_facet

string

 

✓

Automatically generated field to support faceting on postal codes.

sem_address_district

text_minimal

 

✓

✓

Organizational or geographical district.

sem_address_district _facet

string

 

✓

Automatically generated field to support faceting on district names.

sem_address_country

text_minimal

 

✓

✓

Country (can be full name or country code).

sem_address_country_facet

string

 

✓

Automatically generated field to support faceting on country names (or codes).

Contact-related fields

Fields related to contact details (person, telephone, email, etc.)

R = required, I = indexed, S = stored, M = multi-valued

Field Type R I S M Description

sem_contact_info

string

 

 

✓

Opaque field for frontend-related contact information.

sem_contact_phone

tel_number

 

✓

✓

✓

Main (landline) telephone number.

sem_contact_mobile

tel_number

 

✓

✓

✓

Mobile telephone number.

sem_contact_fax

tel_number

 

✓

✓

✓

Fax number.

sem_contact_email

email

 

✓

✓

✓

Email address.

sem_contact_social

string

 

✓

✓

✓

Opaque field that can be used to store contact information for social networking accounts.

sem_contact_name

text_minimal

 

✓

✓

The name of the contact (may be a person’s name but not necessary).

Document-related fields

Fields that captures details for indexed document resources.

R = required, I = indexed, S = stored, M = multi-valued

Field Type R I S M Description

sem_image_url

url

 

✓

✓

Document image

sem_image_thumbnail_url

url 

✓

 

Document thumbnail image

sem_links

url

 

✓

✓

✓

URLs related to the document (e.g., of links inside the document)

sem_headings

text_general

 

✓

✓

✓

Document section headings

sem_meta_description

text_general

 

✓

✓

Document meta-data description

sem_meta_title

text_general

 

✓

✓

Document meta-data title

sem_host

string

 

✓

✓

Host part of the document’s URL (useful when documents may be indexed from multiple hosts).

sem_path

string

 

✓

✓

Path part of the document’s URL.

sem_mime_type

mime

 

✓

✓

Document MIME type.

sem_mime_type_facet

string

 

✓

Automatically generated field that allows faceting by MIME types.

sem_content_size

long

 

✓

✓

Document content size (in bytes).

Event-related fields

Fields for capturing details of one-time and recurring events.

R = required, I = indexed, S = stored, M = multi-valued

Field Type R I S M Description

sem_event_location_name

text_minimal

 

✓

✓

Name of an event location.

sem_event_location_facet

string

 

✓

Automatically generated field to support faceting on event location names.

sem_event_start

date

 

✓

✓

Start date / time of an event.

sem_event_end

date

 

✓

✓

End date / time of an event.

sem_event_occurences

occurrence

 

✓

✓

✓

For recurring events, pairs of start and end dates and times.

sem_event_organizer

text_minimal

 

✓

✓

Event organizer’s name (person, company, etc.)

sem_event_organizer_facet

string

 

✓

Automatically generated field to support faceting on event organizers.

Web-related fields

Fields related to typical web portals features, such as blogs, FAQs and online newsletters.

R = required, I = indexed, S = stored, M = multi-valued

Field Type R I S M Description

sem_blog_name

text_general

 

✓

✓

Name of the blog

sem_blog_facet

string

 

✓

Automatically generated field to support faceting on blog names.

sem_newsletter_issue

text_general

 

✓

✓

Newsletter issue (as simple text)

sem_newsletter_facet

string

 

✓

Automatically generated field to support faceting on newsletter issues.

sem_faq_question

text_general

 

✓

✓

Question part of a FAQ

sem_faq_answer

text_general

 

✓

✓

Answer part of an FAQ

sem_faq_facet

string

 

✓

Auxiliary field that can be populated for faceting on whole FAQs.

sem_comments

text_general

 

✓

✓

✓

Can hold the text of comments of a particular page, article, blog post, etc.

sem_comments_count

int

 

✓

✓

Number of comments related to the current record (see sem_comments)

sem_comments_info

string

 

✓

Opaque field for storing additional information for the comments related to the current record (see sem_comments)

Shop-related fields

Fields intended for capturing details of products in online shops. See also Categorization fields for fields that can be used to store product category information.

R = required, I = indexed, S = stored, M = multi-valued  

Field Type R I S M Description

sem_product_sku

text_minimal

 

✓

✓

✓

A customer-specific identifier for a product.

sem_product_sku_facet

string

 

✓

✓

Automatically generated field that stores the SKU for use in exact matches and faceting.

sem_product_description

text_general

 

✓

✓

Product description.

sem_product_price

float

 

✓

✓

Product price.

sem_product_price_currency

string

 

✓

✓

Currency in which product price values are expressed.

sem_product_specialoffer_price

float

 

✓

✓

Product price when on special offer.

sem_product_specialoffer_start

date

 

✓

✓

Start date / time of a special offer.

sem_product_specialoffer_end

date

 

✓

✓

End date / time of a special offer.

sem_product_specialoffer_info

string

 

✓

Opaque string to store additional information for a special offer (e.g., to be used in the frontend).

sem_product_weight

float

 

✓

✓

Product weight.

sem_product_weight_unit

string

 

✓

✓

Unit in which the product’s weight is expressed,

sem_product_size

text_minimal

 

✓

✓

Textual description of the the product’s dimensions.

sem_product_size_facet

string

 

✓

Automatically generated field for faceting over product size (usable only for a single dimension, for multiple dimensions additional fields need to be declared).

sem_product_in_stock

boolean

 

✓

✓

Whether a product is in stock.

sem_product_color

text_general

 

✓

✓

✓

The product’s color(s).

sem_product_color_facet

string

 

✓

✓

Automatically generated field for faceting over the product’s color(s).

sem_product_brand

text_general

 

✓

✓

✓

The product’s brand’s name.

sem_product_brand_facet

string

 

✓

✓

Automatically generated field for faceting over the product’s brand’s name.

sem_product_manufacturer

text_general

 

✓

✓

✓

The product’s manufacturer.

sem_product_manufacturer_facet

string

 

✓

✓

Automatically generated field for faceting over the product’s manufacturer.

sem_opening_hours

occurrence

 

✓

✓

✓

Specialized field to store the opening hours of shops, so that "is it open now" or "is it open at any time during the weekend" can be answered.

sem_product_ean

ean

 

✓

✓

A product’s EAN.

sem_product_age_group

text_minimal

 

✓

✓

✓

The age group(s) for which a product is intended.

sem_product_age_group_facet

string

 

✓

✓

Automatically generated field for faceting over the age group(s) for which a product is intended.

sem_product_gender

text_minimal

 

✓

✓

✓

The gender(s) for which a product is intended.

sem_product_gender_facet

string

 

✓

✓

Automatically generated field for faceting over the gender(s) for which a product is intended.

sem_product_material

text_minimal

 

✓

✓

✓

The product’s material(s).

sem_product_material_facet

string

 

✓

✓

Automatically generated field for faceting over the product’s material(s).

Categorization fields

Fields that facilitate the capturing and searching of categorical information, including the automatic management of hierarchical information.

R = required, I = indexed, S = stored, M = multi-valued

Field Type R I S M Description

sem_topic

string

 

✓

✓

✓

Main document topic

sem_topic_*

string

 

✓

✓

✓

Field prefix for dynamic fields that represent additional document topics.

sem_category_main

string

 

✓

✓

✓

Main document category (non-hierarchical)

sem_category_main_*

string

 

✓

✓

✓

Field prefix for dynamic fields that represent additional document categories (non-hierarchical).
HIERACHICAL FIELDS
Field Type R I S M Description

sem_category_ids_branch

descendent_path

 

✓

✓

✓

Hierarchical category branch(es), using category ids

sem_category_ids

cat_minimal

 

✓

✓

Automatically generated field with all category ids in the branch.

sem_category_ids_root

cat_root

 

✓

✓

Automatically generated field with only the root category ids

sem_category_ids_leaf

cat_leaf

 

✓

✓

Automatically generated field with only the bottom (leaf) category ids

sem_category_names_branch

descendent_path

 

✓

✓

✓

Hierarchical category branch(es), using category names

sem_category_names

cat_minimal

 

✓

✓

Automatically generated field with all category names in the branch.

sem_category_names_root

cat_root

 

✓

✓

Automatically generated field with only the root category names

sem_category_names_leaf

cat_leaf

 

✓

✓

Automatically generated field with only the bottom (leaf) category names

sem_category_ids_branch_*

descendent_path

 

✓

✓

✓

Prefix version of sem_category_ids_branch

sem_category_ids_*

cat_minimal

 

✓

✓

Prefix version of sem_category_ids

sem_category_ids_root_*

cat_root

 

✓

✓

Prefix version of sem_category_ids_root

sem_category_ids_leaf_*

cat_leaf

 

✓

✓

Prefix version of sem_category_ids_leaf

sem_category_names_branch_*

descendent_path

 

✓

✓

✓

Prefix version of sem_category_names_branch

sem_category_names_*

cat_minimal

 

✓

✓

Prefix version of sem_category_names

sem_category_names_root_*

cat_root

 

✓

✓

Prefix version of sem_category_names_root

sem_category_names_leaf_*

cat_leaf

 

✓

✓

Prefix version of sem_category_names_leaf

Synthetic fields (automatically generated)

The fields described here are generated automatically and can be used for various search purposes.

R = required, I = indexed, S = stored, M = multi-valued

Field Type R I S M Description

sem_timestamp

date

 

✓

✓

Field set automatically to the time of indexing; can be overridden if desirable.

sem_text_search

text_general

 

✓

✓

✓

"Sink" field into which all searchable fields are copied

sem_text_spell

text_spell

 

✓

✓

"Sink" field into which all fields from which spelling suggestions can be derived are copied.

sem_text_suggest

sem_text_suggest

 

✓

✓

"Sink" field into which all fields from which auto-complete suggestions can be derived are copied.

sem_text_excerpt

sem_text_excerpt

 

✓

✓

✓

This field can be optionally used as a "sink" field from which the search excerpts (or "snippets") are extracted. Normally the field sem_text_search is used for that.

Indexing

Adding, updating and deleting documents

Topics: indexing whole documents; performing atomic partial updates of documents; deleting documents; "hard" commits.

Adding documents

The indexing of documents is done through the end point /ingest of the search server installation.

This endpoint accepts POST requests, containing the document(s) to be indexed in the request’s body. Documents must be encoded in one of these format / mime-types:

  • text/xml or application/xml

  • text/json or application/json

  • text/csv or application/csv

The capabilities of the default Solr update handler are present here also.

Ingested documents are automatically committed to the index after a preset period of time, or after a preset maximum number of documents have been ingested. Note that committing documents to the index does not make them automatically visible to the searches. For that a manual commit or a scheduled or manual optimize call need to be performed.

Performing atomic updates

To perform partial atomic updates to a document’s fields, it is possible to use field modifiers. The supported modifiers are:

Modifier Description

set

Set or replace the field value(s) with the specified value(s), or remove the values if 'null' or empty list is specified as the new value. May be specified as a single value, or as a list for multivalued fields.

removeregex

Removes all occurrences of the specified regex from a multi-valued field. May be specified as a single value, or as a list.

remove    

Removes (all occurrences of) the specified values from a multi-valued field. May be specified as a single value, or as a list.

inc

Increments a numeric value by a specific amount. Must be specified as a single numeric value.

add

Adds the specified values to a multivalued field. May be specified as a single value, or as a list.

 

Field modifiers are specified as single-attribute “objects”, with the modifier name as an attribute, and the parameters as the attribute’s value; this “object” then takes the place of what would normally have been the field’s value.

The following example demonstrates the use of the field modifiers for the partial update of an object:

Example of the use of field modifiers

{
 "id":"mydoc",
 "price":{"set":99},
 "popularity":{"inc":20},
 "categories":{"add":["toys","games"]},
 "promo_ids":{"remove":"a123x"},
 "tags":{"remove":["free_to_try","on_sale"]}
}

Deleting documents

Deletion of documents is done through a POST request  to the endpoint /delete, with the parameter q containing the query that identifies the document or documents to be deleted.

To delete individual documents, q can have the form id:<document id>.

To delete the entire set of documents, q can have the form *:*.

Selective deletion of groups of documents is typically done through a corresponding grouping field (e.g., sem_grouping:<some top-level category>)

Hard commits

Manual invocation of commits (or “hard” commits) can be done through:

  • A POST request to the endpoint /commit

  • A POST request to the ingestion address with content type application/xml, and body: <commit />

The first of the above options may take longer to complete, because the call only returns after all changes have been flushed to disk, and are “visible” in the search results of any subsequent search calls.

Searching

This chapter introduces the basic concepts of the Semanteer Search Layer.

  • It contains an overview of querying endpoints and their parameters, 

  • It explains the response structures for the Semanteer Search Layer.

  • It presents the available search features and describes its configuration options in detail.

The search request

Semanteer Search Layer exposes a default endpoint for searching under /search. The search will take one or more search term(s) as parameter - together with additional optional parameters that are described in this section.

The search will give you back a response that contains the result documents and additional information (e.g. Information about Filters, Grouping, Sorting, and Results with corresponding action links).

Basic Parameters

The basic parameters described below can be applied in every search request.

Moreover every component can add additional parameters to enable configuration options. These additional parameters are described in detail in the corresponding section of the particular component (Features, e.g. Autofilter or Geographic Query)
Search Term

There is only one mandatory parameter that is

  • q: This parameter contains the search terms entered by the user

Special characters allowed in the search queries include:

  • *: matches any character(s) in a indexed term / token. When used in isolation, returns all documents in the index.

  • +: indicates that the term that immediately follows it, must exist in the documents returned

  • -: indicates that the term that immediately follows it, is not allowed to exist in the documents returned

  • "…": terms enclosed in double quotes are searched verbatim in the indexed documents

Try it out!
All examples used in the documentation are retrieved from our online demo, try e.g. http://search-features.demos.contexity.ch/sl/search?q=Sauvignon
Optional parameters

There are additional parameters that will influence the query itself as well as the representation of the results.

  • Parameters that are relevant for the Response Format(s)

    • wt: determines the format of the response; officially supported formats are json and xml

    • indent: if set to true or on, the response is indented for easier reading; default is false

  • Paging parameters can be set by the frontend or by using the actions from the Base URLs.

    • rows: Determines the number of results to be returned in a "batch"

    • start: Determines the offset into the result set of the first document to be included in the current "batch"

  • Parameters for Sorting Results and Grouping Results do not need to be managed separately by the frontend

  • Filter parameters do not need to be managed separately by the frontend. To enable or disable Facets and Filters the frontend can use the corresponding "actions which already contain the full current state of the search, including activated and deactivated filters. See section Action URLs for more details. Filtering parameters typically start with the prefix ‘f.’.

  • Monitoring parameters, like filter parameters, do not need to be managed explicitly by the frontend. Action URLs are provided in two forms, one of which contains all the necessary monitoring parameters pre-populated in the URL. See section Monitoring URLs for more details. Monitoring parameters start with the prefix '`ot.'`.

  • Parameters to activate different configurations of to activate defined prepared queries. See section Advanced configuration options for more details.

Semanteer vers Solr parameters
Users that are familiar with Apache Solr need to be aware that some common features - e.g. Grouping Results - can not be enabled the same way as with Solr (by adding the regarding parameters).

The search response

The search response of Semanteer Search Layer contains a lot of information about documents, facets, as well as Action URLs that enables you to easily navigate through the search. That way you can activating and deactivate features, apply filters, change sorting and grouping, use paging links, select alternative spell-corrected suggestion, retrieve similar (more-like-this) results, and so on.

Response Format(s)

Semanteer officially supports two response format, JSON and XML (other formats supported by Solr are possible, please inquire if you need one of those returned).

The default format returned is JSON and conforms to the arrmap variant of the Solr JSON response subtypes. To retrieve results in XML format instead, add the parameter wt=xml to the requests sent to the search server. To activate indentation in the results (turned off by default), add the parameter indent=on to the requests (applies both to JSON and XML).

The description of response structures in the rest of this document uses the JSON format, but all descriptions transfer exactly to the XML format as well (including naming, nesting, etc.

Structure of the response

The overall structure of a (ungrouped) search response is as follows (see also Appendix A.1. Full search response for a full sample response)

Main sections in the response

The main sections of the response are:

  • responseHeader: This section contains information about the response as a whole. Fields therein include:

    • semQTime: The total time a request took on the search server ** QTime: The time spent exclusively on the main query by Solr

    • semStatus: Search server status code (0 signifies no errors)

    • status: Solr status code (0 signifies no errors)

  • response: This section contains the actual search results, when no grouping is applied - if any were found. Fields therein include:

    • numFound: The total number of search results found (to be distinguished from the number of results in the current batch returned)

    • start: Zero-based offset of the first result in the current batch of results returned

    • docs: An array of results, with each result represented as an “object” with its own fields

  • grouped: This section contains the actual search results when grouping is applied. (please refer to Grouping Results for more details)

  • facet_counts: This section contains the different types of facets to which the total set of results can be assigned. Fields therein include (please refer to [Facets and filters] for more details)

    • facet_fields: This is the most common type of facet, and refers to index fields that have distinct values (typically strings). (see also Text based facets)

    • facet_queries: This type of facet corresponds to Solr query-based facets.)

    • facet_dates: This is a specialized type of facet that is used for date fields.

    • facet_intervals: This type of facet is activated for fields that can be split into preset intervals (usually numeric fields).

    • facet_ranges: This type of facet is activated for fields that can be split into preset ranges (usually numeric fields) (see also [Range facet])

    • facet_hierarchies: This type of facet is activated for fields that contain a hierarchical information (see also [Hierarchical facet]) 

  • active_filters: This section contains Information about filters that are currently active. That maybe facet based filters, or filters added by the Geographic Query or the Autofilter. Filters get activated through parameters in the incoming request or through the internal logic of involved components. (please refer to Active and suggested Filters for more details)

  • suggested_filters: This section contains information about filters suggested by the Geographic Query or the Autofilter. (please refer to Active and suggested Filters for more details)

  • base_urls: This section contains a set of “base” URLs that can be used to compose other URLs for requests to the search server (please refer to Base URLs for more details)

  • paging: A specialized part of the base_urls that is available also separately for convenience purposes (please refer to section Base URLs for more details)

  • spelling: This section contains an array of spelling suggestions, if any can be made for the current search term(s) (please refer to Spelling suggestions for more details)

Note that there is only either a `response `or a `grouped `section in the response depending on whether grouping is applied or not!

The individual sections are described in more detail in the rest of this document.

The Context section
Available since version 1.6.

The context section reveals information about what and how tokens have been used in the search:

  • originalTokens: Lists all tokens of the original searchterm

  • originalSearchTerm: The original search term (as supplied with the parameter q).

  • tokens: Lists all tokens that remain after tokens within matched terms have been removed.

  • searchTerm: The searchterm after matched terms have been removed - that is the term that is sent to solr then.

  • matchedTerms:  A list of terms that have been identified as matches by certain components (Autofilter and Geographic Query) an may have been removed from the searchterm.

    • componentName: The name of the component that using the term 

    • componentClass: The class of the component that using the term

    • matchType:` AUTOFILTER` or GEOQUERY

    • tokens: List of tokens that make up the term

    • tokenPositions: Positions of the tokens within the originalTokens list

    • remove: Whether or not the tokens have been removed in the searchTerm 

Additional sections in the response

Apart from the main sections described above some components add their own section to the response:

  • autofilter: This section is added by the Autofilter

  • grouping: This section is added for Grouping Results in addition to the grouped section mentioned above.

  • geoquery: This section is added by the Geographic Query.

Example

The example below illustrates the overall structure of a (ungrouped) search response.

Structure of an ungrouped search response
 {
 "responseHeader": {
  "QTime": 103,
  "solrQTime": 5,
  "status": 0,
  "solrStatus": 0,
  "params": {
   "q": "dummy",
   "...": "other request parameters"
  }
 },
 "response": {
  "numFound": 13,
  "start": 0,
  "docs": [{
   "id": "10",
   "...": "other document fields"
  }, {
   "other documents": "..."
  }]
 },
 "facet_counts": {
  "facet_queries": "< facet structure >",
  "facet_fields": "< facet structure >",
  "facet_dates": "< facet structure >",
  "facet_ranges": "< facet structure >",
  "facet_intervals": "< facet structure >"
 },
 "autofilter": "{ ...  autofilter fields ... }",
 "geoquery": "{ ... geoquery fields ... }",
 "spelling": "[ < spelling structure > , ...]",
 "active_filters": "{ < action structure >}",
 "suggested_filters": "{ < action structure >}",
 "base_urls": "{ < named URL structures >}",
 "paging": "{ < named URL structure > }"
}

Base URLs

Base URLs are URLs that are intended to facilitate implementation of the frontend.

The response contains a section base_urls that is made up by a number of URLs. As can be seen in the snippet below, Base URLs have a structure identical to that of individual actions. Base URLs are intended to be used as a basis for implementing features such as paging, resetting search state, etc. in the frontend.

Listing of Base URLs

Base URLs that are always available are:

  • paging: To be used as the basis for URLs that maintain the current search state but without paging parameters.

  • search_with_current_state: To be used as a basis for URLs that maintain the current search state (including paging state)

  • search_with_empty_state: These URLs “reset” the current search state, leaving any additional request parameters intact. Note that these URLs do not contain a q parameter, which needs to be added anew to them

  • remove_all_filters: These URLs, as their name suggests, deactivate all currently active filters, leaving the rest of the search state intact. They are equivalent to the top-level sem_action in the active filters’ section of the response (see Facet values as filters for more details).

Furthermore additional (optional) Base URLs may be added:

  • disable_geoquery or enable_geoquery: En- or disables the Geographic Query

  • dont_apply_geoquery or apply_geoquery: Apply or do not apply places found by the Geographic Query

  • reset_geoquery: Reset the Geographic Query (that means removing related parameters).

  • disable_autofilter or enable_autofilter: En- or disables the Autofilter

  • dont_apply_autofilter or apply_autofilter: Apply or do not apply matches found by the Autofilter

  • reset_autofilter: Reset the Autofilter (that means removing related parameters).

  • disable_grouping and enable_grouping: En- or disables grouped results. That requires a configuration with a GroupFieldComponent. Read more about Grouping Results

Example

The example below shows a snippet of the base_urls section:

Code example

 {
  "base_urls": {
    "paging": {
      "sem_url": "?q=*",
      "sem_url_action": "?q=*&ot.et=search&ot.est=paging&ot.es=source_serp&ot.sse=984a17c4-cb47-4c3e-b2e2-9d6c54519fe3"
    },
    "search_with_current_state": {
      "sem_url": "",
      "sem_url_action": "?ot.et=search&ot.est=normal&ot.es=source_serp&ot.sse=984a17c4-cb47-4c3e-b2e2-9d6c54519fe3"
    },
    "search_with_empty_state": {
      "sem_url": "?",
      "sem_url_action": "?ot.et=search&ot.est=normal&ot.es=source_serp&ot.sse=984a17c4-cb47-4c3e-b2e2-9d6c54519fe3"
    },
    "remove_all_filters": {
      "sem_url": "?q=*",
      "sem_url_action": "?q=*&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=984a17c4-cb47-4c3e-b2e2-9d6c54519fe3"
    },
    "disable_geoquery": {
      "sem_url": "?q=*&geoquery=false",
      "sem_url_action": "?q=*&geoquery=false&ot.et=search&ot.est=autofilter-modified&ot.es=source_serp&ot.sse=984a17c4-cb47-4c3e-b2e2-9d6c54519fe3"
    },
    "reset_geoquery": {
      "sem_url": "?q=*",
      "sem_url_action": "?q=*&ot.et=search&ot.est=autofilter-modified&ot.es=source_serp&ot.sse=984a17c4-cb47-4c3e-b2e2-9d6c54519fe3"
    },
    "dont_apply_geoquery": {
      "sem_url": "?q=*&geoquery.apply=false",
      "sem_url_action": "?q=*&geoquery.apply=false&ot.et=search&ot.est=autofilter-modified&ot.es=source_serp&ot.sse=984a17c4-cb47-4c3e-b2e2-9d6c54519fe3"
    },
    "disable_autofilter": {
      "sem_url": "?q=*&autofilter=false",
      "sem_url_action": "?q=*&autofilter=false&ot.et=search&ot.est=autofilter-modified&ot.es=source_serp&ot.sse=984a17c4-cb47-4c3e-b2e2-9d6c54519fe3"
    },
    "reset_autofilter": {
      "sem_url": "?q=*",
      "sem_url_action": "?q=*&ot.et=search&ot.est=autofilter-modified&ot.es=source_serp&ot.sse=984a17c4-cb47-4c3e-b2e2-9d6c54519fe3"
    },
    "dont_apply_autofilter": {
      "sem_url": "?q=*&autofilter.apply=false",
      "sem_url_action": "?q=*&autofilter.apply=false&ot.et=search&ot.est=autofilter-modified&ot.es=source_serp&ot.sse=984a17c4-cb47-4c3e-b2e2-9d6c54519fe3"
    }
  }
}
For convenience, the paging URLs are also available as a separate top-level section in the response under paging. Note that neither this top-level section, nor the corresponding part in the Base URLs are present if the number of results are less than or equal to the default, or the dynamically set "page" size.

Common conventions

Semanteer Search Layer responses have some elements that are common across all response sections. The most important of them are Encoded document identifiers, Action URLs, and Monitoring URLs.

Encoded document identifiers

Documents in the search response always contain a field named sem_encoded_id. As its name suggests, this field contains an encoded form of the document’s id field. This form is recognized and decoded by search endpoints that expect a document reference (e.g., the [More like this] and Record endpoint). The encoding should be treated as a “black-box” one, and it can be assumed that its representation is always URL-safe.

Action URLs

For certain elements in the response (e.g., documents, facets, etc.), Semanteer can determine what actions are possible with them. For instance, documents can be retrieved through a URL, facets can be activated as filters, etc. The actions understood by Semanteer are dynamically added to the corresponding elements, using the following structure:

Code example

 {
  "sem_actions": {
    "default": "goto",
    "goto": {
      "sem_url": "... url ...",
      "sem_url_action": "... monitoring url ...",
      "sem_is_external": true
    },
    "mlt": {
      "sem_url": "... url ...",
      "sem_url_action": "... monitoring url ..."
    }
  }
}

Key points regarding the above structure:

  • Every action is assigned an own identifier, which signifies the type of the action. For example, goto is the identifier for simple “follow the link” actions, and mlt is the identifier for invoking the More Like This endpoint.

  • If there are more than one actions possible, Semanteer will designate one of them as the default. If not otherwise configured, Semanteer will always select the goto action of an element as the default one.

  • Actions are expressed through URLs that need to be invoked. These could be “internal” URLs that point to specific endpoints of the search server, or “external” URLs that lead to addresses outside the scope of the search server. Which of the two types an action URL belongs to is captured in the field sem_is_external. The URL itself is captured in the field sem_url. 

Action URLs are, in practically all cases, not complete URLs, but rather the query portion of URLs; frontend code needs to compose the full URL by appending the provided query portion to the base search endpoint address of an installation. This approach allows JS-based frontends to adopt a URL schema that mirrors that of the search server for ease of implementation. The sample development frontend demonstrates how this can be implemented in practice.

A variant of the set of actions structure explained above is the structure used for single actions:

Code example

 {
  "sem_actions": {
    "default": "goto",
    "goto": {
      "sem_url": "... url ...",
      "sem_url_action": "... monitoring url ...",
      "sem_is_external": true
    },
    "mlt": {
      "sem_url": "... url ...",
      "sem_url_action": "... monitoring url ..."
    }
  }
}
Note that the above structure is essentially the same as the structure of individual actions in a set, as presented above. For example these type of single action is used in Facets and Filters, Spelling suggestions, Sorting Results, ..

Monitoring URLs

When monitoring is activated, Semanteer generates two dynamic URLs for elements that support actions. The “simple” URL is still present, and is captured in the field sem_url (just as when monitoring is not activated). The additional URL can be found in the field sem_url_action and may contain additional parameters intended to be intercepted by the search server’s monitoring component. Such parameters may indicate, for instance, the position of a clicked result in the total list of search results, or that the user’s current search is using terms that were offered as spelling suggestions to the previous search.

Note that the “simple” URLs are typically the ones shown to users in the frontend, while the “action” URLs are the ones actually called for performing the search.

Document fields

For every single document in the result list a set of fields with values and Action URLs are returned.

The fields returned belong to two categories:

  • Fields stored in the index: These fields are identical to what Solr would return as well. It is possible to define in the search configuration which fields should be returned, if not all stored fields are needed in the response.

  • Fields generated by Semanteer: These fields are dynamically generated by the search server.

In the example below  we can see a response with the following Semanteer fields:

  • sem_encoded_id: This is an encoded form of the document id (see Encoded document identifiers)

  • sem_excerpt: This is a (set of) highlighted excerpt(s) that contain the search term(s), derived from the main “sink” field for searching. 

  • sem_actions: This is a set of structured actions related to the present document. Please refer to Action URLs and Monitoring URLs described in Common conventions for details.

A single entry for a document may look like the following:

Code example

 {
  "wine_producer": "Domaine de la Romanée-Conti",
  "sem_text_search": [
    "Domaine de la Romanée-Conti",
    "France",
    "Red Wine",
    "Vosne-Romanée",
    "La Tâche AOC",
    "1999 Domaine Romanee Conti La Tache"
  ],
  "sem_text_suggest": [
    "Domaine de la Romanée-Conti",
    "France",
    "La Tâche AOC",
    "1999 Domaine Romanee Conti La Tache"
  ],
  "sem_suggest_prefix": [
    "Domaine de la Romanée-Conti",
    "France",
    "La Tâche AOC",
    "1999 Domaine Romanee Conti La Tache"
  ],
  "sem_suggest_phrase": [
    "Domaine de la Romanée-Conti",
    "France",
    "La Tâche AOC",
    "1999 Domaine Romanee Conti La Tache"
  ],
  "wine_sub_region": "Côte de Nuits",
  "wine_style": [
    "Red Burgundy"
  ],
  "wine_style_facet": "Red Burgundy",
  "fruit_source": "Producer owned",
  "country": "France",
  "/common/topic/image": [
    "DRC99LaTache.JPG"
  ],
  "id": "/en/1999_domaine_romanee_conti_la_tache",
  "region": "Burgundy",
  "wine_type": "Still Wine",
  "wine_type_facet": "Still Wine",
  "percentage_alcohol": 13.5,
  "color": "Red Wine",
  "sem_grouping": "Red Wine",
  "color_facet": "Red Wine",
  "appellation": "Vosne-Romanée",
  "appellation_facet": "Vosne-Romanée",
  "vintage_hierachy": "Old:1999",
  "vintage": 1999,
  "vineyard": "La Tâche AOC",
  "sem_location": "47.161664, 4.955435",
  "sem_location_latlng": "47.161664, 4.955435",
  "sem_publication_date": "2017-05-29T16:33:28.258Z",
  "sem_opening_hours": [
    "1701010000 2501010000"
  ],
  "sem_title": "1999 Domaine Romanee Conti La Tache",
  "sem_title_alpha": "1999 Domaine Romanee Conti La Tache",
  "sem_record_type": "wine",
  "sem_lang": "it",
  "sem_timestamp": "2018-11-13T16:35:03.868Z",
  "sem_actions": {
    "default": "goto",
    "goto": {
      "sem_url": "/record?id=SEML2VuLzE5OTlfZG9tYWluZV9yb21hbmVlX2NvbnRpX2xhX3RhY2hl",
      "sem_url_action": "/record?id=SEML2VuLzE5OTlfZG9tYWluZV9yb21hbmVlX2NvbnRpX2xhX3RhY2hl&ot.et=result&ot.est=serp-result-click&ot.q=*&ot.es=source_serp&ot.nf=100&ot.s=0&ot.r=20&ot.rank=0&ot.sse=984a17c4-cb47-4c3e-b2e2-9d6c54519fe3",
      "sem_url_logging": "http://search-features.demos.contexity.ch/sl/log?id=SEML2VuLzE5OTlfZG9tYWluZV9yb21hbmVlX2NvbnRpX2xhX3RhY2hl&ot.et=result&ot.est=serp-result-click&ot.q=*&ot.es=source_serp&ot.nf=100&ot.s=0&ot.r=20&ot.rank=0&ot.sse=984a17c4-cb47-4c3e-b2e2-9d6c54519fe3",
      "sem_is_external": false
    },
    "mlt": {
      "sem_url": "http://search-features.demos.contexity.ch/sl/morelikethis?id=SEML2VuLzE5OTlfZG9tYWluZV9yb21hbmVlX2NvbnRpX2xhX3RhY2hl",
      "sem_url_action": "http://search-features.demos.contexity.ch/sl/morelikethis?id=SEML2VuLzE5OTlfZG9tYWluZV9yb21hbmVlX2NvbnRpX2xhX3RhY2hl&ot.et=mlt&ot.est=mlt&ot.es=source_serp&ot.nf=100&ot.s=0&ot.r=20&ot.rank=0&ot.q=*&ot.sse=984a17c4-cb47-4c3e-b2e2-9d6c54519fe3"
    }
  },
  "sem_encoded_id": "SEML2VuLzE5OTlfZG9tYWluZV9yb21hbmVlX2NvbnRpX2xhX3RhY2hl"
}
This field sem_excerpt is not generated when the search term(s) contain only wildcard characters (e.g., q=dum* will still provide excerpts, but q=* will not)

Configure your search

The behavior of the Semanteer Search Layer, including the activation of its features, can easily be configured in one configuration file: search-layer-configuration.json.

Endpoints

There are two endpoints to read the actual configuration: /config and /config/file

The difference between the endpoints /config and /config/file is that /config shows a processed version where inheritance of configuration has been processed (see Advanced configuration options and The extends property and configuration inheritance)

The configuration file

The search-layer-configuration.json configuration file is made up different sections.

  1. properties: This section contains the basic properties of the semanteer search. Read more about the available properties, and what they are used for. See Configuration Properties for a complete Listing of available properties.

  2. configurations: This section contains one or more configuration(s), each of it is composed by predefined parameters, predefined queries, and a set of different components. See Advanced configuration options to learn how it works!

  3. collection: This sections contains information about different collection that can be used for searching. See Distributed Search - multiple collections on how to use it!

Structure

The code below shows the basic structure of search-layer-configuration.json.

Code example

 {
 "collections": {
  "empty": {
   "solrUrl": "http://localhost:8035/solr/empty",
   "monitoringSolrUrl": "http://localhost:8035/solr/sem_monitoring_core_empty",
   "analyticsSolrUrl": "http://localhost:8035/solr/sem_monitoring_core_empty"
  },
  "wines": {
   "solrUrl": "http://localhost:8035/solr/winedemo",
   "monitoringSolrUrl": "http://localhost:8035/solr/sem_monitoring_core",
   "analyticsSolrUrl": "http://localhost:8035/solr/sem_monitoring_core"
  }
 },
 "properties": {
  "search.config.autocomplete.group": "sem_grouping",
  "... more properties ": "..."
 },
 "configurations": {
  "default": {
   "extends": [{
    "configuration": "base",
    "mode": "merge"
   }],
   "requestParams": {
    "defaults": [{
     "name": "sph.fq",
     "value": "sem_record_type:\"wine\""
    }, {
     "... defined default request parameters ": "..."
    }],
    "invariants": [{
     "... defined invariants request parameters ": "..."
    }],
    "appends": [{
     "... defined appends request parameters ": "..."
    }]
   },
   "preparedQueries": {
    "oldwines": [{
     "name": "q",
     "value": "*:*"
    }, {
     " ... more parameters for the prepared Query": "..."
    }]
   },
   "components": [{
    "type": "FacetFieldComponent",
    "... component definition ... ": "..."
   }, {
    "... more components ... ": "..."
   }]
  }
 }
}
Try it out: Let’s have a look at the configuration of our demo search: http://search-features.demos.contexity.ch/sl/config

Configuration Properties

Within the properties section of search-layer-configuration.json the basic configuration of the search is set up.

There a set of options can be configured, e.g. fields to be used for retrieving ID, thumbnail and title of your documents, default setup and field for grouping, autocomplete features, and so on.

Listing of all configuration properties
Name Type Default value See also Description

search.config.autocomplete

String

autocomplete

Autocomplete

Response section name for autocomplete

search.config.autocomplete.config

String

autocomplete

Autocomplete

Name of the configuration that is used for prefetching of results in autocomplete

search.config.autocomplete.highlight

boolean

false

Autocomplete

En-/Disables highlighting of results.

search.config.autocomplete.lasttermquery

boolean

false

Autocomplete

En-/Disables retrieving suggestions for the last term of the query only (in case no other suggestions for the full query string are present)

search.config.autocomplete.maxsuggestions

int

5

Autocomplete

Max. number of suggestions in autocomplete

search.config.autocomplete.response

String

response

Autocomplete

Name of main response section in the autocomplete

search.config.autocomplete.prefetchedResults

String

prefetchedResults

Autocomplete

Response section name for prefetched results

search.config.autocomplete.prefetchResults

boolean

true

Autocomplete

En-/Disables prefetching of results.

search.config.autocomplete.sortsuggestions

String

natural

Autocomplete

"natural" of "hits": sorting mode that will be applied for suggestions

search.config.autocomplete.spellcheckresults

boolean

false

Autocomplete

En-/Disables retrieving results from spellcheck and include them in the list of suggestions (in case no other suggestions are present)

search.config.autocomplete.suggestions

boolean

true

Autocomplete

En-/Disables retrieving the list of suggestions.

search.config.caching.solr

boolean

false

 

Turn on caching for solr results. Default cache size when enabled 500 entries, ttl=1800sec. Can be devativated on request basis with sph.sl.nocache=false

search.config.field.id

String

id

 

Fieldname where the ID is stored

search.config.field.similarity

String

sem_similarity

 

Fieldname where similarity is shown

search.config.field.thumbnail

String

sem_image_thumbnail_url

 

Fieldname where the thumbnail is stored

search.config.field.title

String

sem_title

 

Fieldname where the title is stored

search.config.field.url

String

sem_url

 

Fieldname where the url is stored

search.config.fields.exclude

String

-

 

Names of the fields that should be excluded from the response

search.config.fields.noncontent

String

*

Autocomplete

Names of the fields that should be shown in the autocomplete prefetched Results

search.config.geoquery.parallel

boolean

false

Geographic Query

Whether to prefetching request in the Geographic Query should be parallelized

search.config.mlt.doctype.field

String

-

More Like This

Name of the Field that is used to restrict MoreLikeThis results to documents with the same value in this field.

search.config.mlt.similarityfields

String

sem_text_search

More Like This

Name of the field(s) that is/are used to get MoreLikeThis results

search.config.monitoring

boolean

true

 

(De)activate logging / monitoring for all endpoints

search.config.operator

String

default

 

Operator (AND or OR) that is used. Values can be: default, AND, OR or AND_OR (Use AND first and repeat the search with OR when the number of results is below a specified threshold (search.config.operator.and.mincount)Can be overriden with the sem.q.op parameter

search.config.operator.and.mincount

int

1

 

Threshold for repeating an AND search with the OR operator (see search.config.operator)

search.config.result.field.url

String

sem_url

 

Name of the property where the url(s) are shown in the response

search.config.spellchecker

List

default

Autocomplete

List of the types of spellchecker to be used (1.6.5+)

search.config.suggester

List

prefix, default

Autocomplete

List of the types of suggester to be used: e.g. 'prefix'

search.config.suggest.prefix

String

sem_text_suggest

Autocomplete

Name of the field that is used for facet prefix suggester

search.config.suggest.prefix.fq

String

-

Autocomplete

Filter query that should be applied when using the facet prefix suggester (e.g. "sem_record_type:wine")

search.config.urls.base_url

String

-

 

Optional baseurl for urls that are constructed as absolute urls. (1.6.2+)

search.config.urls.mlt_urls.mode

String

absolute

 

Determines how mlt urls are constructed. Valid options: no_path, absolute, full_path, last_path_segment. (1.6.2+)

search.config.urls.result_urls.mode

String

absolute

 

Determines how result urls are constructed. Valid options: no_path, absolute, full_path, last_path_segment. (1.6.2+)

search.config.urls.search_urls.mode

String

no_path

 

Determines how search urls are constructed. Valid options: no_path, absolute, full_path, last_path_segment. (1.6.2+)

search.config.urls.use_forwarded_headers

boolean

false

 

Read X-Forwarded-Proto and X-Forwarded-Host from Request Header and use in the generate of absolute urls (see above) (1.6.2+)
Code Example
Take a look at the properties in our example configuration at http://search-features.demos.contexity.ch/sl/config 
{
  "properties": {
    "search.config.autocomplete": "autocomplete",
    "search.config.autocomplete.config": "autocomplete.ungrouped",
    "search.config.autocomplete.highlight": true,
    "search.config.autocomplete.lasttermquery": true,
    "search.config.autocomplete.maxsuggestions": 5,
    "search.config.autocomplete.response": "response",
    "search.config.autocomplete.prefetchedResults": "prefetchedResults",
    "search.config.autocomplete.prefetchResults": true,
    "search.config.autocomplete.sortsuggestions": "natural",
    "search.config.autocomplete.spellcheckresults": true,
    "search.config.autocomplete.suggestions": true,
    "search.config.caching.solr": false,
    "search.config.field.id": "id",
    "search.config.field.similarity": "sem_similarity",
    "search.config.field.thumbnail": "sem_image_thumbnail_url",
    "search.config.field.title": "sem_title",
    "search.config.field.url": "sem_url",
    "search.config.fields.exclude": "_version_",
    "search.config.fields.noncontent": "id,sem_image_thumbnail_url,sem_url,sem_title",
    "search.config.geoquery.parallel": false,
    "search.config.mlt.doctype.field": "sem_record_type",
    "search.config.mlt.similarityfields": "sem_text_search",
    "search.config.monitoring": true,
    "search.config.operator": "default",
    "search.config.operator.and.mincount": 1,
    "search.config.result.field.url": "sem_url",
    "search.config.suggester": "prefix,suggest_prefix",
    "search.config.suggest.prefix": "sem_text_suggest",
    "search.config.suggest.prefix.fq": "sem_record_type:wine",
    "search.config.urls.base_url": "http://search-features.demos.contexity.ch/sl/",
    "search.config.urls.mlt_urls.mode": "absolute",
    "search.config.urls.result_urls.mode": "absolute",
    "search.config.urls.search_urls.mode": "no_path"
  }
}

An example of a complete configuration file can be found in Appendix B: Semanteer configuration file: search-layer-configurations.json.

Advanced configuration options

The configurations section of the configuration file contains one or more configuration(s), each of it is composed by predefined parameters, predefined queries, and a set of different components.

Components

Components are added to a configuration by simply adding them to the components list of the regarding configuration. Features and Advanced Features (Autofilter and Geographic Query to learn more about the specific components.

Different configurations

Semanteer can use more than one configuration that can be defined in search-layer-configurations.json. The main advantage of using different configurations is the possibility to switch between them depending on the use-case (e.g. frontend and backend) that must fit different requirement.

Switching between configurations

In the normal case Semanteer would apply the default configuration. To use another configuration an additional parameter _semc has to be applied.

It is also possible to supply _semc more than once, in which the configrations will be merged together, e.g. http://search-features.demos.contexity.ch/sl/search?q=*&_semc=map&_semc=minimal (1.7+)
The extends property and configuration inheritance

A form of "inheritance" to allow for writing complex configurations without unnecessary repetitions. 

Multiple inheritance

"Parent" referred to with extends.configuration. Order not important.

The property extends.mode can have the values:

  • merge: the contents of the known sections of the present configuration will be added to the contents of the corresponding sections of the parent configurations. Components that can be defined only once or component with the same name would be replaced.

  • replace: the contents of the known sections of the present configuration will replace the contents of the corresponding sections of the parent configurations

If extends.mode is omitted, the default value merge is assumed.

A configuration can be based on a parent configuration and thus inherit properties and components defined already in the parent.

The code example below shows the configuration grouped that inherits from a parent base and then adds another component:

Configuration of grouped (exteding default)
 {
  "grouped": {
    "extends": [
      {
        "configuration": "default",
        "mode": "merge"
      }
    ],
    "components": [
      {
        "type": "GroupFieldComponent",
        "componentName": "color",
        "fieldName": "color_facet",
        "groupLimit": 3,
        "doc": true,
        "docField": "id",
        "docFilter": true,
        "docFq": "sem_record_type:color",
        "docExclude": false
      }
    ]
  }
}
Predefined parameters

A configuration can contain a list of default parameters. This way these parameters are applied automatically when the search configuration is invoked.

There are three different types of parameters:

  • defaults: Default parameters can be overridden.

  • invariants: Invariants can never be overridden.

  • appends: Appends parameter will always be added.

The code below shows a configuration redwines that sets the default number of rows to 100 (that can be overridden), a filter query for "Red Wines" that will always apply and a fieldlist that can not be changed.

Configuration example using default request params
 {
  "redwines": {
    "requestParams": {
      "defaults": [
        {
          "name": "rows",
          "value": "100"
        },
        {
          "name": "sph.fq",
          "value": "color_facet:\"Red Wine\""
        },
        {
          "name": "sph.fl",
          "value": "sem_title, country"
        }
      ]
    }
  }
}
The parameter prefixes sp. and sph. mean solr parameter and solr parameter hidden and make sure that the parameters are passed through to to solr server. When sph. is used this parameter won’t show up in the action URLs in the response.
Prepared queries

Every configuration can also contain one or more predefined queries that can than be easily envoked from the frontend.

Configuration

A predefined query is defined by a name (id) and a value (all parameters and their values).

The example below adds a prepared query to the default configuration that does puts a filter for old vines (with vintage before 1996) to the query.

Configuration example for preparedQueries
 {
  "preparedQueries": {
    "oldwines": [
      {
        "name": "q",
        "value": "*:*"
      },
      {
        "name": "sph.fq",
        "value": "vintage:[* TO 1995]"
      },
      {
        "name": "rows",
        "value": "10"
      }
    ]
  }
}
Autoactivate

Optionally an autoActivateCondition can be defined as a Spring Expression Language expression; whenever the expression evaluates to true, the predefined query is activated automatically. A typical side-effect of such activation is the use of the parameters defined in the prepared query, even though the query itself is not explicitly activated. 

The expression evaluation environment does not currently contain any context objects, but if there is interest in such a feature (e.g. exposing request parameters or search configuration objects in the evaluation context) please get in touch with us.

The expression below depends on the current date and returns true during summer months. This way e.g. some special boosting depending on seasons can be defined. 
"autoActivateCondition": "(T(org.joda.time.LocalDate).now().monthOfYear().get()+12-4)%12 < 9"
Usage

The predefined query can then be used by adding the parameter q.pre with the name of the prepared query.

The example configuration above thus can be activated here: http://search-features.demos.contexity.ch/sl/search?q.pre=oldwines

Placeholders and functions

Available since version 1.4.

Placeholders and functions can be used in the search configuration that will be resolved for each search request.

Configuration

Placeholders can be simply used in some parts (see below) of of the search configuration by using the notation ${myparam:defaultparamvalue}. The value of placeholder is then looked up in the parameters supplied to the search.

If no parameter with the name is found:

  • The default value is used if one is specified. The definition of a default value is optional, as well as it can be defined empty, e.g. ${myparam:}

  • The part of configuration where the placeholder is used is not applied. That is the case if no default value was supplied, e.g. ${myparam}

Valid configuration places

Placeholders and functions can be used in the requestParams section or in the FacetQueryComponent 

Functions

Functions are methods supplied by Semanteer that can be invoked using the same notation as placeholders ${myFuntion()}

Currently the following function are available:

  • getNowAsOccurence()

  • getDateFromNowAsOccurence(gap)

  • getDateFromTodayAsOccurence(gap)

  • getDateFromThisWeekAsOccurence(gap)

  • getRandomField(seed)

Example configuration and usage

The examples below illustrate how placeholders and function can be used within a Facet Query for faceting documents that are nearby - by using a parameter pt

Code example

 {
  "type": "FacetQueryComponent",
  "componentName": "custom_geoqueries",
  "queries": {
    "nearby": "{!geofilt pt=${pt} sfield=sem_location d=0.2}"
  },
  "excludeOwnFilters": true,
  "excludeAllFilters": false,
  "multiSelect": true,
  "multiSelectMode": "OR",
  "useExclusiveSetAction": true
}

or documents that are open - by using the getNowAsOccurence()

Code example

 {
  "type": "FacetQueryComponent",
  "componentName": "custom_queries",
  "queries": {
    "old_american": "country_facet:'United States of America' AND vintage:[0 TO 1999]",
    "open": "sem_opening_hours:\"Intersects(ENVELOPE(1501010000,${getNowAsOccurence()},2512122359,${getNowAsOccurence()}))\""
  },
  "excludeOwnFilters": true,
  "excludeAllFilters": false,
  "multiSelect": true,
  "multiSelectMode": "OR",
  "addNegations": true,
  "useExclusiveSetAction": true
}
Usage: Supply the parameter pt to activate the nearby facet: http://search-features.demos.contexity.ch/sl/search?q=*&pt=47.161664,4.955434

Features

Semanteer Search Layer offers a huge list of features that can be added to your search. Generally speaking features of the search are activated by adding them to the components elements of a certain configuration in your configuration file. Every component then enhances the search by modifying or adding a section to the response of the search.

Basic structure of a component / feature

The properties of a component are described with a JSON Object, whereas the properties type must be specified to determine the type of the component. The structure of regarding section in search-layer-configuration can be seen in the code example below:

Code example

 {
  "minimal": {
    "components": [
      {
        "type": "FacetFieldComponent",
        "componentName": "color",
        "fieldName": "color_facet",
        "uitype": "FilterListWidget",
        "minCount": 1
      }
    ]
  }
}

Available Features / Components

More about the configuration of components can be found in the section of the corresponding component: 

The advanced features Autofilter and the Geographic Query are more sophisticated components that imply some additional modifications to the original search.

Dependent Components

Components (e.g. Filters) can be configured as dependent on other filters or a parameter value.That means the regarding filter only shows up, when one or more other filter(s) are activated and set to a determined value.

Configuration

Dependencies are added by adding a dependsOnAny element to the component’s configuration that contains a list of component (or parameters) and their values. The dependencies is satisfied and thus the component is activated when one of these dependencies is satisfied.

Dependency on component

A dependency on a component is described by the component name and the value.

Code example

 {
  "type": "FacetFieldComponent",
  "uitype": "FilterListWidget",
  "componentName": "wine_style",
  "minCount": 1,
  "fieldName": "wine_style_facet",
  "dependsOnAny": [
    {
      "component": "color",
      "value": "Red\\ Wine"
    },
    {
      "component": "color",
      "value": "White\\ Wine"
    }
  ]
}

As it can easily be seen, in the above example the filter component would only be available if the (filter) component color has been applied and set to the value "Red Wine" or "White Wine"

Try it out: http://search-features.demos.contexity.ch/sl/search?q=*&f.color=color_facet%3ARed\+Wine
In versions prior to 1.6. defined values may need to be escaped, e.g "value": "Red\\ Wine" instead of  "value": "Red Wine"``
Dependency on a range

Dependencies on Range facets can be described using a custom range. The dependency is satisfied whenever the range covered by the set filter(s) contains the configured range.

Custom range definition for date ranges are available since version 1.6.
Dependency on parameter
Available since version 1.6.

A dependency is described by the parameter name and it’s value.

Wildcards
Available since version 1.4.

In the configuration of a dependency on Hierarchical facets, wildcards can be used to indicate a dependency on a branch

Supported components

Currently these components can be defined as dependent components

Spelling suggestions

Spell-checking is activated by default and does not need any further configuration. Suggestions are added to a section spelling in the response, whenever alternative spelling-corrected terms would let in more results.

Structure

The top-level element suggestions is an array containing the individual suggestions. The fields therein are as follows:

  • suggestion: This is the actual suggested search term(s)

  • hits: The number of documents in which the suggested search term(s) occur

  • sem_action: An action that “switches” the current search term(s) with the suggested one(s), leaving the rest of the search state, as represented in the search URL, intact

Example
The spelling section may look as like this example from http://search-features.demos.contexity.ch/sl/search?q=Savignon

Code example

 {
  "spelling": {
    "suggestions": [
      {
        "suggestion": "sauvignon",
        "hits": "4",
        "sem_action": {
          "sem_url": "?q=sauvignon",
          "sem_url_action": "?q=sauvignon&ot.et=search&ot.est=spell-term-select&ot.es=source_serp&ot.orig=Savignon&ot.sg=sauvignon&ot.sgidx=0&ot.sse=447073bc-76b1-40a8-8603-43e300e71668"
        }
      }
    ]
  }
}
Note that, if there are no spelling suggestions for the current search term(s), the suggestions’ array is empty.

Grouping Results

Grouped search results are activated whenever a GroupFieldComponent is added to the configuration.

Note that grouping can be deactivated on a single request basis by submitting the parameter g.group=false.

Configuration

Thus in any case to be able to use a field for grouping there needs to be a configuration for the regarding field as GroupFieldComponent:

Code example

 {
  "type": "GroupFieldComponent",
  "componentName": "color",
  "fieldName": "color_facet",
  "groupLimit": 3,
  "doc": true,
  "docField": "id",
  "docFilter": true,
  "docFq": "sem_record_type:color",
  "docExclude": false
}
Listing of configuration parameters

Apart from the common parameters used in other components the additional parameters used by GroupFieldComponent are described below:

Parameter Type Description Solr parameter

groupLimit

Integer

Specifies the number of results to return for each group. The default value is 1.

group.limit

doc

Boolean

En-/Disable parent group documents

docField

String

Name of the field that is used to get the parent group documents (if possible an id or another field with unique values)

docFilter

boolean

Exclude parent group documents from other places by adding an additional FQ to the query during prepareQuery

 

docFq

String

Filter query that defines the type of documents that are group parent documents, e.g. sem_record_type:color

docExclude

Boolean

Exclude parent group documents from other places by removing them during processResponse.
Adding and showing a parent document for each group

When doc is set to true, Semanteer tries to fetch a parent document for each result group.

  1. Therefore an additional query is sent to Solr that tries to get the corresponding parent document (that is the document with the group’s value in the field that is specified in docField)

  2. The parent group documents attached to the groups can be excluded from being listed on other places. Therefore two parameters are use:

    1. docExclude excludes the document from the list 

    2. docFilter thus applies a filter query already to the first query (for the grouped results) that filters out all documents with certain criteria (as specified with docFilter)

docExclude may not be work fully as expected because when SolrDocuments are removed from groups the resultlist is not refilled up to the expected group.limit and the correct value of matches and numFound can not be calculated either.
Additional (Solr) Parameters

Parameters that are passed through from the request to the Solr query:

  • group.offset: Specifies an initial offset for the document list of each group.

  • group.sort: Specifies how Solr sorts documents within each group. The default behavior if group.sort is not specified is to use the same effective value as the sort parameter.

Refer also to the Solr Documentation: Result Grouping

Response

As mentioned in the section The search response the response structure of a grouped search differs from an ungrouped search, whereas a section grouped is shown instead of the section response.

Elements of the response

The elements of the grouped response are:

  • fieldname: In the example above color_facet, this is the name of the field that is used for grouping

    • matches: The total number of matches (results)

    • nGroups: The total number of groups (for that we have results)

    • groups: The container that actually contains the groups and result documents

Every group in groups then contains the following elements:

  • groupvalue: The value of the field that is used for grouping

  • doclist: This is the list of results in the present group.

    • numFound: The total number of results for the group

    • start: The index of the first document in the list

    • docs: The container that actually contains the documents

  • paging: Action URLs for paging

  • sem_actions: Additional Action URLs

    • filterby: Action URLs to activate a filter for this group

  • groupDoc: The parent document of the group (if doc and docField have been configured - see above)

Example

The code example shows an response from http://search-features.demos.contexity.ch/sl/search?q=200?&_semc=grouped&sp.fl=sem_title (we use sp.fl=sem_title here to make the example shorter and thus more readable)

Code example

 {
  "matches": 93,
  "ngroups": 2,
  "groups": [
    {
      "groupValue": "White Wine",
      "doclist": {
        "numFound": 2,
        "start": 0,
        "docs": [
          {
            "sem_title": "2003 Cloudy Bay Sauvignon Blanc",
            "sem_actions": {
              "default": "goto",
              "goto": {
                "sem_url": "/record?id",
                "sem_url_action": "/record?id&ot.et=result&ot.est=serp-result-click&ot.q=200%3F&ot.es=source_serp&ot.nf=93&ot.s=0&ot.r=20&ot.rank=0&ot.sse=3e3f9b4b-fb9b-4e3e-9e10-98909bbdd025",
                "sem_url_logging": "http://search-features.demos.contexity.ch/sl/log?id&ot.et=result&ot.est=serp-result-click&ot.q=200%3F&ot.es=source_serp&ot.nf=93&ot.s=0&ot.r=20&ot.rank=0&ot.sse=3e3f9b4b-fb9b-4e3e-9e10-98909bbdd025",
                "sem_is_external": false
              },
              "mlt": {
                "sem_url": "http://search-features.demos.contexity.ch/sl/morelikethis?id",
                "sem_url_action": "http://search-features.demos.contexity.ch/sl/morelikethis?id&ot.et=mlt&ot.est=mlt&ot.es=source_serp&ot.nf=93&ot.s=0&ot.r=20&ot.rank=0&ot.q=200%3F&ot.sse=3e3f9b4b-fb9b-4e3e-9e10-98909bbdd025"
              }
            },
            "sem_encoded_id": null
          },
          {
            "sem_title": "2004 Domaine Prieur Brunet Jehanne de Chantal",
            "sem_actions": {
              "default": "goto",
              "goto": {
                "sem_url": "/record?id",
                "sem_url_action": "/record?id&ot.et=result&ot.est=serp-result-click&ot.q=200%3F&ot.es=source_serp&ot.nf=93&ot.s=0&ot.r=20&ot.rank=1&ot.sse=3e3f9b4b-fb9b-4e3e-9e10-98909bbdd025",
                "sem_url_logging": "http://search-features.demos.contexity.ch/sl/log?id&ot.et=result&ot.est=serp-result-click&ot.q=200%3F&ot.es=source_serp&ot.nf=93&ot.s=0&ot.r=20&ot.rank=1&ot.sse=3e3f9b4b-fb9b-4e3e-9e10-98909bbdd025",
                "sem_is_external": false
              },
              "mlt": {
                "sem_url": "http://search-features.demos.contexity.ch/sl/morelikethis?id",
                "sem_url_action": "http://search-features.demos.contexity.ch/sl/morelikethis?id&ot.et=mlt&ot.est=mlt&ot.es=source_serp&ot.nf=93&ot.s=0&ot.r=20&ot.rank=1&ot.q=200%3F&ot.sse=3e3f9b4b-fb9b-4e3e-9e10-98909bbdd025"
              }
            },
            "sem_encoded_id": null
          }
        ]
      },
      "paging": {
        "sem_generic_urls": {
          "sem_url": "?q=200%3F&_semc=grouped&sp.fl=sem_title",
          "sem_url_action": "?q=200%3F&_semc=grouped&sp.fl=sem_title&ot.et=search&ot.est=paging&ot.es=source_serp&ot.sse=3e3f9b4b-fb9b-4e3e-9e10-98909bbdd025"
        }
      },
      "sem_actions": {
        "default": "filterby",
        "filterby": {
          "sem_url": "?_semc=grouped&sp.fl=sem_title&f.color=color_facet%3AWhite%5C+Wine&q=200%3F",
          "sem_url_action": "?_semc=grouped&sp.fl=sem_title&f.color=color_facet%3AWhite%5C+Wine&q=200%3F&ot.et=search&ot.est=suggest-category-select&ot.es=source_autocomplete&ot.orig=200%3F&ot.sse=3e3f9b4b-fb9b-4e3e-9e10-98909bbdd025"
        }
      },
      "groupdoc": {
        "id": "White Wine",
        "sem_title": "White Wine",
        "sem_suggest_prefix": [
          "White Wine"
        ],
        "sem_suggest_phrase": [
          "White Wine"
        ],
        "sem_text_search": [
          "White Wine"
        ],
        "sem_text_suggest": [
          "White Wine"
        ],
        "sem_title_alpha": "White Wine",
        "sem_record_type": "color",
        "sem_url": "http://www.freebase.com/m/02wsb25",
        "sem_lang": "de",
        "_version_": 1617037278531551200,
        "sem_timestamp": "2018-11-13T16:35:21.163Z",
        "sem_actions": {
          "default": "goto",
          "goto": {
            "sem_url": "http://search-features.demos.contexity.ch/sl/redirect?url=http%3A%2F%2Fwww.freebase.com%2Fm%2F02wsb25&id=SEMV2hpdGUgV2luZQ",
            "sem_url_action": "http://search-features.demos.contexity.ch/sl/redirect?url=http%3A%2F%2Fwww.freebase.com%2Fm%2F02wsb25&id=SEMV2hpdGUgV2luZQ&ot.et=result&ot.est=serp-result-click&ot.q=200%3F&ot.es=source_serp&ot.nf=2&ot.s=0&ot.r=20&ot.rank=0&ot.sse=3e3f9b4b-fb9b-4e3e-9e10-98909bbdd025",
            "sem_url_logging": "http://search-features.demos.contexity.ch/sl/log?url=http%3A%2F%2Fwww.freebase.com%2Fm%2F02wsb25&id=SEMV2hpdGUgV2luZQ&ot.et=result&ot.est=serp-result-click&ot.q=200%3F&ot.es=source_serp&ot.nf=2&ot.s=0&ot.r=20&ot.rank=0&ot.sse=3e3f9b4b-fb9b-4e3e-9e10-98909bbdd025",
            "sem_is_external": true
          },
          "mlt": {
            "sem_url": "http://search-features.demos.contexity.ch/sl/morelikethis?id=SEMV2hpdGUgV2luZQ",
            "sem_url_action": "http://search-features.demos.contexity.ch/sl/morelikethis?id=SEMV2hpdGUgV2luZQ&ot.et=mlt&ot.est=mlt&ot.es=source_serp&ot.nf=2&ot.s=0&ot.r=20&ot.rank=0&ot.q=200%3F&ot.sse=3e3f9b4b-fb9b-4e3e-9e10-98909bbdd025"
          }
        },
        "sem_encoded_id": "SEMV2hpdGUgV2luZQ"
      }
    },
    {
      "groupValue": "Red Wine",
      "doclist": {
        "numFound": 91,
        "start": 0,
        "docs": [
          {
            "sem_title": "2005 Ridge \"Pagani Ranch\" Sonoma Valley Zinfandel",
            "sem_actions": {
              "default": "goto",
              "goto": {
                "sem_url": "/record?id",
                "sem_url_action": "/record?id&ot.et=result&ot.est=serp-result-click&ot.q=200%3F&ot.es=source_serp&ot.nf=93&ot.s=0&ot.r=20&ot.rank=2&ot.sse=3e3f9b4b-fb9b-4e3e-9e10-98909bbdd025",
                "sem_url_logging": "http://search-features.demos.contexity.ch/sl/log?id&ot.et=result&ot.est=serp-result-click&ot.q=200%3F&ot.es=source_serp&ot.nf=93&ot.s=0&ot.r=20&ot.rank=2&ot.sse=3e3f9b4b-fb9b-4e3e-9e10-98909bbdd025",
                "sem_is_external": false
              },
              "mlt": {
                "sem_url": "http://search-features.demos.contexity.ch/sl/morelikethis?id",
                "sem_url_action": "http://search-features.demos.contexity.ch/sl/morelikethis?id&ot.et=mlt&ot.est=mlt&ot.es=source_serp&ot.nf=93&ot.s=0&ot.r=20&ot.rank=2&ot.q=200%3F&ot.sse=3e3f9b4b-fb9b-4e3e-9e10-98909bbdd025"
              }
            },
            "sem_encoded_id": null
          },
          {
            "sem_title": "2004 Passalacqua \"TR Passalacqua Vineyard\" Dry Creek Valley Cabernet Sauvignon",
            "sem_actions": {
              "default": "goto",
              "goto": {
                "sem_url": "/record?id",
                "sem_url_action": "/record?id&ot.et=result&ot.est=serp-result-click&ot.q=200%3F&ot.es=source_serp&ot.nf=93&ot.s=0&ot.r=20&ot.rank=3&ot.sse=3e3f9b4b-fb9b-4e3e-9e10-98909bbdd025",
                "sem_url_logging": "http://search-features.demos.contexity.ch/sl/log?id&ot.et=result&ot.est=serp-result-click&ot.q=200%3F&ot.es=source_serp&ot.nf=93&ot.s=0&ot.r=20&ot.rank=3&ot.sse=3e3f9b4b-fb9b-4e3e-9e10-98909bbdd025",
                "sem_is_external": false
              },
              "mlt": {
                "sem_url": "http://search-features.demos.contexity.ch/sl/morelikethis?id",
                "sem_url_action": "http://search-features.demos.contexity.ch/sl/morelikethis?id&ot.et=mlt&ot.est=mlt&ot.es=source_serp&ot.nf=93&ot.s=0&ot.r=20&ot.rank=3&ot.q=200%3F&ot.sse=3e3f9b4b-fb9b-4e3e-9e10-98909bbdd025"
              }
            },
            "sem_encoded_id": null
          },
          {
            "sem_title": "2003 Beaulieu \"Georges de Latour Private Reserve\" Napa Valley Cabernet Sauvignon",
            "sem_actions": {
              "default": "goto",
              "goto": {
                "sem_url": "/record?id",
                "sem_url_action": "/record?id&ot.et=result&ot.est=serp-result-click&ot.q=200%3F&ot.es=source_serp&ot.nf=93&ot.s=0&ot.r=20&ot.rank=4&ot.sse=3e3f9b4b-fb9b-4e3e-9e10-98909bbdd025",
                "sem_url_logging": "http://search-features.demos.contexity.ch/sl/log?id&ot.et=result&ot.est=serp-result-click&ot.q=200%3F&ot.es=source_serp&ot.nf=93&ot.s=0&ot.r=20&ot.rank=4&ot.sse=3e3f9b4b-fb9b-4e3e-9e10-98909bbdd025",
                "sem_is_external": false
              },
              "mlt": {
                "sem_url": "http://search-features.demos.contexity.ch/sl/morelikethis?id",
                "sem_url_action": "http://search-features.demos.contexity.ch/sl/morelikethis?id&ot.et=mlt&ot.est=mlt&ot.es=source_serp&ot.nf=93&ot.s=0&ot.r=20&ot.rank=4&ot.q=200%3F&ot.sse=3e3f9b4b-fb9b-4e3e-9e10-98909bbdd025"
              }
            },
            "sem_encoded_id": null
          }
        ]
      },
      "paging": {
        "sem_generic_urls": {
          "sem_url": "?q=200%3F&_semc=grouped&sp.fl=sem_title",
          "sem_url_action": "?q=200%3F&_semc=grouped&sp.fl=sem_title&ot.et=search&ot.est=paging&ot.es=source_serp&ot.sse=3e3f9b4b-fb9b-4e3e-9e10-98909bbdd025"
        },
        "sem_next_page_urls": {
          "sem_url": "?q=200%3F&_semc=grouped&sp.fl=sem_title&group.offset=3",
          "sem_url_action": "?q=200%3F&_semc=grouped&sp.fl=sem_title&ot.et=search&ot.est=paging&ot.es=source_serp&ot.sse=3e3f9b4b-fb9b-4e3e-9e10-98909bbdd025&group.offset=3"
        }
      },
      "sem_actions": {
        "default": "filterby",
        "filterby": {
          "sem_url": "?_semc=grouped&sp.fl=sem_title&f.color=color_facet%3ARed%5C+Wine&q=200%3F",
          "sem_url_action": "?_semc=grouped&sp.fl=sem_title&f.color=color_facet%3ARed%5C+Wine&q=200%3F&ot.et=search&ot.est=suggest-category-select&ot.es=source_autocomplete&ot.orig=200%3F&ot.sse=3e3f9b4b-fb9b-4e3e-9e10-98909bbdd025"
        }
      },
      "groupdoc": {
        "id": "Red Wine",
        "sem_title": "Red Wine",
        "sem_suggest_prefix": [
          "Red Wine"
        ],
        "sem_suggest_phrase": [
          "Red Wine"
        ],
        "sem_text_search": [
          "Red Wine"
        ],
        "sem_text_suggest": [
          "Red Wine"
        ],
        "sem_title_alpha": "Red Wine",
        "sem_record_type": "color",
        "sem_url": "http://www.freebase.com/m/02wsb20",
        "sem_lang": "de",
        "_version_": 1617037278527357000,
        "sem_timestamp": "2018-11-13T16:35:21.163Z",
        "sem_actions": {
          "default": "goto",
          "goto": {
            "sem_url": "http://search-features.demos.contexity.ch/sl/redirect?url=http%3A%2F%2Fwww.freebase.com%2Fm%2F02wsb20&id=SEMUmVkIFdpbmU",
            "sem_url_action": "http://search-features.demos.contexity.ch/sl/redirect?url=http%3A%2F%2Fwww.freebase.com%2Fm%2F02wsb20&id=SEMUmVkIFdpbmU&ot.et=result&ot.est=serp-result-click&ot.q=200%3F&ot.es=source_serp&ot.nf=2&ot.s=0&ot.r=20&ot.rank=1&ot.sse=3e3f9b4b-fb9b-4e3e-9e10-98909bbdd025",
            "sem_url_logging": "http://search-features.demos.contexity.ch/sl/log?url=http%3A%2F%2Fwww.freebase.com%2Fm%2F02wsb20&id=SEMUmVkIFdpbmU&ot.et=result&ot.est=serp-result-click&ot.q=200%3F&ot.es=source_serp&ot.nf=2&ot.s=0&ot.r=20&ot.rank=1&ot.sse=3e3f9b4b-fb9b-4e3e-9e10-98909bbdd025",
            "sem_is_external": true
          },
          "mlt": {
            "sem_url": "http://search-features.demos.contexity.ch/sl/morelikethis?id=SEMUmVkIFdpbmU",
            "sem_url_action": "http://search-features.demos.contexity.ch/sl/morelikethis?id=SEMUmVkIFdpbmU&ot.et=mlt&ot.est=mlt&ot.es=source_serp&ot.nf=2&ot.s=0&ot.r=20&ot.rank=1&ot.q=200%3F&ot.sse=3e3f9b4b-fb9b-4e3e-9e10-98909bbdd025"
          }
        },
        "sem_encoded_id": "SEMUmVkIFdpbmU"
      }
    }
  ],
  "start": 0
}

Sorting Results

By default results are sorted by their score. Custom sorting of results is activated whenever a SortingComponent is added to the configuration.

Configuration

In the SortingComponent all fields that can be used for sorting are added to sortableFields array, e.g.

Code example

 {
  "type": "SortingComponent",
  "uitype": "SortingModule",
  "sortableFields": [
    "sem_title_alpha",
    "appellation",
    "vineyard"
  ]
}

Response

The SortingComponent adds a top-level section sorting to the response that contains the actions for activating the defined sorting. See the code example from taken from http://search-features.demos.contexity.ch/sl/search?q=*

Elements of the response

As can be seen in the example below, the top-level elements of the section are:

  • sem_action: An action that provides URLs for removing all sorting parameters from the current URL, leaving all other parameters (e.g. filters) intact

  • fields: A list of the fields configured as sortable fields. For each sortable field, actions are provided for sorting the field in ascending (asc) and descending (desc) order (the semantics of that order and their implementation depend on the respective field’s type).

    • asc: Action URL to sort ascending

    • desc: Action URL to sort ascending

Example

The sorting section may look like the following example:

Code example

 {
  "sorting": {
    "sem_action": {
      "sem_url": "?q=*",
      "sem_url_action": "?q=*&ot.et=search&ot.est=sort-modified&ot.es=source_serp&ot.sse=f38b4cdb-b83d-4197-b904-847629218237"
    },
    "fields": {
      "sem_title_alpha": {
        "asc": {
          "sem_url": "?q=*&sort=sem_title_alpha+asc",
          "sem_url_action": "?q=*&sort=sem_title_alpha+asc&ot.et=search&ot.est=sort-modified&ot.es=source_serp&ot.sse=f38b4cdb-b83d-4197-b904-847629218237",
          "active": true
        },
        "desc": {
          "sem_url": "?q=*&sort=sem_title_alpha+desc",
          "sem_url_action": "?q=*&sort=sem_title_alpha+desc&ot.et=search&ot.est=sort-modified&ot.es=source_serp&ot.sse=f38b4cdb-b83d-4197-b904-847629218237"
        }
      },
      "appellation": {
        "asc": {
          "sem_url": "?q=*&sort=appellation+asc",
          "sem_url_action": "?q=*&sort=appellation+asc&ot.et=search&ot.est=sort-modified&ot.es=source_serp&ot.sse=f38b4cdb-b83d-4197-b904-847629218237"
        },
        "desc": {
          "sem_url": "?q=*&sort=appellation+desc",
          "sem_url_action": "?q=*&sort=appellation+desc&ot.et=search&ot.est=sort-modified&ot.es=source_serp&ot.sse=f38b4cdb-b83d-4197-b904-847629218237"
        }
      },
      "vineyard": {
        "asc": {
          "sem_url": "?q=*&sort=vineyard+asc",
          "sem_url_action": "?q=*&sort=vineyard+asc&ot.et=search&ot.est=sort-modified&ot.es=source_serp&ot.sse=f38b4cdb-b83d-4197-b904-847629218237"
        },
        "desc": {
          "sem_url": "?q=*&sort=vineyard+desc",
          "sem_url_action": "?q=*&sort=vineyard+desc&ot.et=search&ot.est=sort-modified&ot.es=source_serp&ot.sse=f38b4cdb-b83d-4197-b904-847629218237"
        }
      }
    }
  }
}
Unlike actions for filters, sorting actions are not toggles but rather assertive applications of sorting fields and orders; to determine whether a particular sorting configuration is currently applied, the server identifies them with the additional field active, which is then set to true (see the part of the response for sem_title_alpha, ascending, above).

 

Facets and Filters

Facet components are showing faceting information (values and counts) and are used to trigger filters.
Facet components can be configured in your configuration file the same way as every other component

Currently the following different kinds of facet components are available in Semanteer:

Common configuration options

Regardless of the concrete type of facet component there are a couple ofcommon configuration options available.

R = required

Parameter Type Default Description R Version support

type

text

-

the component type

✓

 

componentName

text

-

name that appears in the response

✓

 

fieldName

text

-

name of the field in the index

✓

 

minCount

integer

-

min. number of documents in the index

 

multiSelect

boolean

false

more than one option is selectable at once

 

multiSelectMode

AND, OR

OR

determine multiselected filter should be combined with AND or OR

 

uitype

text

-

Type that is used in the UI (e.g. FilterListWidget)

 

excludeOwnFilters

boolean

false

exclude the filter applied by this component when calculating facet counts

 

excludeAllFilters

boolean

false

exclude all filter applied by this and other component when calculating facet counts

 

useExclusiveSetAction

boolean

false

other filters are removed when this filter is set (requires multiSelect to be false)

 

addMissingFilters

boolean

true

 

 

excludeTags

text array

-

List of tags from other filters that should be excluded when faceting

1.6.1+

localParams

text

-

localParams to be added when the filter is applied

1.6.1+

dependsOnAny

list of components

-

see Dependent Components

 
Dependent Filters

Filters can be configured to depend on other filters. See Dependent Components

Facet Search Response

For configured facets, the search response contains facet values in the appropriate sections, in accordance to the total set of search results.

The following is an example of a response using Text based facets configured for the field color, from http://search-features.demos.contexity.ch/sl/search?q=*&_semc=minimal

Code example

 {
  "facet_counts": {
    "facet_queries": {},
    "facet_fields": {
      "color": {
        "Red Wine": {
          "value": "Red Wine",
          "count": 97,
          "sem_action": {
            "sem_url": "?q=*&_semc=minimal&f.color=color_facet%3ARed%5C+Wine",
            "sem_url_action": "?q=*&_semc=minimal&f.color=color_facet%3ARed%5C+Wine&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=c37f159c-7e47-4790-8456-4cba119c0b8e"
          }
        },
        "White Wine": {
          "value": "White Wine",
          "count": 2,
          "sem_action": {
            "sem_url": "?q=*&_semc=minimal&f.color=color_facet%3AWhite%5C+Wine",
            "sem_url_action": "?q=*&_semc=minimal&f.color=color_facet%3AWhite%5C+Wine&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=c37f159c-7e47-4790-8456-4cba119c0b8e"
          }
        },
        "Rosé": {
          "value": "Rosé",
          "count": 1,
          "sem_action": {
            "sem_url": "?q=*&_semc=minimal&f.color=color_facet%3ARos%C3%A9",
            "sem_url_action": "?q=*&_semc=minimal&f.color=color_facet%3ARos%C3%A9&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=c37f159c-7e47-4790-8456-4cba119c0b8e"
          }
        }
      }
    },
    "facet_ranges": {},
    "facet_intervals": {},
    "facet_heatmaps": {}
  }
}

As can be seen in the preceding example, the structure sets out with “named” structured objects, with each “name” corresponding to a field, range, interval, etc. In the example, we only see a top-level objects color within facet_fields that correspond to the configured facet field.

Inside these top-level objects, are the individual values for that facet. In the example, these include Red Wine and White Wine for the field color.

Depending on the type of the facet component it shows up in the corresponding section, as facet_fields, facet_ranges, facet_hierarchies, and so on. Have a look at the facet_counts section of full response - e.g. here: http://search-features.demos.contexity.ch/sl/search?q=* - and refer to the description of the regarding component for more details.

For each value, the following fields are returned:

  • value: This is the value’s “name” and is repeated as a field for convenience.

  • count: This is the number of occurrences of the value amongst the search results.

  • sem_action: This is a “toggle” action for enabling or disabling the use of the present facet value as a filter. Please refer to the sections Action URLs and Monitoring URLs for details on action fields.

Action URLs in the facet values toggle only the corresponding value on or off as a filter; the rest of the active filters are not affected (in other words, Semanteer generates the action URLs so that the rest of the filtering is maintained as is).
Facet values as filters

When a facet value is activated as a filter (through the corresponding Action URL), then the facet value’s structure in the response is modified as follows, the example is taken from http://search-features.demos.contexity.ch/sl/search?q=*&_semc=minimal&f.color=color_facet%3ARed%5C+Wine

Code example

 {
  "facet_counts": {
    "facet_queries": {},
    "facet_fields": {
      "color": {
        "Red Wine": {
          "value": "Red Wine",
          "count": 97,
          "active": true,
          "sem_action": {
            "sem_url": "?q=*&_semc=minimal",
            "sem_url_action": "?q=*&_semc=minimal&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=5cda47e4-d167-4daf-be8b-c81ebbcba28c"
          }
        }
      }
    },
    "facet_ranges": {},
    "facet_intervals": {},
    "facet_heatmaps": {}
  }
}

In the above snippet note that:

  • The facet value’s structure has been extended to include the field active, which has the value true.

  • The actions associated with the facet value have been modified to toggle the filter off if invoked (in this example, the filter for field color is assumed to be the only one that is active)

Active Facet based Filters

Active filters are also represented in the corresponding section of the response. See Active and suggested Filters.

Text based facets

Facets and filters based on text fields can be added by using a FacetFieldComponent.

Configuration

In order to use text based facets a FacetFieldComponent has to be defined in the configuration.

While most of the elements of the configuration as shown above have already been described in the section Common configuration options, some additional configuration options are available for FacetFieldComponent:

R = required

Name Type Default Description R

sortBy

COUNT, INDEX, DEFAULT

COUNT

Determines the sorting that is applied to the facet.

The example code below defines a FacetFieldComponent for the field wine_type:

Code example

 {
  "type": "FacetFieldComponent",
  "componentName": "wine_type",
  "fieldName": "wine_type_facet",
  "uitype": "FilterListWidget",
  "minCount": 1,
  "sortBy": "index",
  "multiSelect": false,
  "excludeOwnFilters": true,
  "excludeAllFilters": false,
  "useExclusiveSetAction": true
}

Response

The response of the FacetFieldComponent shows up under the the section facet_fields section of facet_count. An example of a response based on the FacetFieldComponent wine_type configured above may look like this:

Code example

 {
  "wine_type": {
    "Dessert wine": {
      "value": "Dessert wine",
      "count": 1,
      "sem_action": {
        "sem_url": "?q=*&f.wine_type=wine_type_facet%3ADessert%5C+wine",
        "sem_url_action": "?q=*&f.wine_type=wine_type_facet%3ADessert%5C+wine&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=984a17c4-cb47-4c3e-b2e2-9d6c54519fe3"
      }
    },
    "Fortified wine": {
      "value": "Fortified wine",
      "count": 1,
      "sem_action": {
        "sem_url": "?q=*&f.wine_type=wine_type_facet%3AFortified%5C+wine",
        "sem_url_action": "?q=*&f.wine_type=wine_type_facet%3AFortified%5C+wine&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=984a17c4-cb47-4c3e-b2e2-9d6c54519fe3"
      }
    },
    "Sparkling wine": {
      "value": "Sparkling wine",
      "count": 1,
      "sem_action": {
        "sem_url": "?q=*&f.wine_type=wine_type_facet%3ASparkling%5C+wine",
        "sem_url_action": "?q=*&f.wine_type=wine_type_facet%3ASparkling%5C+wine&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=984a17c4-cb47-4c3e-b2e2-9d6c54519fe3"
      }
    },
    "Still Wine": {
      "value": "Still Wine",
      "count": 97,
      "sem_action": {
        "sem_url": "?q=*&f.wine_type=wine_type_facet%3AStill%5C+Wine",
        "sem_url_action": "?q=*&f.wine_type=wine_type_facet%3AStill%5C+Wine&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=984a17c4-cb47-4c3e-b2e2-9d6c54519fe3"
      }
    }
  }
}

Child based facet

v1.7+

Child based facets are a special special implentation of Text based facets that can be applied to a field of child documents.
The facet count indicates the number of parent documents that have one or more childs with the given value and the applied filter weould show these parent documents.

Configuration

In order to use text based facets a FacetChildDocFieldComponent has to be defined in the configuration.

While most of the elements of the configuration as shown above have already been described in the section Common configuration options, some additional configuration options are available for FacetChildDocFieldComponent:

R = required

Name Type Default Description R

sortBy

COUNT, INDEX, DEFAULT

COUNT

Determines the sorting that is applied to the facet.

parentDocs

String

-

A filter query that defines all parent documents (e.g. sem_record_type:wine)

✓

The example code below defines a FacetChildDocFieldComponent for the field rating used by child documents:

Configuration example of a child based facet
{
  "type": "FacetChildDocFieldComponent",
  "componentName": "review_rating",
  "fieldName": "rating",
  "parentDocs": "sem_record_type:wine",
  "excludeOwnFilters": true,
  "minCount": 1
}

Response

The response of the FacetChildDocFieldComponent looks like tha response of a regular FacetFieldComponent. It shows up under the the section facet_fields section of facet_count. An example of a response based on the FacetChildDocFieldComponent review_rating configured above may look like this:

Example response of a child based facet
{
  "facet_fields": {
    "review_rating": {
      "Good": {
        "value": "Good",
        "count": 1,
        "sem_action": {
          "sem_url": "?q=*&f.review_rating=rating%3AGood",
          "sem_url_action": "?q=*&f.review_rating=rating%3AGood&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=62af546c-2a40-443d-85c6-92a7b15c42d8"
        }
      }
    }
  }
}

Additional information on child documents

Indexing child documents

Information about indexing child documents can be found in the Solr Documentation - Nested Child Documents

Returning child documents

The ChildDocTransformerFactory is needed to retrieve child documents. See Solr Documentation - ChildDocTransformerFactory

In our demo installation parameter fl=*,[child parentFilter=sem_record_type:wine childFilter=sem_record_type:review limit=100] would include the child documents in the result.

Range facets

Range facets can be configured for integer, decimal or date fields. A range facet will group values into defined ranges.

Configuration

Depending on the type of the underlying field range facets are defined as

  • IntegerRangeComponent

  • FloatRangeComponent

  • DateRangeComponent

From the configuration elements described in the section Common configuration options Range facets support minCount, multiSelect, and minCount (DateRangeComponent only).

These elements are not supported: addMissingFilters, excludeAllFilters, excludeOwnFilters, maxOptions, multiselectMode, sortBy and useExclusiveSetAction.

In addition Range facets configuration introduces the following elements:

R = required

Name Type Default Description R

start

int/float/date

- (lowest value found)

Value to start with

end

int/float/date

- (highest value found)

Value to end with

gap

 

-

Gap used

✓

prune

boolean

false

Remove empty facet options on lower and higher end

slider

boolean

 

Used by the UI

In order to use range facet based on an integer field an IntegerRangeComponent has to be defined in the configuration. The example code below defines a IntegerRangeComponent for the field vintage:

Code example

 {
  "type": "IntegerRangeComponent",
  "componentName": "vintage",
  "fieldName": "vintage",
  "uitype": "RangeModule",
  "slider": true,
  "start": 1970,
  "end": 2010,
  "gap": 5,
  "prune": true,
  "multiSelect": true,
  "multiSelectMode": "OR",
  "excludeOwnFilters": true
}

Response

The response any RangeComponent shows up under the facet_ranges section of facet_counts. An example of a response based on the IntegerRangeComponent vintage configured above may look like this:

Code example

 {
  "facet_ranges": {
    "vintage": {
      "counts": {
        "* TO 1990": {
          "value": "* TO 1990",
          "count": 1,
          "sem_action": {
            "sem_url": "?q=*&f.vintage=vintage%3A%5B*+TO+1990%7D",
            "sem_url_action": "?q=*&f.vintage=vintage%3A%5B*+TO+1990%7D&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=984a17c4-cb47-4c3e-b2e2-9d6c54519fe3"
          }
        },
        "1990": {
          "value": "1990",
          "count": 1,
          "sem_action": {
            "sem_url": "?q=*&f.vintage=vintage%3A%5B1990+TO+1995%7D",
            "sem_url_action": "?q=*&f.vintage=vintage%3A%5B1990+TO+1995%7D&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=984a17c4-cb47-4c3e-b2e2-9d6c54519fe3"
          }
        },
        "1995": {
          "value": "1995",
          "count": 4,
          "sem_action": {
            "sem_url": "?q=*&f.vintage=vintage%3A%5B1995+TO+2000%7D",
            "sem_url_action": "?q=*&f.vintage=vintage%3A%5B1995+TO+2000%7D&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=984a17c4-cb47-4c3e-b2e2-9d6c54519fe3"
          }
        },
        "2000": {
          "value": "2000",
          "count": 41,
          "sem_action": {
            "sem_url": "?q=*&f.vintage=vintage%3A%5B2000+TO+2005%7D",
            "sem_url_action": "?q=*&f.vintage=vintage%3A%5B2000+TO+2005%7D&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=984a17c4-cb47-4c3e-b2e2-9d6c54519fe3"
          }
        },
        "2005": {
          "value": "2005",
          "count": 52,
          "sem_action": {
            "sem_url": "?q=*&f.vintage=vintage%3A%5B2005+TO+2010%5D",
            "sem_url_action": "?q=*&f.vintage=vintage%3A%5B2005+TO+2010%5D&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=984a17c4-cb47-4c3e-b2e2-9d6c54519fe3"
          }
        }
      },
      "groupedCounts": {
        "* TO 1990": {
          "value": "* TO 1990",
          "count": 1,
          "sem_action": {
            "sem_url": "?q=*&f.vintage=vintage%3A%5B*+TO+1990%7D",
            "sem_url_action": "?q=*&f.vintage=vintage%3A%5B*+TO+1990%7D&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=984a17c4-cb47-4c3e-b2e2-9d6c54519fe3"
          }
        },
        "1990 TO 1995": {
          "value": "1990 TO 1995",
          "count": 1,
          "sem_action": {
            "sem_url": "?q=*&f.vintage=vintage%3A%5B1990+TO+1995%7D",
            "sem_url_action": "?q=*&f.vintage=vintage%3A%5B1990+TO+1995%7D&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=984a17c4-cb47-4c3e-b2e2-9d6c54519fe3"
          }
        },
        "1995 TO 2000": {
          "value": "1995 TO 2000",
          "count": 4,
          "sem_action": {
            "sem_url": "?q=*&f.vintage=vintage%3A%5B1995+TO+2000%7D",
            "sem_url_action": "?q=*&f.vintage=vintage%3A%5B1995+TO+2000%7D&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=984a17c4-cb47-4c3e-b2e2-9d6c54519fe3"
          }
        },
        "2000 TO 2005": {
          "value": "2000 TO 2005",
          "count": 41,
          "sem_action": {
            "sem_url": "?q=*&f.vintage=vintage%3A%5B2000+TO+2005%7D",
            "sem_url_action": "?q=*&f.vintage=vintage%3A%5B2000+TO+2005%7D&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=984a17c4-cb47-4c3e-b2e2-9d6c54519fe3"
          }
        },
        "2005 TO 2010": {
          "value": "2005 TO 2010",
          "count": 52,
          "sem_action": {
            "sem_url": "?q=*&f.vintage=vintage%3A%5B2005+TO+2010%5D",
            "sem_url_action": "?q=*&f.vintage=vintage%3A%5B2005+TO+2010%5D&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=984a17c4-cb47-4c3e-b2e2-9d6c54519fe3"
          }
        }
      },
      "gap": 5,
      "start": 1990,
      "adjustedStart": 1990,
      "end": 2010,
      "adjustedEnd": 2010,
      "after": 0,
      "before": 1,
      "between": 98,
      "sem_action": {
        "sem_url": "?q=*&f.vintage=vintage%3A%5B%24%7Bstart%7D+TO+%24%7Bend%7D%5D",
        "sem_url_action": "?q=*&f.vintage=vintage%3A%5B%24%7Bstart%7D+TO+%24%7Bend%7D%5D&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=984a17c4-cb47-4c3e-b2e2-9d6c54519fe3"
      }
    }
  }
}

Hierarchical facets

Semanteer supports facet hierarchies of two types, “Pivot” style hierarchies and Single-field hierarchies 

  • “Pivot” style hierarchies: formed by the hierarchical combination of two or more fields (e.g., combining country and region into a hierarchy)

  • Single-field hierarchies: for fields that already contain hierarchical information

“Pivot” style hierarchies

Configuration

In order to use pivot faceting a FacetPivotComponent has to be defined in the configuration.

From the configuration elements described in the section Common configuration options FacetPivotComponent supports minCount, multiSelect, multiSelectMode, and sortBy. (These elements are not supported: addMissingFilters, excludeAllFilters, excludeOwnFilters, maxOptions, useExclusiveSetAction`)

In addition FacetPivotComponent introduces the following elements:

R = required

Name Type Default Description R

fieldNames

string array

-

Names of the fields that contain the hierarchical information (Replaces fieldName)

✓

filterFirstLevel

boolean

true

en-/disables filter actions be available at the top level

The code below defines a pivot facet component region composed by the fields country_facet, region_facet, wine_sub_region_facet

Code example

 {
  "type": "FacetPivotComponent",
  "componentName": "region",
  "fieldNames": [
    "country_facet",
    "region_facet",
    "wine_sub_region_facet"
  ],
  "uitype": "FilterHierarchyModule",
  "minCount": 1,
  "multiSelect": false,
  "sortBy": "index",
  "useExclusiveSetAction": true,
  "excludeOwnFilters": true
}
Response

The response of the FacetPivotComponent shows up under the the section facet_hierarchies section of facet_counts. An example of a response based on the FacetPivotComponent place configured above may look like in the the response returned from http://search-features.demos.contexity.ch/sl/search?q=*&f.region=country_facet:France

Code example

 {
  "facet_hierarchies": {
    "region": {
      "Australia": {
        "value": "Australia",
        "count": 1,
        "sem_action": {
          "sem_url": "?q=*&f.region=country_facet%3AAustralia",
          "sem_url_action": "?q=*&f.region=country_facet%3AAustralia&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=6489377f-40d2-48a4-b734-bae9bbece7a7"
        },
        "options": {
          "Clare Valley wine region": {
            "value": "Clare Valley wine region",
            "count": 1,
            "sem_action": {
              "sem_url": "?q=*&f.region=region_facet%3AClare%5C+Valley%5C+wine%5C+region",
              "sem_url_action": "?q=*&f.region=region_facet%3AClare%5C+Valley%5C+wine%5C+region&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=6489377f-40d2-48a4-b734-bae9bbece7a7"
            }
          }
        }
      },
      "Chile": {
        "value": "Chile",
        "count": 1,
        "sem_action": {
          "sem_url": "?q=*&f.region=country_facet%3AChile",
          "sem_url_action": "?q=*&f.region=country_facet%3AChile&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=6489377f-40d2-48a4-b734-bae9bbece7a7"
        },
        "options": {
          "Cachapoal Province": {
            "value": "Cachapoal Province",
            "count": 1,
            "sem_action": {
              "sem_url": "?q=*&f.region=region_facet%3ACachapoal%5C+Province",
              "sem_url_action": "?q=*&f.region=region_facet%3ACachapoal%5C+Province&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=6489377f-40d2-48a4-b734-bae9bbece7a7"
            }
          }
        }
      },
      "France": {
        "value": "France",
        "count": 6,
        "active": true,
        "sem_action": {
          "sem_url": "?q=*",
          "sem_url_action": "?q=*&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=6489377f-40d2-48a4-b734-bae9bbece7a7"
        },
        "options": {
          "Burgundy": {
            "value": "Burgundy",
            "count": 4,
            "sem_action": {
              "sem_url": "?q=*&f.region=region_facet%3ABurgundy",
              "sem_url_action": "?q=*&f.region=region_facet%3ABurgundy&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=6489377f-40d2-48a4-b734-bae9bbece7a7"
            },
            "options": {
              "Côte de Nuits": {
                "value": "Côte de Nuits",
                "count": 2,
                "sem_action": {
                  "sem_url": "?q=*&f.region=wine_sub_region_facet%3AC%C3%B4te%5C+de%5C+Nuits",
                  "sem_url_action": "?q=*&f.region=wine_sub_region_facet%3AC%C3%B4te%5C+de%5C+Nuits&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=6489377f-40d2-48a4-b734-bae9bbece7a7"
                }
              },
              "Côte de Beaune": {
                "value": "Côte de Beaune",
                "count": 1,
                "sem_action": {
                  "sem_url": "?q=*&f.region=wine_sub_region_facet%3AC%C3%B4te%5C+de%5C+Beaune",
                  "sem_url_action": "?q=*&f.region=wine_sub_region_facet%3AC%C3%B4te%5C+de%5C+Beaune&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=6489377f-40d2-48a4-b734-bae9bbece7a7"
                }
              }
            }
          },
          "Bordeaux": {
            "value": "Bordeaux",
            "count": 1,
            "sem_action": {
              "sem_url": "?q=*&f.region=region_facet%3ABordeaux",
              "sem_url_action": "?q=*&f.region=region_facet%3ABordeaux&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=6489377f-40d2-48a4-b734-bae9bbece7a7"
            },
            "options": {
              "Médoc": {
                "value": "Médoc",
                "count": 1,
                "sem_action": {
                  "sem_url": "?q=*&f.region=wine_sub_region_facet%3AM%C3%A9doc",
                  "sem_url_action": "?q=*&f.region=wine_sub_region_facet%3AM%C3%A9doc&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=6489377f-40d2-48a4-b734-bae9bbece7a7"
                }
              }
            }
          },
          "Champagne": {
            "value": "Champagne",
            "count": 1,
            "sem_action": {
              "sem_url": "?q=*&f.region=region_facet%3AChampagne",
              "sem_url_action": "?q=*&f.region=region_facet%3AChampagne&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=6489377f-40d2-48a4-b734-bae9bbece7a7"
            }
          }
        }
      },
      "Italy": {
        "value": "Italy",
        "count": 5,
        "sem_action": {
          "sem_url": "?q=*&f.region=country_facet%3AItaly",
          "sem_url_action": "?q=*&f.region=country_facet%3AItaly&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=6489377f-40d2-48a4-b734-bae9bbece7a7"
        },
        "options": {
          "Montalcino": {
            "value": "Montalcino",
            "count": 3,
            "sem_action": {
              "sem_url": "?q=*&f.region=region_facet%3AMontalcino",
              "sem_url_action": "?q=*&f.region=region_facet%3AMontalcino&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=6489377f-40d2-48a4-b734-bae9bbece7a7"
            }
          },
          "Piedmont": {
            "value": "Piedmont",
            "count": 2,
            "sem_action": {
              "sem_url": "?q=*&f.region=region_facet%3APiedmont",
              "sem_url_action": "?q=*&f.region=region_facet%3APiedmont&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=6489377f-40d2-48a4-b734-bae9bbece7a7"
            },
            "options": {
              "Piedmont": {
                "value": "Piedmont",
                "count": 1,
                "sem_action": {
                  "sem_url": "?q=*&f.region=wine_sub_region_facet%3APiedmont",
                  "sem_url_action": "?q=*&f.region=wine_sub_region_facet%3APiedmont&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=6489377f-40d2-48a4-b734-bae9bbece7a7"
                }
              }
            }
          }
        }
      },
      "New Zealand": {
        "value": "New Zealand",
        "count": 1,
        "sem_action": {
          "sem_url": "?q=*&f.region=country_facet%3ANew%5C+Zealand",
          "sem_url_action": "?q=*&f.region=country_facet%3ANew%5C+Zealand&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=6489377f-40d2-48a4-b734-bae9bbece7a7"
        },
        "options": {
          "Marlborough": {
            "value": "Marlborough",
            "count": 1,
            "sem_action": {
              "sem_url": "?q=*&f.region=region_facet%3AMarlborough",
              "sem_url_action": "?q=*&f.region=region_facet%3AMarlborough&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=6489377f-40d2-48a4-b734-bae9bbece7a7"
            }
          }
        }
      },
      "Spain": {
        "value": "Spain",
        "count": 1,
        "sem_action": {
          "sem_url": "?q=*&f.region=country_facet%3ASpain",
          "sem_url_action": "?q=*&f.region=country_facet%3ASpain&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=6489377f-40d2-48a4-b734-bae9bbece7a7"
        },
        "options": {
          "Ribero del Duero": {
            "value": "Ribero del Duero",
            "count": 1,
            "sem_action": {
              "sem_url": "?q=*&f.region=region_facet%3ARibero%5C+del%5C+Duero",
              "sem_url_action": "?q=*&f.region=region_facet%3ARibero%5C+del%5C+Duero&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=6489377f-40d2-48a4-b734-bae9bbece7a7"
            }
          }
        }
      },
      "United States of America": {
        "value": "United States of America",
        "count": 85,
        "sem_action": {
          "sem_url": "?q=*&f.region=country_facet%3AUnited%5C+States%5C+of%5C+America",
          "sem_url_action": "?q=*&f.region=country_facet%3AUnited%5C+States%5C+of%5C+America&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=6489377f-40d2-48a4-b734-bae9bbece7a7"
        },
        "options": {
          "California": {
            "value": "California",
            "count": 84,
            "sem_action": {
              "sem_url": "?q=*&f.region=region_facet%3ACalifornia",
              "sem_url_action": "?q=*&f.region=region_facet%3ACalifornia&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=6489377f-40d2-48a4-b734-bae9bbece7a7"
            },
            "options": {
              "Sonoma County": {
                "value": "Sonoma County",
                "count": 59,
                "sem_action": {
                  "sem_url": "?q=*&f.region=wine_sub_region_facet%3ASonoma%5C+County",
                  "sem_url_action": "?q=*&f.region=wine_sub_region_facet%3ASonoma%5C+County&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=6489377f-40d2-48a4-b734-bae9bbece7a7"
                }
              },
              "Contra Costa County": {
                "value": "Contra Costa County",
                "count": 5,
                "sem_action": {
                  "sem_url": "?q=*&f.region=wine_sub_region_facet%3AContra%5C+Costa%5C+County",
                  "sem_url_action": "?q=*&f.region=wine_sub_region_facet%3AContra%5C+Costa%5C+County&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=6489377f-40d2-48a4-b734-bae9bbece7a7"
                }
              },
              "Napa County": {
                "value": "Napa County",
                "count": 3,
                "sem_action": {
                  "sem_url": "?q=*&f.region=wine_sub_region_facet%3ANapa%5C+County",
                  "sem_url_action": "?q=*&f.region=wine_sub_region_facet%3ANapa%5C+County&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=6489377f-40d2-48a4-b734-bae9bbece7a7"
                }
              },
              "San Joaquin County": {
                "value": "San Joaquin County",
                "count": 3,
                "sem_action": {
                  "sem_url": "?q=*&f.region=wine_sub_region_facet%3ASan%5C+Joaquin%5C+County",
                  "sem_url_action": "?q=*&f.region=wine_sub_region_facet%3ASan%5C+Joaquin%5C+County&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=6489377f-40d2-48a4-b734-bae9bbece7a7"
                }
              },
              "Amador County": {
                "value": "Amador County",
                "count": 2,
                "sem_action": {
                  "sem_url": "?q=*&f.region=wine_sub_region_facet%3AAmador%5C+County",
                  "sem_url_action": "?q=*&f.region=wine_sub_region_facet%3AAmador%5C+County&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=6489377f-40d2-48a4-b734-bae9bbece7a7"
                }
              },
              "Mendocino County": {
                "value": "Mendocino County",
                "count": 2,
                "sem_action": {
                  "sem_url": "?q=*&f.region=wine_sub_region_facet%3AMendocino%5C+County",
                  "sem_url_action": "?q=*&f.region=wine_sub_region_facet%3AMendocino%5C+County&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=6489377f-40d2-48a4-b734-bae9bbece7a7"
                }
              },
              "Riverside County": {
                "value": "Riverside County",
                "count": 1,
                "sem_action": {
                  "sem_url": "?q=*&f.region=wine_sub_region_facet%3ARiverside%5C+County",
                  "sem_url_action": "?q=*&f.region=wine_sub_region_facet%3ARiverside%5C+County&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=6489377f-40d2-48a4-b734-bae9bbece7a7"
                }
              },
              "Santa Cruz County": {
                "value": "Santa Cruz County",
                "count": 1,
                "sem_action": {
                  "sem_url": "?q=*&f.region=wine_sub_region_facet%3ASanta%5C+Cruz%5C+County",
                  "sem_url_action": "?q=*&f.region=wine_sub_region_facet%3ASanta%5C+Cruz%5C+County&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=6489377f-40d2-48a4-b734-bae9bbece7a7"
                }
              }
            }
          },
          "Napa Valley AVA": {
            "value": "Napa Valley AVA",
            "count": 1,
            "sem_action": {
              "sem_url": "?q=*&f.region=region_facet%3ANapa%5C+Valley%5C+AVA",
              "sem_url_action": "?q=*&f.region=region_facet%3ANapa%5C+Valley%5C+AVA&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=6489377f-40d2-48a4-b734-bae9bbece7a7"
            }
          }
        }
      }
    }
  }
}

There are two elements in the above structure that merit particular attention:

  • When a facet option has “child” options, these are to be found under the field options.

  • Apart from the aforementioned additional field for the “children”, hierarchical facet options are identical to single-level facet options with one exception: for additional flexibility the option’s label is the same as the current “level” in the hierarchy where the option resides, while the field value contains the full branch. In the above snippet, the label for the nested option is "Clare Valley wine region" and its value field is also "Clare Valley wine region"

    • Difference between “Pivot” style hierarchies and Single-field hierarchies: value contains the full path, in the example above this would then be "Australia:Clare Valley wine region"

Single-field hierarchies

Configuration

In order to use singe-field hierarchical faceting a FacetHierarchyComponent has to be defined in the configuration.

From the configuration elements described in the section Common configuration options FacetHierarchyComponent supports addMissingFilters, excludeAllFilters, excludeOwnFilters, maxOptions, minCount, multiSelect, multiSelectMode, sortBy and useExclusiveSetAction.

In addition FacetHierarchyComponent introduces the following elements:

R = required

Name Type Default Description R

divider

string

/

Divider character or string used to get the hierarchical information from the field

✓

filterFirstLevel

boolean

true

En-/disables filter actions be available at the top level

exclusiveSetInBranch

boolean

false

When true, applying a filter within a branch resets any other active filter(s) "above" or "below" in the branch; multiple filters at the same branch level are still possible in multi-select mode.
Attention! The component needs to be defined with a field of the type descendant_path and the component must be configured to have the same divider as the field!

Facet Query

Custom queries can be defined and used as Facet and filter by adding a FacetQueryComponent.

Configuration

In order to use custom query based facets a FacetQueryComponent has to be defined in the configuration.

While most of the elements of the configuration as shown above have already been described in the section Common configuration options some additional configuration options are available for FacetQueryComponent:

R = required

Name Type Default Description R

queries

key/value

 

A map of keys and values whereas the unique key identifies the query described in value.

✓

addNegations

boolean

false

true will add additional negated query / queries for every prefixed with not_
Examples
FacetQueryComponent example - custom_queries
{
  "type": "FacetQueryComponent",
  "componentName": "custom_queries",
  "queries": {
    "old_american": "country_facet:'United States of America' AND vintage:[0 TO 1999]",
    "open": "sem_opening_hours:\"Intersects(ENVELOPE(1501010000,${getNowAsOccurence()},2512122359,${getNowAsOccurence()}))\""
  },
  "excludeOwnFilters": true,
  "excludeAllFilters": false,
  "multiSelect": true,
  "multiSelectMode": "OR",
  "addNegations": true,
  "useExclusiveSetAction": true
}
FacetQueryComponent example - custom_geoqueries
{
  "type": "FacetQueryComponent",
  "componentName": "custom_geoqueries",
  "queries": {
    "nearby": "{!geofilt pt=${pt} sfield=sem_location d=0.2}"
  },
  "excludeOwnFilters": true,
  "excludeAllFilters": false,
  "multiSelect": true,
  "multiSelectMode": "OR",
  "useExclusiveSetAction": true
}
Various facet queries can be defined in one or more components. Note that when they are within one component that would share the same configuration - e.g. multiselect - and would be treated together, meaning that selecting one would have effect on value / query as a filter would have effects on the other facet queries. Thus queries that should be independent should be defined in separate components.

Response

The response of the FacetQueryComponent shows up under the the section facet_queries section of facet_count. An example of a response based on examples configuration above may look like this:

Facet Query response example
{
  "facet_queries": {
    "custom_geoqueries": [],
    "custom_queries": {
      "old_american": {
        "value": "old_american",
        "count": 3,
        "sem_action": {
          "sem_url": "?q=*%0D&f.custom_queries=old_american",
          "sem_url_action": "?q=*%0D&f.custom_queries=old_american&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=71968056-a645-4e3e-9cc8-bf4aaa656782"
        }
      },
      "open": {
        "value": "open",
        "count": 2,
        "sem_action": {
          "sem_url": "?q=*%0D&f.custom_queries=open",
          "sem_url_action": "?q=*%0D&f.custom_queries=open&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=71968056-a645-4e3e-9cc8-bf4aaa656782"
        }
      },
      "not_open": {
        "value": "not_open",
        "count": 98,
        "sem_action": {
          "sem_url": "?q=*%0D&f.custom_queries=not_open",
          "sem_url_action": "?q=*%0D&f.custom_queries=not_open&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=71968056-a645-4e3e-9cc8-bf4aaa656782"
        }
      },
      "not_old_american": {
        "value": "not_old_american",
        "count": 97,
        "sem_action": {
          "sem_url": "?q=*%0D&f.custom_queries=not_old_american",
          "sem_url_action": "?q=*%0D&f.custom_queries=not_old_american&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=71968056-a645-4e3e-9cc8-bf4aaa656782"
        }
      }
    }
  }
}

 

Active and suggested Filters

Facets and Filters can add active filters while Autofilter and Geographic Query can add active and suggested filters to the response.

Suggested Filters are available since version 1.6.
Active filters from Geographic Query are available since version 1.6.

Active Filters

Active filters describe all filter that have been set by the user or by the system (e.g. by the Autofilter or the Geographic Query and are represented in the corresponding section of the response.

Continuing from the example in Facets and Filters the query http://search-features.demos.contexity.ch/sl/search?q=*&_semc=minimal&f.color=color_facet%3ARed%5C+Wine activates a filter on the field color and adds a filter to that section:

Code example

 {
  "active_filters": {
    "sem_action": {
      "sem_url": "?q=*&_semc=minimal",
      "sem_url_action": "?q=*&_semc=minimal&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=5cda47e4-d167-4daf-be8b-c81ebbcba28c"
    },
    "filters": [
      {
        "componentName": "color",
        "source": "USER",
        "valueLabel": "Red Wine",
        "active": true,
        "value": "color_facet:Red\\ Wine",
        "sem_action": {
          "sem_url": "?q=*&_semc=minimal",
          "sem_url_action": "?q=*&_semc=minimal&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=5cda47e4-d167-4daf-be8b-c81ebbcba28c"
        }
      }
    ]
  }
}

In the above snippet:

  • The top-level sem_action is one that disables all currently active filters. An equivalent action is available in the form of Base URLs (see section Base URLs).

  • filters: An array of the currently active filters. The fields that describe each filter are:

    • componentName: This, by default, the name of the corresponding document field, except if otherwise configured (e.g., so that two different facet components can be applied on the same document field).

    • source: USER if the filter was set by the user), SYSTEM if the filter was set by the system (Autofilter or Geographic Query)

    • valueLabel: This is the actual unprocessed facet value, typically used as a label in the front end.

    • value: This is a representation of the filter in Solr format, which can be used if the frontend needs to do custom filter query management directly.

    • sem_action: This is the “toggle the filter off” action for the current filter.

Suggested Filters

Suggested filters are added by Autofilter and Geographic Query. If present they are also represented in the corresponding section of the response.

E.g. the autofilter adds a filter as suggested filter when it is not in apply mode: http://search-features.demos.contexity.ch/sl/search?q=Red%20Wine&autofilter.apply=false

Code example

 {
  "suggested_filters": {
    "filters": [
      {
        "componentName": "color",
        "source": "SYSTEM",
        "valueLabel": "Red Wine",
        "active": false,
        "value": "color_facet:Red\\ Wine",
        "sem_action": {
          "sem_url": "?q=Red+Wine&autofilter.apply=false&indent=true&f.color=color_facet%3ARed%5C+Wine",
          "sem_url_action": "?q=Red+Wine&autofilter.apply=false&indent=true&f.color=color_facet%3ARed%5C+Wine&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=554b9952-7bb3-4bef-8030-a446629ba126"
        }
      }
    ]
  }
}

In the above snippet:

  • filters: An array of suggested filters. The fields that describe each filter are:

    • componentName: This, by default, the name of the corresponding document field, except if otherwise configured (e.g., so that two different facet components can be applied on the same document field).

    • source: SYSTEM indicates that the filter was suggested by the system (Autofilter or Geographic Query)

    • valueLabel: This is the actual unprocessed facet value, typically used as a label in the front end.

    • value: This is a representation of the filter in Solr format, which can be used if the frontend needs to do custom filter query management directly.

    • sem_action: This is the “toggle the filter on” action for the suggested filter.

Autofilter

Available since version 1.4.

The autofilter is a feature that automatically applies a filters for certain fields. The autofilter feature is provided by the AutofilterSearchLayerComponent.

Basic functionality

The AutofilterSearchLayerComponent looks up each query term in the index of the fields that were configured.

Query terms that were found in the defined autofilter fields are then

  • activated as filters

Configuration

The Autofilter feature is available and enabled per default as soon as an AutofilterSearchLayerComponent is added to your configuration.

Solr configuration

As the AutofilterSearchLayerComponent works together with a plugin in solr some feature need to be edited directly in solr’s main configuration file solr-config.xml:

Name

Type

Default

Description

autofilter

boolean

false

enable/disable if autofilter

apply

boolean

true

apply or sugguest only

synonyms

String

-

Name of the file that is used to retrieve the synonyms.

field

String

-

The names of the field(s) to be used for the autofilter (Note that the field type of the fields should be string!)
Note that every field that is defined in the plugin should have corresponding Text based facets defined and should be used with addMissingFilters = true!

The example below shows the default configuration for the Autofilter plugin:

Configuration in solr
	<searchComponent name="autofilter"
		class="org.apache.solr.handler.component.QueryAutoFilteringComponent">
		<str name="autofilter">false</str>
		<str name="apply">true</str>
		<str name="synonyms">synonyms_autofilter.txt</str>
		<str name="field">country_facet,region_facet,wine_type_facet,wine_style_facet</str>
	</searchComponent>
Component configuration

In your search-layer-configuration.json configuration file the following options can be configured.

R = required

Name Type Default Description R Overridden by searchtime parameter

autofilterEnabled

boolean

true

enable/disable if autofilter

autofilter

autofilterApplied

boolean

true

apply or sugguest only

autofilter.apply

mode

"filter", "boost", "replace"

filter

Set mode to "filter", "boost" or "replace"

autofilter.mode

afb

Integer

1

Boost factor when boost mode is activated

autofilter.afb

omitTermsMatchingMultipleFields

boolean

false

if true, terms that match in more than one field are not processed. (filter mode)

omitUnkownFields

boolean

true

If true, matches in fields that are not defined as FacetFieldComponents are not processed. (filter mode)

 

applyFirstFieldMatchOnly

boolean

false

if true only the first match for a field is applied whereby any succeeding matches are ignored

removeMatchingQueryTerms

boolean

false

If true the matched query terms are removed from the query - requires version 1.6

mappings

key/value

-

One or more mappings from a field that is configured in solrconfig.xml and a component that is configured in the configuration can be defined.
Note that some of the configuration options can be overridden on search time using the mentioned parameters. Use autofilter = true / false to en- or disable the component (enabled per default whenever the AutofilterSearchLayerComponent was added).

The code example below defines an AutofilterSearchLayerComponent in filter mode:

Configuration of an AutofilterSearchLayerComponent
{
  "type": "AutofilterSearchLayerComponent",
  "autofilterApplied": true,
  "mode": "filter",
  "omitTermsMatchingMultipleFields": false,
  "omitUnkownFields": true,
  "applyFirstFieldMatchOnly": false,
  "mappings": {
    "vintage_facet": "vintage_hierachy"
  }
}

Modes

As mentioned above the Autofilter supports three different modes.

  • Filter: The matched word/phrase is taken out of the search terms (q) and added as filter query (fq).

  • Replace: The matched word/phrase is replaced with a fieldname:fieldvalue notation within the search terms (q).

  • Boost: The matched word/phrase is left in the search terms but completed by an additional fieldname:fieldvalue notation with a boost factor 

Try out the different modes by simply adding the autofilter.mode parameter, e.g. http://search-features.demos.contexity.ch/sl/search?q=Still Wine 1999&autofilter.mode=boost vers http://search-features.demos.contexity.ch/sl/search?q=Still Wine 1999&autofilter.mode=replace

Observe how the query gets modified by having a look at autofilter > fields > all ! 

Response

A top-level section autofilter is added to the response that shows the found matches.

Response in filter mode

In the mode filter the subsection matches would list all possible matches in the same format that is also used in Active and suggested Filters whereby the field active informs if the filter has been applied or not. Furthermore the same filters would show up in the active_filters and suggested_filter section.

Example
The example below shows the autofilter section from the response for http://search-features.demos.contexity.ch/sl/search?q=Still Wine Red Wine. Note that the property source shows SYSTEM indicating that the filter was automatically set (in contrast to USER)
Autofilter filter mode response
{
  "autofilter": {
    "enable": true,
    "apply": true,
    "matches": [
      {
        "componentName": "wine_type",
        "source": "SYSTEM",
        "valueLabel": "Still Wine",
        "active": true,
        "value": "wine_type_facet:Still\\ Wine",
        "sem_action": {
          "sem_url": "?q=Still+Wine+Red+Wine&f.color=color_facet%3ARed%5C+Wine&f.autofilterexclude.f.wine_type=wine_type_facet%3AStill%5C+Wine",
          "sem_url_action": "?q=Still+Wine+Red+Wine&f.color=color_facet%3ARed%5C+Wine&f.autofilterexclude.f.wine_type=wine_type_facet%3AStill%5C+Wine&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=181dcbfd-eb53-4f86-96b2-cc8d1bedc5fa"
        }
      },
      {
        "componentName": "color",
        "source": "SYSTEM",
        "valueLabel": "Red Wine",
        "active": true,
        "value": "color_facet:Red\\ Wine",
        "sem_action": {
          "sem_url": "?q=Still+Wine+Red+Wine&f.wine_type=wine_type_facet%3AStill%5C+Wine&f.autofilterexclude.f.color=color_facet%3ARed%5C+Wine",
          "sem_url_action": "?q=Still+Wine+Red+Wine&f.wine_type=wine_type_facet%3AStill%5C+Wine&f.autofilterexclude.f.color=color_facet%3ARed%5C+Wine&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=181dcbfd-eb53-4f86-96b2-cc8d1bedc5fa"
        }
      }
    ]
  }
}
Response in boost mode

As in boost mode there are no filter the response differs. The query term q itself is changed which is exposed in fields.all.q

The example below shows the autofilter section from the response using the same query as above but activating Autofilter’s boost mode: http://search-features.demos.contexity.ch/sl/search?q=Still Wine Red Wine&autofilter.mode=boost
Autofilter boost mode response
{
  "autofilter": {
    "enable": true,
    "apply": true,
    "tokens": [
      "Still",
      "Wine",
      "Red",
      "Wine"
    ],
    "fieldlist": "color_facet, wine_type_facet, wine_style_facet, appellation_facet, vintage_facet, producer_region, producer_country",
    "fields": {
      "all": {
        "q": "(Still Wine OR wine_type_facet:\"Still Wine\") (Red Wine OR color_facet:\"Red Wine\")",
        "fq": [],
        "apply": true
      }
    },
    "matches": {
      "wine_type_facet:Still Wine": {
        "usedTokenPositions": [
          0,
          1
        ],
        "field": "wine_type_facet",
        "terms": [
          "Still Wine"
        ],
        "query": "wine_type_facet:\"Still Wine\"",
        "usedTokens": [
          "Still",
          "Wine"
        ]
      },
      "color_facet:Red Wine": {
        "usedTokenPositions": [
          2,
          3
        ],
        "field": "color_facet",
        "terms": [
          "Red Wine"
        ],
        "query": "color_facet:\"Red Wine\"",
        "usedTokens": [
          "Red",
          "Wine"
        ]
      }
    }
  }
}
Response in replace mode

The response in replace mode is very similar to the boost mode, only that the resulting q would differ.

For the query above q would be "q": "wine_type_facet:\"Still Wine\" color_facet:\"Red Wine\""
Base url section

Additionally the Autofilter adds some entries to the base_urls section (see Base URLs)

Autofilter base urls
{
  "disable_autofilter": {
    "sem_url": "?q=Still+Wine+Red+Wine%0D&autofilter=false",
    "sem_url_action": "?q=Still+Wine+Red+Wine%0D&autofilter=false&ot.et=search&ot.est=autofilter-modified&ot.es=source_serp&ot.sse=e92155d7-4c68-49a2-9ef8-42b31bcf26ed"
  },
  "reset_autofilter": {
    "sem_url": "?q=Still+Wine+Red+Wine%0D",
    "sem_url_action": "?q=Still+Wine+Red+Wine%0D&ot.et=search&ot.est=autofilter-modified&ot.es=source_serp&ot.sse=e92155d7-4c68-49a2-9ef8-42b31bcf26ed"
  },
  "dont_apply_autofilter": {
    "sem_url": "?q=Still+Wine+Red+Wine%0D&autofilter.apply=false",
    "sem_url_action": "?q=Still+Wine+Red+Wine%0D&autofilter.apply=false&ot.et=search&ot.est=autofilter-modified&ot.es=source_serp&ot.sse=e92155d7-4c68-49a2-9ef8-42b31bcf26ed"
  }
}

Geographic Query

The Geographic Query feature augments the original search with geographic reference information, before it is executed against the search index.The Geographic Query feature is provided by the GeoQueryComponent.

Basic functionality

The user can either:

  1. explicitly supply a geographic location (lat,long) through a request parameter: geoquery.coordinates (refer to Search time parameters below)

  2. use the geographic information of a record from the index. Furthermore the component supports two alternative modes for establishing the index record(s) from which the reference information is extracted:

    1. by matching terms in the user’s query against the names of geographic places in the index, or

    2. by using a record in the index explicitly specified through a dedicated request parameter: geoquery.geoid (refer to Search time parameters below)

If the component can establish at least one candidate reference record, it modifies the search query as follows:

  1. If the record resulted from the matching of terms against the names of geographic places, then the terms used for that match are removed from the user’s query string.

  2. It adds parameters to the query that filter or boost results on the basis of their relation to the geographic shape that is extracted from the reference record (e.g., restricts results to only those that are located within a polygonal geographic area).

The starting geographic information in the reference record can be either a single point or a (multi-)polygon. Operations that can be additionally applied on these starting shapes include: for points, the creation of a circle with the point as its center; for (multi-)polygons, "buffering" of the the polygon.

Configuration

The component configuration controls how geographic data is retrieved from the index, what fields are reviewed for geographic information, which operations are applied and in which cases.

That means the component can be configured so that different settings are applied depending on the reference geographic record.

Name

Type

Scope

Default

Description

Overriden by searchtime parameter

qf

string

component

 

field name of the title of the places

 

fq

string (fieldname:fieldvalue)

component

 

filter query that restricts results to relevant geo records

 

fl

String

component

*

Field list of fields to be listed in the document section

 

geoqueryApplied

boolean

component

 

Enable / disable if the geoquery is directly applie

geoquery.apply

distErrPct

double

component

0.05

distErrPct at query time, lower value gives more precise results at the cost of slower performance. A value of 0.01 may give good results without affecting performance too much. See also Solr Spatial Documentation.

 

prefetch

boolean

component

 

en/disable prefetching that is showing number of his for alternative places (up to max alternative 5 places are shown). Refer to Configuration Properties for the search.config.geoquery.parallel property. See also the minCountApply and minCountList for further fine tuning.

geoquery.prefetch

removeModeAll (from version 1.6)

boolean

component

false

true: when the selected/applied place is removed, the geoquery will not apply any other place(s) automatically (meaning it will only suggested alternative places) / false: when the selected/applied place is removed, only this place will be excluded, while other places may still get applied automatically

 

proactiveSimplifyingEnabled (from version 1.7.5)

boolean

component

true

true: before applying a geometric operation, the component will check whether the geometry at hand is "too complex" and will attempt to simplify it if possible / false: no proactive simplification is performed before geometric operations

 

configurations

list

component

 

list of configurations activated conditionally on the basis of a field:value combination

 

configurations.default

configuration

component

 

default configuration – gets applied if none of the conditional configurations (see below) is applicable

 

shapefield

string

document

 

name of the field where the geographic information of the geo records is stored

 

searchfields

string array

document

 

name(s) of the field(s) where the geographic information of the main documents is stored

 

params

double array

document

 

one or more parameters for the geographic operation applied on the starting geographic shape of the reference record(currently only the first value is typically used, as a distance expressed in km)

geoquery.distance

operation

"buffer"

document

-

the geographic operation to be applied on the starting geographic shape of the reference record (if any)

 

mode

"filter" OR "boost"

document

filter

add filtering, or, alternatively, boosting parameters to the main search query.

geoquery.mode

simplify

double

document

 

setting a value for simplification would apply a simplification with the given value of the document’s shape (useful for complex Polygons or Multipolygons)

 

minCountApply

integer

component

1

Minimum number of hits (results) a place needs to give back to be applied (when prefetch = true) - requires version 1.4

 

minCountList

integer

component

1

Minimum number of hits (results) a place needs to give back to be included to the list of alternatives (when prefetch = true) - requires version 1.4

 
Sample Configuration

The code example below illustrates how a complex configuration of the GeoQueryComponent can look like in practice. Different reference fields, operations and parameters are used depending on the values in the field sem_record_type and sem_record_subtype.

Code example

 {
  "type": "GeoQueryComponent",
  "qf": "sem_geo_title",
  "fq": "sem_record_type:sem_geo",
  "prefetch": true,
  "geoqueryApplied": true,
  "distErrPct": 0.01,
  "configurations": [
    {
      "default": {
        "shapefield": "sem_geo_object",
        "titlefield": "sem_geo_title",
        "searchfields": [
          "sem_location"
        ]
      }
    },
    {
      "field": "sem_record_type",
      "values": {
        "sem_geo": {
          "subtypes": {
            "field": "sem_record_subtype",
            "values": {
              "region": {
                "shapefield": "sem_geo_object",
                "mode": "filter"
              },
              "city": {
                "shapefield": "sem_geo_object",
                "mode": "filter",
                "params": [
                  30
                ]
              }
            }
          },
          "default": {
            "shapefield": "sem_geo_object",
            "mode": "filter",
            "params": [
              5
            ]
          }
        }
      }
    }
  ]
}
Search time parameters

Apart from parameters geoquery.apply, geoquery.distance, geoquery.prefetch, and geoquery.mode that can be used to override configuration option.

These additional search time parameters are available

Parameter Type Description

geoquery

boolean

Enable / disable the Geoquery component itself (enabled by default)

geoquery.coordinates

lat,long

Provide coordinates for the search directly

geoquery.geoid

string

Supply a reference geographic record directly by id

geoquery.alternativeid

string

Supply a reference geographic record directly by id. The parameter is triggered if an alternative from the list is selected, the other alternatives are shown in the response list (in contrary to the use of geoquery.geoid)

Response

The component adds a section geoquery to the response where it exposes information about the matched place and its name and location, together with the operations that have been applied. Moreover the list of alternative places with additional information and their corresponding actions is added.

Elements of the response

The listing below shows the elements that you will find within the geoquery section:

Name Type Description

enable

boolean

en/disabled

match

boolean

match / no match

apply

boolean

value of the configuration/parameter apply

error

boolean

Indicates an error

msg

String

Error message in case of an error

place

place

an object describing the place that is the prefered match

alternatives

list of places

lists alternative geoquery search results (same strutures as place with additional active = true/false and corresponding sem_urls for selecting the alternative place directly

The place elements in place and alternatives is made up by the following fields:

Name Type Description

type

string

type of the place

id

string

id

mode

string ("filter" OR "boost")

filter or boost

name

string

title of the place

name_hl

string

title of the place with highlighting

query_shape

string

the (simplified) geolocation used in the search (lat,long location or WKT (Multi)polygon)

distance

string

distance that is used in the search

unit

string

unit of the distance

document

document

The document as it comes from solr

active

boolean

true if the place is currently used in the search

hits

integer

The number of results when the place is applied (requires activated prefetching)

sem_action

sem_url

lists alternative geoquery search results (same strutures as place with additional active = true/false and corresponding sem_urls for selecting the alternative place directly
Active and suggested places
Active place as filter

If the Geographic Query was used with geoquery.apply=true, or a place has been explicitly supplied by geoquery.geoid, geoquery.alternativeid or geoquery.coordinates the matched place is added as active filter in active_filters section.

Otherwise (with geoquery.apply=false) the matched place would be shown in the place section but it would be not be activated. When the matched place does not satisfy the requirement (e.g. minCountApply) the place would no even be shown in the place section but only in the list of alternatives.

Alternative places as suggested filters

All places would however by listed in the alternatives section, and all but the active one would be listed as suggested_filters.

See the 2nd and the 3rd code block below.

Sample Response

The code blocks below shows a sample response section for http://search-features.demos.contexity.ch/sl/search?q=Siena that encountered Siena (region) as best match and Siena (place) as alternative place that can be selected by their action_urls.

Code example

 {
  "geoquery": {
    "enable": true,
    "match": true,
    "apply": true,
    "totalTime": 43,
    "QTime": 2,
    "prefetchTime": 39,
    "prefetchCount": 2,
    "place": {
      "type": "city",
      "mode": "filter",
      "name": "Siena",
      "id": "3166548",
      "name_hl": "<em>Siena</em>",
      "query_shape": "POINT (11.33064 43.31822)",
      "distance": 30,
      "unit": "km",
      "queryTerms": [
        "Siena"
      ],
      "document": {
        "sem_address_country": "IT",
        "sem_text_search": [
          "IT"
        ],
        "sem_text_suggest": [
          "IT"
        ],
        "sem_geo_object": "POINT(11.33064 43.31822)",
        "sem_location": "43.31822,11.33064",
        "sem_location_latlng": "43.31822,11.33064",
        "sem_record_type": "sem_geo",
        "sem_geo_title": [
          "Siena"
        ],
        "id": "3166548",
        "sem_record_subtype": "city",
        "sem_source_id": "geonames",
        "_version_": 1617037263104901000,
        "sem_timestamp": "2018-11-13T16:35:06.495Z"
      },
      "hits": 2,
      "sem_action": {
        "sem_url": "?q=Siena&geoquery.excludeid=3166548",
        "sem_url_action": "?q=Siena&geoquery.excludeid=3166548&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=7b9d9f5a-be62-406c-8ce2-14db86a9bfcd"
      }
    },
    "alternatives": [
      {
        "type": "city",
        "mode": "filter",
        "name": "Siena",
        "id": "3166548",
        "name_hl": "<em>Siena</em>",
        "query_shape": "POINT (11.33064 43.31822)",
        "distance": 30,
        "unit": "km",
        "queryTerms": [
          "Siena"
        ],
        "document": {
          "sem_address_country": "IT",
          "sem_text_search": [
            "IT"
          ],
          "sem_text_suggest": [
            "IT"
          ],
          "sem_geo_object": "POINT(11.33064 43.31822)",
          "sem_location": "43.31822,11.33064",
          "sem_location_latlng": "43.31822,11.33064",
          "sem_record_type": "sem_geo",
          "sem_geo_title": [
            "Siena"
          ],
          "id": "3166548",
          "sem_record_subtype": "city",
          "sem_source_id": "geonames",
          "_version_": 1617037263104901000,
          "sem_timestamp": "2018-11-13T16:35:06.495Z"
        },
        "hits": 2,
        "active": true,
        "sem_action": {
          "sem_url": "?q=Siena&geoquery.excludeid=3166548",
          "sem_url_action": "?q=Siena&geoquery.excludeid=3166548&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=7b9d9f5a-be62-406c-8ce2-14db86a9bfcd"
        }
      },
      {
        "type": "region",
        "mode": "filter",
        "name": "Siena",
        "id": "relation/42172",
        "name_hl": "<em>Siena</em>",
        "location": "POLYGON ((11.271991 43.5291843, 11.2464189 43.4982957, 11.2081468 43.4803563, 11.1744572 43.48979, 11.1964081 43.5170662, 11.1834702 43.5226534, 11.1236584 43.4746149, 11.111303 43.4982136, 11.0342552 43.5467771, 10.9656079 43.5139505, 10.9516306 43.4491434, 10.9790126 43.4060745, 11.0018463 43.401544, 11.0133182 43.364993, 10.9977098 43.3440441, 10.9720679 43.3442642, 10.9699866 43.3142418, 10.9547604 43.3185573, 10.9520614 43.3020514, 10.9836571 43.2886969, 10.9965987 43.2397811, 10.9889427 43.2311218, 10.9394654 43.245154, 10.9272381 43.2311125, 10.9414104 43.2040072, 10.9087977 43.1639283, 10.9424257 43.1603547, 11.0218013 43.1891437, 11.0590956 43.183877, 11.0501528 43.1446165, 11.0834579 43.0831987, 11.1223446 43.0915513, 11.1689284 43.0785124, 11.1750957 43.089993, 11.2305854 43.0922019, 11.2972852 43.0770502, 11.3285459 43.0916001, 11.3619888 43.0829725, 11.3677882 43.0562176, 11.3540174 43.0403389, 11.3641915 43.0326014, 11.351963 43.0270546, 11.3805162 42.996083, 11.3652464 42.9945538, 11.3756843 42.979246, 11.3528976 42.9692627, 11.4570154 42.957349, 11.5176551 42.9812159, 11.5293577 42.9654341, 11.5716011 42.9702551, 11.6101518 42.951121, 11.6268779 42.9146753, 11.6196858 42.8814067, 11.6372624 42.8528187, 11.6292112 42.7966085, 11.6797401 42.7879626, 11.7024642 42.7990508, 11.7460177 42.7857594, 11.7742431 42.8210689, 11.8068483 42.794854, 11.8145721 42.8233525, 11.8493907 42.8387459, 11.8945934 42.8342863, 11.9290187 42.8692902, 11.9600251 42.8692394, 11.9562799 42.9003921, 11.9334807 42.9054182, 11.9347337 42.9270234, 11.9824016 43.0509482, 11.9567018 43.0686646, 11.9360654 43.0559259, 11.9123518 43.1284024, 11.9615158 43.1670678, 11.9034459 43.1745975, 11.8878478 43.1563413, 11.8482381 43.1887306, 11.8338218 43.2243783, 11.7493943 43.2339123, 11.7015948 43.2855464, 11.6766923 43.2859702, 11.6580243 43.3217013, 11.5665632 43.3622767, 11.5315205 43.3923367, 11.5506326 43.4595802, 11.5017622 43.4606988, 11.4608131 43.5102917, 11.4358772 43.5116138, 11.4029754 43.5485101, 11.3640231 43.5261199, 11.3071753 43.5182895, 11.2696302 43.5461491, 11.271991 43.5291843))",
        "distance": 5,
        "unit": "km",
        "queryTerms": [
          "Siena"
        ],
        "document": {
          "id": "relation/42172",
          "sem_geo_object": "POLYGON ((11.271991 43.5291843, 11.2464189 43.4982957, 11.2081468 43.4803563, 11.1744572 43.48979, 11.1964081 43.5170662, 11.1834702 43.5226534, 11.1236584 43.4746149, 11.111303 43.4982136, 11.0342552 43.5467771, 10.9656079 43.5139505, 10.9516306 43.4491434, 10.9790126 43.4060745, 11.0018463 43.401544, 11.0133182 43.364993, 10.9977098 43.3440441, 10.9720679 43.3442642, 10.9699866 43.3142418, 10.9547604 43.3185573, 10.9520614 43.3020514, 10.9836571 43.2886969, 10.9965987 43.2397811, 10.9889427 43.2311218, 10.9394654 43.245154, 10.9272381 43.2311125, 10.9414104 43.2040072, 10.9087977 43.1639283, 10.9424257 43.1603547, 11.0218013 43.1891437, 11.0590956 43.183877, 11.0501528 43.1446165, 11.0834579 43.0831987, 11.1223446 43.0915513, 11.1689284 43.0785124, 11.1750957 43.089993, 11.2305854 43.0922019, 11.2972852 43.0770502, 11.3285459 43.0916001, 11.3619888 43.0829725, 11.3677882 43.0562176, 11.3540174 43.0403389, 11.3641915 43.0326014, 11.351963 43.0270546, 11.3805162 42.996083, 11.3652464 42.9945538, 11.3756843 42.979246, 11.3528976 42.9692627, 11.4570154 42.957349, 11.5176551 42.9812159, 11.5293577 42.9654341, 11.5716011 42.9702551, 11.6101518 42.951121, 11.6268779 42.9146753, 11.6196858 42.8814067, 11.6372624 42.8528187, 11.6292112 42.7966085, 11.6797401 42.7879626, 11.7024642 42.7990508, 11.7460177 42.7857594, 11.7742431 42.8210689, 11.8068483 42.794854, 11.8145721 42.8233525, 11.8493907 42.8387459, 11.8945934 42.8342863, 11.9290187 42.8692902, 11.9600251 42.8692394, 11.9562799 42.9003921, 11.9334807 42.9054182, 11.9347337 42.9270234, 11.9824016 43.0509482, 11.9567018 43.0686646, 11.9360654 43.0559259, 11.9123518 43.1284024, 11.9615158 43.1670678, 11.9034459 43.1745975, 11.8878478 43.1563413, 11.8482381 43.1887306, 11.8338218 43.2243783, 11.7493943 43.2339123, 11.7015948 43.2855464, 11.6766923 43.2859702, 11.6580243 43.3217013, 11.5665632 43.3622767, 11.5315205 43.3923367, 11.5506326 43.4595802, 11.5017622 43.4606988, 11.4608131 43.5102917, 11.4358772 43.5116138, 11.4029754 43.5485101, 11.3640231 43.5261199, 11.3071753 43.5182895, 11.2696302 43.5461491, 11.271991 43.5291843))",
          "sem_geo_title": [
            "Siena",
            "Provincia di Siena"
          ],
          "ISO3166-2": [
            "IT-SI"
          ],
          "admin_level": [
            "6"
          ],
          "boundary": [
            "administrative"
          ],
          "name:ru": [
            "Сиена"
          ],
          "official_name": [
            "Provincia di Siena"
          ],
          "ref:ISTAT": [
            "052"
          ],
          "short_name": [
            "SI"
          ],
          "wikipedia": [
            "it:Provincia di Siena"
          ],
          "sem_record_type": "sem_geo",
          "sem_record_subtype": "region",
          "sem_source_id": "osm",
          "sem_geo_country": "Italy",
          "sem_lang": "unknown",
          "_version_": 1617037282008629200,
          "sem_timestamp": "2018-11-13T16:35:24.482Z"
        },
        "hits": 3,
        "active": false,
        "sem_action": {
          "sem_url": "?q=Siena&geoquery.alternativeid=relation%2F42172",
          "sem_url_action": "?q=Siena&geoquery.alternativeid=relation%2F42172&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=7b9d9f5a-be62-406c-8ce2-14db86a9bfcd"
        }
      }
    ]
  }
}
Active Filter

Code example

 {
  "active_filters": {
    "sem_action": {
      "sem_url": "?q=Siena",
      "sem_url_action": "?q=Siena&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=7b9d9f5a-be62-406c-8ce2-14db86a9bfcd"
    },
    "filters": [
      {
        "componentName": "GeoQueryComponent125664502",
        "source": "SYSTEM",
        "valueLabel": "place",
        "active": true,
        "value": "Siena (city)",
        "sem_action": {
          "sem_url": "?q=Siena&geoquery.excludeid=3166548",
          "sem_url_action": "?q=Siena&geoquery.excludeid=3166548&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=7b9d9f5a-be62-406c-8ce2-14db86a9bfcd"
        }
      }
    ]
  }
}
Suggested Filter

Code example

 {
  "suggested_filters": {
    "filters": [
      {
        "componentName": "GeoQueryComponent125664502",
        "source": "SYSTEM",
        "valueLabel": "place",
        "active": false,
        "value": "Siena (region)",
        "sem_action": {
          "sem_url": "?q=Siena&geoquery.alternativeid=relation%2F42172",
          "sem_url_action": "?q=Siena&geoquery.alternativeid=relation%2F42172&ot.et=search&ot.est=filter-modified&ot.es=source_serp&ot.sse=7b9d9f5a-be62-406c-8ce2-14db86a9bfcd"
        }
      }
    ]
  }
}
Error handling

In any case of an error (due to misconfiguration of missing data: e.g. the field that is supposed to contain information about the location is empty. etc.) the search would be processed "normally" (that is without further implication of GeoQueryComponent). An error message will be added to the response section of GeoQueryComponent.

Postprocess

The postprocess feature can be used to fetch additional information (documents) from your solr index. The Postprocess feature is provided by the PostprocessComponent.

Configuration

The component configuration controls which fields contain information for the additional query and where this information should be added to.

Name Type Default Description Version support

target

string

 -

name of the field that will be added and filled with the information.

 

source

string

 -

name of the field that contains the information use to perform the query.

 

qf

string

 

name of the field(s) that will be queried.

 

result

"documents"

"documents"

"documents" means that all documents from the result will be added the target field

 

requestParams

object (string)

-

An object containing a list of parameter names and values, that will be used in the post query e.g. { "fl": "id,sem_title", "fq" : "sem_record_tyle": "color" }

1.4+
Sample Configuration

The code example below illustrates how a configuration of the PostprocessComponent can look like in practice.

Code example

 {
  "type": "PostProcessComponent",
  "configurations": [
    {
      "target": "color_document",
      "source": "color",
      "qf": "id",
      "result": "documents",
      "requestParams": {
        "rows": "50",
        "fl": "id,sem_title,sem_url,sem_record_type",
        "fq": "sem_record_type:color"
      }
    }
  ]
}

Conditional Search Parameters

Available since version 1.6.

The ConditionalSearchParamsComponent can be used to define additional search parameter.As the component can be defined as [Dependent Component] these parameters are only added under the condition that the dependencies are satisfied.

Configuration

The component configuration is similar to the Predefined parameters definition (see Predefined parameters in Advanced configuration options and thus uses these three different types of parameters:

  • defaults: Default parameters can be overridden.

  • invariants: Invariants can never be overridden.

  • appends: Appends parameter will always be added.

Sample Configuration

The code examples below illustrates how a configuration of the ConditionalSearchParamsComponent can look like in practice.

Code example

 {
  "type": "ConditionalSearchParamsComponent",
  "componentName": "conditional_appends",
  "appends": [
    {
      "name": "sp.fq",
      "value": "id:\"/en/2005_ravenswood_old_vine_mendocino_county_zinfandel\""
    }
  ],
  "dependsOnAny": [
    {
      "param": "q",
      "value": "ravenswood"
    }
  ]
}

Code example

 {
  "type": "ConditionalSearchParamsComponent",
  "componentName": "conditional_invariants",
  "invariants": [
    {
      "name": "f.wine_type",
      "value": "wine_type_facet:Still\\ Wine"
    }
  ],
  "dependsOnAny": [
    {
      "component": "wine_producer",
      "value": "Clos\\ du\\ Bois\\ Wines"
    }
  ]
}

 

Distributed Search - multiple collections

Available since version 1.4.

With the distributed search feature one or more collection(s) that is/are different from the default one can be searched. 

That enables the use cases for

  • Distributed search where a set of collection can be used for the same search

  • Different collections can be used for e.g. language/country versions while they all share the same search configuration.

Configuration

The collections that would be searchable have to be defined in the section collections of the search-layer-configurations.json`. Therefore a map that consists in the collection name and their properties is supplied. The properties for each collection are the following:

R = required

Name Type Description R

solrUrl

String

Full URL of the solr collection

✓

monitoringSolrUrl

String

Full URL of the solr monitoring collection (If empty the default one would be used)

analyticsSolrUrl

String

Url that points to the solr collection for analytics (If empty the default one would be used)

Configuration example

Take a look at the collections in our example configuration at <http://search-features.demos.contexity.ch/sl/config>. The example code below defines the collections wines and empty

Code example

 {
  "collections": {
    "empty": {
      "solrUrl": "http://localhost:8035/solr/empty",
      "monitoringSolrUrl": "http://localhost:8035/solr/sem_monitoring_core_empty",
      "analyticsSolrUrl": "http://localhost:8035/solr/sem_monitoring_core_empty"
    },
    "wines": {
      "solrUrl": "http://localhost:8035/solr/winedemo",
      "monitoringSolrUrl": "http://localhost:8035/solr/sem_monitoring_core",
      "analyticsSolrUrl": "http://localhost:8035/solr/sem_monitoring_core"
    }
  }
}

Usage

The collection to be used for the search can be specified either

  • by using the parameter _collection

  • by accessing the collections specific endpoint(s):
    Specific endpoints for search, suggest, similar, etc. are available under {collectionname}/search, {collectionname}/suggest, {collectionname}/similar, etc.

Search multiple collections

In order to search over multiple collection the parameter _collection can be supplied with a comma-separated list of collections

Autocomplete

As the name suggests Autocomplete can be used to get auto-complementation suggestion for terms entered by the user.

  • Therefore a range of different types of suggesters are available that can be used.

  • Results can be prefetched for the suggested terms (therefore an additional search is triggered)

Usage

The default auto-complete endpoint is /suggest. The single mandatory parameter is:

  • term / q: This parameter contains the so far entered search term(s) for which suggestions should be returned. (The parameter q, if present, is treated as a synonym to term.)

Optional parameters:

  • Filter parameters (specifically for the auto-complete calls) need to be explicitly managed by the frontend, unlike what is the case of normal search. In other words, the search server defaults to not applying filters in the context of auto-complete calls. The filter parameters that can be used are identical to those of the normal search, and frontend developers can adjust the corresponding action URLs for the auto-completion calls as necessary. Note that auto-complete responses do again contain filtering capabilities (see the Example below for details).

Configuration

In the properties section of the configuration file the following properties are used to control the behavior of the Autocomplete search:

Name Type Default value Description Overridden by searchtime parameter

search.config.autocomplete

String

autocomplete

Response section name for autocomplete

search.config.autocomplete.maxsuggestions

int

5

Max. number of suggestions in autocomplete

search.config.autocomplete.config

String

autocomplete

Name of the configuration that is used for prefetching of results

autocomplete.config

search.config.autocomplete.sortsuggestions

String

natural

Available options: natural (based on string length), hits (based on number of hits)

search.config.autocomplete.prefetchedResults

String

prefetchedResults

Response section name for prefetched results

search.config.autocomplete.prefetchResults

boolean

true

En-/Disables prefetching of results.

search.config.autocomplete.highlight

boolean

false

En-/Disables highlighting of results.

search.config.spellchecker

List

default

List of the types of spellchecker to be used (1.6.5+)

spellchecker

search.config.suggest.prefix

String

sem_text_suggest

Name of the field that is used for facet prefix suggester

search.config.suggest.prefix.fq

List

-

Filter queries that should be applied when using the facet prefix suggester (e.g. sem_record_type:wine)

suggest.prefix.fq (1.6.5+)

search.config.suggester

List

prefix

The name(s) of the suggester that are used (e.g. prefix, suggest, or sem_suggest_phrase)

suggester (1.6.5+)

search.config.autocomplete.spellcheckresults

boolean

false

Spellcheck results are added as suggestions when the suggesters do not return results

search.config.autocomplete.lasttermquery

boolean

false

Get suggestions only for the last term when the suggesters do not return results

Response

The autocomplete response differs from the search response, it writes information to the section autocomplete and prefetchedResults.

Elements of the Response

The main sections of the response are:

  • autocomplete:

    • suggestions: This section contains the suggestions for the term(s) that can be used to “complete” the thus far user-entered query. The structure of the individual suggestions in this case is identical to the structure used for spelling suggestions – please refer to section Spelling suggestions for details.

  • prefetchedResults: This section contains “prefetched” documents that match the top suggestion, grouped by a predefined grouping field. The fields inside this section are:

    • term: This is the actual suggestion for which documents were prefetched

    • sem_action: The action URL that selects the term

    • response or grouped: Contains the prefetched documents either as document list or grouped. This sections correspond to the sections in the chapter The search response.

  • responseHeader: Same as for the search response.

Example

The overall structure of a search response is shown in the example below (see also A.2. Full auto-complete response for a full sample response)

Autocomplete response section
 {
 "response": {
  "autocomplete": {
   "suggestions": [{
    "suggestion": "brunet",
    "hits": "1",
    "sem_action": {
     "sem_url": "?q=brunet",
     "sem_url_action": "?q=brunet&ot.et=search&ot.est=suggest-term-select&ot.es=source_autocomplete&ot.orig&ot.sg=brunet&ot.sgidx=0&ot.sse=499783ee-d608-40eb-a502-094d235e5c21"
    }
   }, {
    "suggestion": "another suggestion ..."
   }]
  },
  "prefetchedResults": {
   "autofilter": "{ ...  autofilter fields ... }",
   "response": {
    "numFound": 1,
    "start": 0,
    "docs": [
     "{ ... Documents ... }"
    ]
   },
   "geoquery": "{ ... geoquery fields ... }",
   "term": "brunet",
   "sem_action": {
    "sem_url": "?q=brunet",
    "sem_url_action": "?q=brunet&ot.et=search&ot.est=normal&ot.es=source_serp&ot.sse=499783ee-d608-40eb-a502-094d235e5c21"
   }
  }
 },
 "responseHeader": " ... response header fields ... "
}

More Like This

More Like This is equivalent to a search for similar documents to a given one.The search can be used either by submitting the encoded ID to the default search handler in the form of q=mlt:$ID or by using the dedicated endpoint (see below).

Normally the search response would contain mlt action URLs that link to the morelikethis search for every document (refer to Document fields).

Usage

The default more-like-this endpoint is /morelikethis. The mandatory parameter for this call is:

  • id: This is the encoded identifier of the document for which similar documents should be returned. This encoded form is automatically generated and returned when a document is retrieved – see section Document fields for details.

Optional parameters:

  • Monitoring parameters, like in the case of normal searches, are prepared and delivered by the search server in prepopulated action URLs – see section Common conventions for details.

Configuration

In the properties section of the configuration file the following properties are used to control the behavior of the More Like This search:

Property name Type Default value Description

search.config.mlt.similarityfields

String

sem_text_search

Name of the field(s) that is/are used to get MoreLikeThis results

search.config.mlt.doctype.field

String

-

Name of the Field that is used to restrict MoreLikeThis results to documents with the same value in this field.

Response

Elements of the Response

The main sections of the response are

  • response: Contains identifying information for the document(s) used as a basis for similarity matching

  • moreLikeThis: Contains the similar documents, grouped by the reference document they are similar to

Within each group of similar documents, the fields returned are:

  • numFound: The total number of documents found that are “similar” to the original document associated with the present group

  • docs: An array of up to 10 documents from the set of similar ones, ordered by decreasing similarity to the original document. Note that the set of fields returned is a subset of the document fields returned in other types of requests; this set is configurable.

Example

Try it out! More like this for the wine 1995 Bruno Giacosa Gallina di Neive Barbaresco: http://search-features.demos.contexity.ch/sl/morelikethis?id=SEML2VuLzE5OTVfYnJ1bm9fZ2lhY29zYV9nYWxsaW5hX2RpX25laXZlX2JhcmJhcmVzY28

The overall structure of an more-like-this response is as follows (see section A.3. Full more like this response for a full sample response)

Code example

 {
 "response": "<information on the document(s) used as a reference>",
 "moreLikeThis": {
  "<id of the reference document>": {
   "numFound": "<total number of similar documents found>",
   "docs": [{
     "id": "...",
     "sem_source_id": "...",
     "sem_record_type": "...",
     "sem_url": "..."
    },
    "<other documents>"
   ]
  }
 },
 "responseHeader": "<response header fields>"
}

 

Record endpoint

The record endpoint is used for retrieving individual documents from the index. The default endpoint is /record.

Parameters

The mandatory parameter for this call is:

  • id: This is the encoded identifier of the document for which similar documents should be returned. This encoded form is automatically generated and returned when a document is retrieved – see section Document fields for details.

Optional parameters:

  • Monitoring parameters, like in the case of normal searches, are prepared and delivered by the search server in prepopulated action URLs – see section Common conventions for details.

Appendix A: Sample responses in full

A.1. Full search response

Example of a full search response (for version 1.3)

Search Response

{
  "response":{
    "numFound":13,
    "start":0,
    "docs":[
      {
        "id":"10",
        "sem_lang":"de",
        "sem_source_id":"www.tips.at",
        "sem_record_type":"Webpage",
        "sem_grouping":"Webpage",
        "sem_title":"Dummy Webpage",
        "sem_text_search":["Dummy Webpage"],
        "sem_title_alpha":"Dummy Webpage",
        "sem_url":"http://www.tips.at/10",
        "sem_category_names_branch":["Blaulicht"],
        "_version_":1497442836416561152,
        "sem_timestamp":"2015-04-03T14:52:53.362Z",
        "sem_excerpt":"... <em>Dummy</em> Webpage ... ",
        "sem_actions":{
          "default":"goto",
          "goto":{
            "sem_url":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F10&id=MTA&ot.et=result&ot.est=serp-result-click&ot.es=source_serp&ot.nf=13&ot.s=0&ot.r=3&ot.rank=0&ot.q=dummy",
            "sem_url_action":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F10&id=MTA&ot.et=result&ot.est=serp-result-click&ot.es=source_serp&ot.nf=13&ot.s=0&ot.r=3&ot.rank=0&ot.q=dummy",
            "sem_is_external":true},
          "mlt":{
            "sem_url":"http://localhost:8091/morelikethis?id=MTA&ot.et=mlt&ot.est=mlt&ot.es=source_serp&ot.nf=13&ot.s=0&ot.r=3&ot.rank=1&ot.q=dummy",
            "sem_url_action":"http://localhost:8091/morelikethis?id=MTA&ot.et=mlt&ot.est=mlt&ot.es=source_serp&ot.nf=13&ot.s=0&ot.r=3&ot.rank=1&ot.q=dummy"}},
        "sem_encoded_id":"MTA"},
      {
        "id":"11",
        "sem_lang":"de",
        "sem_source_id":"www.tips.at",
        "sem_record_type":"News",
        "sem_grouping":"News",
        "sem_title":"Dummy News beta",
        "sem_text_search":["Dummy News beta"],
        "sem_title_alpha":"Dummy News beta",
        "sem_url":"http://www.tips.at/11",
        "sem_category_names_branch":["Das ist los"],
        "_version_":1497442836417609728,
        "sem_timestamp":"2015-04-03T14:52:53.362Z",
        "sem_excerpt":"... <em>Dummy</em> News beta ... ",
        "sem_actions":{
          "default":"goto",
          "goto":{
            "sem_url":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F11&id=MTE&ot.et=result&ot.est=serp-result-click&ot.es=source_serp&ot.nf=13&ot.s=0&ot.r=3&ot.rank=2&ot.q=dummy",
            "sem_url_action":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F11&id=MTE&ot.et=result&ot.est=serp-result-click&ot.es=source_serp&ot.nf=13&ot.s=0&ot.r=3&ot.rank=2&ot.q=dummy",
            "sem_is_external":true},
          "mlt":{
            "sem_url":"http://localhost:8091/morelikethis?id=MTE&ot.et=mlt&ot.est=mlt&ot.es=source_serp&ot.nf=13&ot.s=0&ot.r=3&ot.rank=3&ot.q=dummy",
            "sem_url_action":"http://localhost:8091/morelikethis?id=MTE&ot.et=mlt&ot.est=mlt&ot.es=source_serp&ot.nf=13&ot.s=0&ot.r=3&ot.rank=3&ot.q=dummy"}},
        "sem_encoded_id":"MTE"},
      {
        "id":"7",
        "sem_lang":"de",
        "sem_source_id":"www.tips.at",
        "sem_record_type":"User",
        "sem_grouping":"Kontakt",
        "sem_title":"Dummy User",
        "sem_text_search":["Dummy User",
          "Otis",
          "Ut enim nisi ut aliquid ex ea commodi consequatur?",
          "ad minima veniam, quis nostrum exercitationem ullam corporis suscipit laboriosam"],
        "sem_title_alpha":"Dummy User",
        "sem_author":["Otis"],
        "sem_abstract":"Ut enim nisi ut aliquid ex ea commodi consequatur?",
        "sem_content":"ad minima veniam, quis nostrum exercitationem ullam corporis suscipit laboriosam",
        "sem_url":"http://www.tips.at/7",
        "_version_":1497442836412366848,
        "sem_timestamp":"2015-04-03T14:52:53.362Z",
        "sem_excerpt":"... <em>Dummy</em> User ... ",
        "sem_actions":{
          "default":"goto",
          "goto":{
            "sem_url":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F7&id=Nw&ot.et=result&ot.est=serp-result-click&ot.es=source_serp&ot.nf=13&ot.s=0&ot.r=3&ot.rank=4&ot.q=dummy",
            "sem_url_action":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F7&id=Nw&ot.et=result&ot.est=serp-result-click&ot.es=source_serp&ot.nf=13&ot.s=0&ot.r=3&ot.rank=4&ot.q=dummy",
            "sem_is_external":true},
          "mlt":{
            "sem_url":"http://localhost:8091/morelikethis?id=Nw&ot.et=mlt&ot.est=mlt&ot.es=source_serp&ot.nf=13&ot.s=0&ot.r=3&ot.rank=5&ot.q=dummy",
            "sem_url_action":"http://localhost:8091/morelikethis?id=Nw&ot.et=mlt&ot.est=mlt&ot.es=source_serp&ot.nf=13&ot.s=0&ot.r=3&ot.rank=5&ot.q=dummy"}},
        "sem_encoded_id":"Nw"}]
  },
  "facet_counts":{
    "facet_queries":{},
    "facet_fields":{
      "sem_category_names_root":{
        "Das ist los":{
          "value":"Das ist los",
          "count":2,
          "sem_action":{
            "sem_url":"?q=dummy&rows=3&indent=on&f.sem_category_names_root=sem_category_names_root%3ADas%5C+ist%5C+los",
            "sem_url_action":"?q=dummy&rows=3&indent=on&f.sem_category_names_root=sem_category_names_root%3ADas%5C+ist%5C+los&ot.et=search&ot.est=filter-modified&ot.es=source_serp"}},
        "Dies & Das":{
          "value":"Dies & Das",
          "count":2,
          "sem_action":{
            "sem_url":"?q=dummy&rows=3&indent=on&f.sem_category_names_root=sem_category_names_root%3ADies%5C+%5C%26%5C+Das",
            "sem_url_action":"?q=dummy&rows=3&indent=on&f.sem_category_names_root=sem_category_names_root%3ADies%5C+%5C%26%5C+Das&ot.et=search&ot.est=filter-modified&ot.es=source_serp"}},
        "Blaulicht":{
          "value":"Blaulicht",
          "count":1,
          "sem_action":{
            "sem_url":"?q=dummy&rows=3&indent=on&f.sem_category_names_root=sem_category_names_root%3ABlaulicht",
            "sem_url_action":"?q=dummy&rows=3&indent=on&f.sem_category_names_root=sem_category_names_root%3ABlaulicht&ot.et=search&ot.est=filter-modified&ot.es=source_serp"}},
        "Land & Leute":{
          "value":"Land & Leute",
          "count":1,
          "sem_action":{
            "sem_url":"?q=dummy&rows=3&indent=on&f.sem_category_names_root=sem_category_names_root%3ALand%5C+%5C%26%5C+Leute",
            "sem_url_action":"?q=dummy&rows=3&indent=on&f.sem_category_names_root=sem_category_names_root%3ALand%5C+%5C%26%5C+Leute&ot.et=search&ot.est=filter-modified&ot.es=source_serp"}},
        "Test Kategorie eins":{
          "value":"Test Kategorie eins",
          "count":1,
          "sem_action":{
            "sem_url":"?q=dummy&rows=3&indent=on&f.sem_category_names_root=sem_category_names_root%3ATest%5C+Kategorie%5C+eins",
            "sem_url_action":"?q=dummy&rows=3&indent=on&f.sem_category_names_root=sem_category_names_root%3ATest%5C+Kategorie%5C+eins&ot.et=search&ot.est=filter-modified&ot.es=source_serp"}},
        "Wirtschaft & Politik":{
          "value":"Wirtschaft & Politik",
          "count":1,
          "sem_action":{
            "sem_url":"?q=dummy&rows=3&indent=on&f.sem_category_names_root=sem_category_names_root%3AWirtschaft%5C+%5C%26%5C+Politik",
            "sem_url_action":"?q=dummy&rows=3&indent=on&f.sem_category_names_root=sem_category_names_root%3AWirtschaft%5C+%5C%26%5C+Politik&ot.et=search&ot.est=filter-modified&ot.es=source_serp"}}},
      "sem_author_facet":{
        "Timo":{
          "value":"Timo",
          "count":3,
          "sem_action":{
            "sem_url":"?q=dummy&rows=3&indent=on&f.sem_author_facet=sem_author_facet%3ATimo",
"sem_url_action":"?q=dummy&rows=3&indent=on&f.sem_author_facet=sem_author_facet%3ATimo&ot.et=search&ot.est=filter-modified&ot.es=source_serp"}},
        "Otis":{
          "value":"Otis",
          "count":2,
          "sem_action":{
            "sem_url":"?q=dummy&rows=3&indent=on&f.sem_author_facet=sem_author_facet%3AOtis",
            "sem_url_action":"?q=dummy&rows=3&indent=on&f.sem_author_facet=sem_author_facet%3AOtis&ot.et=search&ot.est=filter-modified&ot.es=source_serp"}}},
      "sem_category_main":[],
      "sem_record_type":{
        "News":{
          "value":"News",
          "count":3,
          "sem_action":{
            "sem_url":"?q=dummy&rows=3&indent=on&f.sem_record_type=sem_record_type%3ANews",
            "sem_url_action":"?q=dummy&rows=3&indent=on&f.sem_record_type=sem_record_type%3ANews&ot.et=search&ot.est=filter-modified&ot.es=source_serp"}},
        "Bildergalerie":{
          "value":"Bildergalerie",
          "count":2,
          "sem_action":{
            "sem_url":"?q=dummy&rows=3&indent=on&f.sem_record_type=sem_record_type%3ABildergalerie",
            "sem_url_action":"?q=dummy&rows=3&indent=on&f.sem_record_type=sem_record_type%3ABildergalerie&ot.et=search&ot.est=filter-modified&ot.es=source_serp"}},
        "Event":{
          "value":"Event",
          "count":1,
          "sem_action":{
            "sem_url":"?q=dummy&rows=3&indent=on&f.sem_record_type=sem_record_type%3AEvent",
            "sem_url_action":"?q=dummy&rows=3&indent=on&f.sem_record_type=sem_record_type%3AEvent&ot.et=search&ot.est=filter-modified&ot.es=source_serp"}},
        "Gewinnspiel":{
          "value":"Gewinnspiel",
          "count":1,
          "sem_action":{
"sem_url":"?q=dummy&rows=3&indent=on&f.sem_record_type=sem_record_type%3AGewinnspiel",
            "sem_url_action":"?q=dummy&rows=3&indent=on&f.sem_record_type=sem_record_type%3AGewinnspiel&ot.et=search&ot.est=filter-modified&ot.es=source_serp"}},
        "TipsOffice":{
          "value":"TipsOffice",
          "count":1,
          "sem_action":{
            "sem_url":"?q=dummy&rows=3&indent=on&f.sem_record_type=sem_record_type%3ATipsOffice",
            "sem_url_action":"?q=dummy&rows=3&indent=on&f.sem_record_type=sem_record_type%3ATipsOffice&ot.et=search&ot.est=filter-modified&ot.es=source_serp"}},
        "TipsPerson":{
          "value":"TipsPerson",
          "count":1,
          "sem_action":{
            "sem_url":"?q=dummy&rows=3&indent=on&f.sem_record_type=sem_record_type%3ATipsPerson",
            "sem_url_action":"?q=dummy&rows=3&indent=on&f.sem_record_type=sem_record_type%3ATipsPerson&ot.et=search&ot.est=filter-modified&ot.es=source_serp"}},
        "User":{
          "value":"User",
          "count":1,
          "sem_action":{
            "sem_url":"?q=dummy&rows=3&indent=on&f.sem_record_type=sem_record_type%3AUser",
            "sem_url_action":"?q=dummy&rows=3&indent=on&f.sem_record_type=sem_record_type%3AUser&ot.et=search&ot.est=filter-modified&ot.es=source_serp"}},
        "Webpage":{
          "value":"Webpage",
          "count":1,
          "sem_action":{
            "sem_url":"?q=dummy&rows=3&indent=on&f.sem_record_type=sem_record_type%3AWebpage",
"sem_url_action":"?q=dummy&rows=3&indent=on&f.sem_record_type=sem_record_type%3AWebpage&ot.et=search&ot.est=filter-modified&ot.es=source_serp"}},
        "Zeitung":{
          "value":"Zeitung",
          "count":1,
          "sem_action":{
            "sem_url":"?q=dummy&rows=3&indent=on&f.sem_record_type=sem_record_type%3AZeitung",
            "sem_url_action":"?q=dummy&rows=3&indent=on&f.sem_record_type=sem_record_type%3AZeitung&ot.et=search&ot.est=filter-modified&ot.es=source_serp"}},
        "ZeitungSeite":{
          "value":"ZeitungSeite",
          "count":1,
          "sem_action":{
            "sem_url":"?q=dummy&rows=3&indent=on&f.sem_record_type=sem_record_type%3AZeitungSeite",
            "sem_url_action":"?q=dummy&rows=3&indent=on&f.sem_record_type=sem_record_type%3AZeitungSeite&ot.et=search&ot.est=filter-modified&ot.es=source_serp"}}},
      "sem_grouping":{
        "Kontakt":{
          "value":"Kontakt",
          "count":3,
          "sem_action":{
            "sem_url":"?q=dummy&rows=3&indent=on&f.sem_grouping=sem_grouping%3AKontakt",
            "sem_url_action":"?q=dummy&rows=3&indent=on&f.sem_grouping=sem_grouping%3AKontakt&ot.et=search&ot.est=filter-modified&ot.es=source_serp"}},
        "News":{
          "value":"News",
          "count":3,
          "sem_action":{
            "sem_url":"?q=dummy&rows=3&indent=on&f.sem_grouping=sem_grouping%3ANews",
            "sem_url_action":"?q=dummy&rows=3&indent=on&f.sem_grouping=sem_grouping%3ANews&ot.et=search&ot.est=filter-modified&ot.es=source_serp"}},
        "Bildergalerie":{
          "value":"Bildergalerie",
          "count":2,
          "sem_action":{
"sem_url":"?q=dummy&rows=3&indent=on&f.sem_grouping=sem_grouping%3ABildergalerie",
            "sem_url_action":"?q=dummy&rows=3&indent=on&f.sem_grouping=sem_grouping%3ABildergalerie&ot.et=search&ot.est=filter-modified&ot.es=source_serp"}},
        "Zeitung":{
          "value":"Zeitung",
          "count":2,
          "sem_action":{
            "sem_url":"?q=dummy&rows=3&indent=on&f.sem_grouping=sem_grouping%3AZeitung",
            "sem_url_action":"?q=dummy&rows=3&indent=on&f.sem_grouping=sem_grouping%3AZeitung&ot.et=search&ot.est=filter-modified&ot.es=source_serp"}},
        "Event":{
          "value":"Event",
          "count":1,
          "sem_action":{
"sem_url":"?q=dummy&rows=3&indent=on&f.sem_grouping=sem_grouping%3AEvent",
            "sem_url_action":"?q=dummy&rows=3&indent=on&f.sem_grouping=sem_grouping%3AEvent&ot.et=search&ot.est=filter-modified&ot.es=source_serp"}},
        "Gewinnspiel":{
          "value":"Gewinnspiel",
          "count":1,
          "sem_action":{
            "sem_url":"?q=dummy&rows=3&indent=on&f.sem_grouping=sem_grouping%3AGewinnspiel",
            "sem_url_action":"?q=dummy&rows=3&indent=on&f.sem_grouping=sem_grouping%3AGewinnspiel&ot.et=search&ot.est=filter-modified&ot.es=source_serp"}},
        "Webpage":{
          "value":"Webpage",
          "count":1,
          "sem_action":{
"sem_url":"?q=dummy&rows=3&indent=on&f.sem_grouping=sem_grouping%3AWebpage",
            "sem_url_action":"?q=dummy&rows=3&indent=on&f.sem_grouping=sem_grouping%3AWebpage&ot.et=search&ot.est=filter-modified&ot.es=source_serp"}}}},
    "facet_dates":{},
    "facet_ranges":{},
    "facet_intervals":{}
  },
  "spelling":[],
  "active_filters":{
    "sem_action":{
      "sem_url":"?q=dummy&rows=3&indent=on",
      "sem_url_action":"?q=dummy&rows=3&indent=on&ot.et=search&ot.est=filter-modified&ot.es=source_serp"}
  },
  "base_urls":{
    "paging":{
      "sem_url":"?q=dummy&indent=on",
      "sem_url_action":"?q=dummy&indent=on&ot.et=search&ot.est=paging&ot.es=source_serp"},
    "search_with_current_state":{
      "sem_url":"?rows=3&indent=on",
      "sem_url_action":"?rows=3&indent=on&ot.et=search&ot.est=normal&ot.es=source_serp"},
    "search_with_empty_state":{
      "sem_url":"?rows=3&indent=on",
      "sem_url_action":"?rows=3&indent=on&ot.et=search&ot.est=normal&ot.es=source_serp"},
    "remove_all_filters":{
      "sem_url":"?q=dummy&rows=3&indent=on",
      "sem_url_action":"?q=dummy&rows=3&indent=on&ot.et=search&ot.est=filter-modified&ot.es=source_serp"},
    "remove_source_filters":{
      "sem_url":"?q=dummy&rows=3&indent=on",
      "sem_url_action":"?q=dummy&rows=3&indent=on&ot.et=search&ot.est=filter-modified&ot.es=source_serp"}
  },
  "paging":{
    "sem_generic_urls":{
      "sem_url":"?q=dummy&indent=on",
      "sem_url_action":"?q=dummy&indent=on&ot.et=search&ot.est=paging&ot.es=source_serp"}
  },
  "responseHeader":{
    "QTime":42,
    "solrQTime":2,
    "status":0,
    "solrStatus":0,
    "params":{
      "q":"dummy",
      "rows":"3",
      "indent":"on",
      "json.nl":"arrmap"
    }
  }
}

A.2. Full auto-complete response

Example of a full autocomplete response (for version 1.3)

{
  "response":{
    "autocomplete":{
      "suggestions":[{
          "suggestion":"dummy",
          "hits":"13",
          "sem_action":{
            "sem_url":"?q=dummy",
            "sem_url_action":"?q=dummy&ot.et=search&ot.est=suggest-term-select&ot.es=source_autocomplete&ot.orig&ot.sg=dummy&ot.sgidx=0"}},
        {
          "suggestion":"dumme",
          "hits":"1",
          "sem_action":{
            "sem_url":"?q=dumme",
            "sem_url_action":"?q=dumme&ot.et=search&ot.est=suggest-term-select&ot.es=source_autocomplete&ot.orig&ot.sg=dumme&ot.sgidx=1"}}]},
    "prefetchedResults":{
      "term":"dummy",
      "grouped":{
        "sem_grouping":{
          "matches":13,
          "groups":[{
              "groupValue":"Webpage",
              "doclist":{"numFound":1,"start":0,"docs":[
                  {
                    "id":"10",
                    "sem_title":"Dummy Webpage",
                    "sem_url":"http://www.tips.at/10",
                    "sem_actions":{
                      "default":"goto",
                      "goto":{
                        "sem_url":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F10&id=MTA&ot.et=result&ot.est=suggest-result-click&ot.es=source_autocomplete&ot.orig=dumm&ot.sg=dummy&ot.sgidx=0&ot.rank=0&ot.tot=0",
                        "sem_url_action":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F10&id=MTA&ot.et=result&ot.est=suggest-result-click&ot.es=source_autocomplete&ot.orig=dumm&ot.sg=dummy&ot.sgidx=0&ot.rank=0&ot.tot=0",
                        "sem_is_external":true}},
                    "sem_encoded_id":"MTA"}]
              },
              "sem_actions":{
                "default":"filterby",
                "filterby":{
                  "sem_url":"?indent=on&f.sem_grouping=sem_grouping%3AWebpage&q=dummy",
                  "sem_url_action":"?indent=on&f.sem_grouping=sem_grouping%3AWebpage&q=dummy&ot.et=search&ot.est=suggest-category-select&ot.es=source_autocomplete&ot.orig=dumm"}}},
            {
              "groupValue":"News",
              "doclist":{"numFound":3,"start":0,"docs":[
                  {
                    "id":"11",
                    "sem_title":"Dummy News beta",
                    "sem_url":"http://www.tips.at/11",
                    "sem_actions":{
                      "default":"goto",
                      "goto":{
"sem_url":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F11&id=MTE&ot.et=result&ot.est=suggest-result-click&ot.es=source_autocomplete&ot.orig=dumm&ot.sg=dummy&ot.sgidx=0&ot.rank=0&ot.tot=0",
                        "sem_url_action":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F11&id=MTE&ot.et=result&ot.est=suggest-result-click&ot.es=source_autocomplete&ot.orig=dumm&ot.sg=dummy&ot.sgidx=0&ot.rank=0&ot.tot=0",
                        "sem_is_external":true}},
                    "sem_encoded_id":"MTE"},
                  {
                    "id":"1",
                    "sem_title":"Dummy News alpha",
                    "sem_url":"http://www.tips.at/1",
                    "sem_actions":{
                      "default":"goto",
                      "goto":{
                        "sem_url":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F1&id=MQ&ot.et=result&ot.est=suggest-result-click&ot.es=source_autocomplete&ot.orig=dumm&ot.sg=dummy&ot.sgidx=0&ot.rank=1&ot.tot=0",
                        "sem_url_action":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F1&id=MQ&ot.et=result&ot.est=suggest-result-click&ot.es=source_autocomplete&ot.orig=dumm&ot.sg=dummy&ot.sgidx=0&ot.rank=1&ot.tot=0",
                        "sem_is_external":true}},
                    "sem_encoded_id":"MQ"},
                  {
                    "id":"50",
                    "sem_title":"Dummy News alpha",
                    "sem_url":"http://www.tips.at/1",
                    "sem_actions":{
                      "default":"goto",
                      "goto":{
                        "sem_url":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F1&id=NTA&ot.et=result&ot.est=suggest-result-click&ot.es=source_autocomplete&ot.orig=dumm&ot.sg=dummy&ot.sgidx=0&ot.rank=2&ot.tot=0",
                        "sem_url_action":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F1&id=NTA&ot.et=result&ot.est=suggest-result-click&ot.es=source_autocomplete&ot.orig=dumm&ot.sg=dummy&ot.sgidx=0&ot.rank=2&ot.tot=0",
                        "sem_is_external":true}},
                    "sem_encoded_id":"NTA"}]
              },
              "sem_actions":{
                "default":"filterby",
                "filterby":{
                  "sem_url":"?indent=on&f.sem_grouping=sem_grouping%3ANews&q=dummy",
                  "sem_url_action":"?indent=on&f.sem_grouping=sem_grouping%3ANews&q=dummy&ot.et=search&ot.est=suggest-category-select&ot.es=source_autocomplete&ot.orig=dumm"}}},
            {
              "groupValue":"Kontakt",
              "doclist":{"numFound":3,"start":0,"docs":[
                  {
                    "id":"7",
                    "sem_title":"Dummy User",
                    "sem_url":"http://www.tips.at/7",
                    "sem_actions":{
                      "default":"goto",
                      "goto":{
                        "sem_url":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F7&id=Nw&ot.et=result&ot.est=suggest-result-click&ot.es=source_autocomplete&ot.orig=dumm&ot.sg=dummy&ot.sgidx=0&ot.rank=0&ot.tot=0",
                        "sem_url_action":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F7&id=Nw&ot.et=result&ot.est=suggest-result-click&ot.es=source_autocomplete&ot.orig=dumm&ot.sg=dummy&ot.sgidx=0&ot.rank=0&ot.tot=0",
                        "sem_is_external":true}},
                    "sem_encoded_id":"Nw"},
                  {
                    "id":"8",
                    "sem_title":"Dummy Tips Geschäftsstelle",
                    "sem_url":"http://www.tips.at/8",
                    "sem_actions":{
                      "default":"goto",
                      "goto":{
                        "sem_url":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F8&id=OA&ot.et=result&ot.est=suggest-result-click&ot.es=source_autocomplete&ot.orig=dumm&ot.sg=dummy&ot.sgidx=0&ot.rank=1&ot.tot=0",
                        "sem_url_action":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F8&id=OA&ot.et=result&ot.est=suggest-result-click&ot.es=source_autocomplete&ot.orig=dumm&ot.sg=dummy&ot.sgidx=0&ot.rank=1&ot.tot=0",
                        "sem_is_external":true}},
                    "sem_encoded_id":"OA"},
                  {
                    "id":"9",
                    "sem_title":"Dummy Tips Person",
                    "sem_url":"http://www.tips.at/9",
                    "sem_actions":{
                      "default":"goto",
                      "goto":{
                        "sem_url":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F9&id=OQ&ot.et=result&ot.est=suggest-result-click&ot.es=source_autocomplete&ot.orig=dumm&ot.sg=dummy&ot.sgidx=0&ot.rank=2&ot.tot=0",
          "sem_url_action":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F9&id=OQ&ot.et=result&ot.est=suggest-result-click&ot.es=source_autocomplete&ot.orig=dumm&ot.sg=dummy&ot.sgidx=0&ot.rank=2&ot.tot=0",
                        "sem_is_external":true}},
                    "sem_encoded_id":"OQ"}]
              },
              "sem_actions":{
                "default":"filterby",
                "filterby":{
                  "sem_url":"?indent=on&f.sem_grouping=sem_grouping%3AKontakt&q=dummy",
                  "sem_url_action":"?indent=on&f.sem_grouping=sem_grouping%3AKontakt&q=dummy&ot.et=search&ot.est=suggest-category-select&ot.es=source_autocomplete&ot.orig=dumm"}}},
            {
              "groupValue":"Bildergalerie",
              "doclist":{"numFound":2,"start":0,"docs":[
                  {
                    "id":"2",
                    "sem_title":"Dummy Bildergalerie eins",
                    "sem_url":"http://www.tips.at/2",
                    "sem_actions":{
                      "default":"goto",
                      "goto":{
                        "sem_url":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F2&id=Mg&ot.et=result&ot.est=suggest-result-click&ot.es=source_autocomplete&ot.orig=dumm&ot.sg=dummy&ot.sgidx=0&ot.rank=0&ot.tot=0",
                  "sem_url_action":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F2&id=Mg&ot.et=result&ot.est=suggest-result-click&ot.es=source_autocomplete&ot.orig=dumm&ot.sg=dummy&ot.sgidx=0&ot.rank=0&ot.tot=0",
                        "sem_is_external":true}},
                    "sem_encoded_id":"Mg"},
                  {
                    "id":"12",
                    "sem_title":"Dummy Bildergalerie zwei",
                    "sem_url":"http://www.tips.at/12",
                    "sem_actions":{
                      "default":"goto",
                      "goto":{
                        "sem_url":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F12&id=MTI&ot.et=result&ot.est=suggest-result-click&ot.es=source_autocomplete&ot.orig=dumm&ot.sg=dummy&ot.sgidx=0&ot.rank=1&ot.tot=0",
                        "sem_url_action":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F12&id=MTI&ot.et=result&ot.est=suggest-result-click&ot.es=source_autocomplete&ot.orig=dumm&ot.sg=dummy&ot.sgidx=0&ot.rank=1&ot.tot=0",
                        "sem_is_external":true}},
                    "sem_encoded_id":"MTI"}]
              },
              "sem_actions":{
                "default":"filterby",
                "filterby":{
                  "sem_url":"?indent=on&f.sem_grouping=sem_grouping%3ABildergalerie&q=dummy",
                  "sem_url_action":"?indent=on&f.sem_grouping=sem_grouping%3ABildergalerie&q=dummy&ot.et=search&ot.est=suggest-category-select&ot.es=source_autocomplete&ot.orig=dumm"}}},
            {
              "groupValue":"Event",
              "doclist":{"numFound":1,"start":0,"docs":[
                  {
                    "id":"5",
                    "sem_title":"Dummy Event",
                    "sem_url":"http://www.tips.at/5",
                    "sem_actions":{
                      "default":"goto",
                      "goto":{
                        "sem_url":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F5&id=NQ&ot.et=result&ot.est=suggest-result-click&ot.es=source_autocomplete&ot.orig=dumm&ot.sg=dummy&ot.sgidx=0&ot.rank=0&ot.tot=0",
                        "sem_url_action":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F5&id=NQ&ot.et=result&ot.est=suggest-result-click&ot.es=source_autocomplete&ot.orig=dumm&ot.sg=dummy&ot.sgidx=0&ot.rank=0&ot.tot=0",
                        "sem_is_external":true}},
                    "sem_encoded_id":"NQ"}]
              },
              "sem_actions":{
                "default":"filterby",
                "filterby":{
                  "sem_url":"?indent=on&f.sem_grouping=sem_grouping%3AEvent&q=dummy",
                  "sem_url_action":"?indent=on&f.sem_grouping=sem_grouping%3AEvent&q=dummy&ot.et=search&ot.est=suggest-category-select&ot.es=source_autocomplete&ot.orig=dumm"}}},
            {
              "groupValue":"Gewinnspiel",
              "doclist":{"numFound":1,"start":0,"docs":[
                  {
                    "id":"6",
                    "sem_title":"Dummy Gewinnspiel",
                    "sem_url":"http://www.tips.at/6",
                    "sem_actions":{
                      "default":"goto",
                      "goto":{
                        "sem_url":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F6&id=Ng&ot.et=result&ot.est=suggest-result-click&ot.es=source_autocomplete&ot.orig=dumm&ot.sg=dummy&ot.sgidx=0&ot.rank=0&ot.tot=0",
                        "sem_url_action":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F6&id=Ng&ot.et=result&ot.est=suggest-result-click&ot.es=source_autocomplete&ot.orig=dumm&ot.sg=dummy&ot.sgidx=0&ot.rank=0&ot.tot=0",
                        "sem_is_external":true}},
                    "sem_encoded_id":"Ng"}]
              },
              "sem_actions":{
                "default":"filterby",
                "filterby":{
                  "sem_url":"?indent=on&f.sem_grouping=sem_grouping%3AGewinnspiel&q=dummy",
                  "sem_url_action":"?indent=on&f.sem_grouping=sem_grouping%3AGewinnspiel&q=dummy&ot.et=search&ot.est=suggest-category-select&ot.es=source_autocomplete&ot.orig=dumm"}}},
            {
              "groupValue":"Zeitung",
              "doclist":{"numFound":2,"start":0,"docs":[
                  {
                    "id":"3",
                    "sem_title":"Dummy Zeitung",
                    "sem_url":"http://www.tips.at/3",
                    "sem_actions":{
                      "default":"goto",
                      "goto":{
                        "sem_url":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F3&id=Mw&ot.et=result&ot.est=suggest-result-click&ot.es=source_autocomplete&ot.orig=dumm&ot.sg=dummy&ot.sgidx=0&ot.rank=0&ot.tot=0",
                        "sem_url_action":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F3&id=Mw&ot.et=result&ot.est=suggest-result-click&ot.es=source_autocomplete&ot.orig=dumm&ot.sg=dummy&ot.sgidx=0&ot.rank=0&ot.tot=0",
                        "sem_is_external":true}},
                    "sem_encoded_id":"Mw"},
                  {
                    "id":"4",
                    "sem_title":"Dummy Zeitungsseite",
                    "sem_url":"http://www.tips.at/4",
                    "sem_actions":{
                      "default":"goto",
                      "goto":{
                        "sem_url":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F4&id=NA&ot.et=result&ot.est=suggest-result-click&ot.es=source_autocomplete&ot.orig=dumm&ot.sg=dummy&ot.sgidx=0&ot.rank=1&ot.tot=0",
                        "sem_url_action":"http://localhost:8091/redirect?url=http%3A%2F%2Fwww.tips.at%2F4&id=NA&ot.et=result&ot.est=suggest-result-click&ot.es=source_autocomplete&ot.orig=dumm&ot.sg=dummy&ot.sgidx=0&ot.rank=1&ot.tot=0",
                        "sem_is_external":true}},
                    "sem_encoded_id":"NA"}]
              },
              "sem_actions":{
                "default":"filterby",
                "filterby":{
                  "sem_url":"?indent=on&f.sem_grouping=sem_grouping%3AZeitung&q=dummy",
"sem_url_action":"?indent=on&f.sem_grouping=sem_grouping%3AZeitung&q=dummy&ot.et=search&ot.est=suggest-category-select&ot.es=source_autocomplete&ot.orig=dumm"}}}]}}}},
  "responseHeader":{
    "QTime":90,
    "solrQTime":3,
    "status":0,
    "solrStatus":0,
    "params":{
      "q":"dumm",
      "indent":"on",
      "json.nl":"arrmap"}}}

A.3. Full more like this response

Example of a full more like this response (for version 1.3)

{
  "response":{"numFound":1,"start":0,"docs":[
      {
        "id":"312061",
        "sem_source_id":"www.tips.at",
        "sem_record_type":"news",
        "sem_url":"http://www.tips.at//news/gmuend/das-ist-los/312061-niedetzky-marschiert"}]
  },
  "moreLikeThis":{
    "312061":{"numFound":995,"start":0,"docs":[
        {
          "id":"312201",
          "sem_source_id":"www.tips.at",
          "sem_record_type":"news",
          "sem_url":"http://www.tips.at//news/steyr/land-leute/312201-kabarett-severin-groebner-abgesagt"},
        {
          "id":"311983",
          "sem_source_id":"www.tips.at",
          "sem_record_type":"news",
          "sem_url":"http://www.tips.at//news/waidhofen-an-der-thaya/das-ist-los/311983-von-den-tuecken-des-hausmannlebens"},
        {
          "id":"312380",
          "sem_source_id":"www.tips.at",
          "sem_record_type":"news",
          "sem_url":"http://www.tips.at//news/steyr/das-ist-los/312380-elke-winkens-putz-dich"},
        {
          "id":"311947",
          "sem_source_id":"www.tips.at",
          "sem_record_type":"news",
          "sem_url":"http://www.tips.at//news/steyr/das-ist-los/311947-weinzettl-rudle-im-stadtsaal"},
        {
          "id":"312444",
          "sem_source_id":"www.tips.at",
          "sem_record_type":"news",
          "sem_url":"http://www.tips.at//news/freistadt/land-leute/312444-klangmaerchen-an-der-volksschule"}
      ]
    }
  },
  "responseHeader":{
    "QTime":0,
    "solrQTime":0,
    "status":0,
    "solrStatus":0,
    "params":{
      "id":"MzEyMDYx",
      "indent":"on",
      "json.nl":"arrmap"
    }
  }
}

Appendix B: Semanteer configuration file: search-layer-configurations.json

Code example

 {
 "collections": {
  "empty": {
   "solrUrl": "http://localhost:8035/solr/empty",
   "monitoringSolrUrl": "http://localhost:8035/solr/sem_monitoring_core_empty",
   "analyticsSolrUrl": "http://localhost:8035/solr/sem_monitoring_core_empty"
  },
  "wines": {
   "solrUrl": "http://localhost:8035/solr/winedemo",
   "monitoringSolrUrl": "http://localhost:8035/solr/sem_monitoring_core",
   "analyticsSolrUrl": "http://localhost:8035/solr/sem_monitoring_core"
  }
 },
 "properties": {
  "search.config.autocomplete": "autocomplete",
  "search.config.autocomplete.config": "autocomplete.ungrouped",
  "search.config.autocomplete.highlight": true,
  "search.config.autocomplete.lasttermquery": true,
  "search.config.autocomplete.maxsuggestions": 5,
  "search.config.autocomplete.response": "response",
  "search.config.autocomplete.prefetchedResults": "prefetchedResults",
  "search.config.autocomplete.prefetchResults": true,
  "search.config.autocomplete.sortsuggestions": "natural",
  "search.config.autocomplete.spellcheckresults": true,
  "search.config.autocomplete.suggestions": true,
  "search.config.caching.solr": false,
  "search.config.field.id": "id",
  "search.config.field.similarity": "sem_similarity",
  "search.config.field.thumbnail": "sem_image_thumbnail_url",
  "search.config.field.title": "sem_title",
  "search.config.field.url": "sem_url",
  "search.config.fields.exclude": "_version_",
  "search.config.fields.noncontent": "id,sem_image_thumbnail_url,sem_url,sem_title",
  "search.config.geoquery.parallel": false,
  "search.config.mlt.doctype.field": "sem_record_type",
  "search.config.mlt.similarityfields": "sem_text_search",
  "search.config.monitoring": true,
  "search.config.operator": "default",
  "search.config.operator.and.mincount": 1,
  "search.config.result.field.url": "sem_url",
  "search.config.suggest.prefix": "sem_text_suggest",
  "search.config.suggest.prefix.fq": "sem_record_type:wine",
  "search.config.suggester": "prefix,suggest_prefix"
 },
 "configurations": {
  "base": {
   "requestParams": {
    "appends": [{
     "name": "sph.fq",
     "value": "sem_record_type:\"wine\""
    }],
    "invariants": [{
     "name": "sph.rows",
     "value": "20"
    }]
   }
  },
  "base_geoquery_autofilter": {
   "extends": [{
    "configuration": "base",
    "mode": "merge"
   }],
   "components": [{
    "type": "AutofilterSearchLayerComponent",
    "autofilterApplied": true,
    "mode": "filter",
    "omitTermsMatchingMultipleFields": false,
    "omitUnkownFields": true,
    "applyFirstFieldMatchOnly": false,
    "mappings": {
     "vintage_facet": "vintage_hierachy"
    }
   }, {
    "type": "GeoQueryComponent",
    "qf": "sem_geo_title",
    "fq": "sem_record_type:sem_geo",
    "prefetch": true,
    "geoqueryApplied": true,
    "distErrPct": 0.01,
    "configurations": [{
     "default": {
      "shapefield": "sem_geo_object",
      "titlefield": "sem_geo_title",
      "searchfields": ["sem_location"]
     }
    }, {
     "field": "sem_record_type",
     "values": {
      "sem_geo": {
       "subtypes": {
        "field": "sem_record_subtype",
        "values": {
         "region": {
          "shapefield": "sem_geo_object",
          "mode": "filter"
         },
         "city": {
          "shapefield": "sem_geo_object",
          "mode": "filter",
          "params": [30.0]
         }
        }
       },
       "default": {
        "shapefield": "sem_geo_object",
        "mode": "filter",
        "params": [5.0]
       }
      }


     }
    }]
   }]
  },

  "autocomplete": {
   "extends": [{
    "configuration": "base_geoquery_autofilter",
    "mode": "merge"
   }],
   "requestParams": {
    "defaults": [{
     "name": "sph.fl",
     "value": "id, sem_title"
    }]
   }
  },
  "autocomplete.ungrouped": {
   "extends": [{
    "configuration": "autocomplete",
    "mode": "merge"
   }]
  },
  "autocomplete.grouped": {
   "extends": [{
    "configuration": "autocomplete",
    "mode": "merge"
   }],
   "components": [{
    "type": "GroupFieldComponent",
    "componentName": "color",
    "fieldName": "color_facet",
    "groupLimit": "5"
   }, {
    "type": "FacetFieldComponent",
    "componentName": "color",
    "fieldName": "color_facet"
   }]
  },

  "default": {
   "extends": [{
    "configuration": "base_geoquery_autofilter",
    "mode": "merge"
   }],
   "components": [{
     "type": "FacetPivotComponent",
     "componentName": "region",
     "fieldNames": [
      "country_facet",
      "region_facet",
      "wine_sub_region_facet"
     ],
     "uitype": "FilterHierarchyModule",
     "minCount": 1,
     "multiSelect": false,
     "sortBy": "index",
     "useExclusiveSetAction": true,
     "excludeOwnFilters": true
    }, {
     "type": "SortingComponent",
     "uitype": "SortingModule",
     "sortableFields": [
      "sem_title_alpha", "appellation", "vineyard"
     ]
    }, {
     "type": "IntegerRangeComponent",
     "componentName": "vintage",
     "fieldName": "vintage",
     "uitype": "RangeModule",
     "slider": true,
     "start": 1970,
     "end": 2010,
     "gap": 5,
     "prune": true,
     "multiSelect": true,
     "multiSelectMode": "OR",
     "excludeOwnFilters": true
    }, {
     "type": "FacetFieldComponent",
     "componentName": "color",
     "fieldName": "color_facet",
     "uitype": "FilterListWidget",
     "minCount": 1,
     "multiSelect": true,
     "multiSelectMode": "OR",
     "addMissingFilter": false,
     "excludeAllFilters": false,
     "excludeOwnFilters": true,
     "useExclusiveSetAction": true
    }, {
     "type": "FacetFieldComponent",
     "componentName": "vineyard",
     "fieldName": "vineyard_facet",
     "uitype": "FilterListWidget",
     "minCount": 1,
     "multiSelect": true,
     "excludeOwnFilters": true,
     "excludeAllFilters": false,
     "useExclusiveSetAction": false
    }, {
     "type": "FacetFieldComponent",
     "componentName": "wine_producer",
     "fieldName": "wine_producer_facet",
     "uitype": "FilterListWidget",
     "minCount": 1,
     "multiSelect": true,
     "excludeOwnFilters": false,
     "excludeAllFilters": false,
     "useExclusiveSetAction": false
    }, {
     "type": "FacetFieldComponent",
     "uitype": "FilterListWidget",
     "componentName": "wine_style",
     "minCount": 1,
     "fieldName": "wine_style_facet",
     "dependsOnAny": [{
      "component": "color",
      "value": "Red\\ Wine"
     }, {
      "component": "color",
      "value": "White\\ Wine"
     }]
    }, {
     "type": "FacetFieldComponent",
     "componentName": "wine_type",
     "fieldName": "wine_type_facet",
     "uitype": "FilterListWidget",
     "minCount": 1,
     "sortBy": "index",
     "multiSelect": false,
     "excludeOwnFilters": true,
     "excludeAllFilters": false,
     "useExclusiveSetAction": true
    }, {
     "type": "FloatRangeComponent",
     "componentName": "alcohol",
     "fieldName": "percentage_alcohol",
     "uitype": "RangeModule",
     "slider": true,
     "start": 10,
     "end": 20,
     "gap": 1,
     "prune": true,
     "minCount": 101,
     "multiSelect": false,
     "excludeOwnFilters": false,
     "excludeAllFilters": true
    }, {
     "type": "DateRangeComponent",
     "componentName": "date",
     "fieldName": "sem_timestamp",
     "minCount": 101,
     "gap": "+7DAY",
     "multiSelect": false
    },
    {
     "type": "FacetQueryComponent",
     "componentName": "custom_geoqueries",
     "queries": {
      "nearby": "{!geofilt pt=${pt} sfield=sem_location d=0.2}"
     },
     "excludeOwnFilters": true,
     "excludeAllFilters": false,
     "multiSelect": true,
     "multiSelectMode": "OR",
     "useExclusiveSetAction": true
    },
    {
     "type": "FacetQueryComponent",
     "componentName": "custom_queries",
     "queries": {
      "old_american": "country_facet:'United States of America' AND vintage:[0 TO 1999]",
      "open": "sem_opening_hours:\"Intersects(ENVELOPE(1501010000,${getNowAsOccurence()},2512122359,${getNowAsOccurence()}))\""
     },
     "excludeOwnFilters": true,
     "excludeAllFilters": false,
     "multiSelect": true,
     "multiSelectMode": "OR",
     "addNegations": true,
     "useExclusiveSetAction": true
    }
   ],
   "preparedQueries": {
    "oldwines": [{
     "name": "q",
     "value": "*:*"
    }, {
     "name": "sph.fq",
     "value": "vintage:[* TO 1995]"
    }, {
     "name": "rows",
     "value": "10"
    }]
   }
  },
  "grouped": {
   "extends": [{
    "configuration": "default",
    "mode": "merge"
   }],
   "components": [{
    "type": "GroupFieldComponent",
    "componentName": "color",
    "fieldName": "color_facet",
    "groupLimit": 3,
    "doc": true,
    "docField": "id",
    "docFilter": true,
    "docFq": "sem_record_type:color",
    "docExclude": false
   }]
  },
  "map": {
   "extends": [{
    "configuration": "base_geoquery_autofilter",
    "mode": "merge"
   }],
   "requestParams": {
    "defaults": [{
     "name": "rows",
     "value": "1000"
    }, {
     "name": "sph.fl",
     "value": "id, sem_title, sem_location"
    }],
    "appends": [{
     "name": "sph.fq",
     "value": "sem_location:*"
    }]
   }

  },
  "minimal": {
   "components": [{
    "type": "FacetFieldComponent",
    "componentName": "color",
    "fieldName": "color_facet",
    "uitype": "FilterListWidget",
    "minCount": 1
   }]
  },
  "hierachical": {
   "extends": [{
    "configuration": "base_geoquery_autofilter",
    "mode": "merge"
   }],
   "components": [{
    "type": "FacetHierarchyComponent",
    "componentName": "vintage_hierachy",
    "fieldName": "vintage_hierachy",
    "divider": ":",
    "minCount": 1,
    "multiSelect": true,
    "multiSelectMode": "AND",
    "excludeOwnFilters": false,
    "filterFirstLevel": true
   }]
  },
  "redwines": {
   "requestParams": {
    "defaults": [{
     "name": "rows",
     "value": "100"
    }, {
     "name": "sph.fq",
     "value": "color_facet:\"Red Wine\""
    }, {
     "name": "sph.fl",
     "value": "sem_title, country"
    }]
   }
  }
 }
}

Monitoring and Search Analytics

Redirect URLs

When monitoring is activated in a search installation, then the search server creates “URLs” for external links. These URLs point to the search server, and include the target URL as a parameter, along with the necessary monitoring parameters as well. This convention is necessary to ensure that browser transitions from the frontend’s search results page to other pages are properly monitored and logged.

© Contexity. All rights reserved.