Designing precise queries across disciplines

Field tags decide where to look

A field tag restricts a query to part of a record. scopus_field_tags() lists the common ones.

scopus_field_tags()

The most generally useful tag is TITLE-ABS-KEY, which searches the title, abstract and keywords together, broad enough to catch a topic without the noise of a full-text match.

One term, many disciplines

The same builder serves any field. Each call below returns the exact query string that would be sent to ‘Scopus’.

# molecular biology
scopus_query("CRISPR", .field = "TITLE-ABS-KEY")

[1] "TITLE-ABS-KEY(CRISPR)"

scopus_query("gravitational waves", .field = "TITLE-ABS-KEY") # physics

[1] "TITLE-ABS-KEY(gravitational waves)"

# environmental science
scopus_query("microplastics", .field = "TITLE-ABS-KEY")

[1] "TITLE-ABS-KEY(microplastics)"

scopus_query("blockchain", .field = "TITLE-ABS-KEY")          # computer science

[1] "TITLE-ABS-KEY(blockchain)"

scopus_query("digital humanities", .field = "AUTHKEY")        # humanities

[1] "AUTHKEY(digital humanities)"

The last example uses AUTHKEY, the author-supplied keywords, which isolates work that self-identifies with a field and so cuts incidental mentions.

Combining terms with boolean operators

Passing several terms joins them. The default operator is AND, and OR or AND NOT are available through .op.

# Two concepts that must co-occur (materials science).
scopus_query("perovskite", "solar cell", .field = "TITLE-ABS-KEY")

[1] "TITLE-ABS-KEY(perovskite) AND TITLE-ABS-KEY(solar cell)"

# Two names for one condition, either of which will do (medicine). writeLines()
# shows the quotation marks without the backslashes R prints to escape them.
writeLines(scopus_query('"heart attack"', '"myocardial infarction"', .op = "OR"))

"heart attack" OR "myocardial infarction"

# A family of related tools (molecular biology).
scopus_query("CRISPR", "Cas9", "Cas12", .op = "OR")

[1] "CRISPR OR Cas9 OR Cas12"

# Excluding a dominant homonym, so the snake stays out of the results.
scopus_query("python", "snake", .op = "AND NOT", .field = "TITLE-ABS-KEY")

[1] "TITLE-ABS-KEY(python) AND NOT TITLE-ABS-KEY(snake)"

Phrases

‘Scopus’ joins the words of an unquoted term with AND, so heart attack finds records that mention both words anywhere in the searched field. Double quotation marks ask for a loose phrase, whose words must sit next to each other, although punctuation is ignored and plurals are included. Braces ask for an exact phrase, including any stop words, spaces and punctuation. Elsevier’s search tips describe all three.

writeLines(scopus_query('"heart attack"', .field = "TITLE-ABS-KEY")) # loose phrase

TITLE-ABS-KEY("heart attack")

writeLines(scopus_query("{heart attack}", .field = "TITLE-ABS-KEY")) # exact phrase

TITLE-ABS-KEY({heart attack})

scopus_query() leaves both kinds of phrase as written. When terms are joined without a field tag, a term of several unquoted words is put in brackets, so it stays one operand of the join. Joined bare, the first query below would read as machine AND (learning OR deep) AND learning.

writeLines(scopus_query("machine learning", "deep learning", .op = "OR"))

(machine learning) OR (deep learning)

writeLines(scopus_query('"machine learning"', '"deep learning"', .op = "OR"))

"machine learning" OR "deep learning"

Brackets in a query written by hand

A query that mixes operators needs brackets to say what it means. The search tips state that ‘Scopus’ applies OR first, then AND, then AND NOT, so A AND NOT B AND C reads as A AND NOT (B AND C). Elsevier has announced a new order, with AND NOT first, then AND, then OR (Feldner, 2025), under which A OR B AND C reads as A OR (B AND C). Brackets give the same reading under either order.

The package brackets the strings it composes. A query built by an earlier scopus_query() call keeps its own join together when it is joined again, as below, and scopus_compare_topics() and scopus_intersections() bracket each side of their joins. The year limit of a plan, a count or a trend never enters the query string, since it travels as a separate request parameter.

young <- scopus_query("children", "adolescents", .op = "OR")
scopus_query("vaccine", young)

[1] "vaccine AND (children OR adolescents)"

From a query to a plan

A composed query drops straight into the rest of the workflow. Here it anchors a year-partitioned plan, which keeps each cell under the API’s 5000-record ceiling.

q <- scopus_query("gut microbiome", "immunology", .field = "TITLE-ABS-KEY")
q

[1] "TITLE-ABS-KEY(gut microbiome) AND TITLE-ABS-KEY(immunology)"

plan <- scopus_plan(q, years = 2015:2022, partition = "year")
plan

The plan is ready to size and run, which contacts the API.

scopus_count(q, years = 2015:2022)
records <- scopus_fetch_plan(plan)

Without a key, vignette("scopusflow") carries the same two steps through to a record set, standing the bundled corpus of real articles in for the harvest, since ‘Scopus’ records may not be redistributed.

Searching by affiliation

Field tags reach beyond topics. AFFILORG searches the affiliation, which turns a query into an institution-level view of output.

scopus_query("Max Planck", .field = "AFFILORG")

[1] "AFFILORG(Max Planck)"

When a term is empty

The builder validates its input, so a stray empty term is caught at construction, well before it can produce a malformed query.

tryCatch(
  scopus_query("graphene", ""),
  scopus_error_bad_input = function(e) conditionMessage(e)
)

[1] "`...` must be one or more non-empty character terms."

Checks before any request

scopus_plan(), scopus_count(), scopus_fetch(), scopus_trend(), scopus_compare_topics() and scopus_intersections() check each query offline before anything is sent, and refuse three mistakes. Refinement syntax copied from the web interface, such as LIMIT-TO(DOCTYPE, "ar"), is ignored by the Search API, so the search would run unfiltered with no sign of a problem. A field tag wrapped around a query that already opens with one is rejected by the API. A bracket, quotation mark or brace left unclosed changes what the rest of the query means. Each raises scopus_error_bad_input with a message saying what to write instead.

tryCatch(
  scopus_plan('working memory AND (LIMIT-TO (DOCTYPE, "ar"))'),
  scopus_error_bad_input = function(e) writeLines(strwrap(conditionMessage(e), 76))
)

The query contains LIMIT-TO() or EXCLUDE(), which belong to the Scopus web
interface's refinement panel. The Search API does not apply them, so the
search would run unfiltered (pybliometrics documents LIMIT-TO() as
affecting the display only). Write the limit as a search field instead,
such as DOCTYPE(ar), LANGUAGE(english), SUBJAREA(PSYC), SRCTYPE(j) or
PUBYEAR > 2015.

Written as search fields, the same limits pass the check.

scopus_plan("working memory AND DOCTYPE(ar) AND LANGUAGE(english)")$query

[1] "working memory AND DOCTYPE(ar) AND LANGUAGE(english)"

A strategy pasted from the advanced search with its own tags, such as TITLE-ABS-KEY ( "working memory" ), is sent as written when field is left as NULL. INDEXTERMS(), which pybliometrics documents as not working through the Search API, raises a warning, and KEY() searches the index terms together with author keywords, trade names and chemical names.

Checking recall against known records

A search strategy can be tested against records already known to be relevant, for instance the studies included in an earlier review or those named by experts, collected without the strategy’s help. The share of them the strategy retrieves is its relative recall (Sampson et al., 2006), and Bramer et al. (2018) make the check a step in developing a search. It takes two searches per chain of DOIs, run under the same year limit. The chain alone finds which known records ‘Scopus’ indexes, and the strategy joined to the chain finds which of those it retrieves. A known record the chain does not find, because ‘Scopus’ does not index it or holds it without that DOI, cannot show whether the strategy finds it. It is left out of the denominator and listed, so it can be checked by hand.

The sketch below contacts the API, so it is not run here. The DOIs are placeholders for your own, and scopus_extract_dois() cleans them first, so a resolver prefix or a difference in case does no harm. Chains of at most 25 DOIs keep each request short. How the API treats much longer chains has not been checked.

known <- scopus_extract_dois(
  c("10.xxxx/known-1", "https://doi.org/10.xxxx/known-2", "doi:10.xxxx/known-3")
)
strategy <- scopus_query("working memory", "training", .field = "TITLE-ABS-KEY")
years <- 2010:2024

chains <- vapply(
  split(known, ceiling(seq_along(known) / 25)),
  function(dois) paste(sprintf('DOI("%s")', dois), collapse = " OR "),
  character(1)
)
found <- function(query) {
  tolower(scopus_extract_dois(scopus_fetch(query, years = years)))
}

indexed <- unique(unlist(lapply(chains, found)))
retrieved <- unique(unlist(lapply(chains, function(chain) {
  found(sprintf("(%s) AND (%s)", strategy, chain))
})))
not_found <- setdiff(tolower(known), indexed)
relative_recall <- length(retrieved) / length(indexed)

The figure describes the strategy against this reference set alone, so report the set, its source and the DOIs left out alongside it.