Global model search

Description

Searches a model space globally rather than one axis at a time. Every structural category the space names becomes an axis of a grid with that category’s values as its alleles, every optional COVARIATE? pair becomes an axis whose alleles are none and each of its forms, and the search walks the grid: "exhaustive" fits every point, "ga" - the default - runs pyDarwin’s genetic algorithm over it and fits a fraction of them.

Usage

ferx_globalsearch(
  model = NULL,
  data = NULL,
  search_space = NULL,
  config = NULL,
  algorithm = NULL,
  iiv_strategy = NULL,
  max_models = NULL,
  ga = NULL,
  penalties = NULL,
  rank = NULL,
  cutoff = NULL,
  threads = NULL,
  retries = NULL,
  directory = NULL,
  resume = FALSE,
  progress = interactive()
)

# S3 method for ferx_globalsearch
print(x, digits = 4, ...)

# S3 method for ferx_globalsearch
summary(object, digits = 4, ...)

Arguments

model: Path to a .ferx model, or a ferx_model object. Omit when config is given.
data: Path to the dataset. Defaults to the model’s [data] block. Names the dataset the search runs on, so like model it cannot be given beside config.
search_space: MFL text naming the grid, e.g. "PERIPHERALS(0..1); LAGTIME([OFF,ON]); COVARIATE?(CL, WT, pow)".
config: Path to a .ferxsearch file. Mutually exclusive with the arguments that state the search.
algorithm: "ga" (the default) or "exhaustive". NULL keeps the engine default. An unrecognised value is an error naming it - there is no partial matching here, because algorithm = "e" choosing an exhaustive grid over a four-axis space is not a typo worth guessing at.
iiv_strategy: How a candidate’s new structural parameters are given a random effect: "absorption_delay" (the default), "add_diagonal" or "no_add". NULL keeps the engine default.
max_models: The largest grid "exhaustive" will enumerate (500 unless the file says otherwise). A bigger grid is an error naming the size, not a truncated search; the genetic algorithm has no such cap.
ga: A named list of [globalsearch.ga] knobs - population_size, generations, crossover_rate, mutation_rate, gene_mutation_probability, elites, tournament_size, downhill_period, niches, niche_radius, niche_penalty, sharing_alpha, final_downhill, seed. The key list and its defaults come from the engine, so an unknown knob is refused by name. Only meaningful for algorithm = "ga".
penalties: A named list of [rank.penalties] charges to override - theta, omega, sigma, convergence, covariance, correlation, max_correlation, condition_number, max_condition_number, non_influential, crash, gate. The first nine are the "penalized" criterion’s own schedule; the last three are the search-level charges, which apply under any criterion.
rank: [rank] type - the criterion the fitness is built on, e.g. "penalized", "bic", "aic", "ofv". NULL keeps the tool default ("penalized").
cutoff: [rank] cutoff: the margin, on the fitness scale, by which a candidate must beat the input model to be selected - not the base model, as in the stepwise tools, because a global search has no base beyond where it started. NULL selects the best eligible candidate outright.
threads: Total worker threads. NULL lets the runner choose.
retries: Perturbed restarts per candidate on top of the exact one. NULL keeps the engine default.
directory: Where the per-batch journals, models.csv, generations.csv, models/<id>.ferx and final.ferx are written. NULL keeps the run in memory, which also means it cannot be resumed.
resume: Reuse the fits already journalled in directory. The genetic algorithm proposes the same genomes for the same seed on the same grid, which is what makes a resume reuse its journal. Refused before the first fit when there is nothing to resume from - no directory, or a directory no earlier run wrote - rather than quietly searching from scratch.
progress: Print the engine’s batch progress to the console.
x: A ferx_globalsearch object.
digits: Significant digits for the printed tables.
...: Ignored.
object: A ferx_globalsearch object.

Details

Where [ferx_modelsearch](ferx_modelsearch.qmd) decides the structure and [ferx_covsearch](ferx_covsearch.qmd) the covariate relations, each conditional on the other being fixed, a global search decides them together. That is the case for using it - a covariate that only pays off once a second compartment is in the model is invisible to a stepwise search - and the cost is the fit count: an exhaustive grid is the product of its axes, and is refused outright above max_models rather than silently truncated.

A different name from ferx_fit()’s global_search

ferx_fit(settings = list(global_search = TRUE)) is a global optimizer phase inside the estimation of one model - a wider search for the parameter values of a fixed model. ferx_globalsearch() is a global search over models. The two are unrelated; the name here is the engine’s, the .ferxsearch file’s ([globalsearch]), the ferx globalsearch command’s and Pharmpy/pyDarwin’s, so a search is portable between them.

Ranking, and the charges the criterion cannot see

[rank] type defaults to "penalized" here - pyDarwin’s fitness, the objective function plus a charge per estimated parameter and per unhealthy fit - where every other tool defaults to a BIC. Any criterion may be named instead. Whatever the criterion, the search charges three things on top of it, and ranks on the sum (fitness, always finite):

charge_non_influential: Per gene of a genome that changed nothing in the rendered model - a covariate on a parameter the structural choice removed. A tie-break towards the simpler genotype among models that are identical in every other way.
charge_gate: A fit the [strictness] gate refused. It cannot be selected, but it still steers a genetic algorithm.
charge_crash: A candidate that produced no fit at all - it does not compile, or the fit errored. Large, and finite, so the algorithm can still rank it.

The three are columns of $models beside criterion and fitness, so a genome that lost to a tie-break penalty does not read as though it lost on OFV. $penalties is the effective schedule the run charged.

Two entry forms

Pass either config - a .ferxsearch file, which is the reproducible artifact - or the inline arguments. The inline form is rendered into the same configuration and handed to the engine’s own loader, so the two cannot disagree; search_space is MFL text, quoted verbatim. The run arguments (threads, retries, resume, directory, progress) say how to run a search rather than what to search, and are available to both forms.

What the space may say

ABSORPTION, ELIMINATION, PERIPHERALS, TRANSITS, LAGTIME and COVARIATE?. A variability statement (IIV? / IOV?) or ALLOMETRY is another tool’s axis and is refused by name before any fit starts, as is a feature the engine cannot build. Check a space with [ferx_search_space](ferx_search_space.qmd) / [ferx_search_coverage](ferx_search_coverage.qmd) first: both answer without fitting anything.

[ferx_modelsearch](ferx_modelsearch.qmd) and [ferx_covsearch](ferx_covsearch.qmd)for the stepwise searches of the same two axes, [ferx_search_space](ferx_search_space.qmd), [ferx_search_coverage](ferx_search_coverage.qmd), [ferx_search_results](ferx_search_results.qmd)Other search: [ferx_allometry](ferx_allometry.qmd), [ferx_amd](ferx_amd.qmd), [ferx_amd_plan](ferx_amd_plan.qmd), [ferx_covsearch](ferx_covsearch.qmd), [ferx_iivsearch](ferx_iivsearch.qmd), [ferx_iovsearch](ferx_iovsearch.qmd), [ferx_modelsearch](ferx_modelsearch.qmd), [ferx_ruvsearch](ferx_ruvsearch.qmd), [ferx_search_config](ferx_search_config.qmd), [ferx_search_coverage](ferx_search_coverage.qmd), [ferx_search_results](ferx_search_results.qmd), [ferx_search_space](ferx_search_space.qmd)

Concept

Value

An object of class ferx_globalsearch:

models: The model table, one row per grid point evaluated (and the input), in the engine’s own column order: id, parent, step, genome, absorption, elimination, peripherals, transits, lagtime, covariates, n_parameters, ofv, criterion, fitness, rank, converged, passed, failures, error, seconds, selected, non_influential, duplicate_of, reused - followed by charge_non_influential, charge_gate and charge_crash, the decomposition of what the search added to the criterion to reach the fitness.
fit: The winning model’s fit as a ferx_fit, or NULL when a degraded resume could not recover it.
model_text: Every evaluated model’s text, named by model id.
axes, space_size: The grid: a named list of each axis’s allele labels, and the number of points they multiply out to. A double rather than an integer, because the product of enough axes exceeds R’s integer range while remaining a grid the genetic algorithm can search.
generations: The genetic algorithm’s trajectory - index, best (the model id), best_fitness, mean_fitness, polished - or NULL for an exhaustive search.
input_model, input_ofv, input_criterion, input_fitness: The model the search started from, and what it scored.
final_model, final_model_path, final_model_id, final_criterion, final_fitness: The winning model’s text, the final.ferx written beside the run, its id, and the two values it won on.
criterion, algorithm, iiv_strategy, max_models, cutoff: What the search ranked on and how it searched.
penalties, ga: The effective [rank.penalties] schedule and [globalsearch.ga] settings, as named numeric vectors.
n_fitted: Models actually fitted, as against grid points evaluated: a duplicate genome and a cached one cost no fit.
candidates: The runner’s candidate table for every batch the run wrote one - see [ferx_search_results](ferx_search_results.qmd).
notes, cancelled: What the search wants said once, and whether it was stopped early.

Examples

ex <- ferx_example("two_cpt_oral_global")

# Inline: a small grid, enumerated exhaustively
res <- ferx_globalsearch(
  model        = ex$model,
  data         = ex$data,
  search_space = "PERIPHERALS(0..1); COVARIATE?(CL, WT, pow)",
  algorithm    = "exhaustive",
  directory    = "globalsearch-run-1"
)
res
res$models[, c("id", "genome", "criterion", "charge_gate", "fitness", "rank")]

# The genetic algorithm over the same grid, seeded so a resume can reuse
# the journal it writes
res2 <- ferx_globalsearch(
  model        = ex$model,
  data         = ex$data,
  search_space = "PERIPHERALS(0..1); COVARIATE?(CL, WT, pow)",
  algorithm    = "ga",
  ga           = list(population_size = 8, generations = 3, seed = 1),
  directory    = "globalsearch-run-2"
)
res2$generations

# Reproducible: the bundled .ferxsearch is the artifact, and states the
# whole search - a four-axis grid, enumerated, ranked on penalized fitness
res3 <- ferx_globalsearch(config = ex$search, directory = "globalsearch-run-3")
summary(res3)