Experimental features
Where you are
Two model extensions in ferx are explicitly experimental: stochastic differential equations, which add random noise to the model states between observations, and neural networks for the covariate model, known as deep compartment models. Both fit from R with the bundled examples. This chapter runs them, compares them with their conventional counterparts, and shows what the fits report. It closes with parameter priors, which carry the same label without being a model extension.
ferx-core labels [diffusion] (SDE) and [covariate_nn] (neural networks) experimental: validated on a small set of examples, with syntax that may still change. Every fit that uses them carries an experimental warning. The feature maturity table’s third experimental entry is the Vern7 ODE stepper (Structural models: analytical and ODE). Parameter priors, at the end of this chapter, are labelled experimental on their own page but are absent from that table, and a priored fit raises no experimental warning.
The data
warfarin_sde uses the warfarin concentrations (10 subjects). warfarin_dcm uses the 30-subject dataset of the covariate thread, with weight and creatinine clearance:
sde <- ferx_example("warfarin_sde")
dcm <- ferx_example("warfarin_dcm")
experimental_dir <- file.path(tempdir(), "experimental")
dir.create(experimental_dir, showWarnings = FALSE)
c(sde_subjects = n_distinct(read.csv(sde$data)$ID), dcm_subjects = n_distinct(read.csv(dcm$data)$ID))
#> sde_subjects dcm_subjects
#> 10 30Minimal runnable call: an SDE model
An SDE model is an ODE model with a [diffusion] block. Each line gives a state a diffusion variance, estimated as the parameter DIFF_<STATE>. ferx evaluates the likelihood with an extended Kalman filter, which adds the propagated state uncertainty to the residual variance at each observation:
invisible(ferx_model_get_section(sde$model, "odes"))
#> # [odes]
#> # `central` is concentration (mg/L): the absorption term carries `/V` so the
#> # ODE state matches the concentration-scale DV column. See the existing
#> # `examples/bioavailability_ode.ferx` in ferx-core for the same convention.
#> d/dt(depot) = -KA * depot
#> d/dt(central) = KA * depot / V - (CL / V) * central
invisible(ferx_model_get_section(sde$model, "diffusion"))
#> # [diffusion]
#> central ~ 0.01
fit_sde <- ferx_fit(sde$model, sde$data, verbose = FALSE)
fit_sde$uses_sde
#> [1] TRUE
round(fit_sde$estimates[, c("estimate", "rse_pct")], 6)
#> estimate rse_pct
#> TVCL 0.133070 5.039476
#> TVV 7.727857 3.039366
#> TVKA 0.727295 17.166207
#> DIFF_CENTRAL 0.000001 590.837652
#> ETA_CL 0.028469 44.648539
#> ETA_V 0.009575 44.861669
#> ETA_KA 0.348907 46.057132
#> PROP_ERR 0.010709 9.004399Reading the result
The fit carries the experimental notice with its other warnings:
ferx_get_warnings(fit_sde, as_df = TRUE)[, c("severity", "category")]
#> severity category
#> 1 warning experimental
#> 2 warning inflated_rse
#> 3 info ode_solver
#> 4 warning dw_autocorrelation
substr(ferx_get_warnings(fit_sde, as_df = TRUE)$message[1], 1, 150)
#> [1] "Stochastic differential equations ([diffusion] / Extended Kalman Filter) are an EXPERIMENTAL feature: validated only on a small set of toy examples, w"Here the fit converged and the covariance step needed no regularization: the relative standard errors are ordinary (at most 46%) except that of DIFF_CENTRAL, at 591% (inflated_rse). Converging is not the same as reaching the optimum, and even this much is not guaranteed. Where the default tolerances leave this fit depends on rounding in the solver, and on other machines and builds the same file stops at a saddle point: the covariance step finds a direction of negative curvature and floors it (covariance_regularized), every standard error runs to hundreds of percent, and standard errors computed there mean nothing. [diffusion] needs finite-difference gradients, so the model file sets gradient_method = fd, and with it optimizer = auto resolves to bobyqa (Estimation methods and controlling the fit). When the covariance step regularizes, the last sentence of its covariance_regularized message points at the ODE tolerances: the finite differences of the covariance step amplify the integration errors of the solver, and the message suggests ode_reltol = 1e-6 and ode_abstol = 1e-8, tighter than the defaults of 1e-4 and 1e-6. A refit at those tolerances does not depend on where the default ones leave the fit:
regularized <- filter(ferx_get_warnings(fit_sde, as_df = TRUE), category == "covariance_regularized")
if (nrow(regularized) > 0) strwrap(sub(".*(Note that)", "\\1", regularized$message), 80)
fit_sde_tight <- ferx_fit(sde$model, sde$data, settings = list(ode_reltol = 1e-6, ode_abstol = 1e-8),
verbose = FALSE)
sde_fits <- list(`model file` = fit_sde, `tighter tolerances` = fit_sde_tight)
do.call(rbind, lapply(names(sde_fits), function(fit) {
f <- sde_fits[[fit]]
data.frame(fit = fit, ofv = f$ofv, DIFF_CENTRAL = f$theta[["DIFF_CENTRAL"]],
covariance_regularized = "covariance_regularized" %in% ferx_get_warnings(f, as_df = TRUE)$category)
}))
#> fit ofv DIFF_CENTRAL covariance_regularized
#> 1 model file -280.3082 7.451050e-07 FALSE
#> 2 tighter tolerances -280.3525 1.203163e-07 FALSE
round(fit_sde_tight$estimates[, c("estimate", "rse_pct")], 6)
#> estimate rse_pct
#> TVCL 0.133005 5.001203
#> TVV 7.730437 3.031046
#> TVKA 0.725158 17.183694
#> DIFF_CENTRAL 0.000000 1654.587578
#> ETA_CL 0.028611 44.761844
#> ETA_V 0.009582 44.872691
#> ETA_KA 0.349573 46.124173
#> PROP_ERR 0.010745 8.818357With those tolerances the same model, optimizer and data need no regularization in the covariance step. The standard errors are ordinary, except for the diffusion variance: at 1e-07 it cannot be told apart from zero. The OFV of the refit is 0.04 lower than the bundled fit’s. Scored without moving, the bundled fit’s own estimates give OFVs 0.00 apart at the default and at the tighter tolerances. So the gap is part solver error in evaluating the objective and part the two fits ending at different points. Where the default tolerances leave the fit depends on rounding in the solver, so the bundled fit’s OFV and estimates differ from one machine to the next; the refit is the one to read. Tighter default tolerances for finite-difference fits of ODE models are tracked in ferx-core #520.
Removing the [diffusion] block gives the ordinary ODE model. That model needs no finite differences: with gradient = "auto" ferx uses the analytic gradient. The table also fits it with the model file’s finite differences at the default tolerances:
sde_lines <- readLines(sde$model)
start <- grep("^\\[diffusion\\]", sde_lines)
ode_model <- file.path(experimental_dir, "warfarin_ode_without_diffusion.ferx")
writeLines(sde_lines[-(start:(start + 2))], ode_model)
fit_ode_fd <- ferx_fit(ode_model, sde$data, verbose = FALSE)
# suppress the R warning that reports overriding the model file's gradient
fit_ode <- suppressWarnings(ferx_fit(ode_model, sde$data, gradient = "auto", verbose = FALSE))
sde_vs_ode <- list(`SDE, model file` = fit_sde, `SDE, tighter tolerances` = fit_sde_tight,
`ODE, finite differences` = fit_ode_fd, `ODE, gradient = "auto"` = fit_ode)
do.call(rbind, lapply(names(sde_vs_ode), function(model) {
f <- sde_vs_ode[[model]]
data.frame(model = model, ofv = f$ofv, durbin_watson = f$dw_statistic,
regularized = "covariance_regularized" %in% ferx_get_warnings(f, as_df = TRUE)$category,
seconds = f$wall_time_secs)
}))
#> model ofv durbin_watson regularized seconds
#> 1 SDE, model file -280.3082 2.733029 FALSE 15.19279607
#> 2 SDE, tighter tolerances -280.3525 2.731168 FALSE 17.60095087
#> 3 ODE, finite differences -280.3595 2.730794 FALSE 0.28478117
#> 4 ODE, gradient = "auto" -280.3595 2.730794 FALSE 0.06019153The diffusion term does not improve on the ordinary ODE model. The SDE fit with the tighter tolerances has a diffusion variance of essentially zero and an OFV 0.007 above the ODE model’s analytic fit. At a zero diffusion variance the two models coincide, so the gap is no gain from the diffusion term; the ODE fits run at the default tolerances, whose solver error is of that size, so the gap does not say which fit stopped short. The ODE model’s own finite-difference fit, at the same default tolerances, ends within 1e-05 of its analytic optimum. These data show no system noise. A [diffusion] term is not a remedy for residuals that are correlated in time within a subject (Diagnosing the model). The filter propagates the state variance and adds it to the residual variance; the predicted state itself stays the ODE solution and is never pulled towards a subject’s observations, so the term cannot follow a drift. Here the Durbin-Watson statistic does not move: 2.73 with the block and 2.73 without. What [diffusion] does estimate is how much unexplained variance builds up between observations. The ferx-core page explains what the filter does not do.
Options that matter
| Block or setting | Meaning |
|---|---|
[diffusion] STATE ~ variance
|
Diffusion variance for an ODE state, estimated as DIFF_STATE (FIX to hold it) |
[covariate_nn NAME] |
A neural network with inputs (covariates), outputs (parameter names), hidden layers, activation and output activation |
NAME.OUTPUT in [individual_parameters]
|
A network output, used like any other quantity |
| Setting | Values | Default | Description |
|---|---|---|---|
nn_l2 |
float ≥ 0 | 0.0 |
L2 (weight-decay) strength. Adds nn_l2 · Σ wᵢ² over the network’s weight matrices only (bias terms are left free). |
nn_smooth |
float ≥ 0 | 0.0 |
Smoothness (curvature) strength. Penalizes the finite-difference 2nd derivative C = f(x+h) − 2·f(x) + f(x−h) of each output along every input’s marginal partial-dependence curve, i.e. nn_smooth · Σ C². |
Constraints from the ferx-core diffusion page: [diffusion] needs an ODE model, cannot be combined with SAEM or [scaling] (Observation scaling with [scaling]), is estimated with FOCE or FOCEI, and uses finite-difference gradients.
Variants
Simulating from an SDE fit
Simulation from an SDE fit works like any other simulation (Simulating scenarios), here from the fit that reached the lowest OFV:
head(ferx_simulate(sde$model, sde$data, n_sim = 1, seed = 1, fit = fit_sde_tight), 3)
#> Warning: ferx_simulate produced 1 diagnostic:
#> Stochastic differential equations ([diffusion] / Extended Kalman Filter) are an EXPERIMENTAL feature: validated only on a small set of toy examples, with estimator support limited to FOCE/FOCEI. Standard errors and convergence behaviour are not yet proven across diverse datasets — validate results carefully before relying on them. The filter is covariance-only: the state mean is propagated by the ODE and is never corrected by the observed data, so a [diffusion] fit scores the deterministic ODE prediction with an inflated observation variance rather than a filtered state. There is no filtered state trajectory, and sdtab IWRES from such a fit is scaled by the residual error alone, so it reads over-dispersed. See the Feature Maturity and SDE pages in the documentation.
#> DRAW SIM ID TIME CMT IPRED DV_SIM OBSERVED
#> 1 1 1 1 0.5 1 3.417031 3.394518 NA
#> 2 1 1 1 1.0 1 5.822097 5.765162 NA
#> 3 1 1 1 2.0 1 8.671250 8.849994 NAA neural-network covariate model
warfarin_dcm replaces the covariate expressions of two_cpt_oral_cov with a network. The network reads WT and CRCL and outputs a multiplier for each PK parameter. The multipliers start at 1, so the thetas carry the typical values:
# print the two blocks without their comment lines
section_lines <- function(block, model = dcm$model) {
capture.output(lines <- ferx_model_get_section(model, block, strip = TRUE))
trimws(lines[nzchar(trimws(lines)) & !grepl("^\\s*#", lines)])
}
section_lines("covariate_nn TYPICAL_PK")
#> [1] "inputs = [WT, CRCL]" "outputs = [CL, V1, Q, V2, KA]"
#> [3] "layers = [8, 8]" "activation = tanh"
#> [5] "output = exp"
section_lines("individual_parameters")
#> [1] "CL = TVCL * TYPICAL_PK.CL * exp(ETA_CL)"
#> [2] "V1 = TVV1 * TYPICAL_PK.V1 * exp(ETA_V1)"
#> [3] "Q = TVQ * TYPICAL_PK.Q * exp(ETA_Q)"
#> [4] "V2 = TVV2 * TYPICAL_PK.V2 * exp(ETA_V2)"
#> [5] "KA = TVKA * TYPICAL_PK.KA * exp(ETA_KA)"The network has 2 inputs, two hidden layers of 8 and 5 outputs: 141 weights, estimated as thetas. The covariance step is skipped here:
# suppress the R warning that reports overriding the model file's covariance setting
fit_dcm <- suppressWarnings(ferx_fit(dcm$model, dcm$data, covariance = FALSE, verbose = FALSE))
network <- fit_dcm$neural_networks$TYPICAL_PK
network[c("shape", "hidden_activation", "output_activation", "n_weights", "input_names", "output_names")]
#> $shape
#> [1] 2 8 8 5
#>
#> $hidden_activation
#> [1] "tanh"
#>
#> $output_activation
#> [1] "exp"
#>
#> $n_weights
#> [1] 141
#>
#> $input_names
#> [1] "WT" "CRCL"
#>
#> $output_names
#> [1] "CL" "V1" "Q" "V2" "KA"
ferx_get_warnings(fit_dcm, as_df = TRUE)[, c("severity", "category")]
#> severity category
#> 1 warning experimental
#> 2 warning general
#> 3 warning general
#> 4 warning dw_autocorrelation
dcm_messages <- ferx_get_warnings(fit_dcm, as_df = TRUE)$message
cat(strwrap(grep("^W_NN_UNSCALED_INPUT", dcm_messages, value = TRUE), 90), sep = "\n")
#> W_NN_UNSCALED_INPUT: [covariate_nn TYPICAL_PK] input WT (|z| up to 93.700), CRCL (|z| up
#> to 150.000) reaches past 3.873 after the block's center/scale, where one input alone
#> saturates a tanh unit at the first layer's initial weight scale. A saturated layer barely
#> learns, and which optimum the fit reaches can then depend on the path to it. Declare
#> `center` and `scale` in the block (the input's mean and SD, or its range) so the inputs
#> land within a few units of 0.The two general rows are about the network. W_NN_BARE_OUTPUT_START says the outputs start at 1 because the block declares no init; here the outputs are multipliers on the thetas, so 1 is the intended start. W_NN_UNSCALED_INPUT, printed above, says the inputs are not scaled: the network sees WT and CRCL as they are in the data, far past the values at which one input saturates a tanh unit at its initial weights. A saturated layer barely learns, and which optimum the fit reaches can then depend on the path to it. The block’s center and scale keys fix that (normalize your inputs); Scaling the inputs below fits the network with them.
ferx-r 0.4.0 does not raise W_NN_UNSCALED_INPUT. Its [covariate_nn] block already takes center and scale.
The same data with the base model and with the explicit covariate model:
base <- ferx_example("two_cpt_oral_base")
cov <- ferx_example("two_cpt_oral_cov")
fit_base <- suppressWarnings(ferx_fit(base$model, base$data, covariance = FALSE, verbose = FALSE))
fit_cov <- suppressWarnings(ferx_fit(cov$model, cov$data, covariance = FALSE, verbose = FALSE))
data.frame(model = c("two_cpt_oral_base (no covariates)", "two_cpt_oral_cov (WT, CRCL)", "warfarin_dcm (network)"),
parameters = c(fit_base$n_parameters, fit_cov$n_parameters, fit_dcm$n_parameters),
ofv = c(fit_base$ofv, fit_cov$ofv, fit_dcm$ofv),
aic = c(fit_base$aic, fit_cov$aic, fit_dcm$aic))
#> model parameters ofv aic
#> 1 two_cpt_oral_base (no covariates) 11 -1185.462 -1163.4620
#> 2 two_cpt_oral_cov (WT, CRCL) 13 -1199.326 -1173.3264
#> 3 warfarin_dcm (network) 152 -1185.462 -881.4621This fit of the network reaches an OFV of -1185.5 against -1185.5 without covariates, and its 141 weights put its AIC at -881.5, against -1163.5 and -1173.3 for the alternatives. The explicit covariate model captures the covariate effects with two parameters.
How much the network departs from multipliers of 1 can be read from the individual parameters. For clearance, CL / (TVCL * exp(ETA_CL)) is the network’s multiplier for each subject:
subjects <- read.csv(dcm$data) |> distinct(ID, WT, CRCL)
# The network's clearance multiplier per subject: CL / (TVCL * exp(ETA_CL)).
clearance_multiplier <- function(f) {
mutate(f$individual_estimates, ID = as.integer(ID)) |>
select(ID, CL) |>
left_join(mutate(f$ebe_etas, ID = as.integer(ID)) |> select(ID, ETA_CL), by = "ID") |>
mutate(multiplier = CL / (f$theta[["TVCL"]] * exp(ETA_CL)))
}
multipliers <- left_join(clearance_multiplier(fit_dcm), subjects, by = "ID")
summary(multipliers$multiplier)
#> Min. 1st Qu. Median Mean 3rd Qu. Max.
#> 1.02 1.02 1.02 1.02 1.02 1.02
range(subjects$WT)
#> [1] 45.0 93.7
range(subjects$CRCL)
#> [1] 46.5 150.0
spread <- diff(range(multipliers$multiplier)) / mean(multipliers$multiplier)The multiplier is the same for every subject to 7 significant digits, across the whole range of weight and creatinine clearance: this fit of the network learned no covariate dependence at all, and only rescaled the typical clearance. Whether that reflects the data or where the optimizer stopped is the question of the next two sections.
Regularizing the network
nn_l2 (weight decay) and nn_smooth (a penalty on curvature) keep a network from fitting spurious covariate effects. They are set in [fit_options] or with settings and apply to the FOCE-family methods and to VI; SAEM, importance sampling and Bayes do not apply them and say so in a warning. The penalty only shapes the search: the reported OFV stays the unpenalized one, so regularized and unregularized fits remain comparable (regularization). The same network with weight decay:
# suppress the R warning that reports overriding the model file's covariance setting
fit_dcm_l2 <- suppressWarnings(ferx_fit(dcm$model, dcm$data, covariance = FALSE,
settings = list(nn_l2 = 0.01), verbose = FALSE))
data.frame(fit = c("network", "network, nn_l2 = 0.01", "two_cpt_oral_cov"),
converged = c(fit_dcm$converged, fit_dcm_l2$converged, fit_cov$converged),
parameters = c(fit_dcm$n_parameters, fit_dcm_l2$n_parameters, fit_cov$n_parameters),
ofv = c(fit_dcm$ofv, fit_dcm_l2$ofv, fit_cov$ofv),
aic = c(fit_dcm$aic, fit_dcm_l2$aic, fit_cov$aic))
#> fit converged parameters ofv aic
#> 1 network TRUE 152 -1185.462 -881.4621
#> 2 network, nn_l2 = 0.01 TRUE 152 -1365.649 -1061.6493
#> 3 two_cpt_oral_cov TRUE 13 -1199.326 -1173.3264
multipliers_l2 <- clearance_multiplier(fit_dcm_l2)$multiplier
summary(multipliers_l2)
#> Min. 1st Qu. Median Mean 3rd Qu. Max.
#> 0.4919 0.9683 1.3743 1.2847 1.5466 2.0248With the penalty, the network’s clearance multipliers range from 0.49 to 2.02 across subjects, and the fit reports an unpenalized OFV of -1365.6, against -1185.5 without the penalty. A penalty cannot make the unpenalized objective better than its optimum, so whenever the penalized fit scores lower, the fit without it stopped at least that far above its optimum, although it reports convergence. Where an unpenalized network fit stops is fragile at this release: this one stops at a different point from one machine to the next, and ferx-core lists DCM FOCEI fits that report convergence far above a point of their own parameter space as a known issue (ferx-core #1561). Fit a network more than once, with and without a penalty, and compare the OFVs before reading anything into it.
Neither network fit beats the explicit covariate model on AIC (-881.5 without the penalty and -1061.6 with it, against -1173.3): 152 parameters cost more than the objective gains.
Scaling the inputs
The same network with its inputs centred and scaled by the mean and SD of each covariate across the subjects, written into a copy of the model:
input_center <- round(c(mean(subjects$WT), mean(subjects$CRCL)))
input_scale <- round(c(sd(subjects$WT), sd(subjects$CRCL)))
scaled_lines <- readLines(dcm$model)
inputs_at <- grep("inputs = [WT, CRCL]", scaled_lines, fixed = TRUE)
stopifnot(length(inputs_at) == 1)
scaled_lines <- append(scaled_lines, after = inputs_at,
c(sprintf(" center = [%s]", paste(input_center, collapse = ", ")),
sprintf(" scale = [%s]", paste(input_scale, collapse = ", "))))
dcm_scaled_model <- file.path(experimental_dir, "warfarin_dcm_scaled.ferx")
writeLines(scaled_lines, dcm_scaled_model)
section_lines("covariate_nn TYPICAL_PK", dcm_scaled_model)
#> [1] "inputs = [WT, CRCL]" "center = [70, 86]"
#> [3] "scale = [13, 24]" "outputs = [CL, V1, Q, V2, KA]"
#> [5] "layers = [8, 8]" "activation = tanh"
#> [7] "output = exp"
# suppress the R warning that reports overriding the model file's covariance setting
fit_dcm_scaled <- suppressWarnings(ferx_fit(dcm_scaled_model, dcm$data, covariance = FALSE, verbose = FALSE))
scaled_warnings <- ferx_get_warnings(fit_dcm_scaled, as_df = TRUE)
multipliers_scaled <- clearance_multiplier(fit_dcm_scaled)$multiplier
data.frame(fit = c("network", "network, scaled inputs"),
converged = c(fit_dcm$converged, fit_dcm_scaled$converged),
ofv = c(fit_dcm$ofv, fit_dcm_scaled$ofv),
min_cl_multiplier = c(min(multipliers$multiplier), min(multipliers_scaled)),
max_cl_multiplier = c(max(multipliers$multiplier), max(multipliers_scaled)))
#> fit converged ofv min_cl_multiplier
#> 1 network TRUE -1185.462 1.019689
#> 2 network, scaled inputs TRUE -1677.484 17.322964
#> max_cl_multiplier
#> 1 1.019689
#> 2 84.301147With scaled inputs the unscaled-input warning is gone, and the fit reaches an OFV 492 units below the unscaled one. The flat multiplier above was where the optimizer stopped, not what the data say. That does not make this a better model. This fit reports convergence, it holds an estimate on an optimizer bound (boundary_estimate), and the clearance multiplier now ranges from 17 to 84 across subjects, so the network and TVCL share the typical clearance between them instead of the network describing a deviation from it. With 141 weights and 30 subjects, an OFV this far below the explicit covariate model’s (-1199.3) says more about what the network can fit than about the covariates, even though its AIC, -1373.5, is now below that model’s -1173.3. Scale the inputs so the network can learn at all, then judge it as the previous section did: refit it, with and without a penalty, and compare it with an explicit covariate model.
A fitted network can be written out as an ordinary model. ferx_model_from_dcm(model, fit) replaces each [covariate_nn] block with its forward pass, as plain equations in [individual_parameters] at the fitted weights, writes it to output_model, and returns a ferx_model for prediction and simulation. Without output_model the file goes next to the model as <stem>_expanded.ferx, which for a ferx_example() model is inside the installed package, so give a path under tempdir(). The exported model is not meant to be refitted. weights = "fixed_theta" keeps each weight as a named FIX theta instead of a number, network_only = TRUE keeps the model file’s other values, and theta = takes the weights from a named vector instead of a fit. ?ferx_model_from_dcm has the details.
ferx_model_from_dcm() is not in ferx-r 0.4.0.
Parameter priors
A prior pins a parameter near a value you already believe and lets the data move it only as far as the data can pay for – penalized maximum likelihood, the mechanism behind updating a published model on local data. It is written inline on the [parameters] row it belongs to, as a central value and a relative standard error, both on the scale the parameter was declared on:
warf <- ferx_example("warfarin")
priored <- file.path(experimental_dir, "warfarin_prior.ferx")
writeLines(sub("theta TVCL(0.134, 0.001, 10.0)",
"theta TVCL(0.134, 0.001, 10.0) prior(0.25, rse = 10%)",
readLines(warf$model), fixed = TRUE), priored)
fit_plain <- ferx_fit(warf$model, warf$data, verbose = FALSE)
fit_prior <- ferx_fit(priored, warf$data, verbose = FALSE)
c(no_prior = fit_plain$theta[["TVCL"]], with_prior = fit_prior$theta[["TVCL"]])
#> no_prior with_prior
#> 0.1329743 0.1627376These data want a TVCL near 0.133; a prior centred at 0.25 with a 10% relative standard error pulls the estimate to 0.1627. The penalty is added to the objective, so fit$ofv is the penalized total and does not compare with the unpenalized fit. The fit reports the two halves separately: fit$ofv_data is the data’s share, fit$ofv_prior the penalty, and they sum to fit$ofv. The information criteria are built from the data half alone, so fit$aic is fit$ofv_data + 2 * fit$n_parameters and not fit$ofv + 2 * fit$n_parameters once a prior is in the model:
data.frame(fit = c("no prior", "prior on TVCL"),
ofv = c(fit_plain$ofv, fit_prior$ofv),
ofv_data = c(fit_plain$ofv_data, fit_prior$ofv_data),
ofv_prior = c(fit_plain$ofv_prior, fit_prior$ofv_prior),
aic = c(fit_plain$aic, fit_prior$aic),
ofv_data_plus_2k = c(fit_plain$ofv_data + 2 * fit_plain$n_parameters,
fit_prior$ofv_data + 2 * fit_prior$n_parameters))
#> fit ofv ofv_data ofv_prior aic ofv_data_plus_2k
#> 1 no prior -280.3595 -280.3595 0.00000 -266.3595 -266.3595
#> 2 prior on TVCL -250.4422 -268.9659 18.52374 -254.9659 -254.9659A fit without a prior has ofv_data equal to ofv and an ofv_prior of 0. ofv_data is the number to set beside the unpenalized fit: -268.97 against -280.36, so pulling TVCL towards 0.25 costs the data 11.39 points of objective, on top of a penalty of 18.52. print() and summary() state the split under the OFV line:
grep("OFV|prior", capture.output(print(fit_prior)), value = TRUE)
#> [1] " Model: warfarin_prior Dataset: warfarin"
#> [2] " OFV: -250.4422 AIC: -254.9659 BIC: -236.0626"
#> [3] " (prior on 1 parameter: data OFV -268.9659 + prior penalty 18.5237; AIC/BIC use the data half)"fit$prior_summary has one row per priored parameter, and is NULL for a model without one:
fit_prior$prior_summary
#> name prior_value estimate shift_in_prior_sds penalty family
#> 1 TVCL 0.25 0.1627376 -4.303922 18.52374 lognormal
#> prior_lower_95 prior_upper_95
#> 1 0.205604 0.3039824
is.null(fit_plain$prior_summary)
#> [1] TRUEshift_in_prior_sds is how far the estimate landed from the prior’s centre, with its sign, in prior standard deviations on the scale the prior is evaluated on. For a lognormal family that is the log scale, so it is not (estimate - 0.25) / 0.025. penalty is its square, and that parameter’s share of ofv_prior. prior_lower_95 and prior_upper_95 are the interval that rse = 10% implies for the family the engine chose (?ferx_fit describes every column). An estimate 4.3 prior standard deviations from the centre, outside that interval, says the prior and these data disagree. The standard errors of a priored fit are those of the penalized objective. What the prior cost in total is fit_prior$ofv - fit_plain$ofv, 29.92 points: the data’s share plus the penalty.
A [priors] block takes the priors from a previous fit instead of writing them out by hand. It has exactly one key, from_fit, naming a saved fit; from R that is the .fitrx bundle ferx_save_fit() writes. Every theta, omega diagonal and sigma of the saved fit that has a standard error, and whose name matches a parameter of the new model, gets prior(estimate, rse = SE / estimate): exactly the prior you would have typed off the earlier fit’s output. As an example, treat warfarin subjects 1 to 5 as an earlier study and subjects 6 to 10 as new data:
warf_data <- read.csv(warf$data, na.strings = ".")
earlier_data <- file.path(experimental_dir, "warfarin_earlier.csv")
later_data <- file.path(experimental_dir, "warfarin_later.csv")
write.csv(warf_data[warf_data$ID <= 5, ], earlier_data, row.names = FALSE, na = ".", quote = FALSE)
write.csv(warf_data[warf_data$ID > 5, ], later_data, row.names = FALSE, na = ".", quote = FALSE)
fit_earlier <- ferx_fit(warf$model, earlier_data, verbose = FALSE)
saved <- file.path(experimental_dir, "warfarin_earlier.fitrx")
ferx_save_fit(fit_earlier, saved)
# The model goes next to the saved fit, so from_fit can name it by a relative
# path, which is resolved against the model file's directory.
updating_model <- file.path(experimental_dir, "warfarin_from_fit.ferx")
writeLines(c(readLines(warf$model), "", "[priors]",
paste0(" from_fit = ", basename(saved))),
updating_model)
fit_later <- ferx_fit(warf$model, later_data, verbose = FALSE)
fit_updated <- ferx_fit(updating_model, later_data, verbose = FALSE)
fit_updated$prior_summary[, c("name", "prior_value", "estimate", "shift_in_prior_sds")]
#> name prior_value estimate shift_in_prior_sds
#> 1 TVCL 0.127278784 0.129484612 0.4085290
#> 2 TVV 7.314328189 7.776244570 1.2263468
#> 3 TVKA 0.778445365 0.759109256 -0.1323822
#> 4 ETA_CL 0.009113214 0.023192435 1.6050252
#> 5 ETA_V 0.013029792 0.007975718 -0.8417466
#> 6 ETA_KA 0.217405580 0.318514511 0.6576069
#> 7 PROP_ERR 0.009251835 0.010391049 0.9748157All 7 parameters got a prior. The estimates on the new subjects alone, the updated ones, and those of a fit to all ten subjects:
rbind(earlier_only = fit_earlier$theta, later_only = fit_later$theta,
later_with_prior = fit_updated$theta, all_subjects = fit_plain$theta)
#> TVCL TVV TVKA
#> earlier_only 0.1272788 7.314328 0.7784454
#> later_only 0.1380580 8.124370 0.6603863
#> later_with_prior 0.1294846 7.776245 0.7591093
#> all_subjects 0.1329743 7.730568 0.7252586
rbind(later_only = fit_later$estimates[c("TVCL", "TVV", "TVKA"), "rse_pct"],
later_with_prior = fit_updated$estimates[c("TVCL", "TVV", "TVKA"), "rse_pct"])
#> [,1] [,2] [,3]
#> later_only 8.329597 1.721060 29.51608
#> later_with_prior 3.521592 3.347401 14.59176Each updated estimate sits between those of the two studies. The relative standard error is smaller than with the new data alone for TVCL and TVKA, but not for TVV. As with an inline prior, fit_updated$ofv is penalized (ofv_prior = 6.36), and ofv_data is the number to compare with unpenalized fits.
Pitfalls
- Experimental means validate yourself. Compare an SDE or network model with its conventional counterpart, as above, before relying on it.
-
Absolute
from_fitpaths.//starts a comment in a.ferxfile, and R’stempdir()contains one on macOS, so an absolute path built from it is cut short before the engine sees it. Keep the saved fit next to the model and give a relative path, as above; if you need an absolute one, write it withnormalizePath(path, winslash = "/"). - Extra flexibility is not extra information. A diffusion variance or a network with many weights needs data that support it; check standard errors, the OFV change and information criteria.
-
A converged network fit may be far from its optimum at this build. DCM fits can report convergence far above a point of their own parameter space (ferx-core #1561), and where
warfarin_dcmstops without a penalty differs from one machine to the next. Compare fits with and withoutnn_l2. -
Standard errors of network weights are not reliable, and the covariance step with many weights is slow. Skip it with
covariance = FALSEwhile exploring. -
SDE fits are slow. The extended Kalman filter uses finite differences; the warfarin SDE fit above took 53 times as long as the same model without
[diffusion], both with finite differences. -
Check that an SDE fit reached a minimum. With finite differences the default optimizer can stop at a saddle point, and whether the bundled fit does depends on rounding in the solver. A
covariance_regularizedwarning that reports negative curvature is the sign; refit with the ODE tolerances its message names, and compare OFVs.
Warnings you may see here
-
experimental(warning): the model uses an experimental component ([diffusion]or[covariate_nn]). The message states what has and has not been validated. It appears on every such fit.
Summary
-
[diffusion]adds system noise to ODE states, estimated with an extended Kalman filter. Check the covariance step, refit with tighter ODE tolerances, and compare with the model without it. -
[covariate_nn]lets a neural network produce covariate multipliers; compare its OFV and AIC with an explicit covariate model. - Centre and scale a network’s inputs with
centerandscalein the block;W_NN_UNSCALED_INPUTnames inputs that saturate it. Without them the warfarin network stops far above the optimum its scaled copy reaches. -
nn_l2andnn_smoothregularize a network. - An inline
prior(value, rse = …)regularizes a parameter towards a value you already believe, and[priors] from_fitbuilds such priors from a fit saved withferx_save_fit(); the reported OFV is then penalized. -
[diffusion]and[covariate_nn]fits carry anexperimentalwarning.
Next: Function, option and example index lists every function, option and example with the chapter that covers it.