The Optimizer Configuration Class

This class is used to configure the model training parameters. The class closely follows the PyTorch interface parameters.

class OptimizerConfig

Optimizer: str = adam, adagrad, adamw, lbfgs, rmsprop, sgd: Specifies the optimizer to use.

lr: float: Specifies the learning rate of the optimizer.

lr_decay: float: Specifies the learning rate decay.

weight_decay: float: Specifies the decay rate of the weights.

initial_accumulator_value

eps

tolerance_grad

tolerance_change

alpha

momentum

dampening

amsgrad

centered

nesterov

max_iter

max_eval

history_size

betas

The Optimizer Configuration Class in C++

The point of the Python interface is to control the workflow of the Analysis framework, however, some of the loss functions may require additional tuning or modifications. As such, the Optimizer class is given below with its class members.

#include <templates/fx_enums.h>
#include <templates/lossfx.h>

enum opt_enum: adam adagrad adamw lbfgs rmsprop sgd invalid_optimizer

enum mlp_init: uniform normal xavier_normal xavier_uniform kaiming_uniform kaiming_normal

enum loss_enum: bce bce_with_logits cosine_embedding cross_entropy ctc hinge_embedding huber kl_div l1 margin_ranking mse multi_label_margin multi_label_soft_margin multi_margin nll poisson_nll smooth_l1 soft_margin triplet_margin triplet_margin_with_distance invalid_loss

enum graph_enum: data_graph data_node data_edge truth_graph truth_node truth_edge

struct optimizer_params_t

std::string optimizer: Specifies the optimizer by string name..

cproperty<double, optimizer_params_t> lr

cproperty<double, optimizer_params_t> lr_decay

cproperty<double, optimizer_params_t> weight_decay

cproperty<double, optimizer_params_t> initial_accumulator_value

cproperty<double, optimizer_params_t> eps

cproperty<double, optimizer_params_t> tolerance_grad

cproperty<double, optimizer_params_t> tolerance_change

cproperty<double, optimizer_params_t> alpha

cproperty<double, optimizer_params_t> momentum

cproperty<double, optimizer_params_t> dampening

cproperty<bool, optimizer_params_t> amsgrad

cproperty<bool, optimizer_params_t> centered

cproperty<bool, optimizer_params_t> nesterov

cproperty<int, optimizer_params_t> max_iter

cproperty<int, optimizer_params_t> max_eval

cproperty<int, optimizer_params_t> history_size

cproperty<std::tuple<float, float>, optimizer_params_t> betas

cproperty<std::vector<float>, optimizer_params_t> beta_hack

bool m_lr

bool m_lr_decay

bool m_weight_decay

bool m_initial_accumulator_value

bool m_eps

bool m_betas

bool m_amsgrad

bool m_max_iter

bool m_max_eval

bool m_tolerance_grad

bool m_tolerance_change

bool m_history_size

bool m_alpha

bool m_momentum

bool m_centered

bool m_dampening

bool m_nesterov

class lossfx : public tools 

loss_enum loss_string(std::string name): Maps the input loss function string name to the loss_enum.

opt_enum optim_string(std::string name): Maps the input optimizer string name to the opt_enum.

torch::Tensor loss(torch::Tensor *pred, torch::Tensor *truth, loss_enum lss): Computes the loss between the prediction and the underlying truth tensors, given the loss_enum. Not all functions have been fully implemented, but can be easily defined under (src/AnalysisG/modules/lossfx/cxx/loss_config.cxx).

void weight_init(torch::nn::Sequential *data, mlp_init method): Specifies how the MLP weights should be initialized.

torch::optim::Optimizer *build_optimizer(optimizer_params_t *op, std::vector<torch::Tensor> *params): Constructs the optimizer with specified parameters.

bool build_loss_function(loss_enum lss): Constructs the loss function using the enum value.