multiDGD.latent

class multiDGD.latent.GaussianMixture(n_mix_comp: int, dim: int, covariance_type='diagonal', mean_init=(0.0, 1.0), sd_init=(0.5, 1.0), weight_alpha=1)

A mixture of multi-variate Gaussians.

Arguments

m_mix_comp: int

number of components in the mixture

dim: int

dimension of the space

Arguments (optional)

covariance_type: str

can be “fixed”, “isotropic” or “diagonal”

mean_init: tuple

mean_init = (radius, hardness) of the softball prior for the GMM means

sd_init: tuple

sd_init = (mean, sd) of the Gaussian prior for the standard deviations

weight_alpha: float

alpha parameter of the Dirichlet prior on the mixture weights

the mean_prior is initialized as a softball (mollified uniform) with

mean_init(<radius>, <hardness>)

neglogvar_prior is a prior class for the negative log variance of the mixture components

• neglogvar = log(sigma^2)

• If it is not specified, we make this prior a Gaussian from sd_init parameters

• For the sake of interpretability, the sd_init parameters represent the desired mean and (approximately) sd of the standard deviation

• the difference btw giving a prior beforehand and giving only init values is that with a given prior, the neglogvar will be sampled from it, otherwise they will be initialized the same

alpha determines the Dirichlet prior on mixture coefficients Mixture coefficients are initialized uniformly Other parameters are sampled from prior

Attributes

dim: int

dimensionality of the space

n_mix_comp: int

number of mixture components

mean: torch.nn.parameter.Parameter

learnable parameter for the GMM means with shape (n_mix_comp,dim)

neglogvar: torch.nn.parameter.Parameter

learnable parameter for the negative log-variance of the components shape depends on what covariances we take into account

diagonal: (n_mix_comp, dim) isotropic: (n_mix_comp) fixed: 0

weight: torch.nn.parameter.Parameter

learnable parameter for the component weights with shape (n_mix_comp)

Methods

clustering(x)

compute the cluster assignment (as int) for each sample

component_sample(n_sample)

returns a sample from each component

Tensor shape (n_sample,n_mix_comp,dim)

property covariance

return covariances of the components

forward(x)

Forward pass computes the negative log density of the probability of z being drawn from the mixture model

log_prob(x)

return the log density of the probability of z being drawn from the mixture model

mixture_probs()

transform weights to mixture probabilites

sample(n_sample)

draw samples from the GMM distribution

sample_new_points(resample_type='mean', n_new_samples=1)

Creates a Tensor with potential new representations

These can be drawn from component samples if resample_type is ‘sample’ or from the mean if ‘mean’. For drawn samples, n_new_samples defines the number of random samples drawn from each component.

sample_probs(x)

compute probability densities per sample without prior. returns tensor of shape (n_sample, n_mix_comp)

property stddev

return standard deviation of the components

class multiDGD.latent.GaussianMixtureSupervised(Nclass: int, Ncompperclass: int, dim: int, covariance_type='isotropic', mean_init=(0.0, 1.0), sd_init=(0.5, 1.0), alpha=1)

Supervised GaussianMixutre class (child of GaussianMixture)

Arguments

Nclass: int

number of classes to be modeled

Ncpc: int

number of components that should model each class

dim: int

dimension of the space

Arguments (optional)

covariance_type: str

can be “fixed”, “isotropic” or “diagonal”

mean_init: tuple

mean_init = (radius, hardness) of the softball prior for the GMM means

sd_init: tuple

sd_init = (mean, sd) of the Gaussian prior for the standard deviations

alpha: float

alpha parameter of the Dirichlet prior on the mixture weights

Methods

forward(x, label=None)

Forward pass computes the negative log density of the probability of z being drawn from the mixture model

If label is not provided, the function returns the unsupervised loss. Otherwise, it returns the supervised loss.

label_mixture_probs(label)

transform weights to mixture probabilites

log_prob(x, label=None)

return the log density of the probability of z being drawn from the mixture model

supervised_sampling(label, sample_type='random')

Sample from the GMM for supervised learning

class multiDGD.latent.RepresentationLayer(n_rep: int, n_sample: int, value_init='zero')

Implements a representation layer, that accumulates pytorch gradients.

Representations are vectors in n_rep-dimensional real space. By default they will be initialized as a tensor of dimension n_sample x n_rep at origin (zero).

One can also supply a tensor to initialize the representations (values=tensor). The representations will then have the same dimension and will assumes that the first dimension is n_sample (and the last is n_rep).

The representations can be updated once per epoch by standard pytorch optimizers.

Arguments

n_rep: int

dimensionality of the representation space

n_sample: int

number of samples to be modelled (has to match corresponding dataset)

Arguments (optional)

value_init: str or torch.nn.parameter.Parameter

defines how the representations are initialized - ‘zero’: all representations are initialized at zero - torch.Tensor: a tensor of shape (n_sample,n_rep) to initialize the representations - float: a float to initialize all representations (not recommended)

Attributes

n_rep: int

dimensionality of the representation space

n_sample: int

number of samples to be modelled (has to match corresponding dataset)

z: torch.nn.parameter.Parameter

tensor of shape (n_sample,n_rep) that holds the representations as learnable parameters

Methods

forward(idx=None)

Forward pass returns indexed representations