crumpets.torch.metrics module¶
- class crumpets.torch.metrics.AccuracyMetric(top_k=1, output_key='output', target_key='label')[source]¶
Bases:
crumpets.torch.metrics.MetricComputes the top-k accuracy metric for given classification scores, i.e. predicted class probabilities. The metric is computed as {1 if target_i in top_k_predicted_classes_i else 0 for all i in n} / n
- Parameters
output_key – the key with which the output is found in the input dictionary
target_key – the key with which the target is found in the input dictionary
- class crumpets.torch.metrics.AverageMetric(output_key='output', metric_key='average_metric')[source]¶
Bases:
crumpets.torch.metrics.MetricComputes a simple average metric for given values inside the output.
- Parameters
output_key – the key with which the output is found in the input dictionary
metric_key – the key with which the metric is to be stored in the output dictionary
- class crumpets.torch.metrics.CombinedMetric(children)[source]¶
Bases:
objectA simple meta metric. Given metric instances, returns a collection of them.
- Parameters
children – list of metric class instances
- class crumpets.torch.metrics.ConfusionMatrix(nclasses=10, output_key='output', target_key='target_image', metric_key='confusion_matrix')[source]¶
Bases:
crumpets.torch.metrics.MetricComputes the confusion matrix for given classification scores, i.e. predicted class probabilities.
- Parameters
output_key – the key with which the output is found in the input dictionary
target_key – the key with which the target is found in the input dictionary
metric_key – the key with which the metric is to be stored in the output dictionary
- get_true_false_positives()[source]¶
Calculate the true positive and false positive rates per class :return: 2d-array. Cx3 array where the first column corresponds
to the true positives per class, the second column, to the false positives per class and the last one, the number of samples per class in total that have been seen.
- class crumpets.torch.metrics.MSELossMetric(output_key='output', target_key='target_image', metric_key='mse')[source]¶
Bases:
crumpets.torch.metrics.MetricComputes the mean squared error
- Parameters
output_key – the key with which the output is found in the input dictionary
target_key – the key with which the target is found in the input dictionary
metric_key – the key with which the metric is to be stored in the output dictionary
- class crumpets.torch.metrics.Metric(output_key='output', target_key='target_image', metric_key='metric')[source]¶
Bases:
objectAbstract class which is to be inherited by every metric. As usual, this class is designed to handle crumpets dictionaries.
- Parameters
output_key – the key with which the output is found in the input dictionary
target_key – the key with which the target is found in the imput dictionary
metric_key – the key with which the metric is to be stored in the output dictionary
- class crumpets.torch.metrics.NSSMetric(output_key='output', target_key='target_image', metric_key='nss')[source]¶
Bases:
crumpets.torch.metrics.MetricComputes the Normalized Scanpath Saliency (NSS) by Bylinskii et. al. (https://arxiv.org/pdf/1604.03605.pdf)
- Parameters
output_key – the key with which the output is found in the input dictionary
target_key – the key with which the target is found in the input dictionary
metric_key – the key with which the metric is to be stored in the output dictionary
- class crumpets.torch.metrics.NoopMetric(output_key='output', target_key='target_image', metric_key='metric')[source]¶
Bases:
crumpets.torch.metrics.MetricProvides the same API as a real metric but does nothing. Can be used where some metric-like object is required, but no actual metrics should be calculated.