PyTorch releases TorchMetrics version 0.7

Version 0.7 has 60+ metrics that cover eight domains including audio, classification, image, pairwise, detection, regression, information retrieval, and text.

Advertisement

PyTorch has launched TorchMetrics v0.7. The fresh release includes several new metrics (mainly for NLP), naming and import changes, general improvements to the API among other features.

Version 0.7 has 60+ metrics that cover eight domains including audio, classification, image, pairwise, detection, regression, information retrieval, and text. The team has also unified TorchMetrics API across all domains and has reached over 600 stars. Currently, 1,500 repositories use TorchMetrics.

THE BELAMY

Sign up for your weekly dose of what's up in emerging technology.

The changes in NLP metrics include text package which in the v 0.7 includes a couple of machine translation metrics such as chrF, chrF++, Translation Edit Rate, or Extended Edit Distance. It now also supports other metrics — Match Error Rate, Word Information Lost, Word Information Preserved, and SQuAD evaluation metrics. The team has also made possible the evaluation of the ROUGE score using multiple references.

Other updates include Translation Edit Rate (TER) which represents a normalized minimum number of edits required for a hypothesis that matches one of the provided references. Edits comprise insertion, deletion, and substation on a word or n-gram word level. Extended Edit Distance is also one of the updates that operate at a character level, and it extends Levenshtein distance using jump operations. 

Match Error Rate (MRE), a metric used to evaluate automatic speech recognition (ASR) systems, has been added. Word Information Lost (WIL) and Word Information Preserved (WIP) are two interwoven metrics used predominantly to evaluate ASR output. SQuAD, which stands for Stanford Question Answering Dataset, is another update. It is a machine comprehension dataset of 100,000+ pairs of questions and Wikipedia articles with selected answers. The dataset is used for the evaluation of extractive models.

TorchMetrics v0.7 brings more extensive changes to how metrics should be imported. The import changes directly impact v0.7, which will require developers to change the import statement for some specific metrics. All naming changes follow the standard deprecation process. From v0.8, the old metric names will no longer be available.

A complete list of metrics undergoing a naming change can be found in the changelog.

More Great AIM Stories

Meeta Ramnani
Meeta’s interest lies in finding out real practical applications of technology. At AIM, she writes stories that question the new inventions and the need to develop them. She believes that technology has and will continue to change the world very fast and that it is no more ‘cool’ to be ‘old-school’. If people don’t update themselves with the technology, they will surely be left behind.

Our Upcoming Events

Conference, in-person (Bangalore)
MachineCon 2022
24th Jun

Conference, Virtual
Deep Learning DevCon 2022
30th Jul

Conference, in-person (Bangalore)
Cypher 2022
21-23rd Sep

3 Ways to Join our Community

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Telegram Channel

Discover special offers, top stories, upcoming events, and more.

Subscribe to our newsletter

Get the latest updates from AIM
MORE FROM AIM
Amit Raja Naik
Oh boy, is JP Morgan wrong?

The global brokerage firm has downgraded Tata Consultancy Services, HCL Technology, Wipro, and L&T Technology to ‘underweight’ from ‘neutral’ and slashed its target price by 15-21 per cent.