TensorLy is an open-source Python library that eases the task of performing tensor operations. It provides a high-level API for dealing with deep tensorized neural networks and tensor methods. It was created in 2015 by a senior research scientist at NVIDIA Research Group named Jean Kossaifi. It was presented at the NeurIPS workshop and was later published byJean Kossaifi, Yannis Panagakis, Anima Anandkumar and Maja Pantic in a JMLR paper in February 2019.

The project is supported by the following universities and organizations:

#### THE BELAMY

##### Sign up for your weekly dose of what's up in emerging technology.

TensorLy makes it easy to handle tensor decomposition, tensor learning and tensor algebra. It’s robust backend system enables users to perform computations with NumPy, TensorFlow, PyTorch, MXNet, CuPy or JAX. It can run methods that can scale on multiple CPUs or GPUs.

TensorLy was initially built on the top of NumPy and SciPy with a soft dependency on plotting library Matplotlib. A backend system was added to it later on for combining tensor methods with deep learning and for enabling transparent switching between various libraries and platforms.. It provides a simple and efficient API such as that of scikit-learn. However, scikit-learn deals with observations in terms of vectors whereas TensorLy represent them as higher-order arrays called tensors.

The methods, functionalities and libraries supported be summarized in the following diagrams:

## Installation of TensorLy

**NOTE: TensorLy has been developed and tested only on Python 3 version.**

- Install the library using pip command:

`pip install -U tensorly `

(option -U above is optional; use it for packages updates)

- Install it using conda from tensorly channel:

`conda install -c tensorly tensorly`

## Practical implementation

Here, we demonstrate how to use this library for basic tensor operations, tensor decomposition and tensor regression. Step-wise explanation for code of each of the use cases is as follows:

### Some basic tensor operations

- Import required libraries

import numpy as np import tensorly as tl from tensorly.testing import assert_array_equal #assert_array_equal() raises an error if two array-like objects are unequal

- Create a tensor (it is nothing but a numpy array)

`orig_tensor = tl.tensor(np.arange(40).reshape((5, 2, 4)))`

A 3D array containing elements from 0 to 39 will be created’

Print the created tensor

orig_tensor = tl.tensor(np.arange(40).reshape((5, 2, 4))) print('Original tensor:\n{}'.format(orig_tensor)) Output:

- Unfolding the tensor

for m in range(tensor.ndim): #for each dimension of the tensor print('Mode-{} unfolding:\n{}'.format(m, tl.unfold(tensor, m)))

unfold() method performs unfolding starting from mode 0, and unfolds till (n-1)th dimension (where, ‘n’ is the dimension of the tensor).

- Refolding the tensor

for mode in range(tensor.ndim): #for each dimension of the tensor unf = tl.unfold(tensor, mode) #unfold the tensor first f = tl.fold(unf, mode, tensor.shape) #re-fold the tensor #give an error if the refolded tensor and original tensor are unequal assert_array_equal(f, tensor)

The unfolded tensor after executing above lines of code:

The re-folded tensor will be as follows:

### Tensor decomposition example

Here, we demonstrate compression of colored image of a raccoon using parafac decomposition and Tucker decomposition techniques.

- Install required libraries

import matplotlib.pyplot as plt import tensorly as tl import numpy as np from scipy.misc import face #for the raccoon’s face image #to zoom the image represented in the form of an array from scipy.ndimage import zoom from tensorly.decomposition import parafac from tensorly.decomposition import tucker from math import ceil

- Load the raccoon face image

img = face() #Represent the image as a tensor img = tl.tensor(zoom(face(), (0.3, 0.3, 1)), dtype='float64')

- Define a function to convert image tensor’s datatype from float to unsigned integer

def to_img(tensor): image = tl.to_numpy(tensor) #Convert the tensor to a numpy array #Subtract minimum element from the array from each of the elements image -= image.min() #Divide the modified elements by the maximum array element image /= image.max() #Multiply the elements by 255 (0-255 is pixel range for colored images) image *= 255 #Change the datatype to unit8 (unsigned integer) return image.astype(np.uint8)

- Define ranks of decomposition

# Rank of CP decomposition cp_rank = 25 # Rank of Tucker decomposition tucker_rank = [100, 100, 2]

‘Rank’ here refers to the number of directions required to describe it.

- Perform CP decomposition

weights, factors = parafac(img, rank=cp_rank, init='random', tol=10e-6) # Reconstruct the image from the factors cp_rec = tl.cp_to_tensor((weights, factors))

- Perform Tucker decomposition

core, factors = tucker(img, rank=tucker_rank, init='random', tol=10e-5, random_state=12345) #Convert the tucker tensor into a full tensor tucker_rec = tl.tucker_to_tensor((core, factors))

- Plot the original image

fig = plt.figure() ax = fig.add_subplot(1, 3, 1) #arguments represent (number of rows, number of columns, index) ax.set_axis_off() #Turn the X and Y axes off ax.imshow(to_img(img)) #Display the plot ax.set_title('Original Image') #Title of the plot

- Similarly, plot the image compressed using CP decomposition and Tucker decomposition.

ax = fig.add_subplot(1, 3, 2) ax.set_axis_off() ax.imshow(to_img(cp_rec)) ax.set_title('CP decomposition') ax = fig.add_subplot(1, 3, 3) ax.set_axis_off() ax.imshow(to_img(tucker_rec)) ax.set_title('Tucker Decomposition') #Adjust the padding between and around the subplots plt.tight_layout() #Display the plots plt.show()

**Output:**

### Tensor regression example

- Import required libraries

import matplotlib.pyplot as plt from tensorly.base import tensor_to_vec, partial_tensor_to_vec from tensorly.datasets.synthetic import gen_image from tensorly.random import check_random_state from tensorly.regression.cp_regression import CPRegressor import tensorly as tl

- Set image parameters

img_ht = 25 #height of the image img_width = 25 #width of the image # shape of the images patterns = ['swiss', 'circle'] # ranks for testing ranks = [1, 2, 3, 4, 5]

- Generate random samples

r = check_random_state(1) #Tensor representation of the samples X = tl.tensor(r.normal(size=(1000, img_ht, img_width), loc=0, scale=1)) #numpy.random.RandomState.normal() draws random samples from a normal distribution

- Parameters of the plot

n_rows = len(patterns) n_columns = len(ranks) + 1

- Plot the images before and after applying regression

fig = plt.figure() for i, pattern in enumerate(patterns): # Generate the original image for regression testing wt_img = gen_image(region=pattern, image_height=img_ht, image_width=img_width) wt_img = tl.tensor(wt_img) #Convert the image into tensor # Generate the labels y = tl.dot(partial_tensor_to_vec(X, skip_begin=1), tensor_to_vec(wt_img)) # Plot the original weights ax = fig.add_subplot(n_rows, n_columns, i*n_columns + 1) ax.imshow(tl.to_numpy(wt_img), cmap=plt.cm.OrRd, interpolation='nearest') ax.set_axis_off() if i == 0: ax.set_title('Original\nweights') for j, rank in enumerate(ranks): # Create a tensor Regressor estimator est = CPRegressor(weight_rank=rank, tol=10e-7, n_iter_max=100, reg_W=1, verbose=0) # Fit the estimator to the data est.fit(X, y) #Plot the images after regression ax = fig.add_subplot(n_rows, n_columns, i*n_columns + j + 2) ax.imshow(tl.to_numpy(est.weight_tensor_), cmap=plt.cm.OrRd, interpolation='nearest') ax.set_axis_off() if i == 0: ax.set_title('Learned\nrank = {}'.format(rank)) plt.suptitle("CP tensor regression") #Add a centered title to the figure plt.show() #Display the plot

**Output:**

**Google colab notebooks**of the above implementations:

## References

Refer to the following sources to get an in-depth understanding of TensorLy Python library: