Tensor

The Tensor class is probably the most important class in Torch. Almost every package depends on this class. It is the class for handling numeric data. Tensors are serializable.

Multi-dimensional matrix

A Tensor is a potentially multi-dimensional matrix. The number of dimensions is unlimited. Many methods have some convenience methods for for a number of dimensions inferior or equal to 4, but can also be used using LongStorage with more dimensions. Example:

 --- creation of a 4D-tensor 4x5x6x2
 z = torch.Tensor(4,5,6,2)
 --- for more dimensions, (here a 6D tensor) one can do:
 s = torch.LongStorage(6)
 s[1] = 4; s[2] = 5; s[3] = 6; s[4] = 2; s[5] = 7; s[6] = 3;
 x = torch.Tensor(s)

The number of dimensions of a Tensor can be queried by nDimension() or dim(). Size of the i-th dimension is returned by size(i). An LongStorage containing all the dimensions can be returned by size().

> print(x:nDimension())
6
> print(x:size())
 4
 5
 6
 2
 7
 3
[torch.LongStorage of size 6]

Internal data representation

The actual data of a Tensor is contained into a DoubleStorage. It can be accessed using storage(). While the memory of a Tensor has to be contained in this unique Storage, it might not be contiguous: the first position used in the Storage is given by storageOffset() (starting at 1). And the jump needed to go from one element to another element in the i-th dimension is given by stride(i). In other words, given a 3D tensor

x = torch.Tensor(7,7,7)

accessing the element (3,4,5) can be done by

= x[3][4][5]

or equivalently (but slowly!)

= x:storage()[x:storageOffset()
           +(3-1)*x:stride(1)+(4-1)*x:stride(2)+(5-1)*x:stride(3)]

One could say that a Tensor is a particular way of viewing a Storage: a Storage only represents a chunk of memory, while the Tensor interprets this chunk of memory as having dimensions:

> x = torch.Tensor(4,5)
> s = x:storage()
> for i=1,s:size() do -- fill up the Storage
>> s[i] = i
>> end
> print(x) -- s is interpreted by x as a 2D matrix

  1   5   9  13  17
  2   6  10  14  18
  3   7  11  15  19
  4   8  12  16  20
[torch.Tensor of dimension 4x5]

Note also that in Torch elements in the same column [elements along the first dimension] are contiguous in memory for a matrix [tensor]:

> x = torch.Tensor(4,5):zero()
> print(x)

0 0 0 0 0
0 0 0 0 0
0 0 0 0 0
0 0 0 0 0
[torch.Tensor of dimension 4x5]

> return  x:stride()
 1 -- element in the first dimension are contiguous!
 4
[torch.LongStorage of size 2]

This is like in Fortran (and not C), which allows us to efficiently interface Torch with standard numerical library packages.

Tensors of different types

Actually, several types of Tensor exists:

CharTensor -- contains chars
ShortTensor -- contains shorts
IntTensor -- contains ints
FloatTensor -- contains floats
Tensor -- contains doubles

It is recommended using only Tensor, as many of the numeric operations are not implemented for other classes. However, in some cases, you might want to use another class, e.g. to save memory space.

Efficient memory managment

All tensor operations in this class do not make any memory copy. All these methods transform the existing tensor, or return a new tensor referencing the same storage. This magical behavior is internally obtained by good usage of the stride() and storageOffset(). Example:

> x = torch.Tensor(5):zero()
> print(x)
0
0
0
0
0
[torch.Tensor of dimension 5]
> x:narrow(1, 2, 3):fill(1) -- narrow() returns a Tensor
                            -- referencing the same Storage than x
> print(x)
 0
 1
 1
 1
 0
[torch.Tensor of dimension 5]

If you really need to copy a Tensor, you can use the copy() method:

> y = torch.Tensor():resizeAs(x):copy(x)

We now describe all the methods for Tensor, but for the other variants, just replace Tensor by the name of the variant (like CharTensor).

Tensor

Subsections