We now study the case of tensors with and . At worst, the maximum rank is . By unfolding such a tensor to a matrix, one can see that rank is definitely achievable.
We are interested in analyzing the tensor
which has nominal rank 9. Consider the tensor-valued function
and notice that the derivative of at 0 is , i.e. . Since
we see that is a limit of rank 2 tensors. (To avoid loss-of-precision errors due to cancellation, larger rank is needed; see [BEY-MOH2005].)
To visualize , we choose
so that will be in the center of these three tensors. Since these three tensors are in , any tensor on the line connecting any two of them is in , and a tensor in the plane is generically in .
We first produce a visualization of on , and obtain
[data file 2to9pr2L9.dat] Black points are scattered sparsely on the visualization, but without apparent voids. It appears that the ALS routine had some difficulty (with its fixed number of iterations). There is a light horizontal streak along the line connecting to of unknown cause, probably a numerical artifact.
Next, we produce a visualization of , and obtain
[data file 2to9pr3L9.dat]
Next, we produce a visualization of , which includes the lines connecting any two of our base tensors, and obtain
[data file 2to9pr6L9.dat]
To complement the specific example above, we choose , , and randomly in . Since these three tensors are in , any tensor on the line connecting any two of them is in , and a tensor in the plane is generically in .
We first produce a visualization of on , and obtain
[data file 2to9pr3random.dat] Next, using the same set of random , , and , we produce a visualization of , and obtain
[data file 2to9pr6random.dat] Since in neither case do we find any unexpected black points, we conclude that the specific example above is not typical.
To complement the specific example above, we choose , , and randomly in . Since these three tensors are in , any tensor on the line connecting any two of them is in , and a tensor in the plane is generically in .
We first produce a visualization of on , and obtain
[data file 2to9pr16random2.dat] Next, using the same set of random , , and , we produce a visualization of , and obtain
[data file 2to9pr32random2.dat]
Next, using the same set of random , , and , we produce a visualization of , and obtain
[data file 2to9pr40random2.dat]
Finally, we produce a visualization of , which should be black, and obtain
[data file 2to9pr48random2.dat] Although we do not obtain black, we do obtain dark blue, indicating that the fitting is working fairly well.
The first time we ran the tests for this example, we used the ALS algorithm at fixed rank rather than growing the rank (see ALS Performance Notes). That algorithm performed much worse, as illustrated in the Section A note on ALS failure.