TENSOR NETWORKS (Part 1 )

Anurag

2022/11/29 15:16

#python ##tensornetworks

TERMINOLOGY

Within the blogs we use the following conventions

The order of tensor is defined as the number of indices it has.

The rank of tensor w.r.t. some partition of the indices specifies the minimum dimension of exact factorization on this partition.

The use of dimension will refer almost exclusively to the size of a tensor index.

Tensor Contractions

Diagrammatic Notion

For our present purpose, tensors are simply multi-dimensional arrays of (real or complex) numbers. Below are some useful ways to initialize tensors:


##### Lets initialize some tensors in Python/Numpy
import numpy as np
# tensor with randomly generated entries, order 3, dims: 2-by-3-by-4
A = np.random.rand(2,3,4)

# identity matrix, order 2, dims: 5-by-5 
B = np.eye(5,5)

# tensor of 1's, order 4, dims: 2-by-4-by-2-by-4
C = np.ones((2,4,2,4))

# matrix of 0's, order 2, dims: 3-by-5
D = np.zeros((3,5))

# initialize complex random tensor
E = np.random.rand(2,3,4) + 1j*np.random.rand(2,3,4)


'A',A

('A',

 array([[[0.22071491, 0.59504767, 0.2342386 , 0.2624513 ],

         [0.34106352, 0.5609685 , 0.08887758, 0.668122  ],

         [0.56270383, 0.66318894, 0.45590939, 0.55599489]],

 

        [[0.69943473, 0.97028887, 0.84672101, 0.34292566],

         [0.05030154, 0.5054766 , 0.91616081, 0.19324619],

         [0.39140383, 0.79112529, 0.71370912, 0.83755626]]]))


'E',E

('E',

 array([[[0.70417777+0.61556297j, 0.27641694+0.89418594j,

          0.59738581+0.51387976j, 0.13384181+0.69059992j],

         [0.96717691+0.44700053j, 0.3490143 +0.28283196j,

          0.30949638+0.89504197j, 0.72703288+0.00763742j],

         [0.70391125+0.58582417j, 0.36047394+0.77508743j,

          0.02315564+0.87890043j, 0.19335745+0.27904282j]],

 

        [[0.76326689+0.5338626j , 0.55890551+0.13726491j,

          0.94700548+0.36418387j, 0.85308451+0.15534306j],

         [0.50231328+0.74404516j, 0.17532106+0.07760074j,

          0.49511942+0.98033376j, 0.60910344+0.64696933j],

         [0.66456822+0.58421971j, 0.04340862+0.18556514j,

          0.76630915+0.01360477j, 0.5717322 +0.62277569j]]]))

It is convenient to represent tensor networks using a diagrammatic notation, where individual tensors are represented as a solid shape with a number of 'legs' that corresponds to the rank of the tensor. Each leg is here labelled with a dummy index (usually a Latin letter: i, j, k, l…) necessary to relate the equation to the diagram. Some examples are presented below.

The diagrammatic tensor notation is useful for describing networks comprised of multiple tensors. An index shared by two tensors denotes a contraction (or summation) over this index. Examples:

Notice that example in Fig.1.1(b) is equivalent to a matrix multiplication between matrices A and B, while Fig.1.1(c) produces a rank-3 tensor D via the contraction of a network with three tensors. Even in this relatively the simple example, we see that the diagrammatic notation is already easier to interpret than the corresponding index equation. In practice, once we have established a convention for index ordering, we can omit labeling the diagrams with dummy indices which further enhances their clarity.

Permute and reshape operations

Fundamental to the manipulation of tensors are the 'permute' and ‘reshape’ functions. Permute allows the index ordering of a tensor to be changed (but does not change the number of indices). The reshape function which allows a collection of tensor indices to be combined into a single larger index (or vice-versa), thus can change the number of indices but not the total dimension. In equations, we sometimes bracket subscript indices to denote the combining of indices, while diagrammatically this may be indicated by drawing two tensor indices coming together to form a single index. Often tensor indices of larger dimension are depicted by thicker lines!


#1.2(a):Transpose
A = np.random.rand(4,4,4,4)
Atilda = A.transpose(3,0,1,2)
# 1.2(b):Reshape
B = np.random.rand(4,4,4)
Btilda = B.reshape(4,4**2)

array([[[0.22071491, 0.59504767, 0.2342386 , 0.2624513 ],

        [0.34106352, 0.5609685 , 0.08887758, 0.668122  ],

        [0.56270383, 0.66318894, 0.45590939, 0.55599489]],



       [[0.69943473, 0.97028887, 0.84672101, 0.34292566],

        [0.05030154, 0.5054766 , 0.91616081, 0.19324619],

        [0.39140383, 0.79112529, 0.71370912, 0.83755626]]])


'A_tilda=',Atilda

('A_tilda=',

 array([[[[0.05063179, 0.13344574, 0.21752414, 0.43307946],

          [0.95795843, 0.74128247, 0.34395473, 0.55075083],

          [0.96216076, 0.72420579, 0.02714007, 0.52031218],

          [0.38005031, 0.10748791, 0.53601589, 0.18714466]],

 

         [[0.73124584, 0.10845123, 0.49304379, 0.73750189],

          [0.40342111, 0.68584472, 0.37158738, 0.67621608],

          [0.81572957, 0.52117896, 0.44531824, 0.03538727],

          [0.81619318, 0.38379751, 0.51328942, 0.76740076]],

 

         [[0.36663813, 0.95005559, 0.61538445, 0.85997137],

          [0.85506229, 0.97190106, 0.37343688, 0.47471119],

          [0.05229643, 0.34701343, 0.91118348, 0.32174047],

          [0.7480279 , 0.66761465, 0.71515288, 0.85339643]],

 

         [[0.62664432, 0.32844726, 0.732003  , 0.84540219],

          [0.25732219, 0.02336443, 0.60211847, 0.63829483],

          [0.28426601, 0.97928983, 0.59010281, 0.24631594],

          [0.99080759, 0.91797111, 0.04200871, 0.82929719]]],

 

 

        [[[0.05380305, 0.36733297, 0.78173395, 0.22819548],

          [0.96022688, 0.17921713, 0.21443936, 0.99032404],

          [0.7007594 , 0.58991249, 0.34260435, 0.36261853],

          [0.15635759, 0.18456865, 0.63181277, 0.6913886 ]],

 

         [[0.02289414, 0.36695668, 0.69668956, 0.96699151],

          [0.50880907, 0.11201324, 0.53936114, 0.91717469],

          [0.32088699, 0.21683368, 0.48917752, 0.43572336],

          [0.70153931, 0.14532706, 0.30790469, 0.41328408]],

 

         [[0.55167498, 0.21268863, 0.88783238, 0.24320936],

          [0.22124536, 0.92698375, 0.192379  , 0.16392322],

          [0.33603904, 0.48869718, 0.6113266 , 0.22285526],

          [0.64898157, 0.61094171, 0.7844977 , 0.00336131]],

 

         [[0.29988452, 0.95329866, 0.36550443, 0.84998751],

          [0.20101572, 0.92864226, 0.9361027 , 0.66231203],

          [0.41835254, 0.74948788, 0.74187157, 0.69931728],

          [0.97258988, 0.4186533 , 0.96028212, 0.70155399]]],

 

 

        [[[0.52682043, 0.03539295, 0.42950029, 0.1424351 ],

          [0.48172574, 0.96343456, 0.22501848, 0.06989187],

          [0.53968117, 0.43411205, 0.65159227, 0.0589162 ],

          [0.59743344, 0.85657027, 0.7791293 , 0.82463573]],

 

         [[0.87365156, 0.06281593, 0.85151053, 0.53910877],

          [0.95431811, 0.90799098, 0.3645519 , 0.22596976],

          [0.10806299, 0.18816252, 0.14475751, 0.88097275],

          [0.9678396 , 0.56994075, 0.83540845, 0.00906168]],

 

         [[0.29119121, 0.7157452 , 0.65976344, 0.99171367],

          [0.24917614, 0.20307506, 0.65179207, 0.3524731 ],

          [0.51645859, 0.21844414, 0.12890568, 0.61582453],

          [0.7537333 , 0.20836987, 0.09517765, 0.07121146]],

 

         [[0.77714297, 0.46200291, 0.27552964, 0.47511335],

          [0.61325435, 0.30084784, 0.12114828, 0.89470458],

          [0.370553  , 0.93517838, 0.77562993, 0.15913146],

          [0.21183943, 0.85343722, 0.1871007 , 0.89048146]]],

 

 

        [[[0.31942134, 0.24435695, 0.96303098, 0.64245123],

          [0.7949898 , 0.81695724, 0.21314605, 0.66775016],

          [0.91542406, 0.0207701 , 0.81105694, 0.20748279],

          [0.29511373, 0.65667101, 0.95462243, 0.70012656]],

 

         [[0.2388992 , 0.9437981 , 0.13812801, 0.00786508],

          [0.39681751, 0.87957376, 0.51601844, 0.28542073],

          [0.51212272, 0.16561104, 0.78139806, 0.73900278],

          [0.31956039, 0.78428302, 0.39149058, 0.59589496]],

 

         [[0.99991359, 0.81220234, 0.34470577, 0.35128717],

          [0.08999906, 0.88028316, 0.86430344, 0.80181015],

          [0.97421951, 0.26912719, 0.23687376, 0.50023797],

          [0.35123655, 0.7004479 , 0.35896991, 0.15029894]],

 

         [[0.21265105, 0.07571621, 0.99536556, 0.52406018],

          [0.29790039, 0.0485201 , 0.43837551, 0.06734798],

          [0.3050784 , 0.46150536, 0.96835139, 0.55296364],

          [0.31370961, 0.01752636, 0.43870367, 0.39411275]]]]))

array([[1., 0., 0., 0., 0.],

       [0., 1., 0., 0., 0.],

       [0., 0., 1., 0., 0.],

       [0., 0., 0., 1., 0.],

       [0., 0., 0., 0., 1.]])


Btilda

array([[0.93461258, 0.91754787, 0.18708068, 0.42891464, 0.94253387,

        0.24788901, 0.91413355, 0.06954167, 0.86851196, 0.68505564,

        0.22435909, 0.03939557, 0.0193083 , 0.46292828, 0.83080227,

        0.11599764],

       [0.84371855, 0.78888855, 0.3195639 , 0.52063825, 0.19467885,

        0.12871202, 0.38399244, 0.23485948, 0.5138507 , 0.87004137,

        0.77097442, 0.73391217, 0.81139732, 0.84300132, 0.82770379,

        0.314837  ],

       [0.81522689, 0.18803904, 0.66767431, 0.31061904, 0.54649877,

        0.68972958, 0.90093313, 0.94425553, 0.75107549, 0.74056145,

        0.04851633, 0.49266691, 0.41305132, 0.10049238, 0.90258146,

        0.24913893],

       [0.99634113, 0.28333667, 0.24559663, 0.42829515, 0.23730978,

        0.20182613, 0.68319769, 0.35468402, 0.67548828, 0.96506642,

        0.23883669, 0.83055588, 0.83447176, 0.36858594, 0.56519168,

        0.86411716]])

Binary tensor contractions

The usefulness of permute and reshape functions is that they allow a contraction between a pair of tensors (which we call a binary tensor contraction) to be recast as a matrix multiplication. Although the computational cost (measured in number of scalar multiplications) is the same both ways, it is usually preferable to recast as multiplication as modern hardware performs vectorized operations much faster than when using the equivalent FOR loop. The steps for doing this are outlined below:


# Ex.1.3(a): Binary Tensor Contraction
d = 10
A = np.random.rand(d,d,d,d)  
B = np.random.rand(d,d,d,d)

Ap  = A.transpose(0,2,1,3)
Bp = B.transpose(0,3,1,2)
App = Ap.reshape(d**2,d**2)
Bpp = Bp.reshape(d**2,d**2)
Cpp = App @ Bpp
C   = Cpp.reshape(d,d,d,d)

Contraction costs

The computational cost of multiplying a d1-by-d2 dimensional matrix A with a d2-by-d3 dimensional matrix B is: cost(A×B) = d1∙d2∙d3. Given the equivalence with matrix multiplication, this is also the cost of a binary tensor contraction (where each dimension d1, d2, d3 may now result as the product of several tensor indices from the reshapes).

Broadly speaking, there are two approaches that could be taken to contract a network containing N>2 tensors: (i) in a single step as a direct summation over all internal indices of the network or (ii) as a sequence of N-1 binary contractions. In practice we prefer the latter option, which is either computationally cheaper or an equivalent cost as the former option. Examples:


# 1.4(c): Tensor network evaluation
d = 10
A = np.random.rand(d,d) 
B = np.random.rand(d,d)
C = np.random.rand(d,d)
# Evaluare network via summation over internal indices
F0 = np.zeros((d,d))
for di in range(d):
    for dj in range(d):
        for dk in range(d):
            for dl in range(d):
                F0[di,dj] = F0[di,dj] + A[di,dk]*B[dk,dl]*C[dl,dj]
            
# Evaluare network via sequence of binary contractions
F1 = (A @ B) @ C

Fig.1.4(c), which represents a product of three matrices, illustrates that it is more efficient (in terms of the total number of scalar multiplications) to evaluate the network as a sequence of binary contractions than as a single summation over all internal indices.

Fig.1.4(d) illustrates that the total cost of contracting a tensor network can depend on the sequence of binary contractions used; here the optimal sequence depends on whether D is larger than d.

Contraction of tensor networks

Given a tensor network composed of N tensors, there are two distinct steps needed to contract the network efficiently:

determine the optimal sequence of the (N-1) binary tensor contractions,

evaluate each of the binary contractions in turn as a matrix multiplication by taking the proper tensor permutes and reshapes.

Once the optimal contraction sequence has been determined, a network can be evaluated by implementing each of the binary contractions in turn. However, using ‘reshape’ and ‘permute’ commands for each binary tensor contraction, although computationally effective, has two significant drawbacks: (i) it results in lengthy code that is error prone and difficult to check and (ii) it does not allow for the contraction sequence to be easily changed (as, in general, the entire code for the contraction would need to be rewritten to accommodate a different ordering).

Network contractor ‘ncon’:

The ‘ncon’ function is a useful tool to lessen the programming effort required to implement a tensor network contraction. This function works by automatically performing a desired sequence of permutes, reshapes and matrix multiplications required to evaluate a tensor network. The ‘ncon’ code and detailed instructions for its usage can be found here, or alternatively the code is also presented on the example code page. The first step in using ‘ncon’ to evaluate a network is to make a labelled diagram of the network such that:

Each internal index is labelled with a unique positive integer (typically sequential integers starting from 1, although this is not necessary).

External indices of the diagram (if there are any) are labelled with sequential negative integers [-1,-2,-3,…] which denote the desired index order on the final tensor (with -1 as the first index, -2 as the second etc).

Following this, the ‘ncon’ routine is called as follows,

OutputTensor = ncon(TensorArray, IndexArray, ContOrder) with input arguments defined:

TensorArray: 1D cell array containing the tensors comprising the network

IndexArray: 1D cell array of vectors, where the kth element is a vector of the integer labels from the diagram on the kth tensor from ‘TensorArray’ (ordered following the corresponding index order on this tensor).

ContOrder: a vector containing the positive integer labels from the diagram, used to specify order in which ‘ncon’ contracts the indices. Note that ‘ContOrder’ is an optional input that can be omitted if desired, in which case ‘ncon’ will contract in ascending order of index labels.


!pip install ncon

Collecting ncon

  Downloading ncon-1.0.0-py3-none-any.whl (7.8 kB)

Requirement already satisfied: numpy>=1.11.0 in /opt/conda/lib/python3.10/site-packages (from ncon) (1.21.0)

Installing collected packages: ncon

Successfully installed ncon-1.0.0


#Contraction using ncon
from ncon import ncon
d = 10
A = np.random.rand(d,d,d); B = np.random.rand(d,d,d,d)
C = np.random.rand(d,d,d); D = np.random.rand(d,d)

TensorArray = [A,B,C,D]
IndexArray = [[1,-2,2],[-1,1,3,4],[5,3,2],[4,5]]

E = ncon(TensorArray,IndexArray,[5,3,4,1,2])

array([[5856.73472948, 6382.79801301, 6289.63750687, 6179.42973476,

        6028.69527438, 6910.19314052, 6446.63465133, 6965.20920493,

        6738.25785533, 6155.25778531],

       [5883.96655847, 6418.09942668, 6328.27167962, 6206.72048511,

        6089.29655223, 6964.32389442, 6465.58598973, 6990.12771246,

        6810.45073336, 6166.61459786],

       [5579.25887576, 6106.84149898, 6020.47668542, 5900.53568125,

        5753.76081567, 6622.76323996, 6096.44000681, 6696.45914798,

        6485.23472692, 5839.02851294],

       [5751.911624  , 6277.79410828, 6218.43354263, 6050.69757035,

        5977.97617768, 6785.74261998, 6329.8758422 , 6822.03714655,

        6686.29737372, 6057.61786122],

       [5851.72667474, 6384.41211773, 6293.871696  , 6149.59968021,

        6035.54476359, 6951.84183382, 6402.4259244 , 7012.33609077,

        6799.78027371, 6099.4999679 ],

       [5854.27385381, 6404.13991157, 6276.44893798, 6191.88070797,

        6003.15456136, 6931.33920325, 6428.04352152, 6951.00815823,

        6775.79105336, 6118.73573932],

       [5761.36857412, 6301.95003187, 6217.84595594, 6095.03370217,

        5986.25501698, 6852.95918833, 6299.35347365, 6899.99070762,

        6719.85584999, 6036.83860602],

       [5701.88640744, 6246.54089534, 6127.48067351, 6031.40624712,

        5889.65319875, 6741.04596923, 6257.02275113, 6798.72592272,

        6628.4206779 , 5994.25281472],

       [5768.34021275, 6291.64195359, 6225.03026421, 6095.84521075,

        5951.52192788, 6843.71217377, 6335.94548907, 6875.95569841,

        6691.57959334, 6050.377313  ],

       [5837.62240988, 6426.66282938, 6293.83522196, 6203.5958726 ,

        5971.67080959, 6901.75991934, 6406.68887163, 6918.5853792 ,

        6762.72104507, 6153.16688741]])


#  Partial trace
d = 10
A = np.random.rand(d,d,d,d,d,d)

B = ncon([A],[[-1,-2,1,-3,-4,1]])

#  Disjoint networks
d = 10
A = np.random.rand(d,d)
B = np.random.rand(d,d)

C = ncon([A,B],[[-1,-2],[-3,-4]])

array([[[7.77495379e-01, 4.08601966e-01, 5.98819058e-01, 8.25968349e-01,

         9.19363410e-01, 3.26636546e-01, 9.07372519e-01, 4.57832392e-02,

         3.85671016e-02, 5.73085085e-01],

        [9.92778297e-01, 2.15599700e-01, 6.44562499e-01, 3.31444425e-01,

         4.82761674e-01, 9.93179199e-01, 9.28644973e-01, 8.81916475e-01,

         5.12734111e-02, 6.04657326e-01],

        [3.16098051e-01, 7.57449673e-01, 7.16645100e-01, 2.88477796e-01,

         6.80569833e-01, 3.01271700e-01, 7.06758920e-01, 2.53720908e-01,

         3.11772798e-01, 8.01146088e-01],

        [6.36768432e-01, 5.06123477e-01, 7.73979563e-01, 8.94598206e-01,

         4.51316950e-01, 2.88015206e-01, 8.30087771e-02, 1.84401003e-01,

         1.27139575e-01, 2.87839317e-01],

        [9.31431094e-01, 8.30553832e-01, 7.70124258e-01, 3.57891355e-01,

         2.51857928e-01, 7.88073985e-01, 8.76046101e-01, 5.13190142e-01,

         6.64183721e-01, 5.28190525e-01],

        [6.81466301e-01, 3.14307699e-02, 9.32358198e-01, 7.26165628e-01,

         1.94448079e-01, 5.86560174e-01, 8.07679027e-01, 1.64487257e-02,

         9.25203227e-01, 1.15574525e-01],

        [2.63233958e-01, 8.68752171e-01, 1.47650331e-01, 5.95303169e-02,

         2.78081797e-01, 2.36474121e-02, 1.20919363e-01, 6.76455339e-01,

         6.56901510e-01, 6.11847811e-01],

        [7.91436909e-01, 4.01187581e-01, 7.44697217e-01, 6.99005813e-01,

         8.07947392e-01, 4.48422081e-01, 6.26371449e-01, 8.30489520e-01,

         4.01703967e-01, 6.03648449e-01],

        [4.94483095e-01, 7.07995918e-01, 3.19788098e-01, 8.96047945e-01,

         2.60821311e-01, 8.37115248e-02, 4.80644409e-01, 5.01016565e-01,

         8.96979894e-01, 4.87194072e-01],

        [5.05973124e-01, 5.69098017e-01, 6.02714387e-01, 3.68212319e-01,

         3.79218418e-02, 8.61525675e-01, 2.20650824e-01, 4.15911835e-02,

         3.58819157e-01, 5.55621075e-01]],



       [[4.57167481e-01, 5.12868390e-01, 4.21362772e-01, 4.62120532e-01,

         4.78075434e-01, 5.82866044e-01, 4.80800524e-02, 1.30049948e-04,

         9.23710795e-02, 8.79589968e-01],

        [4.70070144e-01, 2.86995478e-01, 5.62734075e-03, 5.74988660e-01,

         1.61493492e-01, 5.05786880e-01, 3.34274478e-01, 2.70070272e-01,

         8.34386052e-03, 8.33134141e-01],

        [5.52076562e-01, 3.11916150e-01, 3.85403490e-01, 9.81367353e-02,

         3.13549311e-01, 6.63864083e-01, 5.23692837e-01, 3.02405064e-01,

         3.13138788e-01, 7.02424389e-01],

        [5.15215572e-01, 7.56672035e-01, 6.83067591e-01, 9.19983750e-02,

         7.92201364e-01, 7.25450058e-02, 9.86873695e-01, 4.61248279e-01,

         4.36048086e-01, 4.64198674e-01],

        [7.54329222e-01, 5.24436517e-01, 9.98835492e-01, 8.68671122e-01,

         6.90397960e-01, 1.99994823e-01, 6.60259654e-01, 1.45494804e-01,

         2.85639991e-02, 7.03491487e-01],

        [2.79430431e-01, 4.02411904e-01, 7.98108097e-01, 2.48419608e-01,

         8.21776264e-01, 3.52936770e-01, 6.50394071e-01, 5.39674338e-01,

         2.63533532e-01, 8.94972377e-01],

        [8.24436984e-01, 4.20557504e-01, 9.40078286e-01, 4.04314122e-01,

         2.70584136e-01, 1.53482622e-01, 8.48493588e-01, 3.74145503e-01,

         1.02702026e-01, 4.50123256e-01],

        [3.67721242e-01, 5.36878811e-01, 5.88842951e-02, 5.88928069e-01,

         2.96584467e-01, 7.79466125e-01, 1.10771149e-01, 3.59839701e-01,

         8.75393042e-01, 8.21976269e-01],

        [5.21453802e-02, 1.66354552e-01, 8.27561528e-02, 1.89704124e-01,

         4.04256542e-01, 6.11030733e-01, 4.73238475e-01, 5.44670436e-01,

         1.28323452e-01, 1.62484904e-02],

        [3.17719800e-01, 7.56502125e-01, 8.66477661e-01, 3.75496168e-01,

         3.52659318e-01, 1.57497556e-01, 8.85140160e-01, 4.82430855e-01,

         9.69872872e-01, 2.11911603e-04]],



       [[1.49985286e-02, 1.46182025e-02, 7.36881808e-01, 3.27210734e-01,

         2.31264333e-01, 4.95127246e-01, 6.07026238e-01, 3.18706798e-01,

         9.27013192e-01, 1.28443855e-01],

        [6.25043222e-01, 9.39123257e-01, 7.55986485e-01, 2.60197978e-01,

         1.28876459e-01, 9.14740466e-01, 4.78087030e-01, 3.69139725e-01,

         7.67595452e-01, 8.54234768e-01],

        [3.98352231e-01, 8.28163942e-01, 6.90849007e-01, 7.12727602e-01,

         3.26647568e-01, 2.84725488e-01, 6.50029676e-01, 8.66947479e-01,

         4.17449510e-01, 4.41074451e-01],

        [4.81578542e-01, 6.29513078e-01, 5.67927438e-01, 6.44082469e-01,

         8.94781758e-01, 2.60601835e-01, 9.58440523e-01, 5.89380873e-01,

         3.93766091e-01, 5.22333690e-01],

        [9.97502669e-01, 8.34761800e-01, 5.38195599e-01, 7.22319055e-01,

         2.89772550e-01, 7.93178714e-01, 4.65075918e-02, 5.51339690e-01,

         6.12097619e-01, 8.60548950e-01],

        [5.28694944e-01, 5.85554717e-01, 5.96903478e-01, 5.04857543e-01,

         4.12884194e-01, 1.46641961e-02, 9.63180541e-01, 9.22993140e-01,

         6.30060267e-01, 2.22033018e-01],

        [1.35499109e-01, 8.11983638e-01, 9.68124848e-01, 4.88417070e-01,

         9.36005150e-01, 5.48221879e-01, 3.82144011e-01, 2.69907295e-01,

         6.57540647e-01, 3.24021333e-01],

        [4.96514122e-01, 5.14788240e-01, 3.73271484e-01, 8.92980986e-01,

         3.68363243e-01, 2.09216443e-01, 8.10504244e-01, 9.22174865e-01,

         1.50973943e-01, 9.16720916e-01],

        [8.91967883e-01, 2.65725861e-01, 7.76800744e-01, 8.83485420e-01,

         8.57783387e-02, 5.60318504e-01, 3.32928416e-01, 5.69574713e-01,

         7.79181525e-01, 4.69229110e-01],

        [3.79821655e-01, 8.22971379e-01, 7.12934483e-02, 2.90562273e-02,

         5.96300416e-01, 7.58441805e-01, 5.97319771e-01, 1.20185838e-01,

         9.55280358e-01, 5.01730360e-01]],



       [[9.95696436e-01, 3.18756428e-01, 7.24178858e-01, 6.89846459e-01,

         7.23291825e-01, 8.43018550e-01, 3.47749760e-01, 4.74711294e-01,

         1.00733823e-01, 8.99129087e-02],

        [4.97020060e-01, 8.26086317e-01, 8.39423969e-01, 4.57389141e-01,

         4.80068658e-01, 1.56565700e-02, 7.09192641e-01, 2.43401869e-01,

         3.70537403e-01, 1.19834737e-01],

        [4.74427099e-01, 6.50585214e-01, 8.29293965e-01, 1.83436590e-01,

         8.72867295e-02, 6.04536460e-01, 5.80763160e-01, 3.68775166e-01,

         6.78076809e-01, 1.13873822e-01],

        [4.87669411e-01, 7.13076819e-01, 7.60038000e-01, 4.92518218e-01,

         2.10655956e-01, 6.87899730e-01, 8.00823258e-01, 3.27926631e-01,

         2.09181328e-01, 8.19569938e-01],

        [7.26300000e-01, 7.76826356e-01, 2.37160909e-03, 1.01418975e-01,

         3.95271752e-01, 9.92223214e-01, 2.67122251e-01, 9.96160940e-01,

         4.95768227e-01, 5.65246053e-02],

        [3.53617937e-01, 2.99053805e-01, 2.72814255e-01, 4.91360590e-01,

         6.57251318e-01, 6.31612453e-01, 3.38284008e-01, 9.64398809e-02,

         1.96912258e-01, 3.26506191e-01],

        [7.55168122e-01, 7.01862831e-01, 1.12994286e-01, 1.64302276e-01,

         1.07115030e-01, 9.07112057e-01, 1.82532937e-01, 5.24171800e-01,

         8.07079861e-01, 6.05442068e-01],

        [6.19664984e-01, 1.50135960e-01, 6.56564736e-01, 5.05727516e-01,

         3.66044841e-01, 1.34294470e-01, 3.55978880e-01, 3.24052271e-01,

         2.73845825e-01, 8.37403610e-01],

        [9.19699624e-01, 2.61520895e-01, 1.46864775e-01, 4.09444304e-01,

         5.04900244e-01, 1.42768181e-01, 1.85938156e-01, 7.77826286e-01,

         5.16617098e-01, 4.49711815e-01],

        [2.57583263e-01, 7.67467806e-01, 9.65347304e-01, 4.02937974e-01,

         4.87376872e-02, 1.70250969e-01, 7.66510683e-01, 2.66239796e-01,

         7.60224225e-01, 6.38954036e-01]],



       [[1.14416898e-01, 3.24899809e-02, 3.18291632e-01, 4.45412832e-01,

         1.05145917e-02, 9.75717990e-01, 6.23497943e-01, 6.10552043e-01,

         1.10937005e-01, 2.09859239e-01],

        [4.07956259e-01, 8.20641946e-01, 4.93220999e-01, 6.18486104e-01,

         1.92029696e-02, 7.73396787e-01, 9.90796736e-01, 2.45858468e-01,

         1.05358462e-01, 1.98161752e-01],

        [2.14645140e-01, 8.88936950e-01, 1.36047281e-01, 4.82886518e-01,

         1.24994558e-02, 7.09973007e-01, 3.07957540e-01, 5.96957567e-01,

         4.59288568e-01, 6.77385723e-01],

        [1.33546251e-01, 5.92464836e-01, 5.97991735e-02, 3.02289516e-01,

         8.95541332e-01, 2.19353891e-01, 3.34819111e-01, 7.09572404e-01,

         5.89678293e-01, 6.74221044e-01],

        [8.59308225e-01, 2.07405020e-01, 3.07927328e-01, 9.68431806e-01,

         8.18683644e-02, 5.82765283e-01, 8.71876505e-01, 8.15745171e-01,

         8.80981758e-01, 5.20385223e-01],

        [7.84021403e-01, 9.42788077e-01, 2.81652569e-01, 6.30092460e-01,

         9.03596119e-01, 7.05610423e-01, 2.17500415e-01, 5.55180274e-01,

         1.99767114e-01, 5.02173867e-01],

        [2.13134591e-01, 2.55738499e-01, 1.11454218e-02, 3.09010369e-01,

         6.23741952e-01, 9.18686347e-01, 5.05411620e-01, 5.90222897e-01,

         4.09527797e-01, 5.67333570e-02],

        [5.05127568e-01, 6.19172390e-01, 2.41553404e-01, 4.48464212e-01,

         7.59266055e-01, 6.91816381e-02, 4.77925021e-01, 1.79906797e-01,

         1.83618652e-01, 1.40509174e-01],

        [7.30815253e-01, 7.72646143e-01, 9.80265845e-01, 6.31946707e-01,

         1.07326626e-01, 8.74831918e-01, 8.58581635e-02, 6.02622736e-01,

         1.43699891e-03, 1.70025270e-02],

        [8.73696993e-01, 4.16385866e-01, 6.74902825e-02, 3.08038635e-01,

         3.29456224e-01, 4.86707527e-01, 2.96941452e-01, 2.62646560e-01,

         2.74314896e-01, 9.02310271e-01]],



       [[4.86960891e-01, 5.48700986e-01, 2.29006379e-01, 8.76868756e-01,

         5.87662577e-01, 5.65269460e-01, 7.34067936e-01, 3.33280643e-02,

         5.29397038e-01, 2.62368869e-01],

        [8.50921923e-01, 8.15657946e-01, 2.43420518e-01, 1.17620216e-02,

         2.71618739e-01, 1.59008419e-01, 2.13491412e-02, 7.06702776e-01,

         8.48119190e-02, 5.26189001e-01],

        [8.30800629e-01, 1.37257740e-01, 5.91181277e-01, 7.57501742e-01,

         7.57822664e-01, 5.08760609e-01, 3.69334823e-01, 7.29020033e-01,

         7.50854718e-01, 7.73641423e-01],

        [5.57937958e-01, 7.96391898e-01, 7.12137193e-01, 4.86161537e-01,

         5.48674717e-01, 2.73144670e-01, 6.25993934e-01, 3.70886147e-02,

         2.81550324e-01, 7.20682878e-01],

        [9.04747045e-01, 2.09507641e-01, 2.85034862e-01, 3.08038990e-01,

         2.59222088e-01, 4.17119750e-01, 7.92861994e-01, 6.92399186e-02,

         7.81847210e-01, 8.50790064e-01],

        [1.42441275e-01, 8.99164536e-01, 9.79175406e-01, 3.69338503e-02,

         7.78470883e-01, 9.17554552e-01, 5.87139810e-01, 3.05963252e-01,

         5.10429695e-01, 6.47458804e-01],

        [5.43502408e-02, 4.00246015e-01, 3.53087518e-01, 2.12094225e-01,

         3.32635046e-01, 3.91732638e-02, 3.92548821e-01, 3.91190320e-01,

         1.19228212e-01, 2.94469635e-01],

        [2.82149963e-02, 9.24686067e-01, 2.58281270e-01, 7.93804190e-01,

         3.00940535e-01, 6.03686725e-01, 8.55875810e-01, 8.43366669e-01,

         7.84258505e-01, 4.59209398e-01],

        [2.49620907e-01, 2.17202656e-01, 7.80787654e-01, 6.60571052e-01,

         1.97606245e-01, 9.12223885e-01, 6.33447261e-01, 2.99798275e-02,

         2.00483880e-01, 5.50768887e-01],

        [9.97608040e-01, 5.76900056e-03, 9.87419891e-01, 1.85907207e-01,

         3.99844301e-01, 1.46307038e-01, 2.67863409e-01, 4.24371539e-01,

         8.62732945e-01, 9.07808696e-02]],



       [[9.98105615e-01, 9.51535127e-01, 6.99947444e-03, 1.45535973e-01,

         2.92470519e-02, 5.50531184e-02, 9.44141600e-01, 6.20728411e-01,

         8.71007324e-01, 8.32045186e-01],

        [1.04860691e-01, 7.14073866e-01, 6.69855577e-01, 9.75168044e-01,

         5.25350583e-01, 1.31272770e-01, 1.63485511e-01, 4.43076807e-01,

         6.64333609e-01, 9.93819214e-01],

        [4.15496526e-01, 8.82637473e-01, 9.25731548e-01, 7.72907051e-01,

         7.25748142e-01, 9.95232697e-01, 4.59944614e-01, 5.75810244e-01,

         6.02046314e-03, 4.78873017e-01],

        [3.79113944e-01, 1.13284970e-01, 3.24602714e-01, 1.91738382e-01,

         7.88467623e-01, 9.96835833e-01, 8.76752251e-02, 7.26961442e-01,

         4.57559022e-01, 9.91065192e-01],

        [8.17901363e-02, 8.62767538e-01, 5.70261973e-01, 5.58718049e-01,

         6.45084318e-01, 2.37980307e-01, 3.04875517e-01, 8.87438381e-01,

         1.32194141e-01, 5.44607395e-02],

        [6.29989050e-01, 4.16044310e-01, 9.08640614e-01, 1.71899363e-01,

         6.11724433e-01, 1.71960126e-01, 5.80787616e-01, 8.28765880e-01,

         9.79253955e-02, 4.63682513e-01],

        [2.35198912e-01, 2.79637960e-01, 4.54562902e-01, 7.89327197e-01,

         4.10136000e-01, 8.72636950e-02, 2.55977261e-02, 1.19099910e-02,

         7.68379266e-01, 7.61003375e-01],

        [3.30803903e-01, 7.71703667e-01, 5.28702898e-01, 6.34654768e-01,

         6.70517357e-01, 2.49140205e-01, 3.68422954e-01, 6.98976878e-01,

         8.17636915e-01, 4.53704246e-01],

        [3.76056561e-01, 7.97700728e-01, 3.63258898e-01, 8.07444154e-01,

         5.67447220e-02, 8.84168412e-01, 1.70170794e-01, 9.70651589e-01,

         5.39158081e-01, 2.86156593e-01],

        [7.67341086e-01, 9.68427963e-01, 6.46866104e-01, 7.55504926e-01,

         7.18350796e-01, 4.54945423e-01, 7.14918258e-01, 8.60511312e-01,

         2.23092973e-01, 1.06087838e-01]],



       [[8.07895983e-01, 7.64733582e-01, 2.04324379e-02, 9.46749223e-01,

         2.95868848e-01, 8.92532086e-02, 4.92560009e-01, 9.37262517e-01,

         7.76464979e-01, 5.91790768e-01],

        [8.75791941e-02, 5.31913320e-01, 7.16168107e-01, 1.19221968e-01,

         9.08744173e-01, 2.96912928e-01, 9.81512999e-01, 1.14855880e-01,

         6.73255948e-01, 2.02093615e-01],

        [2.71613225e-01, 2.27753746e-01, 5.74286554e-01, 6.76525557e-01,

         7.78717021e-03, 4.39077165e-01, 7.60938809e-01, 8.12602794e-02,

         6.62403770e-01, 2.54807001e-01],

        [4.09898951e-01, 9.46067743e-01, 3.99962446e-02, 3.91127020e-01,

         7.03565663e-01, 8.74949454e-02, 8.56292514e-01, 7.75772049e-01,

         1.40244798e-01, 8.46945865e-01],

        [1.59500304e-01, 5.56135083e-01, 4.67571394e-02, 6.07547247e-02,

         6.35297416e-01, 2.60403988e-01, 7.88616770e-01, 1.83988438e-01,

         5.88143349e-01, 7.53369609e-01],

        [2.29412230e-01, 5.72793249e-01, 2.33731096e-01, 7.23638008e-01,

         7.61177156e-01, 4.42956722e-01, 2.37155555e-01, 3.41151238e-02,

         6.02567668e-01, 6.56913542e-01],

        [8.71302370e-01, 3.85616023e-01, 6.32694736e-01, 6.12629113e-01,

         5.95533379e-02, 2.71461996e-01, 6.45725257e-02, 4.41525234e-01,

         4.70785328e-01, 2.27133630e-01],

        [9.46202754e-01, 5.14499131e-01, 3.49553801e-01, 9.32901824e-01,

         3.82338628e-01, 3.36406025e-01, 7.86029698e-01, 8.51571624e-01,

         4.49836872e-01, 4.69537163e-01],

        [7.96676707e-01, 9.49865353e-01, 4.05362709e-01, 7.67175007e-01,

         9.62689810e-03, 3.70197483e-01, 8.58270337e-01, 3.54162236e-01,

         4.99899461e-01, 8.94811887e-01],

        [4.47497592e-01, 5.26463933e-01, 7.97300590e-01, 6.57234871e-01,

         6.05446766e-01, 8.14425235e-02, 9.58932069e-01, 3.27032185e-01,

         9.14704149e-02, 8.25080215e-02]],



       [[5.67624587e-01, 3.62671140e-01, 8.95292858e-01, 5.12800228e-01,

         5.08094344e-01, 2.96573624e-01, 3.97238328e-01, 7.67220525e-01,

         1.89338586e-01, 5.30592148e-01],

        [1.49951921e-01, 1.70881918e-01, 9.07801778e-02, 7.55441367e-01,

         8.72939994e-01, 8.62530557e-01, 1.07973350e-01, 2.71323202e-01,

         1.14135684e-01, 9.61107414e-01],

        [1.88734250e-01, 7.87943891e-01, 8.94415581e-01, 9.77598287e-01,

         7.29582395e-01, 2.26160313e-01, 8.50132810e-01, 8.48629951e-01,

         5.60827969e-01, 3.63574624e-01],

        [1.21084303e-01, 7.43113186e-01, 6.93651208e-01, 4.70604201e-01,

         1.64001198e-01, 4.77594812e-02, 7.38215291e-01, 9.06353186e-01,

         2.49743623e-01, 6.57910989e-02],

        [7.71288403e-01, 2.72218259e-01, 5.20457219e-01, 7.09877181e-01,

         3.13694292e-01, 9.51447099e-01, 7.52720241e-02, 3.39079835e-01,

         4.43227459e-01, 2.94586553e-01],

        [8.01305836e-01, 9.28196782e-01, 9.57242196e-01, 6.86453405e-01,

         2.26090537e-01, 8.38574508e-01, 7.59150445e-01, 2.59761841e-01,

         4.49967044e-01, 6.58593153e-01],

        [7.44175524e-01, 1.45802651e-01, 8.86852780e-01, 1.54326266e-01,

         7.83112182e-01, 6.63382985e-01, 7.13700297e-01, 6.97626426e-01,

         8.54403129e-01, 6.67225353e-01],

        [8.66897924e-01, 9.04917866e-01, 9.93211476e-01, 8.88441006e-01,

         5.95547271e-01, 1.54084808e-01, 9.21776064e-01, 3.24019626e-01,

         7.68368022e-01, 6.74831679e-01],

        [6.41611708e-01, 8.46691829e-01, 5.16200331e-01, 5.16081843e-01,

         3.64290115e-02, 2.31944995e-01, 7.54538474e-01, 8.93991461e-01,

         8.23091912e-01, 1.26559693e-01],

        [5.78638086e-01, 3.01907178e-01, 2.98411879e-01, 8.74126497e-01,

         3.31680868e-01, 7.01796444e-02, 4.44338248e-01, 6.92991432e-01,

         9.47784971e-02, 4.60959936e-02]],



       [[2.00832311e-01, 5.57510502e-01, 5.56493621e-01, 2.49548764e-01,

         7.38785390e-01, 3.73230546e-01, 4.89307357e-01, 9.66155314e-02,

         4.44509570e-01, 2.83648741e-01],

        [2.10297681e-02, 6.08092430e-01, 2.31807763e-02, 5.25173938e-01,

         7.75901818e-01, 9.48170599e-01, 6.96105955e-01, 4.83524478e-01,

         7.11447962e-01, 1.00308651e-01],

        [8.97165175e-01, 7.80075956e-01, 3.02305092e-01, 2.18901353e-01,

         2.95121825e-01, 7.54830631e-01, 1.05766072e-01, 2.28032508e-01,

         1.85889323e-01, 7.45292118e-01],

        [2.09509836e-01, 9.59542522e-01, 2.51791761e-01, 1.40504134e-01,

         7.00343252e-02, 7.58913062e-01, 8.84980322e-01, 7.90782612e-02,

         1.29383989e-01, 9.41160997e-01],

        [4.24685342e-01, 7.64728513e-02, 7.98836982e-01, 2.97696170e-01,

         7.44437153e-01, 3.41823594e-01, 6.43248957e-01, 7.14578054e-01,

         6.72555074e-02, 6.94555171e-01],

        [4.00416574e-01, 9.03075606e-01, 6.86947335e-01, 8.27588589e-01,

         5.33290885e-02, 8.13033658e-01, 4.90110924e-01, 8.85749311e-01,

         4.72619709e-01, 6.89952437e-01],

        [4.16279248e-01, 5.35969666e-01, 6.36363746e-01, 1.03637455e-01,

         2.17197792e-01, 2.23538029e-01, 4.35881880e-01, 8.14255960e-01,

         2.73752274e-01, 9.81273297e-01],

        [3.84773673e-01, 8.78314900e-01, 2.08219300e-01, 8.71294306e-01,

         6.96435496e-01, 9.37805595e-01, 4.40624432e-01, 9.92338663e-02,

         5.06165953e-01, 1.60602518e-01],

        [4.80177114e-01, 8.27000719e-02, 6.48567957e-01, 1.45653825e-01,

         1.50614737e-02, 4.68782081e-02, 5.30956051e-02, 6.91342673e-01,

         4.12895132e-01, 4.13474845e-01],

        [9.81423228e-01, 4.48459797e-01, 6.71465338e-01, 2.10599040e-01,

         1.15730566e-01, 4.95869488e-01, 2.86216029e-01, 4.27469662e-01,

         1.41728675e-01, 5.61022770e-01]]])