Raptorlicious

I know this should be stunningly obvious, but I lack the background and insight you do.

I know that we are following the rank-r factorization:
Xk = A*R*A.T

I ran a version of RESCAL called Ext-RESCAL, and chose the tiny-example dataset. It is the graph on the left under “Let's imagine we have the following semantic graph:” at the url: https://github.com/nzhiltsov/Ext-RESCAL . I guessed that it would give the probability of links.

----------------

For the slice for relation 1, member-of:

X1 = ([[0,0,0,0],[1,0,0,0],[1,0,0,0],[0,0,0,0]])

Link-Representation = ([[AA,AB,AC,AD],[BA,BB,BC,BD],[CA,CB,CC,CD],[DA,DB,DC,DD]])

A*R1*A.T =

([[ 4.30000000e-06, -3.81164826e-21, -3.81164826e-21, -1.48892510e-23],
[ 9.99989146e-01, -8.88178420e-16, -8.88178420e-16, -3.46944695e-18],
[ 9.99989146e-01, -8.88178420e-16, -8.88178420e-16, -3.46944695e-18],
[ 3.90620760e-03, -3.46944695e-18, -3.46944695e-18,-1.35525272e-20]])

Most probable links:

AA = 4.30000000e-06 ( dbr:Tristania member-of dbr:Tristania )
BA = 9.99989146e-01 ( dbr:Vibecke member-of dbr:Tristania )
CA = 9.99989146e-01 ( dbr:Morten member-of dbr:Tristania )
DA = 3.90620760e-03 ( dbr:Metal member-of dbr:Tristania )

For the slice for relation 2, genre:

X2 = ([[0,0,0,1],[0,0,0,0],[0,0,0,0],[0,0,0,0]])

Link-Representation = ([[AA,AB,AC,AD],[BA,BB,BC,BD],[CA,CB,CC,CD],[DA,DB,DC,DD]])

A*R2*A.T =

([[ -2.16840434e-19, 1.95311646e-03, 1.95311646e-03, 7.62936119e-06],
[ 1.92592994e-34, -1.72202920e-18, -1.72202920e-18, -6.72667656e-21],
[ 1.92592994e-34 -1.72202920e-18, -1.72202920e-18, -6.72667656e-21],
[ 7.52316385e-37, -6.72667656e-21, -6.72667656e-21, -2.62760803e-23]])

Most probable links:

AB = 1.95311646e-03 ( dbr:Tristania genre dbr:Vibecke )
AC = 1.95311646e-03 ( dbr:Tristania genre dbr:Morten )
AD = 7.62936119e-06 ( dbr:Tristania genre dbr:Metal )

Thanks for your time. Best regards, Brent .

–---------------------------------------------------------------------------

Prediction of Unknown Triples (Section 3.3: Factorizing YAGO)::
or Canonical Relational Learning (section 4.4: A Three-Way Model for Collective Learning on Multi-Relational Data)

A*Rk*aj

Entity A, [AA,AB,AC,AD] :

s1 = np.matmul(A,R1)
np.matmul(s1,A[0,:])

array([ 4.30000000e-06, 9.99989146e-01, 9.99989146e-01,

3.90620760e-03]

Entity B, [BA,BB,BC,BD]:

np.matmul(s1,A[1,:])

array([ -3.81164826e-21, -8.88178420e-16, -8.88178420e-16,

-3.46944695e-18])

Entity C, [CA,CB,CC,CD]:

np.matmul(s1,A[2,:])

array([ -3.81164826e-21, -8.88178420e-16, -8.88178420e-16,

-3.46944695e-18])

Entity D, [DA,DB,DC,DD]:

np.matmul(s1,A[3,:])

array([ -1.48892510e-23, -3.46944695e-18, -3.46944695e-18,

-1.35525272e-20])

Create Ranking:::

AB => 9.99989146e-01 ( dbr:Tristania member-of dbr:Vibecke)
AC => 9.99989146e-01 ( dbr:Tristania member-of dbr:Vibecke )
AD => 3.90620760e-03 (dbr:Tristania member-of dbr:Metal)
AA => 4.30000000e-06 ( dbr:Tristania member-of dbr:Tristania )

==================================================================

Entity A, [AA,AB,AC,AD] :

s2 = np.matmul(A,R2)
np.matmul(s2,A[0,:])

array([ -2.16840434e-19, 1.92592994e-34, 1.92592994e-34,

7.52316385e-37])

Entity B, [BA,BB,BC,BD]:

np.matmul(s2,A[1,:])

array([ 1.95311646e-03, -1.72202920e-18, -1.72202920e-18,

-6.72667656e-21])

Entity C, [CA,CB,CC,CD]:

np.matmul(s2,A[2,:])

array([ 1.95311646e-03, -1.72202920e-18, -1.72202920e-18,

-6.72667656e-21])

Entity D, [DA,DB,DC,DD]:

np.matmul(s2,A[3,:])

array([ 7.62936119e-06, -6.72667656e-21, -6.72667656e-21,

-2.62760803e-23])

Create Ranking:::

BA => 1.95311646e-03 (dbr:Vibecke genre dbr:Tristania)
CA => 1.95311646e-03 ( dbr:Morten genre dbr:Tristania)
DA => 7.62936119e-06 ( dbr:Metal genre dbr:Tristania)

================================================

Retrieval of similar entities (Section 3.3.2: Factorizing YAGO)::
or Link-based clustering (section 4.4: A Three-Way Model for Collective Learning on Multi-Relational Data)

A

array([[-0.70710678, 0.70710678],

[ 0.52943053, 0.52943053],

[ 0.00206809, 0.00206809]])

Corresponds to Entities A. B. C, D in row order

Hypothesis: B and C are alike…

B and C are closer to D than they are to A

=====================================

Collective Classification::

Add an additional slice mapping all entities to classes with the classOf relationship:

which gives us an entity-term matrix to add as a slice??

But possibly this is not the case, since R only has three slices???

member-of

genre

cites

Aha, do it seperately. “The basic idea is to process attribute values just as described above, but to add the <predicate, value> pairs to a separate entity-attributes matrix D and not to tensor X” (Factorizing YAGO)
D = AV

D =

[[ 1. 0. 0. 0.]

[ 0. 1. 0. 0.]

[ 0. 0. 1. 0.]

[ 0. 0. 0. 1.]

[ 0. 0. 0. 1.]]

A =

[[ 5.28740028e-01 -2.64828807e-17]

[ 6.47298575e-01 -1.88352457e-17]

[ 2.52851006e-03 -8.39977963e-20]

[ 9.72366338e-17 7.11843766e-01]

[ 9.72371832e-17 7.11843766e-01]]

V =

[[ 4.73082585e-01 1.15769705e+00 4.52225411e-03 4.82270178e-17]

[ -6.97733700e-17 -1.45176144e-16 -5.86014702e-19 1.40457240e+00]]

import numpy as np

>>> A = np.array([[ 5.28740028e-01, -2.64828807e-17],

... [ 6.47298575e-01, -1.88352457e-17],

... [ 2.52851006e-03, -8.39977963e-20],

... [ 9.72366338e-17, 7.11843766e-01],

... [ 9.72371832e-17, 7.11843766e-01]])

V = np.array([[ 4.73082585e-01, 1.15769705e+00, 4.52225411e-03, 4.82270178e-17],

[ -6.97733700e-17, -1.45176144e-16, -5.86014702e-19, 1.40457240e+00]])

D = np.matmul(A,V)

array([[ 2.50137699e-01, 6.12120771e-01, 2.39109676e-03,

-1.16975686e-17],

[ 3.06225683e-01, 7.49375651e-01, 2.92724864e-03,

4.76181364e-18],

[ 3.06225683e-01, 7.49375651e-01, 2.92724864e-03,

4.76181364e-18],

[ 1.19619408e-03, 2.92724864e-03, 1.14345650e-05,

3.96151333e-21],

[ -3.66678039e-18, 9.22783102e-18, 2.25778544e-20,

9.99836107e-01],

[ -3.66652048e-18, 9.22846706e-18, 2.25803390e-20,

9.99836107e-01]])

[[ 2.50137699e-01, 6.12120771e-01, 2.39109676e-03, -1.16975686e-17],

[ 3.06225683e-01, 7.49375651e-01, 2.92724864e-03, 4.76181364e-18],

[ 1.19619408e-03, 2.92724864e-03, 1.14345650e-05, 3.96151333e-21],

[ -3.66678039e-18, 9.22783102e-18, 2.25778544e-20, 9.99836107e-01],

[ -3.66652048e-18, 9.22846706e-18, 2.25803390e-20,

9.99836107e-01]])

[[ 1. 0. 0. 0.]

[ 0. 1. 0. 0.]

[ 0. 0. 1. 0.]

[ 0. 0. 0. 1.]

[ 0. 0. 0. 1.]]

with

Predicted Triples are in Bold (but some of these look incorrect):

dbr:Tristania, band : 2.50137699e-01

dbr:Vibeke, band : 3.06225683e-01

dbr:Morten, band : 3.06225683e-01

dbr:Metal, band : 1.19619408e-03
dbr:Vibeke, member : 7.49375651e-01
dbr:Morten, member : 7.49375651e-01

dbr:Tristania, member : 6.12120771e-01

dbr:Metal, member : 2.92724864e-03
dbr:Metal, genre : 1.14345650e-05

dbr:Tristania, genre : 2.39109676e-03
dbr:Vibeke, genre : 2.92724864e-03
dbr:Morten, genre : 2.92724864e-03
author1, tensor : 9.99836107e-01

author2, tensor : 9.99836107e-01

python rescal.py --latent 2 --lmbda 0 --input tiny-example --outputentities entity.embeddings.csv --outputfactors latent.factors.csv --log rescal.log
The number of entities: 4
loaded 1: 1-rows
loaded 2: 2-rows
The number of tensor slices: 2
The number of non-zero values in the tensor: 3
Objective function value: 0.166667938222720551921796072747
# of iterations: 8

For matrix A::
cat entity.embeddings.csv
-7.071067811865472397e-01 7.071067811865471286e-01
5.294305317939924649e-01 5.294305317939912436e-01
5.294305317939924649e-01 5.294305317939912436e-01
2.068088014820283066e-03 2.068088014820278295e-03

For matrix R::
cat latent.factors.csv
-6.677943034708067049e-01 6.677943034708085923e-01
-6.677943034708074821e-01 6.677943034708093695e-01

-1.304285748966419346e-03 -1.304285748966421297e-03
1.304285748966423249e-03 1.304285748966424767e-03

cat rescal.log
DEBUG:RESCAL:[Config] rank: 2 | maxIter: 50 | conv: 1.0e-05 | lmbda: 0.0e+00
DEBUG:RESCAL:[Algorithm] The tensor norm: 3.00000
DEBUG:RESCAL:[Algorithm] The eigenvector based initialization will be performed.
DEBUG:RESCAL:Initializing tensor slices by summation required secs: 0.00496
DEBUG:RESCAL:eigenvector decomposition required secs: 0.00453
DEBUG:RESCAL:[Algorithm] Finished initialization.
DEBUG:RESCAL:[Algorithm] Preheating is going on.
DEBUG:RESCAL:[ 0] total fit: 0.0000000000 | delta: 0.0000000000 | secs: 0.00377
DEBUG:RESCAL:[ 1] total fit: 0.1717171717 | delta: 0.1717171717 | secs: 0.00356
DEBUG:RESCAL:[ 2] total fit: 0.1679586563 | delta: 0.0037585154 | secs: 0.00348
DEBUG:RESCAL:[ 3] total fit: 0.1669915530 | delta: 0.0009671034 | secs: 0.00347
DEBUG:RESCAL:[ 4] total fit: 0.1667480072 | delta: 0.0002435458 | secs: 0.00357
DEBUG:RESCAL:[ 5] total fit: 0.1666870092 | delta: 0.0000609979 | secs: 0.00345
DEBUG:RESCAL:[ 6] total fit: 0.1666717528 | delta: 0.0000152565 | secs: 0.00347
DEBUG:RESCAL:[ 7] total fit: 0.1666679382 | delta: 0.0000038146 | secs: 0.00344

===========

cat entity.embeddings.csv
5.287400282344740798e-01 -2.648288074937905172e-17
6.472985753526332431e-01 -1.883524571199644730e-17
6.472985753526332431e-01 -1.883524571199644730e-17
2.528510059971223606e-03 -8.399779633628449750e-20
9.723663381107547794e-17 7.118437655042088030e-01
9.723718319815804104e-17 7.118437655042088030e-01

cat term.embeddings.csv
4.730825851270915039e-01 -6.977337002813972351e-17
1.157697052140589156e+00 -1.451761439325526721e-16
4.522254109924176389e-03 -5.860147020766050828e-19
4.822701779013331954e-17 1.404572401961088790e+00

cat latent.factors.csv
5.476271649231128080e-01 -1.168831819383593920e-16
-6.070133651429397840e-17 1.295001708078205427e-32

1.069584306490454703e-03 -1.659206118320981211e-19
-2.282874647233581876e-19 3.540155448418387818e-35

1.741390346565283288e-33 5.092367925378079864e-17
5.092333426231983632e-17 1.478663144852583988e+00

Raptorlicious

Thursday, May 3, 2018

Notes for Ext-Rescal (may 3rd)

Ext Rescal Run: (try 1)

Followers

Blog Archive

About Me