Fix error with Pandas

raise ValueError("Usecols do not match names.")

https://stackoverflow.com/questions/37395881/pandas-read-in-txt-file-without-headers

https://stackoverflow.com/questions/37395881/pandas-read-in-txt-file-without-headers

other:
https://docs.python.org/3/library/functions.html

Information for missing 32 bit libraries from Steam

https://askubuntu.com/questions/257578/im-trying-to-run-steam-but-32-bit-libraries-are-missing

Looking for inferincing vs training (and I saw this)

https://devblogs.nvidia.com/parallelforall/inference-next-step-gpu-accelerated-deep-learning/

data processing for machine learning ---scratch pad

Grab the 1st three rows of data files to analyze:

cat ex2x.dat | awk 'NR >= 1  && NR <= 3 { print }' > ex2x.short.dat

cat ex2y.dat | awk 'NR >= 1  && NR <= 3 { print }' > ex2y.short.dat

https://stackoverflow.com/questions/6491532/how-to-subset-a-file-select-a-numbers-of-rows-or-columns

working with Unreal Engine to Prep for AirSim

Follow the steps here:

http://www.hpique.com/2013/09/updating-a-fork-directly-from-github/

"
Here’s how to update your fork directly from GitHub (as shown in the video above):

1. Open your fork on GitHub.
2. Click on Pull Requests.
3. Click on New Pull Request. By default, GitHub will compare the original with your fork, and there shouldn’t be anything to compare if you didn’t make any changes.
4. Click on switching the base. Now GitHub will compare your fork with the original, and you should see all the latest changes.
5. Click on Click to create a pull request for this comparison and assign a predictable name to your pull request (e.g., Update from original).
6. Click on Send pull request.
7. Scroll down and click Merge pull request and finally Confirm merge. If your fork didn’t have any changes, you will be able to merge it automatically."

Then git fetch --prune

-----------------------------------------

git clone https://github.com/bshambaugh/UnrealEngine

-----------------------------------------

sh Setup.sh
Registering git hooks... (this will override existing ones!)
Setup.sh: 39: Setup.sh: pushd: not found

How to use pushd and popd:

https://unix.stackexchange.com/questions/77077/how-do-i-use-pushd-and-popd-commands

How to use source:
https://ss64.com/bash/source.html

how do I get search with gitter? well I could build it...initial survey

Get Messages to search for a particular one:

First get the ID:
curl -X POST -i -H "Content--H "Accept: application/json" -H "Authorization: Bearer {{token}}" "https://api.gitter.im/v1/rooms" -d '{"uri":"solid/node-solid-server"}'

Remember authorization: https://developer.gitter.im/docs/authentication

Then get the messages by the room id:
https://developer.gitter.im/docs/rooms-resource
https://api.gitter.im/v1/rooms/:roomId/chatMessages

Then add this JSON object to your index

For a process that updates, use the streaming API: https://developer.gitter.im/docs/streaming-api

For indexing: https://cwiki.apache.org/confluence/display/solr/Transforming+and+Indexing+Custom+JSON

log: gitter:solid/node-solid-server

Brent Shambaugh
@bshambaugh
11:28

I'm trying to learn about how to access SOLiD with FOAF+SSL and other semantic web resources from JavaScript. I can look up the flow diagram for how the protocol works, but I found I did not know about CORS or much about writing code for Authorization or Authentication. Thus I'm going through the process of watching videos about XMLHttpRequests and how to work with OAuth in JavaScript (even though WebID with foaf+ssl is a bit different). I'm learning about setting headers, expanding my knowledge of xhr structure, working with proxy servers for CORS, working with access tokens, ....

[Edit: https://www.youtube.com/watch?v=82hnvUYY6QA , https://www.youtube.com/watch?v=eogjvTurTN8 ]

I find populating my mind with information, and gradually refining over time as I discover things is an effective way to eventually reach a solution when at first there does not appear to be a explicit and verbose algorithm in my mind to do so.

Is a gradual convergence approach a good idea as any?
Dmitri Zagidulin
@dmitrizagidulin
11:30
it's as good an approach as any. (though I would look at 'fetch' api, instead of XmlHttpRequest'
Brent Shambaugh
@bshambaugh
11:35
@dmitrizagidulin Okay, thanks!
Brent Shambaugh
@bshambaugh
11:44

Looping back, it looks with FetchAPI is a WhatWG Standard and it is documented by MDN and Google Developers. I recall this mentioned before by those in this group as an approach, so I have considerable confidence I'm synchronizing to your efforts. Maybe I'll help with the docs if I get any good.

Verified: solid/solid-web-client#20
[Edit: https://github.com/solid/solid-web-client/issues/20]
[Edit: https://fetch.spec.whatwg.org/ , https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API , https://davidwalsh.name/fetch , https://github.com/github/fetch , https://developers.google.com/web/updates/2015/03/introduction-to-fetch , https://www.youtube.com/watch?v=0bec1BmeY4c ( JavaScript Fetch Api Example - A Real World App) ]

Trying to understand headers HTTP

AJAX Crash Course:

https://www.youtube.com/watch?v=82hnvUYY6QA&t=429s&fref=gc&dti=743759285743274

Content Negotiation:
https://en.wikipedia.org/wiki/Content_negotiation

Now that I have that out of my system....I am looking at Content Negotiation, AJAX requests, and HTTP Headers...

Goal
(1) Accept different file types other than the standard html when hitting a URL (e.g, text/turtle, application+jsonld(?))
(2) Use X509 certificates in the HTTP request (understand authentication and authorization over HTTP)
I have no clue what the Headers should look like in this algorithm:

Maybe see: http://raptorlicious.blogspot.com/2017/09/i-want-cors-with-credentials-notes.html


(http://dig.csail.mit.edu/2009/Papers/SPOT/foaf-ssl-spot2009.pdf)

"The act of retrieving a representation of a resource identified by a URI is known as dereferencing that URI."
https://www.w3.org/2001/tag/doc/httpRange-14/2007-05-31/HttpRange-14

Also see: https://web-payments.org/specs/source/http-signatures/

CORS Experiments

test6.html works like a charm...

https://github.com/bshambaugh/corsexp

RESCAL Links

RESCAL Minimization Problem.

Factorizing YAGO
http://www.dbs.ifi.lmu.de/%7Etresp/papers/p271.pdf

A Three-Way Model for Collective Learning on Multi-Relational Data
http://www.icml-2011.org/papers/438_icmlpaper.pdf

"Python module to compute the RESCAL tensor factorization "
https://github.com/mnick/rescal.py

"Scikit Tensor"
https://github.com/mnick/scikit-tensorTensor Factorization for Multi-Relational Learning

http://ecmlpkdd2013.org/wp-content/uploads/2013/07/673.pdf

Logistic Tensor Factorization for Multi-Relational Data

https://arxiv.org/pdf/1306.2084v1.pdf

A Three-Way Model for Collective Learning on

Multi-Relational Data

28th International Conference on Machine Learning

http://www.cip.ifi.lmu.de/~nickel/data/slides-icml2011.pdf

http://web.mit.edu/~mnick/www/   <<< Maximilian Nickel's website

---------------------------------------------------------

Zhiltsov, N; Agichtein, E "Improving entity search over linked data by modeling latent semantics" https://www.researchgate.net/publication/260419630_Improving_entity_search_over_linked_data_by_modeling_latent_semantics

"Scalable tensor factorization"
https://github.com/nzhiltsov/Ext-RESCAL

--------------------------------------------------------

A Semantic Matching Energy Function for Learning

with Multi-relational Data
Xavier Glorot et. al
https://arxiv.org/pdf/1301.3485.pdf


reference for cross-check: https://github.com/researchstudio-sat/webofneeds/issues/1165

I want cors with credentials notes

I have https://github.com/Rob--W/cors-anywhere/
and I want to add credentials:
https://stackoverflow.com/questions/2054316/sending-credentials-with-cross-domain-posts

sidenote: nginx as cors proxy. https://stackoverflow.com/questions/2054316/sending-credentials-with-cross-domain-posts

set up proxy server for cors

https://daveceddia.com/access-control-allow-origin-cors-errors-in-angular/

Set up proxy server ...


edit: (Thursday, September 14, 2017)

http://jquery-howto.blogspot.com/2013/09/jquery-cross-domain-ajax-request.html


http://ryankirkman.com/2012/03/30/pass-through-cross-domain-proxy-jquery.html 
(with this code the cross domain query is still blocked --- I do not know why...)

https://stackoverflow.com/questions/15005500/loading-cross-domain-endpoint-with-jquery-ajax

setting up with websid

curl https://timbl.com/timbl/Public/ does not fail with a 401, whereas my JavaScript does (it has CORS issues already which need to be fixed...so 401 is the least of my worries).

I have JavaScript that hits the URL. I might need to change this JavaScript such that it works with the ACL scheme. Are there non-human clients that work with the webid protocol? I recall reading that there were previously. https://www.w3.org/2011/identity-ws/papers/idbrowser2011_submission_22/webid.html
mmm..maybe I should start here https://github.com/bblfish/read-write-web

Early explorations using Sigma.js to replicate IsaViz/GSS

Viewing the referenced plugin-customshapes:

But it looks like you will have to modify the shape library /sigma.js/plugins/sigma.renderers.customShapes/shape-library.js  to include a GSS custom shape in addition to the star, the cross, the equilateral triangle, etc...offered by Sigma.js

Experiments Displaying Triples and Substitutions along with a Force Directed Graph in Node-arc-d3

Refer to the repository at: https://github.com/bshambaugh/node-arc-d3/

Step (1): Initiate loading the file of your choice.

Step (2): Confirm you want to load it.

Step (3) : View the Loaded Graph

Step (4) : View the data used for the graph in the top right box, and the substitutions used for the graph in the bottom right box

Step (5) :  View the html that contains the divs containing the data used for the graph in the top right box, and the substitutions used for the graph in the bottom right box

 

Step (6) :  View the d3sketch.js called by the Load File / LDP container button event that writes to the divs

 

Step (7) :  Modify the primitives object in createprimitives.js so the d3sketch.js file can expose the triples

Step (8) : Modify the d3sketch.js file so that it exposes the triples from the object passed from createprimitives.js

 

Step (9) : View the triples loaded in the div in the top right hand corner of index.html

Edit: Thursday September 7th:

Modify d3sketch code to change the display of triples and prefixes.

Following CORS -- Getting Started with WebID-TLS Links

Dr. Andrei Sambra's Dissertation (Discusses ACL, WebID-TLS) - [2013]:

https://tel.archives-ouvertes.fr/file/index/docid/917965/filename/SAMBRA_Andrei-2.pdf

TimBL's Socially Aware Cloud Storage:

https://www.w3.org/DesignIssues/CloudStorage.html

James Dylan Hollenbach's Thesis (Discusses ACL + FOAF+SSL) - [2009]:




http://dig.csail.mit.edu/2010/rdf-widgets/thesis.pdf

Getting Started with CORS Links

https://stackoverflow.com/questions/17318426/cors-cross-domain-ajax-without-jsonp-by-allowing-origin-on-server#17319310

https://stackoverflow.com/questions/5584923/a-cors-post-request-works-from-plain-javascript-but-why-not-with-jquery

http://www.test-cors.org/

http://search.cpan.org/dist/RDF-LinkedData/

https://enable-cors.org/client.html

https://www.html5rocks.com/en/tutorials/cors/

Edit: https://stackoverflow.com/questions/24371734/firefox-cors-request-giving-cross-origin-request-blocked-despite-headers

Beginning Examination of the substitutions in node-arc-d3

In https://github.com/bshambaugh/node-arc-d3

Input:

<> <http://purl.org/dc/terms/title> "Food Growing Methods" ;
<http://www.w3.org/2000/01/rdf-schema#comment> "For independence and resource optimization, some
or all of the astronauts' diet must be grown in-situ. Various forms have been proposed, from greenhouses to
hydroponics, but there must also be optimization in the types of food grown to maximize caloric and nutrient
output compared to water and energy input." ;
<http://data.thespaceplan.com/ontologies/lsi#averageEstInvestmentCost> "0.0(investment cost pulled from children pages)" ;
<http://data.thespaceplan.com/ontologies/lsi#averageEstTimetoMaturity> "0.0 (time to maturity pulled from children pages)" ;
<http://data.thespaceplan.com/ontologies/lsi#commercialStatus> "Research" ;
<http://data.thespaceplan.com/ontologies/lsi#relatedIndustriesFields> "Health and Medicine" ;
<http://data.thespaceplan.com/ontologies/lsi#label> <http://localhost:8080/marmotta/ldp/waypaver-lsi/biological-support> ;
<http://data.thespaceplan.com/ontologies/lsi#label> <http://localhost:8080/marmotta/ldp/waypaver-lsi/habitation-infrastructure> .

document.write(JSON.stringify(namsreplaced)); in js/triplemodifications.js :

{"triples":[["base5:old-Food-Growing-Methods.ttl","dcterms:title","\"Food Growing Methods\""],
["base5:old-Food-Growing-Methods.ttl","rdfs:comment","\"For independence and resource..."],
["base5:old-Food-Growing-Methods.ttl","lsi:averageEstInvestmentCost","\"0.0(investment cost pulled fr..."],
["base5:old-Food-Growing-Methods.ttl","lsi:averageEstTimetoMaturity","\"0.0 (time to maturity pulled ..."],
["base5:old-Food-Growing-Methods.ttl","lsi:commercialStatus","\"Research\""],
["base5:old-Food-Growing-Methods.ttl","lsi:relatedIndustriesFields","\"Health and Medicine\""],
["base5:old-Food-Growing-Methods.ttl","lsi:label","ns1:biological-support"],
["base5:old-Food-Growing-Methods.ttl","lsi:label","ns1:habitation-infrastructure"]],
"replacements":[{"prefix":"ns1","uri":"http://localhost:8080/marmotta/ldp/waypaver-lsi/"},
{"prefix":"base5","uri":"http://localhost/cp4-bc-node-arc-d3/node-arc-d3/data/"},
{"prefix":"dcterms","uri":"http://purl.org/dc/terms/"},
{"prefix":"rdfs","uri":"http://www.w3.org/2000/01/rdf-schema#"},
{"prefix":"lsi","uri":"http://data.thespaceplan.com/ontologies/lsi#"}],
"longstrings":[{"match":"\"For independence and resource optimization, some or all of the astronauts'
                           diet must be grown in-situ. Various forms have been proposed, from greenhouses
                           to hydroponics, but there must also be optimization in the types of food grown
                           to maximize caloric and nutrient output compared to water and energy input.\"",
               "replacement":"\"For independence and resource..."},
               {"match":"\"0.0(investment cost pulled from children pages)\"",
               "replacement":"\"0.0(investment cost pulled fr..."},
               {"match":"\"0.0 (time to maturity pulled from children pages)\"",
                "replacement":"\"0.0 (time to maturity pulled ..."}]}

wish list: get Longwell 2.5.1 working ... it is on the computer after all

https://github.com/bshambaugh/longwell

How do I set up the plugins to work with the sigma.js global object in Require.js?

In /node_modules/sigma/plugins/sigma.layout.forceAtlas2/supervisor.js I have:  

/**
   * Interface
   * ----------
   */
  sigma.prototype.startForceAtlas2 = function(config) {

    // Create supervisor if undefined
    if (!this.supervisor)
      this.supervisor = new Supervisor(this, config);

    // Configuration provided?
    if (config)
      this.supervisor.configure(config);

    // Start algorithm
    this.supervisor.start();

    return this;
  };

I want to bind this to the sigma global object.

Otherwise I get this:


How did I set up other libraries in node-arc-d3?
http://www.forbeslindesay.co.uk/post/46324645400/standalone-browserify-builds

or browserify exportN3.js -s N3 -o bun2-exportN3.js

http://raptorlicious.blogspot.com/2017/02/exposing-n3-from-script-tag-in-browser.html

Browserify with sigma: https://github.com/jacomyal/sigma.js/issues/680

so I need to do something like?:

browserify /src/sigma.exports.js -s sigma -o bun-sigma.exports.js

/build/plugins/sigma.layout.forceAtlas2.min.js

I am trying to figure out how to Load Sigma.js Plugins with Require.js

The Grunt file [1] that I modified does not create a sigma.require.js file that loads the ForceAtlas2 Plugin. (maybe the file does not include the Plugin)

I can get this:

But I cannot get this:

I think I need to include the plugin in the require statement and assign it to the global sigma object:

require([ "sigma", "js/node_modules/sigma/plugins/sigma.layout.forceAtlas2/supervisor" 

,"js/node_modules/sigma/plugins/sigma.layout.forceAtlas2/worker"], function( sigma ) {



    // Your code goes here.



 



});





like in https://learn.jquery.com/jquery-ui/environments/amd/

require([ "jquery", "jquery-ui/autocomplete" ], function( $ ) {



    $( "<input>" )



        .autocomplete({ source: [ "One", "Two", "Three" ]})



        .appendTo( "body" );



});

Also, sigma comes from the config file:

requirejs.config({
    baseUrl : 'js',
    paths : {
         d3 : 'libraries/d3',
         jquery: 'libraries/jquery-3.1.1.min',
         sigma: 'node_modules/sigma/build/sigma.require'
    },
    shim: { 'sigma' : { exports: 'sigma' } }
});
 

[1] https://github.com/bshambaugh/sigma.js/blob/master/Gruntfile.js

Playing with Sigma-force-atlas2

 About:

https://github.com/jacomyal/sigma.js/tree/master/plugins/sigma.layout.forceAtlas2

https://www.rapidops.com/blog/big-data-visualization-using-sigma-js/

http://thewhyaxis.info/hairball/

Modify grunt file in sigma.js to include sigma.canvas.edges.curve ??

Yes: I modified the grunt file and put it in my own repo:

https://github.com/bshambaugh/sigma.js

edge labels, curved edges, and no-overlap does not work. ...I guess I need to rebuild

cat settings-edge2-render-d-index-old-aug22.txt

settings
:
Object
animationsTime
:
200
autoRescale
:
true
autoResize
:
true
batchEdgesDrawing
:
false
borderSize
:
0
canvasEdgesBatchSize
:
500
classPrefix
:
"sigma"
clone
:
true
defaultEdgeColor
:
"#000"
defaultEdgeHoverColor
:
"#000"
defaultEdgeLabelActiveColor
:
"#000"
defaultEdgeLabelColor
:
"#000"
defaultEdgeLabelSize
:
10
defaultEdgeType
:
"def"
defaultHoverLabelBGColor
:
"#fff"
defaultLabelColor
:
"#000"
defaultLabelHoverColor
:
"#000"
defaultLabelSize
:
14
defaultNodeBorderColor
:
"#000"
defaultNodeColor
:
"#000"
defaultNodeHoverColor
:
"#000"
defaultNodeType
:
"def"
doubleClickEnabled
:
true
doubleClickTimeout
:
300
doubleClickZoomDuration
:
200
doubleClickZoomingRatio
:
2.2
doubleTapTimeout
:
300
dragTimeout
:
200
drawEdgeLabels
:
true
drawEdges
:
true
drawLabels
:
true
drawNodes
:
true
edgeColor
:
"source"
edgeHoverColor
:
"edge"
edgeHoverExtremities
:
false
edgeHoverPrecision
:
5
edgeHoverSizeRatio
:
1
edgeLabelSize
:
"fixed"
edgeLabelSizePowRatio
:
1
edgeLabelThreshold
:
1
edgesPowRatio
:
0.5
enableCamera
:
true
enableEdgeHovering
:
false
enableHovering
:
true
eventsEnabled
:
true
font
:
"arial"
fontStyle
:
""
hideEdgesOnMove
:
false
hoverFont
:
""
hoverFontStyle
:
""
immutable
:
true
labelColor
:
"default"
labelHoverBGColor
:
"default"
labelHoverColor
:
"default"
labelHoverShadow
:
"default"
labelHoverShadowColor
:
"#000"
labelSize
:
"fixed"
labelSizeRatio
:
1
labelThreshold
:
8
maxEdgeSize
:
1
maxNodeSize
:
8
minArrowSize
:
0
minEdgeSize
:
0.5
minNodeSize
:
1
mouseEnabled
:
true
mouseInertiaDuration
:
200
mouseInertiaRatio
:
3
mouseWheelEnabled
:
true
mouseZoomDuration
:
200
nodeHoverColor
:
"node"
nodesPowRatio
:
0.5
rescaleIgnoreSize
:
false
scalingMode
:
"inside"
sideMargin
:
0
singleHover
:
true
skipErrors
:
false
touchEnabled
:
true
touchInertiaDuration
:
200
touchInertiaRatio
:
3
verbose
:
false
webglEdgesBatchSize
:
1000
webglOversamplingRatio
:
2
zoomMax
:
2
zoomMin
:
0.0625
zoomingRatio
:
1.7
__proto__
:
Object

cat 4r-settings-edge2-render-d-index-old-aug22.txt

settings
:
Object
animationsTime
:
200
autoRescale
:
true
autoResize
:
true
batchEdgesDrawing
:
false
borderSize
:
0
canvasEdgesBatchSize
:
500
classPrefix
:
"sigma"
clone
:
true
defaultEdgeColor
:
"#000"
defaultEdgeHoverColor
:
"#000"
defaultEdgeType
:
"def"
defaultHoverLabelBGColor
:
"#fff"
defaultLabelColor
:
"#000"
defaultLabelHoverColor
:
"#000"
defaultLabelSize
:
14
defaultNodeBorderColor
:
"#000"
defaultNodeColor
:
"#000"
defaultNodeHoverColor
:
"#000"
defaultNodeType
:
"def"
doubleClickEnabled
:
true
doubleClickTimeout
:
300
doubleClickZoomDuration
:
200
doubleClickZoomingRatio
:
2.2
doubleTapTimeout
:
300
dragTimeout
:
200
drawEdgeLabels
:
false
drawEdges
:
true
drawLabels
:
true
drawNodes
:
true
edgeColor
:
"source"
edgeHoverColor
:
"edge"
edgeHoverExtremities
:
false
edgeHoverPrecision
:
5
edgeHoverSizeRatio
:
1
edgesPowRatio
:
0.5
enableCamera
:
true
enableEdgeHovering
:
false
enableHovering
:
true
eventsEnabled
:
true
font
:
"arial"
fontStyle
:
""
hideEdgesOnMove
:
false
hoverFont
:
""
hoverFontStyle
:
""
immutable
:
true
labelColor
:
"default"
labelHoverBGColor
:
"default"
labelHoverColor
:
"default"
labelHoverShadow
:
"default"
labelHoverShadowColor
:
"#000"
labelSize
:
"fixed"
labelSizeRatio
:
1
labelThreshold
:
8
maxEdgeSize
:
1
maxNodeSize
:
8
minArrowSize
:
0
minEdgeSize
:
0.5
minNodeSize
:
1
mouseEnabled
:
true
mouseInertiaDuration
:
200
mouseInertiaRatio
:
3
mouseWheelEnabled
:
true
mouseZoomDuration
:
200
nodeHoverColor
:
"node"
nodesPowRatio
:
0.5
rescaleIgnoreSize
:
false
scalingMode
:
"inside"
sideMargin
:
0
singleHover
:
true
skipErrors
:
false
touchEnabled
:
true
touchInertiaDuration
:
200
touchInertiaRatio
:
3
verbose
:
false
webglEdgesBatchSize
:
1000
webglOversamplingRatio
:
2
zoomMax
:
2
zoomMin
:
0.0625
zoomingRatio
:
1.7
__proto__
:
Object

I can get the sigma object to load, but I get a slient fail upon rendering

Performing a web search gave me:

https://stackoverflow.com/questions/32863277/sigma-js-graph-doesnt-render-anything

Tada!

Actually, I found my canvas had no height, only a width. Specifying a height and other properties, like in the stackoverflow link:

#graph-container {
  max-width: 400px;
  height: 400px;
  margin: auto;
}

solves the problem of displaying a graph...