ETL JSON - Not working as documented in demo

I’m following the ETL Json demo here: Import from JSON

I’m not receiving any errors as you can see below. The database is getting created, but the Account vertex, Friend and Enemy edges, and the data are getting loaded… The user name admin and the password are not being updated either. I have to log into the new database with root.

Hi,
can you post the json files?
Can you precisely describe the problem?

Thanks

Regards,
Michela

eltdemodata.json
[
{
“name”: “Joe”,
“id”: 1,
“friends”: [2,4,5],
“enemies”: [6]
},
{
“name”: “Suzie”,
“id”: 2,
“friends”: [1,4,6],
“enemies”: [5,2]
}
]

etldemo.json
{
“config”: {
“log”: “debug”
},
“source” : {
“file”: { “path”: “…/etl/etldemodata.json” }
},
“extractor” : {
“json”: {}
},
“transformers” : [
{ “merge”: { “joinFieldName”: “id”, “lookup”: “Account.id” } },
{ “vertex”: { “class”: “Account”} },
{ “edge”: {
“class”: “Friend”,
“joinFieldName”: “friends”,
“lookup”: “Account.id”,
“unresolvedLinkAction”: “CREATE”
} },
{ “edge”: {
“class”: “Enemy”,
“joinFieldName”: “enemies”,
“lookup”: “Account.id”,
“unresolvedLinkAction”: “CREATE”
} }
],
“loader” : {
“orientdb”: {
“dbURL”: “plocal:…/databases/etldemo”,
“dbUser”: “admin”,
“dbPassword”: “scada”,
“dbAutoDropIfExists”: true,
“dbAutoCreate”: true,
“standardElementConstraints”: false,
“tx”: false,
“wal”: false,
“batchCommit”: 1000,
“dbType”: “graph”,
“classes”: [{“name”: “Account”, “extends”:“V”}, {“name”: “Friend”, “extends”:“E”}, {“name”: ‘Enemy’, “extends”:“E”}],
“indexes”: [{“class”:“Account”, “fields”:[“id:integer”], “type”:“UNIQUE_HASH_INDEX” }]
}
}
}

According to the Import from JSON documentation, the classes, indexes, vertex, edges, and data should be added to the target database. Here is the expected output from the documentation. Note, the script runs without reporting any errors, the database is created, but none of the classes, or data are added, and the username/password specified in the json file is not valid when trying to connect to the newly created database.

Expected Output from Documentation
OrientDB etl v.2.1-SNAPSHOT www.orientechnologies.com
feb 09, 2015 2:46:42 AM com.orientechnologies.common.log.OLogManager log
INFORMAZIONI: OrientDB auto-config DISKCACHE=10.695MB (heap=3.641MB os=16.384MB disk=42.205MB)
[orientdb] INFO Dropping existent database ‘plocal:/tmp/databases/db’…
BEGIN ETL PROCESSOR
[file] DEBUG Reading from file /tmp/database.json
[orientdb] DEBUG - OrientDBLoader: created vertex class ‘Account’ extends ‘V’
[orientdb] DEBUG orientdb: found 0 vertices in class ‘null’
[orientdb] DEBUG - OrientDBLoader: created edge class ‘Friend’ extends ‘E’
[orientdb] DEBUG orientdb: found 0 vertices in class ‘null’
[orientdb] DEBUG - OrientDBLoader: created edge class ‘Enemy’ extends ‘E’
[orientdb] DEBUG orientdb: found 0 vertices in class ‘null’
[orientdb] DEBUG - OrientDBLoader: created property ‘Account.id’ of type: integer
[orientdb] DEBUG - OrientDocumentLoader: created index ‘Account.id’ type ‘UNIQUE_HASH_INDEX’ against Class ‘Account’, fields [id:integer]
[0:merge] DEBUG Transformer input: {name:Joe,id:1,friends:[3],enemies:[1]}
[0:merge] DEBUG joinValue=1, lookupResult=null
[0:merge] DEBUG Transformer output: {name:Joe,id:1,friends:[3],enemies:[1]}
[0:vertex] DEBUG Transformer input: {name:Joe,id:1,friends:[3],enemies:[1]}
[0:vertex] DEBUG Transformer output: v(Account)[#11:0]
[0:edge] DEBUG Transformer input: v(Account)[#11:0]
[0:edge] DEBUG joinCurrentValue=2, lookupResult=null
[0:edge] DEBUG created new vertex=Account#11:1{id:2} v1
[0:edge] DEBUG created new edge=e[#12:0][#11:0-Friend->#11:1]
[0:edge] DEBUG joinCurrentValue=4, lookupResult=null
[0:edge] DEBUG created new vertex=Account#11:2{id:4} v1
[0:edge] DEBUG created new edge=e[#12:1][#11:0-Friend->#11:2]
[0:edge] DEBUG joinCurrentValue=5, lookupResult=null
[0:edge] DEBUG created new vertex=Account#11:3{id:5} v1
[0:edge] DEBUG created new edge=e[#12:2][#11:0-Friend->#11:3]
[0:edge] DEBUG Transformer output: v(Account)[#11:0]
[0:edge] DEBUG Transformer input: v(Account)[#11:0]
[0:edge] DEBUG joinCurrentValue=6, lookupResult=null
[0:edge] DEBUG created new vertex=Account#11:4{id:6} v1
[0:edge] DEBUG created new edge=e[#13:0][#11:0-Enemy->#11:4]
[0:edge] DEBUG Transformer output: v(Account)[#11:0]
[1:merge] DEBUG Transformer input: {name:Suzie,id:2,friends:[3],enemies:[2]}
[1:merge] DEBUG joinValue=2, lookupResult=Account#11:1{id:2,in_Friend:[#12:0]} v2
[1:merge] DEBUG merged record Account#11:1{id:2,in_Friend:[#12:0],name:Suzie,friends:[3],enemies:[2]} v2 with found record={name:Suzie,id:2,friends:[3],enemies:[2]}
[1:merge] DEBUG Transformer output: Account#11:1{id:2,in_Friend:[#12:0],name:Suzie,friends:[3],enemies:[2]} v2
[1:vertex] DEBUG Transformer input: Account#11:1{id:2,in_Friend:[#12:0],name:Suzie,friends:[3],enemies:[2]} v2
[1:vertex] DEBUG Transformer output: v(Account)[#11:1]
[1:edge] DEBUG Transformer input: v(Account)[#11:1]
[1:edge] DEBUG joinCurrentValue=1, lookupResult=Account#11:0{name:Joe,id:1,friends:[3],enemies:[1],out_Friend:[#12:0, #12:1, #12:2],out_Enemy:[#13:0]} v5
[1:edge] DEBUG created new edge=e[#12:3][#11:1-Friend->#11:0]
[1:edge] DEBUG joinCurrentValue=4, lookupResult=Account#11:2{id:4,in_Friend:[#12:1]} v2
[1:edge] DEBUG created new edge=e[#12:4][#11:1-Friend->#11:2]
[1:edge] DEBUG joinCurrentValue=6, lookupResult=Account#11:4{id:6,in_Enemy:[#13:0]} v2
[1:edge] DEBUG created new edge=e[#12:5][#11:1-Friend->#11:4]
[1:edge] DEBUG Transformer output: v(Account)[#11:1]
[1:edge] DEBUG Transformer input: v(Account)[#11:1]
[1:edge] DEBUG joinCurrentValue=5, lookupResult=Account#11:3{id:5,in_Friend:[#12:2]} v2
[1:edge] DEBUG created new edge=e[#13:1][#11:1-Enemy->#11:3]
[1:edge] DEBUG joinCurrentValue=2, lookupResult=Account#11:1{id:2,in_Friend:[#12:0],name:Suzie,friends:[3],enemies:[2],out_Friend:[#12:3, #12:4, #12:5],out_Enemy:[#13:1]} v6
[1:edge] DEBUG created new edge=e[#13:2][#11:1-Enemy->#11:1]
[1:edge] DEBUG Transformer output: v(Account)[#11:1]
END ETL PROCESSOR
+ extracted 2 entries (0 entries/sec) - 2 entries -> loaded 2 vertices (0 vertices/sec) Total time: 228ms [0 warnings, 0 errors]

Hi,
I’ve been able to reproduce the situation, the import worked as expected.
If you want to report this behaviour feel free to open an issue on github.

Thanks

Regards,
Michela

What do you mean the import worked as expected? Did it create the Account vertex class, Friend, and Enemy edge classes, and import the two vertexs?

Hi,
during the import I didn’t have outputs but, this is the result from a SELECT:

+----+-----+-------+----+-------------+-----+-------+-------+-------------------+-------------+--------+
|#   |@RID |@CLASS |id  |in_Friend    |name |friends|enemies|out_Friend         |out_Enemy    |in_Enemy|
+----+-----+-------+----+-------------+-----+-------+-------+-------------------+-------------+--------+
|0   |#17:0|Account|2   |[#21:0]      |Suzie|[1,4,6]|[5,2]  |[#24:0,#21:1,#22:1]|[#26:0,#27:0]|[#27:0] |
|1   |#17:1|Account|6   |[#22:1]      |     |       |       |                   |             |[#25:0] |
|2   |#18:0|Account|1   |[#24:0]      |Joe  |[2,4,5]|[6]    |[#21:0,#22:0,#23:0]|[#25:0]      |        |
|3   |#19:0|Account|4   |[#22:0,#21:1]|     |       |       |                   |             |        |
|4   |#20:0|Account|5   |[#23:0]      |     |       |       |                   |             |[#26:0] |
+----+-----+-------+----+-------------+-----+-------+-------+-------------------+-------------+--------+