Gremlin with OResultSet.close() exception

#1

Hi

  • BackGround

I use gremlin language to create Vertex and Edge.

  • Version
Driver : orientdb-tp3-3.0.18

Maven:
       <dependency>
           <groupId>org.apache.tinkerpop</groupId>
           <artifactId>tinkergraph-gremlin</artifactId>
           <version>3.3.0</version>
       </dependency>

       <dependency>
           <groupId>com.orientechnologies</groupId>
           <artifactId>orientdb-gremlin</artifactId>
           <version>3.0.18</version>
       </dependency>
  • Method
Vertex:
    /**
     * create vertexs
     *
     * @param nodes
     */
    public void createNodes(OrientGraph orientGraph, List<Node> nodes, List<Relation> relations, KngraphFusionRule kngraphFusionRule) {

        // index configuration
        Configuration configuration = orientGraph.configuration();

        configuration.setProperty("type", OClass.INDEX_TYPE.NOTUNIQUE);

        for (Node node : nodes) {

            String nodeName = node.getNodeName();

            if (StringUtils.isBlank(nodeName)) {
                continue;
            }

            // label / class
            String label = node.getLabel();

            if (StringUtils.isNotBlank(label)) {
                label = label.trim();

                OClass aClass = orientGraph.getRawDatabase().getMetadata().getSchema().getClass(label);

                //create class and index
                if (aClass == null) {

                    orientGraph.createClass(label, V);

                    orientGraph.createVertexIndex("name", label, configuration);

                    orientGraph.createVertexIndex("kngraphId", label, configuration);

                    orientGraph.createVertexIndex("type", label, configuration);
                }

                Vertex vertex = null;

                //check exit
                boolean exit = g.V()
                        .has("kngraphId", kngraphFusionRule.getKngraphId())
                        .hasLabel(label)
                        .has("name", node.getNodeName())
                        .hasNext();
                // not exit 
                if (!exit) {
                    vertex = g.addV(label)
                            .property("name", node.getNodeName())
                            .property("date", new Date())
                            .next();
                } else {
                    //get vertex
                    vertex = g.V()
                            .has("kngraphId", kngraphFusionRule.getKngraphId())
                            .hasLabel(label)
                            .has("name", node.getNodeName())
                            .next();
                }

                log.info(vertex.toString());
            }
        }
    }


Edge:

    /**
     * create Edges
     *
     * @param relations
     */
    public void createRelations(OrientGraph orientGraph, List<Relation> relations, KngraphFusionRule kngraphFusionRule) {

        // index configuration
        Configuration configuration = orientGraph.configuration();

        configuration.setProperty("type", OClass.INDEX_TYPE.NOTUNIQUE.toString());

        for (Relation relation : relations) {

            // label / class
            String eLabel = relation.getLabel();

            if (StringUtils.isNotBlank(eLabel)) {

                eLabel = eLabel.trim();

                OClass aClass = orientGraph.getRawDatabase().getMetadata().getSchema().getClass(eLabel);

                // create class and index
                if (aClass == null) {

                    orientGraph.createClass(eLabel, E);

                    orientGraph.createEdgeIndex("name", eLabel, configuration);

                    orientGraph.createEdgeIndex("kngraphId", eLabel, configuration);

                    orientGraph.createEdgeIndex("type", eLabel, configuration);
                }

                // OutV() of edge
                RelationNode start = relation.getStart();

                // InV() of edge
                RelationNode end = relation.getEnd();

                Edge edge = null;

                // check exit
                boolean exit = g.E()
                        .has("kngraphId", kngraphFusionRule.getKngraphId())
                        .hasLabel(eLabel)
                        .where(outV()
                                .hasLabel(start.getLabel())
                                .has("name", start.getName())
                                .and()
                                .inV()
                                .hasLabel(end.getLabel())
                                .has("name", end.getName())
                        )
                        .hasNext();

                // not exit
                if (!exit) {

                    if (StringUtils.isBlank(start.getName()) || StringUtils.isBlank(end.getName())) {
                        continue;
                    }

                    // outV()
                    Vertex startNode = this.checkNode(start.getLabel(), start.getName(), kngraphFusionRule.getKngraphId());

                    // inV()
                    Vertex endNode = this.checkNode(end.getLabel(), end.getName(), kngraphFusionRule.getKngraphId());

                    // create edge
                    edge = startNode.addEdge(eLabel, endNode);

                } else {

                    // get edge
                    edge = g.E()
                            .has("kngraphId", kngraphFusionRule.getKngraphId())
                            .hasLabel(eLabel)
                            .where(outV()
                                    .hasLabel(start.getLabel())
                                    .has("name", start.getName())
                                    .and()
                                    .inV()
                                    .hasLabel(end.getLabel())
                                    .has("name", end.getName())
                            ).next();
                }

                log.info(edge.toString());
            }
        }

    }

  • Logs
2019-04-19 16:26:27 - v[#94:424]
2019-04-19 16:26:27 - v[#109:245]
2019-04-19 16:26:27 - v[#95:424]
2019-04-19 16:26:27 - v[#110:245]
2019-04-19 16:26:28 - v[#96:424]
2019-04-19 16:26:28 - v[#111:245]
2019-04-19 16:26:28 - v[#97:424]
2019-04-19 16:26:29 - v[#112:245]
2019-04-19 16:26:29 - v[#98:424]
2019-04-19 16:26:29 - v[#113:245]
2019-04-19 16:26:30 - v[#99:424]
2019-04-19 16:26:30 - v[#114:245]
2019-04-19 16:26:30 - v[#100:424]
2019-04-19 16:26:31 - v[#115:245]
2019-04-19 16:26:31 - v[#101:424]
2019-04-19 16:26:31 - v[#116:245]
2019-04-19 16:26:32 - v[#102:424]
2019-04-19 16:26:32 - v[#117:245]
2019-04-19 16:26:33 - v[#103:423]
2019-04-19 16:26:33 - v[#118:244]

--- EOF ---

2019-04-19 16:26:33 - Finished to create Vertexs,Start to create Edges , kngraphId--> GR0000000000946  
2019-04-19 16:26:34 - e[#121:6][#97:218-guanximingchen->#120:124]
2019-04-19 16:26:35 - e[#122:6][#98:218-guanximingchen->#105:125]
2019-04-19 16:26:36 - e[#123:6][#99:217-guanximingchen->#106:124]
2019-04-19 16:26:37 - e[#124:6][#100:217-guanximingchen->#106:124]
2019-04-19 16:26:38 - e[#125:5][#101:217-guanximingchen->#106:124]
2019-04-19 16:26:38 - This database instance has 10 open command/query result sets, please make sure you close them with OResultSet.close()
2019-04-19 16:26:39 - e[#126:5][#102:217-guanximingchen->#106:124]
2019-04-19 16:26:40 - e[#127:5][#103:217-guanximingchen->#105:125]
2019-04-19 16:26:41 - e[#128:5][#104:217-guanximingchen->#107:124]
2019-04-19 16:26:42 - e[#129:5][#89:218-guanximingchen->#107:124]
2019-04-19 16:26:43 - e[#130:5][#90:218-guanximingchen->#108:124]
2019-04-19 16:26:44 - This database instance has 20 open command/query result sets, please make sure you close them with OResultSet.close()
2019-04-19 16:26:45 - e[#131:5][#91:218-guanximingchen->#109:124]
2019-04-19 16:26:46 - e[#132:5][#92:218-guanximingchen->#109:124]
2019-04-19 16:26:48 - e[#133:5][#93:218-guanximingchen->#109:124]
2019-04-19 16:26:53 - e[#134:5][#94:218-guanximingchen->#109:124]
2019-04-19 16:26:54 - e[#135:5][#95:218-guanximingchen->#109:124]
2019-04-19 16:26:54 - This database instance has 30 open command/query result sets, please make sure you close them with OResultSet.close()
2019-04-19 16:26:55 - This database instance has 30 open command/query result sets, please make sure you close them with OResultSet.close()
2019-04-19 16:26:56 - e[#136:5][#96:218-guanximingchen->#110:124]
2019-04-19 16:26:56 - This database instance has 30 open command/query result sets, please make sure you close them with OResultSet.close()
2019-04-19 16:26:56 - e[#121:5][#97:217-guanximingchen->#110:124]
2019-04-19 16:26:56 - e[#122:5][#98:217-guanximingchen->#110:124]
2019-04-19 16:26:57 - e[#123:5][#99:216-guanximingchen->#108:124]
2019-04-19 16:26:57 - e[#124:5][#100:216-guanximingchen->#111:124]
2019-04-19 16:26:58 - e[#125:4][#101:216-guanximingchen->#111:124]
2019-04-19 16:26:58 - This database instance has 40 open command/query result sets, please make sure you close them with OResultSet.close()
2019-04-19 16:27:01 - e[#126:4][#102:216-guanximingchen->#111:124]
2019-04-19 16:27:02 - e[#127:4][#103:216-guanximingchen->#111:124]
2019-04-19 16:27:03 - e[#128:4][#104:216-guanximingchen->#111:124]
2019-04-19 16:27:05 - e[#129:4][#89:217-guanximingchen->#111:124]
2019-04-19 16:27:06 - e[#130:4][#90:217-guanximingchen->#108:124]
2019-04-19 16:27:07 - This database instance has 50 open command/query result sets, please make sure you close them with OResultSet.close()
2019-04-19 16:27:08 - e[#131:4][#91:217-guanximingchen->#112:124]
  • Descriptions

First. Before create Vertex or Edge , I will check whether it exists. if it does ,I get it and if not , I will create it

Second. About the logs. Before the flag called ‘EOF’ ,it is operation about Vertexs , and after the flag, it is operation about Edges.

Finally. I use the method called createNodes() to create Vertexs and the method called createRelations() to create Edges

  • Questions

When I start to deal with Edges , i get some messages ,such as( something after EOF in Logs) :

2019-04-19 16:26:44 - This database instance has 20 open command/query result sets, please make sure you close them with OResultSet.close()

On the one hand , it seems that i should close the OResultSet . In OrientDB Language, I know it .but now ,I did with gremlin , I do not know how to do ?

On the other hand , when i deal with Vertexs ,I get nothing about it .

it is very strangely.

  • Another

I have another ting. My creating vertex and edge process is too long . long than one hour. in other words ,during creating process ,i wil met session expire timeout. what i did is :

when i met session timeout ,I will reopen the connection. codd such as( this is my create vertex and edge process) :

  public void execute(KngraphData kngraphData, KngraphFusionRule kngraphFusionRule) {

        try {

            // vertex collections
            List<Node> nodeList = kngraphData.getNodes();

            // edge collections
            List<Relation> relationList = kngraphData.getRelations();

            orientGraph = OrientdbUtil.openTx(orientGraphFactory);

            g = orientGraph.traversal();

            log.info("Start to create Vertex  , kngraphId--> {}  ", kngraphFusionRule.getKngraphId());

            // create Vertex
            this.createNodes(orientGraph, nodeList, relationList, kngraphFusionRule);

            log.info("Finish to create Vertex ,Start to create Edge , kngraphId--> {}  ", kngraphFusionRule.getKngraphId());

            // create edge
            this.createRelations(orientGraph, relationList, kngraphFusionRule);

            log.info("Finish to create edge , kngraphId--> {} ", kngraphFusionRule.getKngraphId());

        } catch (Exception e) {

            e.printStackTrace();

            // error
            String message = e.getMessage();

            //session expire timeout
            if (e instanceof ODatabaseException && message.contains("expired session")) {

                // commit 
                OrientdbUtil.commitAndClose(orientGraph);

                do {

                    this.execute(kngraphData, kngraphFusionRule);

                } while (true);
 
            } else {

                // update operations
                Kngraph kngraph = new Kngraph();

                kngraph.setGraphId(kngraphFusionRule.getKngraphId());
                kngraph.setStatus(KngraphStatus.error.getValue());
                kngraphService.updateByGraphId(kngraph);

                // rollback
                orientGraph.rollback();

                throw new KngraphBuildErrorException("build exception");
            }

        } finally {

            OrientdbUtil.commitAndClose(orientGraph);
        }
    }

SomeBody help !

52HZ

Thanks

#2

Hi @52HZ

I see that you are using the traversal from OrientGraph. That means that you could end up doing multiple roudtrip client server for a traversal.

For example this one

boolean exit = g.E()
                        .has("kngraphId", kngraphFusionRule.getKngraphId())
                        .hasLabel(eLabel)
                        .where(outV()
                                .hasLabel(start.getLabel())
                                .has("name", start.getName())
                                .and()
                                .inV()
                                .hasLabel(end.getLabel())
                                .has("name", end.getName())
                        ) 

the first part it will be translated in a sql query

has("kngraphId", kngraphFusionRule.getKngraphId())
                    .hasLabel(eLabel)

to select from elabel where kngraphId = kngraphFusionRule.getKngraphId() but then the rest of the traversal will be executed on the client with elements fetched from the server when needed.

Depending from the use case it could be ok for you.

For the OResultSet.close() the query in ODB it is paginated so the result set should be closed. If you use the api .next or hasNext from gremlin traversal the traversal itself will not be closed and the query not closed.

You have two options

  1. you use the api toList() in order to consume the result set that will be automatically closed when consumed

  2. you can use the try with resource in order to close the traversal

For example

try (GraphTraversal traversal = g.V().hasLabel("Countries").has("Id", P.gt(1))) {
        boolean hasNext = traversal.hasNext();
        System.out.println(hasNext);
      };

GraphTraversal is AutoClosable and it will close the query even if it’s not entirely consumed

Let me know if this helps

Thanks

#3

Hi @wolf4ood

  • I did like what you said, and it works for me.

but now I want to know if it’s because of that the traversal executed on the client ? In other words,if the traversal will be closed if i did :

Vertex vertex = g.V()
                 .hasLabel(label)
                 .next()

another question , and which operations will be executed on the client ?

52HZ

Thanks