Pizza Connections

IT Technology Blog

Menu Close

Analyze a class diagram with OrientDB

It's time to share...Share on FacebookTweet about this on TwitterShare on LinkedInShare on Google+Pin on Pinterest
Introduction
A class diagram is a tree that describes the structure of an application. A Tree is just a specialized form of a Graph so why not use a graph database to store a class diagram and analyze it? In this post we will build a graph in OrientDB that contains OrientDB 2.0.6 class diagram. Classes and interfaces will be the vertices of our graph while the edges describe relationships superclassOf and usedBy. So we will use the created graph to analyze the classes and relations between them, by exploiting SQL.

Import data into OrientDB with Java API
We will create a Java application that will use Google Reflections and ASM to extract information about the library OrientDB 2.06, then we will use the JAVA API OrientDB to create the schema and populate it.
Let’s start by creating a new Maven project in Eclipse IDE.
 1
So let’s add the dependencies to our pom.xml:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>net.pizzaconnections</groupId>
  <artifactId>OrientClassDiagramCreator</artifactId>
  <version>1.0</version>
  <dependencies>
  <dependency>
  <groupId>com.orientechnologies</groupId>
  <artifactId>orientdb-graphdb</artifactId>
  <version>2.0.6</version>
  </dependency>
  <dependency>
    <groupId>org.reflections</groupId>
    <artifactId>reflections</artifactId>
    <version>0.9.9-RC1</version>
  </dependency>
  <dependency>
  <groupId>org.ow2.asm</groupId>
  <artifactId>asm</artifactId>
  <version>5.0.2</version>
  </dependency>
  </dependencies>
</project>
Then we create a Collector class that allows you to find out which classes are used by a particular class. To do this we use the library ASM that allows to analyze and modify a class from the bytecode. This is the code of the class:
package net.pizzaconnections.asm;

import java.io.IOException;
import java.util.Comparator;
import java.util.Set;
import java.util.TreeSet;

import org.objectweb.asm.ClassReader;
import org.objectweb.asm.ClassVisitor;
import org.objectweb.asm.commons.EmptyVisitor;
import org.objectweb.asm.commons.Remapper;
import org.objectweb.asm.commons.RemappingClassAdapter;

public class Collector extends Remapper{

    private final Set<Class<?>> classNames;
    private final String prefix;

    public Collector(final Set<Class<?>> classNames, final String prefix){
        this.classNames = classNames;
        this.prefix = prefix;
    }

    /**
     * {@inheritDoc}
     */
    @Override
    public String mapDesc(final String desc){
        if(desc.startsWith("L")){
            this.addType(desc.substring(1, desc.length() - 1));
        }
        return super.mapDesc(desc);
    }

    /**
     * {@inheritDoc}
     */
    @Override
    public String[] mapTypes(final String[] types){
        for(final String type : types){
            this.addType(type);
        }
        return super.mapTypes(types);
    }

    private void addType(final String type){
        final String className = type.replace('/', '.');
        if(className.startsWith(this.prefix)){
            try{
                this.classNames.add(Class.forName(className));
            } catch(final ClassNotFoundException e){
                throw new IllegalStateException(e);
            }
        }
    }

    @Override
    public String mapType(final String type){
    	if(type==null)
    		return "Object";
        this.addType(type);
        return type;
    }


  public static Set<Class<?>> getClassesUsedBy(
      final String name,   // class name
      final String prefix  // common prefix for all classes
                           // that will be retrieved
      ) throws IOException{
      final ClassReader reader = new ClassReader(name);
      final Set<Class<?>> classes =
          new TreeSet<Class<?>>(new Comparator<Class<?>>(){
  
              @Override
              public int compare(final Class<?> o1, final Class<?> o2){
                  return o1.getName().compareTo(o2.getName());
              }
          });
      final Remapper remapper = new Collector(classes, prefix);
      final ClassVisitor inner = new EmptyVisitor();
      final RemappingClassAdapter visitor =
          new RemappingClassAdapter(inner, remapper);
      reader.accept(visitor, ClassReader.EXPAND_FRAMES);
      return classes;
  }
}
Now we create the class OrientDBClassImporter that extracts all data about OrientDB library 2.0.6 classes, then it imports the result into  the OrientClassDiagram schema of our database.
package net.pizzaconnections.importer;

import java.io.IOException;
import java.net.URL;
import java.util.Iterator;
import java.util.Set;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

import net.pizzaconnections.asm.Collector;

import org.reflections.Reflections;
import org.reflections.scanners.SubTypesScanner;

import com.orientechnologies.orient.core.intent.OIntentMassiveInsert;
import com.tinkerpop.blueprints.Compare;
import com.tinkerpop.blueprints.Vertex;
import com.tinkerpop.blueprints.impls.orient.OrientGraphNoTx;
import com.tinkerpop.blueprints.impls.orient.OrientVertex;

public class OrientDBClassImporter {
  
  Pattern pattern = Pattern.compile("(.*)(\\/([a-zA-Z0-9_\\-\\.])*.jar\\!)(.*)");
  
  public static void main(String[] args) throws IOException {
    
    OrientDBClassImporter oi = new OrientDBClassImporter();
    Reflections reflections = new Reflections("com.orientechnologies", new SubTypesScanner(false)); 
    
    Set<Class<? extends Object>> subTypes = 
                 reflections.getSubTypesOf(Object.class);
    
    System.out.println("st "+subTypes);
    OrientGraphNoTx graph = new OrientGraphNoTx("plocal:/your-orientb-path/databases/OrientClassDiagram");

    graph.setRequireTransaction(false);
    graph.getRawGraph().declareIntent(new OIntentMassiveInsert());
    if(graph.getEdgeType("usedBy")==null){
      
      graph.createEdgeType("usedBy");
    }
    
    if(graph.getEdgeType("superclassOf")==null){
      
      graph.createEdgeType("superclassOf");
    }
    
    Iterator<Class<? extends Object>> i = subTypes.iterator();
    
    
    while(i.hasNext()){
      
      Class<? extends Object> c = i.next();
      oi.importClass(graph, c);
    }
    
  }

  private OrientVertex importClass(OrientGraphNoTx graph, Class<? extends Object> c) {
    // TODO Auto-generated method stub
    
    System.out.println("Importing Class : "+ c.getName());
    
    OrientVertex v=null;
    
    //Anonymous, inner class and array aren't imported
    if(c.isAnonymousClass() || c.getName().indexOf('$')!=-1 || c.getName().indexOf('[')!=-1)
      return v;
    
    
    String className = c.getSimpleName();
    String classPackage= c.getPackage().getName();
    String classType = c.isInterface()?"Interface":"Class";
    String classLibrary = "";
    
    //Get library name from class
    URL location = c.getResource('/'+c.getName().replace('.', '/')+".class");
    if(location!=null && location.getFile()!=null && location.getFile().indexOf(".jar")!=-1){
      classLibrary=this.extractLibraryName(location.getFile());
    }
    
    //Creating Vertex Type
    //Interface extends V
    //Class extends V
    if(c.isInterface()){
      if(graph.getVertexType("Interface")==null){
        
        graph.createVertexType("Interface");
        classType="Interface";
      }
    }
    else{
      if(graph.getVertexType("Class")==null){
        
        graph.createVertexType("Class");
        classType="Class";
      }
    }
    
    //Check if the class has already been imported
    Iterable<Vertex> queryVertex=(Iterable<Vertex>) graph.query().has("name", Compare.EQUAL, className).has("package", Compare.EQUAL, classPackage).vertices();
    
    Iterator<Vertex> iter=queryVertex.iterator();
    //If class has already been created returns the imported vertex
    if(queryVertex!=null && iter!=null &&  iter.hasNext()){
      return ((OrientVertex) queryVertex.iterator().next());
    }
      
    //Otherwise I create the vertex
    v = graph.addVertex("class:"+classType);
    v.setProperty("name", className);
    v.setProperty("package", classPackage);
    v.setProperties("library",classLibrary);
    
    //Checks if this class implements interfaces
    if(c.getInterfaces()!=null && c.getInterfaces().length>0){
      
      //Import the interfaces
      //Then for each interface create an edge : interface - superclassOf - v
      for(Class<?> i:c.getInterfaces()){
        if(i.getName()!=c.getName()){
          OrientVertex vi = importClass(graph, i);
          if(vi!=null){
            vi.addEdge("superclassOf", v);
            //graph.commit();
          }
        }
      }
    }
    
    //Check if this class extends another class
    if(c.getSuperclass()!=null){
      
      //Import the extended class (v1)
      //Then create an edge : v1 - superclassOf - v
      OrientVertex v1 = importClass(graph, c.getSuperclass());
      if(v1!=null){
        v1.addEdge("superclassOf", v);
        //graph.commit();
      }
      
    }
    
    //Check classes used by c 
    //only if package prefix is "com.orientechnologies"
    if(c.getPackage()!=null && c.getPackage().getName().indexOf("com.orientechnologies")!=-1){
      Set<Class<?>> usingClasses;
      try{
       usingClasses = Collector.getClassesUsedBy(c.getName(), "");
      }catch(Exception e){
        usingClasses=null;
      }catch(UnsatisfiedLinkError e1){
        usingClasses=null;
      }catch(java.lang.NoClassDefFoundError e2){
        usingClasses=null;
      }
    
      if(usingClasses!=null && !c.isInterface() && !usingClasses.isEmpty() ){
        
        //Imported usedBy classes
        //For each class create an edge : vu - usedBy - v
        for(Class<?> c1:usingClasses){
          
          OrientVertex vu = importClass(graph, c1);
          
          if(vu!=null){
            vu.addEdge("usedBy", v);
          }
        }
        
      } 
    }
    
    
    return v;
  }
  
  private String extractLibraryName(String s){
    
    String extractedString="";
    
    
    Matcher m = pattern.matcher(s);
    
    while(m.find()){
      extractedString=m.group(2);
      extractedString=extractedString.substring(1,extractedString.length()-1);
    }
    
    
    return extractedString;
  }

}
In particular, the OrientDBClassImporter class uses the importClass method recursively to extract and import information about the class name, the origin package, the origin library and relationships with the possible superclass and used classes .
Remember to change the path of the line 34 with the path of your OrientDB installation.
If you are Java 8 and lambda expressions fan I add the OrientDBClassImporterLambda class code that works like as previous class but using the java 8 lambda expressions.
package net.pizzaconnections.importer;

import java.io.IOException;
import java.net.URL;
import java.util.Arrays;
import java.util.Iterator;
import java.util.List;
import java.util.Set;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

import net.pizzaconnections.asm.Collector;

import org.reflections.Reflections;
import org.reflections.scanners.SubTypesScanner;

import com.orientechnologies.orient.core.intent.OIntentMassiveInsert;
import com.tinkerpop.blueprints.Compare;
import com.tinkerpop.blueprints.Vertex;
import com.tinkerpop.blueprints.impls.orient.OrientGraphNoTx;
import com.tinkerpop.blueprints.impls.orient.OrientVertex;

public class OrientDBClassImporterLambda {
  
  Pattern pattern = Pattern.compile("(.*)(\\/([a-zA-Z0-9_\\-\\.])*.jar\\!)(.*)");
  
  public static void main(String[] args) throws IOException {
    
    OrientDBClassImporterLambda oi = new OrientDBClassImporterLambda();
    
    Reflections reflections = new Reflections("com.orientechnologies", new SubTypesScanner(false)); 
    
    Set<Class<? extends Object>> subTypes = 
                 reflections.getSubTypesOf(Object.class);
    
    OrientGraphNoTx graph = new OrientGraphNoTx("plocal:/your-orientdb-path/databases/OrientClassDiagram");
    graph.setRequireTransaction(false);
    graph.getRawGraph().declareIntent(new OIntentMassiveInsert());
    if(graph.getEdgeType("usedBy")==null){
      
      graph.createEdgeType("usedBy");
    }
    
    if(graph.getEdgeType("superclassOf")==null){
      
      graph.createEdgeType("superclassOf");
    }
    
    Iterator<Class<? extends Object>> i = subTypes.iterator();
    
    i.forEachRemaining(c -> oi.importClass(graph, c)); 
    
    
  }

  private OrientVertex importClass(OrientGraphNoTx graph, Class<? extends Object> c) {
    System.out.println("Importing Class : "+ c.getName());
    final OrientVertex  v;
    
    //Anonymous, inner class and array aren't imported
    if(c.isAnonymousClass() || c.getName().indexOf('$')!=-1 || c.getName().indexOf('[')!=-1)
      return null;
    
    String className = c.getSimpleName();
    String classPackage= c.getPackage().getName();
    String classType = c.isInterface()?"Interface":"Class";
    String classLibrary = "";
    
    //Get library name from class
    URL location = c.getResource('/'+c.getName().replace('.', '/')+".class");
    if(location!=null && location.getFile()!=null && location.getFile().indexOf(".jar")!=-1){
      classLibrary=this.extractLibraryName(location.getFile());
    }
    
    //Creating Vertex Type
    //Interface extends V
    //Class extends V
    if(c.isInterface()){
      if(graph.getVertexType("Interface")==null){
        
        graph.createVertexType("Interface");
        classType="Interface";
      }
    }
    else{
      if(graph.getVertexType("Class")==null){
        
        graph.createVertexType("Class");
        classType="Class";
      }
    }
    
    //Check if the class has already been imported
    Iterable<Vertex> queryVertex=(Iterable<Vertex>) graph.query().has("name", Compare.EQUAL, className).has("package", Compare.EQUAL, classPackage).vertices();
    
    Iterator<Vertex> iter=queryVertex.iterator();
    //If class has already been created returns the imported vertex
    if(queryVertex!=null && iter!=null &&  iter.hasNext()){
      return ((OrientVertex) queryVertex.iterator().next());
    }
      
    //Otherwise I create the vertex
      v = graph.addVertex("class:"+classType);
    v.setProperty("name", className);
    v.setProperty("package", classPackage);
    v.setProperties("library",classLibrary);
    
    //Checks if this class implements interfaces
    if(c.getInterfaces()!=null && c.getInterfaces().length>0){
      
      //Import the interfaces
      //Then for each interface create an edge : interface - superclassOf - v
      List<Class<?>> interfaces = Arrays.asList(c.getInterfaces());
      
      interfaces.forEach(i->{if(i.getName()!=c.getName()){
                      OrientVertex vi = importClass(graph, i);
                      if(vi!=null) 
                        vi.addEdge("superclassOf", v);
                  }
                  
                  });
    }
    
    //Check if this class extends another class
    if(c.getSuperclass()!=null){
      
      //Import the extended class (v1)
      //Then create an edge : v1 - superclassOf - v
      OrientVertex v1 = importClass(graph, c.getSuperclass());
      if(v1!=null){
        v1.addEdge("superclassOf", v);
        //graph.commit();
      }
      
    }
    
    //Check classes used by c 
    //only if package prefix is "com.orientechnologies"
    if(c.getPackage()!=null && c.getPackage().getName().indexOf("com.orientechnologies")!=-1){
      Set<Class<?>> usingClasses;
      try{
       usingClasses = Collector.getClassesUsedBy(c.getName(), "");
      }catch(Exception e){
        usingClasses=null;
      }catch(UnsatisfiedLinkError e1){
        usingClasses=null;
      }catch(java.lang.NoClassDefFoundError e2){
        usingClasses=null;
      }
      
      if(usingClasses!=null && !c.isInterface() && !usingClasses.isEmpty() ){
        //Imported usedBy classes
        //For each class create an edge : vu - usedBy - v
        usingClasses.forEach(c1 -> {OrientVertex vu = importClass(graph, c1); 
                        if(vu!=null){
                          vu.addEdge("usedBy", v);
                        }
                       });
        
      } 
    }
    
    
    return v;
  }
  
  private String extractLibraryName(String s){
    
    String extractedString="";
    
    
    Matcher m = pattern.matcher(s);
    
    while(m.find()){
      extractedString=m.group(2);
      extractedString=extractedString.substring(1,extractedString.length()-1);
    }
    
    return extractedString;

  }

}
Now you can execute with Eclipse (Run as -> Java Application) OrientDBClassImporter or OrientDBClassImporterLambda to import the data.
2
You can download the project from github.
 
Play with graph
Let’s start OrientDB server
 3
Then connect to the OrientClassDiagram schema.
4
If you click on the graph tab and execute a simple query on Class you can navigate in your graph.
 5
We begin the graph analysis with a query that allows you to understand the OrientDB library class with more usedBy relationship.
select name, package, out('usedBy').size() as usedCount 
  from Class 
 where package like 'com.orientechnologies%' 
 order by usedCount desc
 6
From this query we can see that a change to the class ODocument affects 253 classes. Then “CAUTION: HOT! DO NOT TOUCH!”
Now we can search the classes of OrientDB library with more relationships with external libraries:
select name, package, count(*) as extRelCount
  from (select expand(out('usedBy'))
  	     from Class 
        where package not like 'com.orientechnologies%' 
          and package not like 'java.%')
group by name, package
order by extRelCount desc
These classes will be more subject to changes if we want to reduce the number of external libraries that we use.
Finally we see a query that returns all classes that use a particular library.
select expand(out('usedBy'))
  from Class 
 where library='mail-1.4.jar'
This query is useful to see the classes that we need to change if we want to replace a library because this library has bugs or has license problems.
 
Conclusions
In this POC we created a graph that contains a class diagram using the Java API OrientDB. The JAVA APIs are very simple to use, yet very powerful, they allow to model and populate a graph without need to know SQL commands.  Moreover, we have seen as this graph could be useful to collect informations about classes relationships and manage possible problems. Since the OrientDB graph is easily extensible, next steps could be achieved by adding information about methods or by importing github meta-data. In this way we can manage information about annotated methods and release revisions, contributors etc…
It's time to share...Share on FacebookTweet about this on TwitterShare on LinkedInShare on Google+Pin on Pinterest

© 2017 Pizza Connections. All rights reserved.

Theme by Anders Norén.