Payloadbuilder

SQL query engine with plugable catalogs (data sources)

Table of Contents

• About the Project
  • Catalogs
  • Built With
• Usage
• Getting Started
  • Editor
• Roadmap
• Contributing
• License

About the project

Query engine with SQL support, easy to extend with catalogs to allow for joining data sets from completely different data sources. A simple analyzer that builds an operator tree from provided query for data retrieval.

Supports:

• JOIN (INNER, LEFT)
• APPLY (CROSS, OUTER)
• TVF's
• Scalar functions
• Key lookup Index
• Multiple statements
• Control-flow statements

Grammar

Catalogs

Catalogs is the extension point where external data is provided. A catalog implementation can provide scalar functions, table valued functions, data operators for tables.

Provided catalogs

Built With

• JDK 8
• Maven

Usage

• mvn install
• Add Reference
  • org.kuse.payloadbuilder.core/target/org.kuse.payloadbuilder.core-{version}.jar
  • org.kuse.payloadbuilder.catalog/target/org.kuse.payloadbuilder.catalog-{version}.jar
    • Contains bundled Catalog implementations
  • Release to Central is coming soon

Getting Started

Simple query using multiple catalogs:

        QuerySession session = new QuerySession(new CatalogRegistry());
        session.setPrintStream(System.out);
        session.getCatalogRegistry().registerCatalog("es", new ESCatalog());
        session.getCatalogRegistry().registerCatalog("fs", new FileSystemCatalog());

        session.setCatalogProperty("es", "endpoint", "http://localhost:9200");
        session.setCatalogProperty("es", "index", "myindex");

        String query = "select top 10 " +
            "        d.fileName " + 
            ",       f.size " + 
            ",       f.lastModifiedTime " + 
            "from es#_doc d " + 
            "cross apply fs#file(concat('/path/to/files/', d.fileName)) f " + 
            "where d._docType = 'pdfFiles'";

        QueryResult queryResult = Payloadbuilder.query(session, query);
        JsonStringWriter writer = new JsonStringWriter();
        while (queryResult.hasMoreResults())
        {
          queryResult.writeResult(writer);
          System.out.println(writer.getAndReset());
        }

Editor

Standalone Query Editor written in Swing

Editor

Raodmap

Alot :)

• Documentation
• Insert/Update/Delete support
• More operators (Merge join, union, except, anti-join, semi-join etc.)
• More catalog implementations (Mongo, Redis etc.)

Contributing

Patches, discussions about features, changes etc. welcome.

License

Distributed under the Apache License Version 2.0. See LICENSE for more information.