cEP-0018: Integration of ANTLR in coala

Propose integration of ANTLR in coala

Closes #118

Author: virresh

cEP-0018.md

@@ -0,0 +1,162 @@
+# Integration of ANTLR into coala core
+
+| Metadata     |                                         |
+| ------------ |-----------------------------------------|
+| cEP          | 0018                                    |
+| Version      | 0.1.0                                   |
+| Title        | Integration of ANTLR into coala core    |
+| Authors      | Viresh Gupta <viresh16118@iiitd.ac.in>  |
+| Status       | Proposed                                |
+| Type         | Feature                                 |
+
+# Abstract
+
+This document describes how an API based on ANTLR will be constructed and
+maintained.
+
+# Introduction
+
+ANTLR provides parsers for various language grammars and thus we can provide
+an interface from within coala and make it available to bear writers so that
+they can write advanced linting bears. This will aim at supporting the more
+flexible visitor based method of AST traversal (as opposed to listener based
+mechanisms).
+
+The proposal is to introduce a parallel concept to the coala-bears library,
+which will be called the coala-ast hereon.
+
+# Proposed Change
+
+Here is the detailed implementation stepwise:
+
+    1. There will be a separate repository named as coala-ast which will be
+    installable via ```pip install coala-ast```.
+    2. A new interface would be introduced in coala-core (```coala/coalib```),
+    which would be the new ast package. This would be the ```coalib.ast``` package
+    3. The ```coalib.ast``` package will provide endpoints relevant to the visitor
+    model of traversing the ast generated via ANTLR.
+    4. The coala-ast repository will hold parser generated (with python targe)
+    beforehand for the supported set of grammars.
+    5. ```coalib.ast``` will have an AST loader class that will be responsible for
+    loading the AST of a given file depending on the extension.
+    For e.g, a .c file will be loaded using the parser from ```coala-ast``` for c
+    language.
+    6. Another AST walker class will be responsible for providing endpoints for
+    traversing the AST using methods such as ```get_next_node``` and
+    ```get_previous_node```.
+
+# Management of the new ```coala-ast``` repository
+
+Managing a new repository is a heavy task, and this will be highly automated as
+follows:
+
+    1. The parsers in coala-ast would be generated and pushed via travis-builds
+    whenever a new grammar is pushed.
+    2. The cib tool can be enhanced to deal with the installation of bears that
+    require only some specified parsers (for e.g a PyUnusedVarBear would only
+    require parser for python)
+    3. The cib tool can also trigger specialised builds and download the newly
+    generated parser on the fly
+
+# Code Samples/Prototypes
+Here is a prototype for the implementations within coalib.ast:
+
+```python
+import parsers
+import antlr4
+
+class ASTLoader():
+    lexerMapping = {
+        'c' : parser.clexer,
+        'py' : parser.pylexer,
+        'js' : parser.jslexer,
+        ...
+    }
+    parserMapping = {
+        'c' : parser.cparser,
+        'py' : parser.pyparser,
+        'js' : parser.jsparser,
+        ...
+    }
+
+    @staticmethod
+    def loadFile(file,filename):
+        """ Loads file's AST into memory """
+        ext = get_extension(filename)
+        if ext is None:
+            raise FileExtensionNotSupported
+        inputStream = file.readlines()
+        lexer = lexerMapping[ext](inputStream)
+        parser = parserMapping[ext](lexer)
+        return parser
+
+class ASTWalker():
+    treeRoot = None
+    tree = None
+    def __init__(file,filename):
+        self.tree = ASTLoader.load(file, filename)
+        self.treeRoot = self.tree
+
+    def get_next_node(self):
+        """ Modify the internal tree variable and return the next node """
+
+    def get_prev_node(self):
+        """ Modify the internal tree variable and return prev node """
+
+    def reset(self):
+        """ Reset the internal tree variable to tree root """
+
+    def isTerminalNode(node):
+        """ Determine if the node is terminal """
+        return isinstance(node, antlr4.tree.Tree.TerminalNode)
+
+    ...
+
+```
+
+
+### Prototype of  `ASTBear` class implementation:
+
+```python
+from coalib.bears.LocalBear import Bear
+from coalaib.ast import ASTWalker
+
+class ASTBear(LocalBear):
+    walker = None
+
+    def initialise(file, filename):
+        walker = ASTWalker(file, filename)
+
+    def run(self,
+            filename,
+            file,
+            tree,
+            *args,
+            dependency_results=None,
+            **kwargs):
+        raise NotImplementedError           # Needs to be done by bear
+
+```
+
+A test bear:
+
+```python
+from coalib.results.Result import Result
+from coalib.bears.ASTBear import ASTBear
+
+class TestBear(ASTBear):
+    def run(self,
+            filename,
+            file,
+            tree,
+            *args,
+            dependency_results=None,
+            **kwargs):
+        self.initialise(file, filename)
+        num_nodes = 0
+        while(tree.get_next_node() != None):
+            num_nodes += 1
+        yield Result(self, 'File {} has {} nodes '
+                           ' in AST.'.format(filename, num_nodes))
+
+```