How to Write Custom Language Support Plugins
January 16th, 2013 by Andrey CheptsovToday we would like to share with you a simple tutorial how to write a plugin with custom language support for IntelliJ IDEA and IntelliJ Platform.
As you know IntelliJ IDEA provide powerful facilities for developers to implement advanced code assistance for custom languages and frameworks. In this step-by-step tutorials you will learn how to use Grammar-Kit to generate a parser and PSI elements, how to use JFlex to generate a lexer, how to implement custom formatting, refactoring support, etc.
The sample code for this tutorial can be found on GitHub.
More steps with other aspects of plugin development are coming soon. In the meanwhile don’t miss the opportunity to register for the second live coding webinar about plugin development, which will take place on Tuesday, January 22.
Develop with Pleasure!
January 16th, 2013 at 8:29 am
This is great! If only this was available a month ago, when I started playing with this. By now, I’ve figured out almost everything of this by myself, using the Erlang plugin and the Developing Custom Language Plugins for IntelliJ IDEA page (http://confluence.jetbrains.net/display/IDEADEV/Developing+Custom+Language+Plugins+for+IntelliJ+IDEA). I haven’t needed the reference contributor, though. What is it needed for?
Also, what I’m playing with now involves navigating across files. Stub indexes etc. Cool stuff, although I’m not sure I fully understand it. The Indexing and PSI Stubs in IntelliJ IDEA page (http://confluence.jetbrains.net/display/IDEADEV/Indexing+and+PSI+Stubs+in+IntelliJ+IDEA) only provides a very basic introduction.
January 16th, 2013 at 8:32 am
Oh, and reference resolution. That’s pretty important topic, I’d suggest covering the scope processors stuff as well. It isn’t needed for such extremely simple language, but it’s absolutely essential once you start playing with something real. Thank you!
January 16th, 2013 at 8:39 am
Please share tutorial how to write a complete plugin for PyCharm;) or API Reference
January 16th, 2013 at 11:01 am
Will such a custom language plugin only work in IntelliJ IDEA or also with the other IDE’s, like PHPStorm or PyCharm?
January 16th, 2013 at 3:28 pm
Could you please also explain, on an example, how the pin and recoverUntil work, as well as how to simplify the AST trees by using the \extends\?
January 16th, 2013 at 3:40 pm
Awesome!
January 16th, 2013 at 5:01 pm
This is great! I noted that the “Lexer and Parser Definition” page is missing a step to create the SimpleFile class that is used in the parser definition. Looking forward to writing my first plugin now.
January 17th, 2013 at 3:59 am
@Alberto: It usually works in all IDEs, depends on how you define the metadata.
January 17th, 2013 at 4:25 am
@Alberto: please see http://confluence.jetbrains.net/display/IDEADEV/Plugin+Compatibility+with+IntelliJ+Platform+Products for more information about making your plugin compatible with other IntelliJ based products
January 17th, 2013 at 4:43 am
@Tobias You are right, thanks. Fixed.
January 17th, 2013 at 9:17 am
+∞ to Luke. The GrammarKit documentation altogether would need some love, but the only important thing I really didn’t understand is pin/recoverUntil. Or actually, I think that I understand them (when a pinned element of the rule is successfully parsed, the rule itself will be considered successfully parsed, and the parser will eat all the following tokens until it reaches some tokens from the recoverUntil set), but I have a really hard time using them correctly. Basically, since I have a C-like syntax, I just settled with a single pinned element in a grammar and a single recoverUntil on ‘;’ or ‘}’. Everytime I try to add some more, the parser stops working and I have no idea why
January 17th, 2013 at 9:53 am
Ladislav,
If you have any specific questions regarding the stubs/indexes, please ask, and we’ll update the document to make sure they’re covered.
January 17th, 2013 at 9:58 am
Konrad,
Check out lib/src/pycharm-openapi-src.zip in the PyCharm distribution and https://github.com/JetBrains/intellij-plugins/tree/master/pycharm-flask
January 18th, 2013 at 5:49 am
Great product! This time its mine
January 18th, 2013 at 9:25 am
@Ladislav, @Luke Please check out http://goo.gl/l7cVj and the 5 lines below. This is a perfectly working example of recoverUntil concept.
January 20th, 2013 at 8:18 am
@Sergey Thanks, but I was able to get to something very similar. I have a working rule with pin/recoverUntil in my grammar, just by following the documentation. That, however, isn’t enough to really understand. Once I get back to it, I will probably end up reverse engineering the concept from differences in generated code.
Oh, and one thing I found just before few minutes. GrammarKit doesn’t support generating classes needed for stub indexes. It looks like it is possible to write all that manually, but it requires working around some GrammarKit bugs: 1. GrammarKit doesn’t like generics (extra interface for each StubBasedPsiElement needed), 2. GrammarKit doesn’t generate proper imports in generated visitor (and for this, there is a pull request with fix for more than 2 months!). And that’s only a beginning for me.
February 8th, 2013 at 12:21 pm
http://goo.gl/19mM0 may shed more light on grammar-kit error recovery
February 14th, 2013 at 3:18 am
The FindUsagesProvider-section uses an static word-scanner-instance.
public WordsScanner getWordsScanner() {
return WORDS_SCANNER;
}
As far as I know this will cause threading issues as the lexer is not thread safe.
February 21st, 2013 at 5:21 pm
Is any way to extend existing ‘language plugin’ with minimal efforts?
For example, i need plugin for Squirrel language:
http://squirrel-lang.org
Its like slightly simplifier and abit extended version of Java.
March 14th, 2013 at 9:55 pm
Thanks for the tutorial, but IntelliJ doesn’t seem to associate the file type specified with my language after following step 2: \Language and file type\.
March 19th, 2013 at 10:45 am
Totally agreed that this is an outstanding tutorial! However, like Luke and Ladislav, I still don’t quite understand how best to use pin and recoverUntil, even after looking at the Erlang grammar. The language I’m implementing is very Java-like, and I’ve made some progress with pins so that I now get “foo expected, found bar” most of the time, but I have a feeling I’m abusing it rather than using it. Also, every time I try to use recoverUntil, primarily by using an expression such as “!(SEMICOLON | RBRACE)”, I just end up tanking the parser.
I’d be happy to share a few simple rules from my grammar as concrete examples of those help the discussion.