First of all, we will use the Xtext project wizard to create the projects for our DSL (we have already experimented with this at the end of Chapter 1, Implementing a DSL).

The wizard will create three projects and it will open the file Entities.xtext, which is the grammar definition.

The main dialog of the wizard is shown in the following screenshot:

The Xtext wizard generates three projects, and, in general, every DSL implemented in Xtext will have these three projects (with a name based on the Project name you specified in the wizard). In our example we have:

We will describe UI functionalities in Chapter 6, Customizations and unit tests in Chapter 7, Testing.

As you may recall from Chapter 1, Implementing a DSL, a default grammar is generated by Xtext. In this section you will learn what this generated grammar contains, and we will modify it to contain the grammar for our Entities DSL. The generated grammar looks like the following:

grammar org.example.entities.Entities with org.eclipse.xtext.common.Terminals

generate entities "http://www.example.org/entities/Entities"

Model:
  greetings+=Greeting*;

Greeting:
  'Hello' name = ID '!';

The first line declares the name of the language (and of the grammar), and it also corresponds to the fully qualified name of the .xtext file (the file is called Entities.xtext, and it is in the package org.example.entities).

The declaration of the grammar also states that it reuses the grammar Terminals, which defines the grammar rules for common things like quoted strings, numbers, and comments, so that in our language we will not have to define such rules. The grammar Terminals is part of the Xtext library; in Chapter 12, Xbase we will see another example of Xtext library grammar (the Xbase grammar).

The generate declaration defines some generation rules for EMF and we will discuss this later.

After the first two declarations, the actual rules of the grammar will be specified. For the complete syntax of the rules, you should refer to the official Xtext documentation (http://www.eclipse.org/Xtext/documentation.html). For the moment, all the rules we will write will have a name, a colon, the actual syntactic form accepted by that rule, and are terminated by a semicolon.

Now we modify our grammar as follows:

grammar org.example.entities.Entities with org.eclipse.xtext.common.Terminals

generate entities "http://www.example.org/entities/Entities"

Model: entities += Entity*;

Entity:
  'entity' name = ID ('extends' superType=[Entity])? '{'
    attributes += Attribute*
  '}'
;

Attribute:
  type=[Entity] array?=('[]')? name=ID ';';

The first rule in every grammar defines where the parser starts and the type of the root element of the model of the DSL, that is, of the Abstract Syntax Tree (AST). In this example, we declare that an Entities DSL program is a collection of Entity elements. This collection is stored in a Model object, in particular in a feature called entities (as we will see later, the collection is implemented as a list). The fact that it is a collection is implied by the operator +=. The star operator, *, states that the number of the elements (in this case Entity) is arbitrary; in particular, it can be any number >= 0. Therefore, a program can also be empty and contain no Entity.

The shape of Entity elements is expressed in its own rule:

Entity:
  'entity' name = ID ('extends' superType=[Entity
])? '{'
    attributes += Attribute*
  '}'
;

First of all, string literals (which in Xtext can be expressed with either single or double quotes) define keywords of the DSL. In this rule we have three keywords, namely 'entity', 'extends', '{', and '}'.

Therefore, a valid entity declaration statement starts with the keyword 'entity' followed by an ID; there is no rule defining ID in our grammar because that is one of the rules that we inherit from the grammar Terminals. If you are curious to know how an ID is defined, you can Ctrl + click on the ID in the Xtext editor and that will bring you to the grammar Terminals, where you can see that an ID starts with an optional '^' character, followed by a letter ('a'..'z'|'A'..'Z'), a '$' character, or an underscore '_' followed by any number of letters, '$' characters, underscores, and numbers ('0'..'9'):

'^'? ('a'..'z'|'A'..'Z'|'$'|'_') 
  ('a'..'z'|'A'..'Z'|'$'|'_'|'0'..'9')*;

The optional '^' character is used to escape an identifier if there are conflicts with existing keywords. The parsed ID will be assigned to the feature name of the parsed Entity model element.

The ( )? operator declares an optional part. Therefore, after the ID, you can write the keyword 'extends' and the name of an Entity. This illustrates one of the powerful features of Xtext, that is, cross-references. In fact, what we want after the keyword 'extends' is not just a name, but the name of an existing Entity. This can be expressed in the grammar using square brackets and the type we want to refer to. Xtext will automatically resolve the cross-reference by searching in the program for an element of that type (in our case an Entity) with the given name. If it cannot find it, it will automatically issue an error. Note that in order for this mechanism to work, the referred element must have a feature called name. As we will see in the following section, the automatic code completion mechanism will also take into consideration cross-references, thus proposing elements to refer to.

Then, the curly brackets '{' '}' are expected and within them Attribute elements can be specified (recall the meaning of += and *); these Attribute elements will be stored in the attributes feature of the corresponding Entity object.

Attribute:
  type=[Entity] array?=('[]')? name=ID ';';

The rule for Attribute requires an Entity name (as explained previously, this is a cross-reference) that will be stored in the type feature and a name for the attribute; attributes must also be terminated with ';'. Note that after the type, an optional '[]' can be specified; in this case, the type of the attribute is considered an array type, and the feature array will be true. This feature is Boolean since we used the ?= assign operator and after such an operator we specify an optional part.

At the end of Chapter 1, Implementing a DSL, we saw how to run the Xtext generator; you should follow the same steps, but instead of right-clicking on the .xtext file and navigating to Run As | Generate Xtext Artifacts, we right-click on the .mwe2 file (in our example it is GenerateEntities.mwe2) and navigate to Run As | MWE2 Workflow. (Remember to accept the request for downloading the ANTLR generator, as explained in Chapter 1, Implementing a DSL).

Before you start the new Eclipse instance, you must make sure that the launch configuration has enough PermGen size, otherwise you will experience "out of memory" errors.

You need to specify the following value as VM arguments in your launch configuration: -XX:MaxPermSize=256m.

You can also simply use the launch configuration that Xtext created for you in your org.example.entities project in the directory .launch; you might not be able to see that directory, since by default the workbench hides resources starting with a dot, so make sure to remove that filter in your workspace preferences. Alternatively, you can right-click on that project and navigate to Run As | Run Configurations...; in the dialog, you can see Launch Runtime Eclipse under Eclipse Application; select it and click on Run.

A new Eclipse instance will be run and a new workbench will appear; in this instance, our Entities DSL implementation is available. So let's create a new General project (call it, for instance, sample). Inside this project, create a new file; the name of the file is not important, but the file extension must be entities (remember that this was the extension we chose in the Xtext project wizard). As soon as the file is created, it will also be opened in a text editor, and you will be asked to add the Xtext nature to your project. You should accept that to make your DSL editor work correctly in Eclipse.

The editor is empty, but there is no error since an empty program is a valid Entities program (remember how the Model rule was defined with the operator *). If you access content assist (with Ctrl + Space bar), you will get no proposal; instead, the entity keyword is inserted for you. This is because the generated content assist is smart enough to know that in that particular program context there is only one valid thing to do: start with the keyword entity.

After that you get an error (refer to the following screenshot), since the entity definition is still incomplete (you can see that the syntax error tells you that an identifier is expected instead of the end of file):

If you access the content assist again, you will get a hint that an identifier is expected (refer to the following screenshot), so let's write an identifier:

If you access the content assist after the identifier, you will see that you get two proposals (refer to the following screenshot). Again, the generated content assist knows that in that program context, you can continue either with an 'extends' specification or with an open curly bracket.

If you choose the open curly bracket, {, you will note some interesting things in the generated editor (refer to the following screenshot):

Continue experimenting with the editor; in particular, in the context where an entity reference is expected (that is, after the extends keyword or when declaring an attribute), you will see that the content assist will provide you with all the Entity elements defined in the current program.

We would also like to stress that all these functionalities, which are far from easy to implement manually, have been generated by Xtext starting from the grammar definition of our DSL.