[Proposal] Combining of Exuberant Ctags and GNU GLOBAL.

[Shigio YAMAGUCHI]

Hello ctags hackers,

I'm a developer of GNU GLOBAL source code tagging system.
<http://www.gnu.org/software/global/>

Exuberant Ctags is a great tool. I wish to use it in combination
with GNU GLOBAL.  Here is a proposal for combining them.
Sorry for a long sentence.

This mail is sent to 'Exuberant Ctags developer mailing list'
with Cc: 'GNU GLOBAL bug mailing list'.

[Notes for people of bug-global mailing list]
Please note that you can not post a message to the ctags mailing
list without subscription.

-------------------------------------------------------------------

Combining of Exuberant Ctags and GNU GLOBAL
                                   by Shigio YAMAGUCHI  2012/Oct/27

0. Introduction

GNU GLOBAL is a source code tagging system that works the same way
across diverse environments (emacs, vi, less, bash, web browser, etc).
The feature of it is an ability to treat not only definitions but
also references.

Though it can be used in combination with Exuberant Ctags, still
it is not enough. This proposal is a thing about the method of better
combination of the two tools.


1. Current Status

Now, you can replace the built-in parser of GNU GLOBAL by
Exuberant Ctags.  With some setup, gtags invokes ctags with
the --format=1 and --filter option, and read tag records from it.

The concrete setup method is as follows.

        (Installation of GLOBAL-5.8.1 or later)
        $ ./configure --with-exuberant-ctags=/usr/local/bin/ctags
        $ sudo make install

        (Execution of GLOBAL)
        $ export GTAGSCONF=/usr/local/share/gtags/gtags.conf
        $ export GTAGSLABEL=exuberant-ctags
        $ gtags                         # invokes ctags internally
        $ global -x func                # locates tags
        func                5 test.c           func() {
        $ _

This means that GLOBAL already supports 41 programming languages
supported by Exuberant Ctags. But it cannot generate reference
tags, because Exuberant Ctags does not pick up them. My proposal
is to bury this lack.


2. Adding a type string to the cross reference format

How about adding --gtags option to Exuberant Ctags?

The --gtags option adds a type string to the cross reference
format of ctags. This enables GLOBAL to treat references.

<a type string>    <cross reference format (level 1 or 2)>
                 ^
                 |
        one or more blanks

Type strings: (explained later)

        "D": definition
        "R": other than definition

Examples
--------

$ ctags -x --format=1 --gtags test.c
D func                5 test.c           func() {

$ ctags -x --format=2 --gtags test.c
D func             function      5 test.c           func() {

We should be able to consider the type string does not belong
to the cross reference format.


3. How to use a type string.

Please assume the following source code.

        [test.c]
        +---------------------
1       |sub() {
2       |       ...
3       |       return 1;
4       |}
5       |func() {
6       |       strcpy(...);
7       |       if (sub()) {
8       |               ...
9       |       }
10      |}
        +---------------------

Now, Exuberant Ctags with the --format=1 option generates
the following output.

$ ctags -x --format=1 test.c
func                5 test.c           func() {
sub                 1 test.c           sub() {

Please care about that the following symbols are ignored,
since they are not a definition.

'strcpy' in the 6th line
'sub' in the 7th line

When the --gtags option is specified, Exuberant Ctags may
pick up reference tags with a type string like follows.

$ ctags -x --format=1 --gtags test.c
D func                5 test.c      func() {
R strcpy              6 test.c             strcpy(...);   <== ADD
D sub                 1 test.c      sub() {
R sub                 7 test.c             if (sub()) {   <== ADD

Algorithm of the parser may be like this.

        foreach of all tokens {
                if (symbol?)
                        if (definition?)
                                Put a record with "D"
                        else
                                Put a record with "R"     <== ADD
        }

GLOBAL can use the type string for classification of tags.
Rewriting GLOBAL to accept above format, GLOBAL commands
will work like follows.

$ gtags                                                 # make tag files
$ global -xd sub
sub                 1 test.c      sub() {               # show definitions
$ global -xr sub
sub                 7 test.c             if (sub()) {   # show references
$ global -xs strcpy
strcpy              6 test.c             strcpy(...);   # show other symbols
$ _

        [-d] shows definitions
        -r   shows references (defined in other places)
        -s   shows other symbols (not defined anywhere)

Please note that definitions, references and other symbols are
treated separately.  This means that Exuberant Ctags become
a perfect parser of GLOBAL.


4. For parsers which don't support reference tags.

The treatment of parsers which don't generate reference tags
is easy. What is necessary is just to add a "D" string
unconditionally at the head of all the records when the --gtags
option is specified.

$ ctags -x --format=1 --gtags test.c
D func                5 test.c           func() {
D sub                 1 test.c           sub() {

Though such output don't have "R" records, it is no problem.


5. Is this combination useful?

for GLOBAL
----------

Although GLOBAL's language support is very poor, we can conquer
the weak point by the combination.

for Exuberant Ctags
-------------------

Although it is difficult for Exuberant Ctags to realize reference
tag support independently, it becomes easy by the combination.

for parser writers
------------------

Parser writers can write a common parser which is applicable
to both of Exuberant Ctags and GLOBAL. This reduces the time
and efforts of them.

for users
---------

Users obtain the third choice, that is, Exuberant Ctags (1st), 
GNU GLOBAL(2nd), and Exuberant Ctags through GNU GLOBAL(3rd).


6. Postscript

Of course, even if this combination is realized, parsers do not
necessarily increase automatically. But it will become an
infrastructure for those who try to write a new parser.

Originally, GNU GLOBAL was developed as a framework for adapting
ctags to large projects. Exuberant Ctags and GNU GLOBAL are
complement partners, I believe.


What do you think?

-- End of proposal
-------------------------------------------------------------------

That's all. Please ask me, if there is an unclear point.
Thank you for reading a long letter.

Best Regards,
Shigio
--
Shigio YAMAGUCHI <address@hidden>
PGP fingerprint: D1CB 0B89 B346 4AB6 5663  C4B6 3CA5 BBB3 57BE DDA3