C/CPreProcessor: In-file macro expanding (CPreProcessor._expand) is dependent on output kind/fields

[ParadoxV5]

Firstly, #2543 is promising.

Expected (by users)

Kinds and fields are toggles for what is present in the output. The expansion of macros (presumably during parsing) is independent of those output configs.

Actual (though understandable to u-ctags devs)

Expanding macros requires all of the following to be active:

• language set to or determined to be C (well, that is expected, although this does exclude C++ (which .h files default to))
• d macro definitions kind on
• signature field on
• macrodef C field on

The param has no effect if any of the above are off.

Demo

Using the example straight from parser-cxx.rst (the raw file, that is – it failed to render both on GitHub and on https://docs.ctags.io)

$ cat input.c
#define DEFUN(NAME) int NAME (int x, int y)
#define BEGIN {
#define END }

DEFUN(myfunc)
  BEGIN
  return -1
  END
$ ctags --param-CPreProcessor._expand=1 --fields-C=+'{macrodef}' --fields=+'{signature}' -o - input.c
BEGIN   input.c /^#define BEGIN /;"     d       file:   macrodef:{
DEFUN   input.c /^#define DEFUN(/;"     d       file:   signature:(NAME)        macrodef:int NAME (int x, int y)
END     input.c /^#define END /;"       d       file:   macrodef:}
myfunc  input.c /^DEFUN(myfunc)$/;"     f       typeref:typename:int    signature:(int x,int y)
$ ctags --param-CPreProcessor._expand=1 --fields-C=+'{macrodef}' -o - input.c
BEGIN   input.c /^#define BEGIN /;"     d       file:   macrodef:{
DEFUN   input.c /^#define DEFUN(/;"     d       file:   macrodef:int NAME (int x, int y)
END     input.c /^#define END /;"       d       file:   macrodef:}
$ ctags --param-CPreProcessor._expand=1 --fields=+'{signature}' -o - input.c
BEGIN   input.c /^#define BEGIN /;"     d       file:
DEFUN   input.c /^#define DEFUN(/;"     d       file:   signature:(NAME)
END     input.c /^#define END /;"       d       file:
$ ctags --param-CPreProcessor._expand=1 --fields-C=+'{macrodef}' --fields=+'{signature}' --kinds-C=-d -o - input.c
$

• The first ctags execution successfully reproduces parser-cxx.rst’s example.
• The second and third executions leave out a field each. The function myfunc is absent.
• The last execution turns the macro definitions kind off, but because neither is the function myfunc present, the output is blank.

ctags --version

Universal Ctags 6.0.0, Copyright (C) 2015-2022 Universal Ctags Team
Universal Ctags is derived from Exuberant Ctags.
Exuberant Ctags 5.8, Copyright (C) 1996-2009 Darren Hiebert
  Compiled: May 18 2023, 13:13:54
  URL: https://ctags.io/
  Output version: 0.0
  Optional compiled features: +wildcards, +regex, +iconv, +option-directory, +xpath, +json, +interactive, +sandbox, +yaml, +packcc, +optscript

[masatake]

That is understandable, but I'm negative about changing the code. I cannot explain the reason well...
If we introduce the parameter <-> kinds dependency, the internal of ctags is more complicated.
I'm afraid of the complication. I have made ctags complicated for years. To do more interesting things on ctags, I think we need overhauling.

How about adding what you wrote in Actual (though understandable to u-ctags devs) to our man page like https://docs.ctags.io/en/latest/man/ctags-lang-asm.7.html .

Thank you for exploring the feature.