configure bug (?): FORTIFY_SOURCE in CPPFLAGS breaks configure on Arch Linux

[msieczka]

Arch Linux build system by default sets CPPFLAGS="-D_FORTIFY_SOURCE=2".

If I unset FORTIFY_SOURCE, configure and make passes. But when it's set, configure in GRASS 6 and 7 fails.

I'm attaching config.log-fail (FORTIFY_SOURCE is set) and config.log-pass (FORTIFY_SOURCE is NOT set) for GRASS 7.0.3. The suspicious bits config.log-fail seem to be:

/usr/include/sys/cdefs.h:30:3: error: #error "You need a ISO C conforming compiler to use the glibc headers"
gcc: error: unrecognized command line option '-nologo'

gcc 5.3.0, make 4.1, autoconf 2.69

[scimmia]

Known issue with the ancient version of autoconf used to generate the configure script.

[neteler]

Replying to msieczka:

gcc 5.3.0, make 4.1, autoconf 2.69

You would need to use autoconf-2.13 if you want to regenerate the configure script. But I guess you don't do that anyway.

I tried on my Fedora box:

Copying config.status to config.status.x86_64-pc-linux-gnu

GRASS is now configured for:  x86_64-pc-linux-gnu

  Source directory:           /home/neteler/software/grass71
  Build directory:            /home/neteler/software/grass71
  Installation directory:     ${prefix}/grass-7.1.svn
  Startup script in directory:${exec_prefix}/bin
  C compiler:                 gcc -O2 -march=native -fdiagnostics-color 
  C++ compiler:               c++ -O2 -D_FORTIFY_SOURCE=2
...

gcc --version
gcc (GCC) 5.3.1 20151207 (Red Hat 5.3.1-2)

I have a link which seems to be missing on your system according to your log file:

ls -la /lib/cpp
lrwxrwxrwx 1 root root 10 Dec  8 18:45 /lib/cpp -> ../bin/cpp*

# its origin:
rpm -qf /lib/cpp
cpp-5.3.1-2.fc23.x86_64

Maybe you simply lost that link?

[msieczka]

Replying to neteler:

Replying to msieczka:

gcc 5.3.0, make 4.1, autoconf 2.69

You would need to use autoconf-2.13 if you want to regenerate the configure script. But I guess you don't do that anyway.

I'd rather rely on an upstream-provided configure script, if possible.

I tried on my Fedora box:

  C++ compiler:               c++ -O2 -D_FORTIFY_SOURCE=2

Is it OK that CPPFLAGS content ends up in CXXFLAGS?

I have a link which seems to be missing on your system according to your log file:

ls -la /lib/cpp
lrwxrwxrwx 1 root root 10 Dec  8 18:45 /lib/cpp -> ../bin/cpp*

# its origin:
rpm -qf /lib/cpp
cpp-5.3.1-2.fc23.x86_64

Maybe you simply lost that link?

There is no CPP package as such on Arch Linux. cpp, gcc and g++ executables come in a single GCC package here. It doesn't provide this `/lib/cpp' link.

Anyway, I created /lib/cpp -> ../bin/cpp symlink manually, but this doesn't help and doesn't seem to be the actual issue. The references to a missing /lib/cpp link only got replaced by yet more "_FORTIFY_SOURCE requires compiling with optimization" warnings. The original errors remain:

/usr/include/sys/cdefs.h:30:3: error: #error "You need a ISO C conforming compiler to use the glibc headers"
gcc: error: unrecognized command line option '-nologo'

I'm attaching a complete config.log-fail-with_cpp_symlink.

[msieczka]

Replying to neteler:

You would need to use autoconf-2.13 if you want to regenerate the configure script. But I guess you don't do that anyway.

I have regenerated the configure script with autoconf-2.13 now. No improvement: configure still fails when CPPFLAGS="-D_FORTIFY_SOURCE=2" is set, and the resulting config.log is pretty much identical to the attached config.log-fail​.

[neteler]

Replying to msieczka:

I have regenerated the configure script

Why do you do this? there is no need for that, all is in SVN.

[neteler]

Replying to msieczka:

Replying to neteler:

Replying to msieczka:

gcc 5.3.0, make 4.1, autoconf 2.69

You would need to use autoconf-2.13 if you want to regenerate the configure script. But I guess you don't do that anyway.

I'd rather rely on an upstream-provided configure script, if possible.

? SVN-grass is upstream.

There is no CPP package as such on Arch Linux. cpp, gcc and g++ executables come in a single GCC package here.

I see.

It doesn't provide this `/lib/cpp' link.

Seems to be a known issue in Archlinux, e.g.: ​https://bbs.archlinux.org/viewtopic.php?id=147199

Suggestion: Try to set CPP, then configure (guessing here):

export CPP=/usr/bin/cpp
configure ...

[glynn]

Replying to msieczka:

Arch Linux build system by default sets CPPFLAGS="-D_FORTIFY_SOURCE=2".

If I unset FORTIFY_SOURCE, configure and make passes. But when it's set, configure in GRASS 6 and 7 fails.

configure:2722: checking how to run the C preprocessor
configure:2741: gcc -E -D_FORTIFY_SOURCE=2 conftest.c >/dev/null 2>conftest.out
In file included from /usr/include/assert.h:35:0,
                 from configure:2736:
/usr/include/features.h:328:4: warning: #warning _FORTIFY_SOURCE requires compiling with optimization (-O) [-Wcpp]
 #  warning _FORTIFY_SOURCE requires compiling with optimization (-O)
    ^
configure: failed program was:

This all arises because autoconf tends to treat ANYTHING written to stderr as indicating a failure.

It then goes on to try:

configure:2758: gcc -E -traditional-cpp -D_FORTIFY_SOURCE=2 conftest.c >/dev/null 2>conftest.out
In file included from /usr/include/assert.h:36:0,
                 from configure:2754:
/usr/include/features.h:328:4: warning: #warning _FORTIFY_SOURCE requires compiling with optimization (-O) [-Wcpp]
 #  warning _FORTIFY_SOURCE requires compiling with optimization (-O)

which fails for the same reason.

Finally it tries:

configure:2775: gcc -nologo -E -D_FORTIFY_SOURCE=2 conftest.c >/dev/null 2>conftest.out
gcc: error: unrecognized command line option '-nologo'

After everything it has tried has failed (by its standards), it unconditionally sets CPP to /lib/cpp and uses that (resulting in every subsequent use of $(CPP) failing).

The first thing it tried (gcc -E) would actually work, but it doesn't know that.

[msieczka]

Replying to neteler:

Replying to msieczka:

I have regenerated the configure script

Why do you do this? there is no need for that, all is in SVN.

I thought I'd try this after your #comment:2, quote:

"You would need to use autoconf-2.13 if you want to regenerate the configure script. But I guess you don't do that anyway."

Nevermind.

[msieczka]

Replying to neteler:

Replying to msieczka:

Replying to neteler:

Replying to msieczka:

gcc 5.3.0, make 4.1, autoconf 2.69

You would need to use autoconf-2.13 if you want to regenerate the configure script. But I guess you don't do that anyway.

I'd rather rely on an upstream-provided configure script, if possible.

? SVN-grass is upstream.

I'm talking from Arch Linux package perspective. In that case "upstream" is GRASS 7.0.3 source code tarball.

There is no CPP package as such on Arch Linux. cpp, gcc and g++ executables come in a single GCC package here.

I see.

It doesn't provide this `/lib/cpp' link.

Seems to be a known issue in Archlinux, e.g.: ​https://bbs.archlinux.org/viewtopic.php?id=147199

Suggestion: Try to set CPP, then configure (guessing here):

export CPP=/usr/bin/cpp
configure ...

Thanks. This doesn't change anything though.

I think Glynn nailed it. I'm going to have a closer look at his comments now.

[msieczka]

Replying to glynn:

Replying to msieczka:

Arch Linux build system by default sets CPPFLAGS="-D_FORTIFY_SOURCE=2".

If I unset FORTIFY_SOURCE, configure and make passes. But when it's set, configure in GRASS 6 and 7 fails.

configure:2722: checking how to run the C preprocessor
configure:2741: gcc -E -D_FORTIFY_SOURCE=2 conftest.c >/dev/null 2>conftest.out
In file included from /usr/include/assert.h:35:0,
                 from configure:2736:
/usr/include/features.h:328:4: warning: #warning _FORTIFY_SOURCE requires compiling with optimization (-O) [-Wcpp]
 #  warning _FORTIFY_SOURCE requires compiling with optimization (-O)
    ^
configure: failed program was:

This all arises because autoconf tends to treat ANYTHING written to stderr as indicating a failure.

Thanks for the explanation. Do you know why this "_FORTIFY_SOURCE requires compiling with optimization (-O)" warning is triggered? I think it shouldn't be, because I do have optimization enabled in my build environment:

$ grep '\-O' /etc/makepkg.conf 
CFLAGS="-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong"
CXXFLAGS="-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong"
LDFLAGS="-Wl,-O1,--sort-common,--as-needed,-z,relro"

config.log-fail contents also reflect this.

[glynn]

Replying to msieczka:

Thanks for the explanation. Do you know why this "_FORTIFY_SOURCE requires compiling with optimization (-O)" warning is triggered? I think it shouldn't be, because I do have optimization enabled in my build environment:

Optimisation switches are normally in CFLAGS or CXXFLAGS, while -D switches are normally in CPPFLAGS. The configure checks for the preprocessor don't pass $(CFLAGS) or $(CXXFLAGS) because most of the switches that they're likely to contain often aren't recognised by a standalone preprocessor, and may result in an error. The comment in autoconf/c.m4 says:

# CFLAGS is not in ac_cpp because -g, -O, etc. are not valid cpp options.

Apart from that, there are valid reasons for compiling without optimisation (e.g. it often makes debugging significantly more difficult), so forcibly enabling _FORTIFY_SOURCE doesn't appear to have been a particularly wise move.

FWIW, on Gentoo this is implemented within the compiler:

$ gcc -E -dM -O2 -x c /dev/null | fgrep FORTIFY
#define _FORTIFY_SOURCE ((defined __OPTIMIZE__ && __OPTIMIZE__ > 0) ? 2 : 0)

I.e. _FORTIFY_SOURCE is enabled automatically if and only if optimisation is enabled.

As for workarounds, the simplest to implement would be to have configure.in check whether $(CPPFLAGS) contains -D_FORTIFY_SOURCE and either attempt to remove it or generate an error telling the user to remove it. It can always be added to CFLAGS and/or CXXFLAGS.

[msieczka]

I forgot to say that the /etc/makepkg.conf contents I quoted are as per default Arch Linux settings, and that makepkg is an official build tool for Arch packages. I'm saying this just to clarify that I'm not making this case up, but that it's a real issue for Arch GRASS users, in case anybody wondered.

So far I've been removing D_FORTIFY_SOURCE from CPPFLAGS before configure as follows, in my Arch GRASS PKGBUILD (which is an input for makepkg, something like a Gentoo ebuild):

CPPFLAGS=`echo $CPPFLAGS | sed 's/-D_FORTIFY_SOURCE=.//g'`

If understand what you are saying, the following would be better (putting aside your point against forcibly enabling _FORTIFY_SOURCE at all):

export CFLAGS="$CPPFLAGS $CFLAGS"
export CXXFLAGS="$CPPFLAGS $CXXFLAGS"
unset CPPFLAGS

By "better" here I mean "preserving the intended makepkg environment at GRASS build-time".

Do you know whether all (sane) CPPFLAGS can be safely be mixed into CXXFLAGS and CFLAGS like this?

And what do you think about scimmia's "ancient version of autoconf used" #comment:1? I mean could using a newer autoconf version to generate the GRASS configure script fix the problem?

[glynn]

Replying to msieczka:

If understand what you are saying, the following would be better (putting aside your point against forcibly enabling _FORTIFY_SOURCE at all):

export CFLAGS="$CPPFLAGS $CFLAGS"
export CXXFLAGS="$CPPFLAGS $CXXFLAGS"
unset CPPFLAGS

The problem with this is that it breaks any case where the preprocessor is run standalone, rather than as part of compilation.

I don't think that happens anywhere within actual the GRASS build process, as Platform.make.in doesn't mention @CPP@ anywhere (so there's no valid way for a Makefile to know how to run the preprocessor by itself).

But it can create problems for the configure script, which uses the pre-processor extensively. E.g. AC_CHECK_HEADERS is based upon AC_TRY_CPP, so if it needs e.g. any -I switches from CPPFLAGS, moving those into CFLAGS/CXXFLAGS will break it.

Do you know whether all (sane) CPPFLAGS can be safely be mixed into CXXFLAGS and CFLAGS like this?

That isn't a problem. The C compiler is always passed both $(CFLAGS) and $(CPPFLAGS), while the C++ compiler is always passed both $(CXXFLAGS) and $(CPPFLAGS). If the preprocessor is run separately, it should only be passed $(CPPFLAGS), hence the distinction.

And what do you think about scimmia's "ancient version of autoconf used" #comment:1? I mean could using a newer autoconf version to generate the GRASS configure script fix the problem?

Maybe. Like many autoconf macros, AC_PROG_CPP runs through a fixed set of candidates until one appears to work. It's possible that a newer version has more candidates. Or it's possible that it's smarter at ignoring warnings from the preprocessor.

[msieczka]

Replying to glynn:

How about this:

CFLAGS="$CPPFLAGS $CFLAGS" CXXFLAGS="$CPPFLAGS $CXXFLAGS" CPPFLAGS=`echo $CPPFLAGS | sed 's/-D_FORTIFY_SOURCE=.//g'` ./configure

This way I'm passing all CPPFLAGS as declared in /etc/makepkg.conf to CXXFLAGS and CFLAGS for make, I don't give up on CPPFLAGS entirely in case it contained anything that configure could need, but I drop D_FORTIFY_SOURCE from CPPFLAGS for configure, to avoid the "_FORTIFY_SOURCE requires compiling with optimization" issue. Seems least intrusive and foolproof?

[glynn]

Replying to msieczka:

How about this:

CFLAGS="$CPPFLAGS $CFLAGS" CXXFLAGS="$CPPFLAGS $CXXFLAGS" CPPFLAGS=`echo $CPPFLAGS | sed 's/-D_FORTIFY_SOURCE=.//g'` ./configure

It would be better if any _FORTIFY_SOURCE definition could be extracted and moved into CFLAGS/CXXFLAGS, e.g.

FORTIFY_FLAGS=`echo "$CPPFLAGS" | sed '/^.*\(-D_FORTIFY_SOURCE=.\).*$/s//\1/'`
CPPFLAGS=`echo "$CPPFLAGS" | sed 's/-D_FORTIFY_SOURCE=.//g'`
CFLAGS="$FORTIFY_FLAGS $CFLAGS"
CXXFLAGS="$FORTIFY_FLAGS $CXXFLAGS"

This avoids any potential problems with duplicating the other elements of $CPPFLAGS.

[msieczka]

Replying to glynn:

It would be better if any _FORTIFY_SOURCE definition could be extracted and moved into CFLAGS/CXXFLAGS, e.g. (...)

Excellent. Thanks a ton.

In summary: there's nothing that GRASS should do about this issue, it's down to the fact that D_FORTIFY_SOURCE should not go into CPPFLAGS, only to CFLAGS and CXXFLAGS. Please let me know whether that's correct.

[scimmia]

No, being a "D" flag, CPPFLAGS is correct, and it works correctly in reasonably current versions of autoconf. Autoconf 2.13 is from Jan 1999.

[glynn]

Replying to msieczka:

In summary: there's nothing that GRASS should do about this issue, it's down to the fact that -D_FORTIFY_SOURCE should not go into CPPFLAGS, only to CFLAGS and CXXFLAGS Please let me know whether that's correct.

It is.

[glynn]

Replying to scimmia:

No, being a "D" flag, CPPFLAGS is correct,

That would be true ... if it wasn't for the fact that <features.h> has an interaction between the setting of _FORTIFY_SOURCE and the setting of OPTIMIZE. The latter is set by the compiler based upon -O switches, which definitely shouldn't go into CPPFLAGS.

_FORTIFY_SOURCE shouldn't be set to a value other than zero unless optimisation is enabled, and CPPFLAGS shouldn't contain any optimisation switches. Therefore, CPPFLAGS shouldn't contain a setting for _FORTIFY_SOURCE.

The fact that -D_FORTIFY_SOURCE=1 is interpreted by the preprocessor doesn't mean that it's acceptable to set that flag when running the preprocessor separately from compilation.

[scimmia]

We'll have to agree to disagree on that, but it's really immaterial. The issue is that the configure script tries to figure out how to call the preprocessor by building a test file, then decides that any output at all is an error. It then defaults to calling /lib/cpp, which is a horrible assumption. All in all, braindead behavior that's not present in modern implementations.

Edit: I'm thinking the cleanest solution for now might be to specifically set CPP to "gcc -E -w" for configure. Not totally confident that won't break some other detection, though.

[neteler]

How hard would it be to update our configure scripts to autoconf-2.69 ?

[martinl]

Still relevant?

[neteler]

Yes, seems we should upgrade to autoconf-2.69: ​https://github.com/OSGeo/grass/issues/560