Intgemm is not compiling with icc (>16).

[akarbown]

While compiling intgemm with one of the latest versions of the ICC (icpc (ICC) 19.1.3.304) I got the following result:

benchmarks/../intgemm/callbacks/implementations.inl(47): error #3632: "target" attribute on special function is not supported
    CPU_ATTR void operator()(vi input, const OutputBufferInfo& info) {
                  ^

as far as I know the attribute ((target ("sse2/sse3/avx2"))) that is used here is a "gnu extension" to the c/c++ programming language and it is not working with ICC compiler.

[kpu]

Does 65276ad fix this?

[kpu]

@sidkashyap-at-Intel Can you setup CI for icc?

[akarbown]

Does 65276ad fix this?

I cherry-picked the intgemm, compiled it and got the following output: compile.log.

[kpu]

Sigh there's a bug in the Intel compiler in which constructors with target attributes compile but fail to link. Minimal example:

/* Compiles then fails to link */
class Foo {
  public:
  __attribute__ ((target ("avx2"))) Foo() {
  }
};

int main() {
  Foo a;
}

Command:

kheafield@var:~$ . /opt/intel/compilers_and_libraries_2020.4.304/linux/bin/compilervars.sh intel64
kheafield@var:~$ icpc bug.cc 
/tmp/icpcqF2ZaF.o: In function `main':
bug.cc:(.text+0x30): undefined reference to `Foo::Foo()'

Whereas a regular method compiles and links fine:

/* Works */
class Foo {
  public:
    Foo() {}
    __attribute__ ((target ("avx2"))) void Bar() {
    }
};

int main() {
  Foo a;
  a.Bar();
}

@akarbown it looks like you work for Intel? Can you help with icc bugs? We collaborate with @sidkashyap-at-Intel.

[kpu]

@XapaJIaMnu says he tried to report this to Intel two years ago via @sidkashyap-at-Intel but couldn't figure out how to raise it to the team.

[snkashya]

This issue was indeed discussed with the team; @akarbown, will ping you offline to take this forward.

@kpu: Yes, will setup a CI for icc.

[akarbown]

Sigh there's a bug in the Intel compiler in which constructors with target attributes compile but fail to link. Minimal example:

/* Compiles then fails to link */
class Foo {
  public:
  __attribute__ ((target ("avx2"))) Foo() {
  }
};

int main() {
  Foo a;
}

Command:

kheafield@var:~$ . /opt/intel/compilers_and_libraries_2020.4.304/linux/bin/compilervars.sh intel64
kheafield@var:~$ icpc bug.cc 
/tmp/icpcqF2ZaF.o: In function `main':
bug.cc:(.text+0x30): undefined reference to `Foo::Foo()'

Whereas a regular method compiles and links fine:

/* Works */
class Foo {
  public:
    Foo() {}
    __attribute__ ((target ("avx2"))) void Bar() {
    }
};

int main() {
  Foo a;
  a.Bar();
}

@akarbown it looks like you work for Intel? Can you help with icc bugs? We collaborate with @sidkashyap-at-Intel.

It compiles when it looks as follows:

/* Compiles then fails to link */
class Foo {
  public:
  __attribute__ ((target ("avx2"))) Foo() {}
  __attribute__ ((target ("default"))) Foo() {}
//  Foo() {
//  }
};

int main() {
  Foo a;
}

I get to know that it is because the compiler creates a resolver function something like this:

Func foo-resolver
  If (target == avx2) return & foo2
  Else return &foo1

It seems to be pretty interesting that for a regular function it works without the attribute default.

[kpu]

Target attributes pre-date function multiversioning. Function multiversioning is a feature that should only be activated if there are two function declarations with the same signature that differ only in target attributes. The bug in the Intel compiler is that it activates function multiversioning for constructors when there is only one version.

We wanted to use function multi-versioning in intgemm but it's actually quite bad for performance due to adding the CPUID dispatch to all the functions, even in contexts where the target architecture is already known: https://hannes.hauswedell.net/post/2017/12/09/fmv/ . So we do target attributes inside the code then a call-by-function-pointer on the external APIs.

[kpu]

I tried to find documentation on icc. To quote page 2212 of https://software.intel.com/content/dam/develop/external/us/en/documents/cpp_compiler_classic.pdf :

On Linux*, in addition to the Intel-defined attributes cpu_specific and cpu_dispatch , C++ compilations with GNU Compiler Collection (GCC*) compatibility 4.8 or higher support creation of multiple function versions using the target attribute. For more information see the GCC documentation on "Function Multiversioning".

So it would seem the defined behavior for icc is to match whatever gcc does. And the gcc documentation https://gcc.gnu.org/wiki/FunctionMultiVersioning states

Determine if two function decls with the same signature are versions.

It further defines this as

Two function decls with the same signature are versions if and only if both are tagged with the function attribute "target" and the target attribute strings differ.

I don't see anything about a constructor being treated differently. And gcc empirically handles a single constructor with a target attribute correctly.

Moreover, icc is moving to a clang-based version of target attributes, which is what we are doing: https://software.intel.com/content/www/us/en/develop/articles/porting-guide-for-icc-users-to-dpcpp-or-icx.html

I tried #pragma intel optimization_parameter target_arch and that might work but it has to appear before template whereas __attribute__ has to appear after template. So it's not a drop-in substitute.

But anyway try 2647d6c ; I just removed the target attributes entirely from icc. Possibly less efficient via icc but deservedly so.

[akarbown]

Thanks for the temporary solution! I'll try to poke a compiler team in that case.

[snkashya]

@akarbown: Have forwarded the meeting minutes with the MKL where we discussed these issues, let us sync offline.

[akarbown]

I've created jira issue internally. If you have any more questions connected with that issue I think that @mibintc could give you more information or direct you to the proper person.

[mibintc]

When the icc compiler sees a target attribute, it thinks you are using gnu function multiversioning. Since your program didn't provide a "default" function, the multiversion definition wasn't complete. The icc compiler really doesn't support the other meaning of target attribute. As I understand it from reading the gcc documentation, __attribute((target("avx")) which is not gnu function multiversioning means that the compiler is entitled to generate avx2 instructions for the function body. If the code was executed on a cpu that didn't support avx2 then you would get an instruction fault. I believe that you could get the effect of avx2 instructions by separating out Foo()'s definition into a separate compilation unit and choose target architecture avx2 using the command line option. Since icc doesn't support the other meaning of target attribute, we could ignore it, I don't know if that would be sufficient to meet your needs. I haven't studied your code base, I just looked at the small example provided.

[kpu]

When the icc compiler sees a target attribute, it thinks you are using gnu function multiversioning. Since your program didn't provide a "default" function, the multiversion definition wasn't complete.

If that is your policy then this should be an error. But it compiles fine.

/* Works */
class Foo {
  public:
    Foo() {}
    __attribute__ ((target ("avx2"))) void Bar() {
    }
};

int main() {
  Foo a;
  a.Bar();
}

The icc compiler really doesn't support the other meaning of target attribute.

Ok. Did I miss some documentation?

As I understand it from reading the gcc documentation, __attribute((target("avx")) which is not gnu function multiversioning means that the compiler is entitled to generate avx2 instructions for the function body. If the code was executed on a cpu that didn't support avx2 then you would get an instruction fault.

Correct. The goal here is to avoid CPUID dispatch for every little internal function call and only do it once at the top-level interface. If every function call had multiversioning, it would be very slow without a compiler that realizes it's already in a achitecture-dependent function and can directly call / inline the next architecture-dependent function.

I believe that you could get the effect of avx2 instructions by separating out Foo()'s definition into a separate compilation unit and choose target architecture avx2 using the command line option.

We have user-defined C++ template arguments. Separate compilation units for each architecture don't really work for that, because it would require our users to separately compile all their code separately for each architecture. Suppose you had some architecture-dependent optimization for std::vector.

Since icc doesn't support the other meaning of target attribute, we could ignore it, I don't know if that would be sufficient to meet your needs. I haven't studied your code base, I just looked at the small example provided.

It appears that icc allows any intrinsic inside any function, regardless of architecture consistency. Therefore this would work for us and be compatible with gcc's behavior, albeit with the possibility of missed optimizations.

[mibintc]

When the icc compiler sees a target attribute, it thinks you are using gnu function multiversioning. Since your program didn't provide a "default" function, the multiversion definition wasn't complete.

If that is your policy then this should be an error. But it compiles fine.

/* Works */
class Foo {
  public:
    Foo() {}
    __attribute__ ((target ("avx2"))) void Bar() {
    }
};

int main() {
  Foo a;
  a.Bar();
}

The icc compiler really doesn't support the other meaning of target attribute.

Ok. Did I miss some documentation?

As I understand it from reading the gcc documentation, __attribute((target("avx")) which is not gnu function multiversioning means that the compiler is entitled to generate avx2 instructions for the function body. If the code was executed on a cpu that didn't support avx2 then you would get an instruction fault.

Correct. The goal here is to avoid CPUID dispatch for every little internal function call and only do it once at the top-level interface. If every function call had multiversioning, it would be very slow without a compiler that realizes it's already in a achitecture-dependent function and can directly call / inline the next architecture-dependent function.
On Linux, with gnu function multiversioning, the resolver function runs when the program is loaded and the call site is patched with the resolved address: the overhead is incurred at load time.

I believe that you could get the effect of avx2 instructions by separating out Foo()'s definition into a separate compilation unit and choose target architecture avx2 using the command line option.

We have user-defined C++ template arguments. Separate compilation units for each architecture don't really work for that, because it would require our users to separately compile all their code separately for each architecture. Suppose you had some architecture-dependent optimization for std::vector.

Since icc doesn't support the other meaning of target attribute, we could ignore it, I don't know if that would be sufficient to meet your needs. I haven't studied your code base, I just looked at the small example provided.

It appears that icc allows any intrinsic inside any function, regardless of architecture consistency. Therefore this would work for us and be compatible with gcc's behavior, albeit with the possibility of missed optimizations.
Yes it's true, icc allows any intrinsic regardless of architecture consistency. Thanks for the followup.