Discussion:
Re: "Illegal instruction" error on 1.8.0.6-jumbo-1-bleeding_omp
(too old to reply)
magnum
2015-10-23 22:09:28 UTC
Permalink
The core JtR tree checks for AVX at runtime and fails gracefully if
it's built for AVX but the system (the CPU and/or the kernel) lacks
AVX support. Ditto for XOP (and even for SSE2 in 32-bit builds).
Or have we broken that in jumbo, maybe along with autoconf'ing it?
This is worth checking.
I tried these on well:
a) make -sj8 -f Makefile.legacy linux-x86-64-xop
b) ./configure CFLAGS="-g -O2 -mxop" --disable-native-tests && make -sj8

Regardless whether autoconf'ed or using legacy Makefile, Jumbo produces
the "Sorry, XOP is required for this build". I also tried reverting to
the very same version as Kali 4.0.0 seem to have used and this still
works. But we've got reports from several places that it doesn't work
for AVX (with Jumbo, at least).

Could it be that the XOP check somehow works while the AVX one doesn't?
I have no way to test that. Do we have a machine somewhere that lacks AVX?

magnum
magnum
2015-10-23 22:34:37 UTC
Permalink
Post by magnum
The core JtR tree checks for AVX at runtime and fails gracefully if
it's built for AVX but the system (the CPU and/or the kernel) lacks
AVX support. Ditto for XOP (and even for SSE2 in 32-bit builds).
Or have we broken that in jumbo, maybe along with autoconf'ing it?
This is worth checking.
a) make -sj8 -f Makefile.legacy linux-x86-64-xop
b) ./configure CFLAGS="-g -O2 -mxop" --disable-native-tests && make -sj8
Regardless whether autoconf'ed or using legacy Makefile, Jumbo produces
the "Sorry, XOP is required for this build". I also tried reverting to
the very same version as Kali 4.0.0 seem to have used and this still
works. But we've got reports from several places that it doesn't work
for AVX (with Jumbo, at least).
Could it be that the XOP check somehow works while the AVX one doesn't?
I have no way to test that. Do we have a machine somewhere that lacks AVX?
Found the problem. When I supplied CFLAGS as above, the reciepe for
john.o got them too but with a pure autoconf with nothing given, we did
not supply any -mavx for that very target. The intentions were good but
not the result: We did not get any CPU_DETECT from arch.h...

I will fix.

magnum
magnum
2015-10-23 23:07:13 UTC
Permalink
Post by magnum
Post by magnum
The core JtR tree checks for AVX at runtime and fails gracefully if
it's built for AVX but the system (the CPU and/or the kernel) lacks
AVX support. Ditto for XOP (and even for SSE2 in 32-bit builds).
Or have we broken that in jumbo, maybe along with autoconf'ing it?
This is worth checking.
a) make -sj8 -f Makefile.legacy linux-x86-64-xop
b) ./configure CFLAGS="-g -O2 -mxop" --disable-native-tests && make -sj8
Regardless whether autoconf'ed or using legacy Makefile, Jumbo produces
the "Sorry, XOP is required for this build". I also tried reverting to
the very same version as Kali 4.0.0 seem to have used and this still
works. But we've got reports from several places that it doesn't work
for AVX (with Jumbo, at least).
Found the problem. When I supplied CFLAGS as above, the reciepe for
john.o got them too but with a pure autoconf with nothing given, we did
not supply any -mavx for that very target. The intentions were good but
not the result: We did not get any CPU_DETECT from arch.h...
Now we're passing -DJOHN_AVX or -DJOHN_XOP as needed. Actually we'll
also pass -DJOHN_AVX2 or -DJOHN_SSSE3 or whatever is applicable so as
soon as core gets eg. an AVX2 check, it should work right away even with
autoconf.

I'll see if I can figure out how to check for AVX2 and beat Solar to
it... it oughta be better than Sudoku.


OT: I wonder why my Thunderbird MUA insists on truncating this subject line.

magnum
magnum
2015-10-24 00:33:34 UTC
Permalink
Post by magnum
I'll see if I can figure out how to check for AVX2 and beat Solar to
it... it oughta be better than Sudoku.
I think/hope I did it right. Seems to work. Here's the asm diff.

magnum
magnum
2015-10-24 00:43:25 UTC
Permalink
Post by magnum
Post by magnum
I'll see if I can figure out how to check for AVX2 and beat Solar to
it... it oughta be better than Sudoku.
I think/hope I did it right. Seems to work. Here's the asm diff.
Had to clear ecx also

#ifdef CPU_REQ_AVX2
xorl %eax,%eax
cpuid
movl $7,%edx
cmpl %edx,%eax
jl CPU_detect_fail
xchgl %edx,%eax
+ xorl %ecx,%ecx
cpuid
andl CF_AVX2,%ebx
cmpl CF_AVX2,%ebx
jne CPU_detect_fail
#else


magnum
Lukas Odzioba
2015-10-26 22:37:16 UTC
Permalink
Post by magnum
Do we have a machine somewhere that lacks AVX?
I tried bleeding jumbo on Q6600 which does not have AVX and I don't
see "AVX no" message on it:
Q6600
checking for MMX... yes
checking for SSE2... yes
checking for SSSE3... yes
checking for SSE4.1... no

3770k
checking for MMX... yes
checking for SSE2... yes
checking for SSSE3... yes
checking for SSE4.1... yes
checking for AVX... yes
checking for XOP... no
checking for AVX2... no

On Q6600 I have 32bit OS whereas on 3770k 64bit.
Let me know if it can help somehow.

Thanks,
Lukas
jfoug
2015-10-27 15:58:12 UTC
Permalink
Post by Lukas Odzioba
Post by magnum
Do we have a machine somewhere that lacks AVX?
I tried bleeding jumbo on Q6600 which does not have AVX and I don't
Q6600
checking for MMX... yes
checking for SSE2... yes
checking for SSSE3... yes
checking for SSE4.1... no
Configure does not attempt AVX probing, since SSE4.1 was not present.
SSSE3 will be the 'highest' CPU used.

Loading...