Discussion:
aix 5 compilation
(too old to reply)
a***@lx42.de
2017-11-28 22:02:22 UTC
Permalink
Hello,

by compiling jtr on an old AIX 5.3 machine (sigh, for compat reasons, ...):

1) there is a conflict for int64, renamed it to jtrint64.
2) there are $ as variable names, which the compiler does not
understand. I renamed them.
3) arg list in the binary linking step was too long -> created a file
"jtr_link" with the command and bashed it
4) no clue

$ git log | head
commit 1b73c7339f385946e287696510909bd6f7950ce8
Author: jfoug <***@openwall.net>
Date: Tue Nov 28 11:49:41 2017 -0600
$ git diff > jtr.diff

... appended, is there a better way to do this? (git send-email ?)

Regards,
Alexander



==== 1
In file included from /usr/include/sys/types.h:57,
from
/opt/freeware/lib/gcc/powerpc-ibm-aix5.3.0.0/4.4.5/include-fixed/stdio.h:431,
from memdbg.h:18,
from math.c:13:
/usr/include/sys/inttypes.h:625: error: conflicting types for
'_john_int64_composite'
math.h:23: note: previous declaration of '_john_int64_composite' was here
math.c:15: error: conflicting types for 'add32to64'
math.h:25: note: previous declaration of 'add32to64' was here
math.c: In function 'add32to64':
math.c:19: error: request for member 'lo' in something not a structure
or union
math.c:20: error: request for member 'lo' in something not a structure
or union
math.c:26: error: request for member 'lo' in something not a structure
or union
math.c:26: error: request for member 'hi' in something not a structure
or union

==== 2

e/cc/laboraix53/curl/lib -D_THREAD_SAFE -I/usr/local/include
-funroll-loops uaf_encode.c -o uaf_encode.o
uaf_encode.c:69: error: stray '$' in program
uaf_encode.c:69: error: expected ':', ',', ';', '}' or '__attribute__'
before 'b_char'
uaf_encode.c: In function 'uaf_test_password':
uaf_encode.c:530: error: stray '$' in program
uaf_encode.c:530: error: expected '=', ',', ';', 'asm' or
'__attribute__' before 'username_dx'
uaf_encode.c:530: error: 'username_dx' undeclared (first use in this function)
uaf_encode.c:530: error: (Each undeclared identifier is reported only once
uaf_encode.c:530: error: for each function it appears in.)
uaf_encode.c:530: error: expected expression before '{' token

==== 3

ique.o gpg2john.o memdbg.o -g
-L/siux_share/ae/cc/laboraix53/curl/lib
-L/siux_share/ae/cc/laboraix53/curl/lib -L/usr/local/lib
-L/usr/local/ssl/lib -lssl -lcrypto -D_THREAD_SAFE -lpthreads
-lm -lz -ldl aes/aes.a secp256k1/secp256k1.a -o ../run/john
make[1]: execvp: gcc: Arg list too long
make[1]: *** [../run/john] Error 127
make[1]: Leaving directory `/siux_share/ae/cc/laboraix53/JohnTheRipper/src'
make: *** [default] Error 2

===== 4

e/cc/laboraix53/curl/lib -D_THREAD_SAFE -I/usr/local/include
-funroll-loops SIPdump.c -o SIPdump.o
In file included from SIPdump.c:51:
tcphdr.h:20: error: expected ':', ',', ';', '}' or '__attribute__'
before '.' token
SIPdump.c: In function 'main':
SIPdump.c:144: error: storage size of 'fp' isn't known
SIPdump.c:147: error: invalid application of 'sizeof' to incomplete
type 'struct bpf_program'
SIPdump.c:144: warning: unused variable 'fp'
SIPdump.c: In function 'sniff_logins':
SIPdump.c:636: error: 'const struct tcp_hdr' has no member named 'ip_ff'

======
a***@lx42.de
2017-11-28 22:36:32 UTC
Permalink
Post by a***@lx42.de
... appended, is there a better way to do this? (git send-email ?)
Maybe this way? (List all new and modified files and pack them to a
tar archive?)
{ git diff --cached --name-only --diff-filter=A; git diff --name-only
--diff-filter=M; } | tar czf altr.tar.gz -T-
Solar Designer
2017-11-29 12:38:13 UTC
Permalink
Post by a***@lx42.de
1) there is a conflict for int64, renamed it to jtrint64.
JtR core tree already includes this workaround in math.h:

#undef int64
#define int64 _john_int64_t

Somehow jumbo has this instead:

#undef int64
#define int64 _john_int64_composite

So it turns out these are insufficient, because AIX header files that we
sometimes (indirectly) include (at least in jumbo) after having already
included our math.h try to refer to AIX's own int64, but instead hit our
workaround macro.

I'll need to fix this in the core tree to avoid unnecessary differences
between it and jumbo.
Post by a***@lx42.de
2) there are $ as variable names, which the compiler does not
understand. I renamed them.
OK.
Post by a***@lx42.de
3) arg list in the binary linking step was too long -> created a file
"jtr_link" with the command and bashed it
Previously discussed in:

https://github.com/magnumripper/JohnTheRipper/issues/2150

Maybe we need to enhance jumbo to exclude opencl_*.o from linking when
we're making non-OpenCL builds. magnum?

Also, we may want to document use of AIX's chdev command:

http://www.in-ulm.de/~mascheck/various/argmax/

"The limit on AIX 5.1 can be changed at run time with "chdev -l sys0 -a
ncargs=value", in the range from 6_4KB to 1024_4KB."

And I don't see how people are bumping into the 24 KB default already.
This default is not that low. Can you find out? Do you already have so
many env vars set or whatever? Try something like "set | wc" in bash.
Post by a***@lx42.de
4) no clue
tcphdr.h:20: error: expected ':', ',', ';', '}' or '__attribute__'
before '.' token
Previously discussed in:

https://github.com/magnumripper/JohnTheRipper/issues/2153

Apparently, libpcap upgrade helps.

Our tcphdr.h:20 is:

uint8_t th_off:4;

I am puzzled as to why this fails to compile, why line 21 apparently
compiles (it's very similar, so maybe the issue is th_off specifically
is already defined?), and and why upgrading libpcap would make a
difference with respect to this error (does it possibly define TCPHDR_H
on its own, which results in our tcphdr.h's content skipped?)
Post by a***@lx42.de
$ git log | head
commit 1b73c7339f385946e287696510909bd6f7950ce8
Date: Tue Nov 28 11:49:41 2017 -0600
$ git diff > jtr.diff
... appended, is there a better way to do this? (git send-email ?)
As you already found out, we use GitHub pull requests. Thank you for
making one:

https://github.com/magnumripper/JohnTheRipper/pull/2935

Please exclude the int64 rename from there: that issue should be dealt
with separately.

The unshadow.c changes, if needed, I should probably make in the core
tree as well, and mine won't be exactly the same as yours. But I don't
mind having this in jumbo as you wrote first.

Alexander
a***@lx42.de
2017-11-29 20:35:55 UTC
Permalink
Post by Solar Designer
Post by a***@lx42.de
1) there is a conflict for int64, renamed it to jtrint64.
So it turns out these are insufficient, because AIX header files that we
sometimes (indirectly) include (at least in jumbo) after having already
included our math.h try to refer to AIX's own int64, but instead hit our
workaround macro.
I'll need to fix this in the core tree to avoid unnecessary differences
between it and jumbo.
I see, thank you.
Post by Solar Designer
Post by a***@lx42.de
3) arg list in the binary linking step was too long -> created a file
"jtr_link" with the command and bashed it
https://github.com/magnumripper/JohnTheRipper/issues/2150
Maybe we need to enhance jumbo to exclude opencl_*.o from linking when
we're making non-OpenCL builds. magnum?
http://www.in-ulm.de/~mascheck/various/argmax/
"The limit on AIX 5.1 can be changed at run time with "chdev -l sys0 -a
ncargs=value", in the range from 6_4KB to 1024_4KB."
And I don't see how people are bumping into the 24 KB default already.
This default is not that low. Can you find out? Do you already have so
many env vars set or whatever? Try something like "set | wc" in bash.
$ getconf ARG_MAX
24576
$ set | wc
2763 7039 69898

surprisingly sourcing the shell script worked, i wrote the script via
"make -n | head 10" and vim, of course. ;)
Post by Solar Designer
Post by a***@lx42.de
4) no clue
tcphdr.h:20: error: expected ':', ',', ';', '}' or '__attribute__'
before '.' token
https://github.com/magnumripper/JohnTheRipper/issues/2153
Apparently, libpcap upgrade helps.
I have to try that.
Post by Solar Designer
uint8_t th_off:4;
I am puzzled as to why this fails to compile, why line 21 apparently
compiles (it's very similar, so maybe the issue is th_off specifically
is already defined?), and and why upgrading libpcap would make a
difference with respect to this error (does it possibly define TCPHDR_H
on its own, which results in our tcphdr.h's content skipped?)
I could neither find that file nor define.
Post by Solar Designer
Post by a***@lx42.de
$ git log | head
commit 1b73c7339f385946e287696510909bd6f7950ce8
Date: Tue Nov 28 11:49:41 2017 -0600
$ git diff > jtr.diff
... appended, is there a better way to do this? (git send-email ?)
As you already found out, we use GitHub pull requests. Thank you for
https://github.com/magnumripper/JohnTheRipper/pull/2935
I am very new to that. It would be nice to hear how it works.
I am compiling john for AIX 5.3/6.1/7.1, HP-UX 11.11/11.31, linux
32bit 64bit, Solaris 5.10/5.11 (64 Bit) , Solaris Sparc 5.8 (32 Bit) /
5.9 (64 Bit) / 5.11 (64 Bit). There are a bunch of issues in the code,
how do you suggest to keep the issues independant? (accept one, reject
others ...)
If one issue is badly solved, it should be independant from the other
ones (to reject it) - so doing a single pull request seems a little
bit clumsy to do that.
But I am new to github so any help or comments are welcome. ;)
Post by Solar Designer
Please exclude the int64 rename from there: that issue should be dealt
with separately.
Yes, that's my question. I did three commits for three different
things and did one pull request - would it have been better to do
three pull requests?
Post by Solar Designer
The unshadow.c changes, if needed, I should probably make in the core
tree as well, and mine won't be exactly the same as yours. But I don't
mind having this in jumbo as you wrote first.
Mine was a solution that solved a problem (for me). My idea was to
share it with you to have the problem saved for future releases.

Feel free to use or change it.

Regards,
Alexander
a***@lx42.de
2017-11-30 00:33:36 UTC
Permalink
Post by Solar Designer
Post by a***@lx42.de
3) arg list in the binary linking step was too long -> created a file
"jtr_link" with the command and bashed it
...
Post by Solar Designer
And I don't see how people are bumping into the 24 KB default already.
This default is not that low. Can you find out? Do you already have so
many env vars set or whatever? Try something like "set | wc" in bash.
Why don't you use ar archives?

ar rc libz.a adler32.o crc32.o deflate.o infback.o inffast.o inflate.o
inftrees.o trees.o zutil.o compress.o uncompr.o gzclose.o gzlib.o
gzread.o gzwrite.o

Regards,
Alexander
Solar Designer
2017-12-01 19:41:01 UTC
Permalink
Post by a***@lx42.de
Why don't you use ar archives?
We do for "aes/aes.a secp256k1/secp256k1.a" as seen right in your
problem report. We could expand on that.

Alexander
jfoug
2017-11-29 13:00:02 UTC
Permalink
Post by Solar Designer
Post by a***@lx42.de
1) there is a conflict for int64, renamed it to jtrint64.
#undef int64
#define int64 _john_int64_t
#undef int64
#define int64 _john_int64_composite
So it turns out these are insufficient, because AIX header files that we
sometimes (indirectly) include (at least in jumbo) after having already
included our math.h try to refer to AIX's own int64, but instead hit our
workaround macro.
I'll need to fix this in the core tree to avoid unnecessary differences
between it and jumbo.
WHY are we not simply using int64_t and be done with it.  So instead of
doing it the right way (replacing all the int64 crap with proper
values), are we simply going to keep chasing our tails???  Building our
own non standard types is so '90s  (lol) and gets us in trouble time and
time again......    Just saying.
Solar Designer
2017-12-02 21:57:56 UTC
Permalink
Post by jfoug
WHY are we not simply using int64_t and be done with it.  So instead of
doing it the right way (replacing all the int64 crap with proper
values), are we simply going to keep chasing our tails???  Building our
own non standard types is so '90s  (lol) and gets us in trouble time and
time again......    Just saying.
I've just switched the core tree to use uint64_t.

Alexander

jfoug
2017-11-30 01:19:47 UTC
Permalink
Post by Solar Designer
Post by a***@lx42.de
3) arg list in the binary linking step was too long -> created a file
"jtr_link" with the command and bashed it
https://github.com/magnumripper/JohnTheRipper/issues/2150
Maybe we need to enhance jumbo to exclude opencl_*.o from linking when
we're making non-OpenCL builds. magnum?
I have checked in a version which now only includes the opencl_*.o files
if building an OpenCL build.  A non OpenCL build will have a reduced
length command line (and faster build time to a small extent).
Loading...