[john-dev] Exclude items from test vectors for --test

[Denis Burykin]

magnum,
I finally got ztex formats to work with --test but displayed measurement results look incorrect. Is there a way to exclude some items from test vectors during --test? Such as in sha512crypt, test vector contains hashes that requires to compute 2 SHA512 blocks each round, hashes with computing difficulty substantially different from standard one, and so on. 
I can arrange test vectors so on average they have standard computing difficulty but maybe there's a better solution? 
Thanks,Denis Burykin

[Frank Dittrich]

Is there a way to exclude some items from test vectors during --test?
Such as in
sha512crypt, test vector contains hashes that requires to compute 2
SHA512 blocks
each round, hashes with computing difficulty substantially different
from standard one,
and so on.

If the number of SHA512 blocks gets added as a 2nd tunable cost (in
addition to iteration count), then it should be possible to filter the
hashes using the --cost switch.

Frank

[Frank Dittrich]

Post by Frank Dittrich
If the number of SHA512 blocks gets added as a 2nd tunable cost (in
addition to iteration count), then it should be possible to filter the
hashes using the --cost switch.

I should have considered that the number of blocks also depends on
password length, not just salt length. So this approach probably doesn't
work.

Frank

[Solar Designer]

Post by Frank Dittrich

Post by Frank Dittrich
If the number of SHA512 blocks gets added as a 2nd tunable cost (in
addition to iteration count), then it should be possible to filter the
hashes using the --cost switch.

I should have considered that the number of blocks also depends on
password length, not just salt length. So this approach probably doesn't
work.

Yes, and this wasn't Denis' question anyway, or at least not the
question I had brought to him in private e-mail before. What we need is
for "--test" to produce meaningful results by default, without any
"--cost" option. Right now, for sha512crypt-ztex it says it's
benchmarking for iteration count 5000, but the actual benchmark is for a
mix of different iteration counts (and maybe a mix of different password
lengths too). This is a bug that we need to fix.

Alexander

[Frank Dittrich]

Post by Solar Designer
What we need is
for "--test" to produce meaningful results by default, without any
"--cost" option. Right now, for sha512crypt-ztex it says it's
benchmarking for iteration count 5000, but the actual benchmark is for a
mix of different iteration counts (and maybe a mix of different password
lengths too). This is a bug that we need to fix.

Currently, all tests of the regular sha512crypt format use 5000
iteraions. (May be we should change that.)


So, I copied the ztex self tests to reproduce this.

When using --test, john first tests all hashes, then it measures the
performance with just the top 2 hashes, i.e. 5000 iterations, unless you
pick other hashes using something like --costs=1021:1021.


Are you suggesting we should support ./john --test --skip-self-test to
just measure the performance without first processing the regular self test?


Frank

[Solar Designer]

This post might be inappropriate. Click to display it.

[Frank Dittrich]

Post by Solar Designer

Post by Frank Dittrich
When using --test, john first tests all hashes, then it measures the
performance with just the top 2 hashes, i.e. 5000 iterations, unless you
pick other hashes using something like --costs=1021:1021.

The current behavior appears to be different from what you simulated.
With the actual sha512crypt-ztex format, the speed reported by --test is
now approx. 4 times higher than what's seen when cracking hashes with
5000 iterations. This suggests that the many lower iteration count test
vectors are used for the benchmark.

I described the behaviour for sha512crypt format after switching to the
ztex self tests. In this case, --test (with or without --costs) works as
expected.

Is the ztex format doinf something "unexpected" to the self tests like
sorting them in a particular order after loading them?
Nevertheless, the reported cost should match reality, and --cost= should
allow picking alternative hashes for the perfomance test.


Frank