HashCat rule handling (in jumbo)
(too old to reply)
2016-04-10 16:26:45 UTC
I have added HashCat rule (and logic changes) handling within
john-bleeding. This change has not been committed to the bleeding-jumbo
branch yet, but there is a full patch file on the github issue where it
is being discussed:

NOTE, there are a couple of issues when doing this.

1. you need to still make a couple of changes to the HC rules files (I
have a pull request to the HC git repo to fix this, but at this time it
is not fixed). These changes are that any rule line starting with a [ or
a . character has to be changed. Simply append a : character to each
such lines. The [ and . are used by john's config.c processing logic
and rules that start with them break that logic.

2. HC rules are owned by the HC project, so it is best to NOT insert
them directly into a john.conf file. That being the case, a good way to
use them (for jumbo), is to use the .include syntax. Also, it is by far
best to create a john-local.conf file to place these in, since the
location of these rules files will be specific to each users system.
Here is how I have setup mine:

!! HashCat logic ON
.include '/c/phpbb/HashCat-fork/rules/d3ad0ne.rule'

!! HashCat logic ON
.include '/c/phpbb/HashCat-fork/rules/dive.rule'

!! HashCat logic ON
.include '/c/phpbb/HashCat-fork/rules/generated.rule'

!! HashCat logic ON
.include '/c/phpbb/HashCat-fork/rules/generated2.rule'

!! HashCat logic ON
.include '/c/phpbb/HashCat-fork/rules/T0XlCv1.rule'

!! HashCat logic ON
.include '/c/phpbb/HashCat-fork/rules/best64.rule'

# ......

3. there are new directives for the rules engine. This is "!! HashCat
logic ON" and then "!! HashCat logic OFF" to switch back out. Much of
the HashCat rules will work fine without the HashCat Logic On being set,
but there are some differences, and some rules would not work (where we
share a rule letter with different logic between john and HC). Of
note: + single vs +N, p vs pN, - reject vs -N, R vs RN, L vs LN.
There are also some edge condition logic for i and x commands that are
different. Some of these differences in the 'shared' logic work without
the hashcat_logic, some partly work without it, and some require the
flag. Again, the easiest way to get it working fully, is to simply use
the HC-logic-ON switch, and include the file.

4. many HC rules, have a trailing space character. Such as $ (append
space). John's config.c logic was stripping all trailing whitespace. I
have added logic to keep 1 space or tab if there was trailing WS.

5. there was a possible bug in wordlist node splitting logic. I think
it should not be called UNLESS there are nodes. If that change is not
made, then the logic I have added to 'skip' the rpp code when running HC
files, will end up generating no words.

The patch has been completed, and moderately tested. Other than an
issue found with HC, and the differences due to length of word buffer
and the fact that jtr uniq the words while HC gives all the dupes, the
patch creates the same word stream as HC does (when using --stdout for
both tools).

I would like some buy in from Solar (and Magnum), on these changes,
since they do significantly impact the rules engine/rpp, and make
changes to config.c and also to wordlist.c Yes, this is jumbo only
code, but these changes are in files which walk in both core and jumbo john.
Community volunteer for John the Ripper project.