Back in 2014, I was very interested in descrypt as a password-hashing algorithm (for reasons that were secret at the time, but are now public). I started looking for ways to increase my hashrate.
After I read about Positive Technologies cracking DES keys for SIM cards using old ZTEX 1.15y boards, I took a gamble that descrypt would not be far behind. While the original German boards and their US clones are now next to impossible to find on eBay, etc., in 2015 the Bitcoin migration from FPGA to ASIC was wrapping up, so a few people were still selling these boards. In January 2015, I lucked out on an eBay lot of ZTEXes in one fell swoop (and then picked up a few more elsewhere).
So I had the boards ... but no code to run on them. They sat in a box for almost two years - until November 2016, when descrypt(!) support for ZTEX 1.15y appeared in the community/"jumbo" version of John the Ripper! So I reactivated the project, and assembled the boards into a little homebrew cluster - powered by a Raspberry Pi.
I use the cluster as part of my password auditing services, for password-hashing competitions such as Crack Me If You Can, and for general password research.
None of this would be possible without Apingis' work on ZTEX support for the community/"jumbo" version of John the Ripper.
| Component | Comment | Cost |
|---|---|---|
| FPGA boards: ZTEX 1.15y clones (x15) |
Quad Xilinx Spartan-6 LX150 FPGA.
Two types of power connector: CON3, 5.5mm barrel (-) with a 2.1mm center pin (+), 4.5V to 16V; and
CON2, a pluggable screw terminal (center is supply, the other two are ground).
On the clones, the capacitor at position C78 is prone to popping off. Ask shippers to take extra care to protect that side of the board. These capacitors are labeled '0G1 / 560 / 2.5v' and are described by a colleague as "the 220uF 25v". Inductors are 1R5. Also, the four tiny fans that they shipped with are terrible. Replace them with more scalable options (120mm on the side, or "box"). Note that these are 2012-era devices, so they're Mini USB. |
~$70/board (a few didn't work) or about $1100 |
| USB distribution hubs: AmazonBasics 7 Port USB 2.0 Hub with 5V/4A Power Adapter (x3) |
Some USB2 hub chipsets are reportedly more stable than USB3 ones. I have had good luck with these hubs, but I don't know what chipset they use, and I have not tested any USB3 hubs to verify reports of problems with those. Scattered Secrets says that hubs with the HS8836 chipset are reliable. See also Marc Merlin's guide on tuning high numbers of USB devices on Linux. | $19 x 3 = $57 |
| USB cables: USB-to-Mini-USB (28AWG/1P + 26AWG/2C) (x15) |
$3.50 x 16 = $56 | |
| Power supply: HP DPS-1200FB-1 A (1200W 220, 800W 110) |
Repurposed modular power supply for servers; available on eBay, Amazon, etc. and apparently popular with the cryptocurrency crowd. | ~$40 |
| Power breakout board: Older (X6) version of the ParallelMiner X7B 12 Port Chain Sync Breakout Board |
Has a small power switch on the board. Newer versions of the board keep the last state of this switch, which allows power to be toggled upstream by an external device. Multiple boards can be chained together for unified on/off control. | $35 |
| Power connectors: 16AWG PCI-E power cables (x5) |
With one end reterminated using 16AWG (1.6mm) European-style bootlace ferrules (color-coded red in the German system - see ZTEX.de instructions or Wayback). The power consumption on these boards is low enough that you can use all three pairs of wires on the cable. | 3 boards/cable = 5 cables = $3.25 x 5 = $16 |
| Power terminal blocks: 5.08mm Angle 3way/pin Screw Terminal Block Connector Pluggable Green, 15mm wide, M2.5 screw (Amazon B00GNTSSU6) (x15) |
Only the terminal half was needed, but it's harder to buy just that half in bulk. | $20 |
| Cooling: single "box" fan |
The loud onboard fans are disconnected. The box fan is much quieter. JtR currently runs these boards much cooler for supported hashes than BTCminer does; my setup is not sufficient for Bitcoin mining! (and in theory, some algorithms have potential efficiency gains remaining - such that I might have to improve my thermal management someday). | $25 |
| Cases: music CD shelves, acrylic, used (x3) |
These can be hard to find in the single-tower format, but you can still find the 30-CD version (vintage) that is half as high but twice as wide. From U.S. Acrylic. The back of the cases are mostly open, but with just enough of an edge to keep the boards in. | $15 x 3 = $45 |
| Host system: Pi 2 Model B Rev 1.1 kit (CanaKit) |
a21041 Embest (4xARMv7l r5 900MHz 1GB - 2015 Q1) + case, heatsinks, AC adapter, 32GB microSD. | $70 |
| Total cost: | Total hardware cost (USD, 2016) | ~$1466 |
Other performance summaries TBD
To improve stability, drop any USB3 ports/devices down to USB2:
lspci -nn | grep USB \
| cut -d '[' -f3 | cut -d ']' -f1 \
| xargs -I@ setpci -H1 -d @ d0.l=0
Initial "proof of concept" cluster stack - all 15 good boards active:
