It's quite interesting to study why crypto_(secret)box requires zero padding. It's not to avoid memory allocation, as tedu suggests, but to make the implementation stupidly simple. For every invocation of crypto_secretbox, NaCl needs to generate a single-use 32 byte key for the Poly1305 MAC. Since the first 32 bytes of the message are zero, NaCl can simply encrypt the entire message with Salsa20, and the first 32 bytes of the resulting ciphertext conveniently become a suitable key for Poly1305. (So when the documentation says the padding has to actually be zero, it's not kidding.)
By making the implementation more complex (some memcpys and special casing of short messages), as libsodium has done for its crypto_secretbox_easy[1], you can avoid the need for this padding.
Personally, I think djb should have avoided exposing this implementation detail in the interface. To simplify the implementation, he could have used the first output block (stream position 0) of Salsa20 only for the Poly1305 key, and then encrypted the message starting with stream position 1, instead of trying to use the first output block for both the Poly1305 key and the message. The implementation would have been only marginally more complex (not as complex as the libsodium implementation, as you wouldn't need special casing). Unfortunately this can't be changed now without breaking interoperability with messages encrypted with the current version of crypto_secretbox.
Is nacl deprecated in favor of the libsodium fork? Or are both projects strong and continuing in parallel?
[libsodium] is a portable, cross-compilable, installable,
packageable fork of NaCl, with a compatible API, and an
extended API to improve usability even further.
The website for nacl seems to be a bit outdated (many dates seem to refer to 2013), although there is still this tantalizing bit about features in the 'next' nacl release:
Major features in the next release of NaCl: full PIC support,
for easy integration into other languages; Ed25519 signatures
(currently available in SUPERCOP); NEON optimizations.
Being listed in the AUTHORS file does not imply active participation. libsodium is based on NaCl's algorithms and code, both of which are publicly reusable, but I think they were placed in libsodium by other people. If you clone the repo, `git log --author=bernstein` and `git log --author=djb` return no results.
It's probably worth treating NaCl as something like research code or a reference implementation. It's certainly way more usable than most research code is, which is extremely commendable, but it's still not quite libsodium in terms of its goal as a product.
As someone who didn't really know what they were doing, I used NaCl recently to write a program to tunnel packets over UDP.
I started by reading the NaCl source and all the introductory material available—the web site, the two NaCl papers ("Cryptography in NaCl", "The security impact of a new cryptography library"), and quickly reviewed a couple of other papers (e.g. to understand deterministic encryption and D-H key exchange). I think the biggest problem I had was to understand nonce generation and handling properly. The "Cryptography …" paper does contain some advice that I ultimately implemented. But I had to think very hard about what it said before I was confident that I was doing what it said. For example, it says:
«…the nonce can be chosen as a simple counter: 0 for Alice’s first packet, 1 for Bob’s first packet, 2 for Alice’s second packet, 3 for Bob’s second packet, 4 for Alice’s third packet, 5 for Bob’s third packet, etc. Choosing the nonce as a counter followed by (e.g.) 32 random bits helps protect some protocols against denial-of-service attacks. In many applications it is better to increase the counter to, e.g., the number of nanoseconds that have passed since a standard epoch in the local clock, so that the current value of the counter does not leak the
traffic rate.»
I managed to figure it out, but I would certainly have welcomed a more detailed explanation, and would have been very happy to have help from the code to do the right thing.
In contrast, the cryptography functions were easy enough to figure out and use (with the C API). The only mistake I kept making was specifying the secret key first and the public key second in all my function calls. Once I got used to doing it the other way around, it was fine. Zero-padding the messages was slightly ugly, but I didn't develop any especially strong feelings about it. (Aside: I actually ended up using TweetNaCl, but of course all the documentation is the same.)
I'm very pleased with the resulting code, anyway.
P.S. I looked at libsodium, but greatly preferred the unadorned library.
This looks good, but from a brief glance I can't seem to find if/how it implements windowing in net.c. I'm guessing it just waits for an ACK before continuing? I'm searching for something very much like this, where I could bind to a TCP socket that uses UDP internally, something like ZeroMQ, for punching through NAT with STUN or something similar.
The easiest way to get round the slightly odd build environment for NaCl is to just use TweetNaCl (http://tweetnacl.cr.yp.to/software.html) which just requires you to drop a single .c/.h file into your project - this isn't as performant as the reference code but in most cases this probably doesn't matter. The libsodium implementation seems overly complicated to me for most uses.
The NaCl API is reasonably simple and is fairly easy to wrap (plug - if anyone is looking for an example have a look at jim-nacl [1] which embeds into Jim TCL)
I like the author of this article and think he knows what he's talking about.
I like libsodium but do worry about the project getting away from them; cleaning up the API is one thing, adding new crypto primitives is another. Still: use libsodium.
The person whose opinion you really want on this article is 'pbsd.
I've been worried about that too. Recently I learned that libsodium includes a way to generate crypto_box keys from a fixed argument entropy chunk. It sounds really convenient but also circumvents all of the labor around centralizing entropy generation.
Is that a good thing to have? Maybe! It's convenient feeling. Is it dangerous? I have no idea—probably? Should it be part of the interface to libsodium? No clue.
It was added after a popular request: providing a simple way to derive a key pair from a secret key, itself derived from a password.
By not providing a simple way to achieve something, users end up copying/pasting random code and inventing their own crypto.
Additional functions that libsodium provides over NaCl were added after observing how developers were using NaCl in actual projects.
When you see projects that allocate temporary buffers to store signed messages, and immediately discard everything because all they actually need is the signature, you realize that developers have to work around limitations of the NaCl API.
When the maintainer of the Go bindings reports memory corruption, which turned out to be due to sign_open() requiring an output buffer larger than the unsigned message you might blame him for not having read the fine print in the NaCl documentation. But maybe the API should better match what feels intuitive to developers instead.
And when developers systematically write wrappers (for example around box() and secretbox() functions) to work around the same issues, it probably means that API could be improved.
Adding convenience functions addressing very common needs to libsodium instead of having everybody rewrite their own implementation makes it less confusing and eventually less dangerous.
These are all very legitimate gripes, and even dangers.
One wrapper I used (think Python/Perl/Ruby), the author did not properly understand how the 0-padding was supposed to be implemented. I was scratching my head for a few hours wondering why messages I encrypted were not being received properly. Turns out, bugs in the C padding code.
The discussion of nonces really needs big flashing warning text and icons. I've seen total disasters with code trying to do nonces correctly, or not even understanding why nonces are important (one VPN package I used had the option of using null nonces, as if it were a feature!)
The ZEROBYTES padding is a source of confusion and bugs, including in bindings. Having to allocate more bytes than the unsealed message in crypto_sign_open() was also the root cause of quite a few unfortunate bugs.
And even when used properly, people always ended up writing wrappers allocating temporary buffers and moving data around in order to add/trim these bytes.
And when people start writing wrappers, it usually means that there is something wrong with the API.
For this reason, libsodium fully supports the crypto_box and crypto_secretbox in a compatible way, but doesn't document them in favor of the "easy" interface, which basically does what everybody was reinventing when writing wrappers for these functions.
How to generate nonces for each operation is described in the libsodium documentation. There are legitimate cases for having full control over the nonce, such as the dnscurve and dnscrypt protocols.
Some bindings such as the Swift bindings (Swift-Sodium) have the box() operation generate and return a nonce in addition to the ciphertext. Less flexible than leaving the responsibility to the developer, but definitely safer, and something that might be added to the library itself. On the other hand, adding more and more functions to do the same thing (because breaking backward compatibility is not an option) isn't good either.
Simply put, use the reference nacl implementation only when you really know what you're doing. As with all things crypto it's easy to shoot yourself in the foot, and libsodium makes it a bit harder than reference nacl.
If you really know what you're doing, you'd use a fast assembly implementation that's guaranteed to run in constant time, like curve25519/ed25519-donna, both of which you get for free if you just use libsodium.
Isn't the fact that nacl is written by such highly reputable guy(in the crypto community) and that it sees relatively lots of review by experts important here ?
He's a highly reputable cryptographer, and a ... controversial ... software developer. I would trust my life to the algorithms. I wouldn't trust my appendix to the code.
The software is delivered over HTTP (not HTTPS) and not cryptographically signed. I've always interpreted this as a statement that, if you're not qualified to review what you just downloaded and make sure it's not malicious and that it faithfully implements the algorithms, that download isn't for you.
In a similar way, libsodium is effectively a nacl fork that's actually usable by normal people. I think I've seen more people review libsodium than nacl's code itself (especially the custom assembly versions of various algorithms).
For years I ran vanilla qmail with two patches (AUTH support for sending authentication, and a patch to allow for a database of valid supported emails so you get rid of the backscatter issue).
It was absolutely fantastic, it ran without issues for years. It was very fast and I never had issues with losing mail or behaviours that baffled me or left me scratching my head.
I currently have a postfix setup, and while it works well I am very wary of upgrading it for fear of breaking something and causing mail to bounce or not be delivered. With qmail it would just queue mail if you accidentally misconfigure something (such as the delivery program is unavailable), with Postfix I've had it simply drop mail on the floor. Sure they are issues that could be solved with better testing, and verification of systems, but qmail is a lot more forgiving without bouncing.
I also quite prefer qmail to postfix in terms of its flexibility and resilience, but it really does highlight one of the biggest problems with long term use of djb software: It always seems to reach a point where djb feels it's done (or he's done with it) and it sits there for years with no updates, while still appearing to be the authoritative version of that product.
Vanilla qmail is genuinely unusable on the internet today (because of the backscatter issue in particular), but netqmail, which is still maintained, still looks like a secondary fork to someone who doesn't know the history. It doesn't help that djb for a long time used licensing that was incompatible with reasonable distribution of these forks as well.
So these days I actually tend to go for the forks and clones of djb software where possible. I use netqmail on my mail server, runit instead of daemontools, etc.
Unmaintained software is a problem and vanilla NaCl also left unfixed issues.
For example, the signature system was a prototype that shouldn't be used any more. The portable AES128 implementation produces incorrect output on some other platforms, one of the Curve25519 implementations performs out of bounds memory accesses, and one of the poly1305 implementations will produce incorrect output if your application changes the FP rounding mode. CurveCP was also a fantastic idea, but the NaCl implementation was just a proof of concept that cannot really be used in actual projects.
But would you rather have djb spend time addressing qmail compilation issues on Ubuntu 14.10, or keep making significant advances in applied cryptography (and security in general) instead?
In addition to organizing competitions, the amount of game-changing publications he made or contributed to is very impressive. And it might not have been the case if he didn't move on from software he wrote years ago.
Have you tried OpenSMTPD? It's really great. My config file is about 30 lines (and most of them are comments) – unlike with Postfix. And it's from OpenBSD, which means it has better security than others :-)
I really want to try OpenSMTPD, but I need it to hook into my Dovecot backend for SMTP auth (SASL would be awesome), and it needs to work with in-line email filtering (i.e. when a message is rejected as spam by amavisd I need that error to propagate back to the sending server). I haven't found good information on how to do any of that yet.
Right now my pipeline is pretty simple:
Postfix accepts -> milter (amavisd) -> dovecot for delivery
I would want to replicate something similar. Do you have any good resources or information?
Somehow these and other "gripes" about djb software over the years have never been as interesting or entertaining as the ones djb himself has been sharing with us since the 90's. But complaints are to be expected.
Personally I do not care for libsodium, but to each his own.
Instead I chose to study curveprotect, tinysshd (tweetnacl), qremote, etc. as examples to learn from. Realizing that everyone is capable of making mistakes, but some are less likely than others.
Good API design is, like "installability," a critical and often neglected aspect of software engineering. I find that bad APIs are the norm in every area, not just crypto. Audio and video stuff is just hideous.
I used the NaCl crypto for ZeroTier One but basically refactored the code into C++ classes that build like normal modern C++.
It's amazing how people can comment without reading the article. It's obvious this happens to some degree, but I was amazed that the first few posts in this thread have exclusively been about a completely different thing.
Please read the article more carefully. The article assumes the reader knows what NaCL is and does not explain it, but it's easy to infer he doesn't talk about the Google native client technology, but about something else.
> If nacl picks up that your new CPU has some advanced instructions, it will use them, resulting in a library that won’t work on older CPUs. It’s hard to ship binaries when you’re uncertain of what CPUs they will run on.
Uhm.. What? Isn't this going exactly against all that nacl was ever designed for?
Also, why can't instructions for one CPU be translated/compiled to instructions suitable for another CPU?
Ah, quite an unfortunate clash of names given the context. I confused it for Chrome's NaCl too. Although, Chrome's NaCl is short for NativeClient and this one is pronounced Salt based on their description.
By making the implementation more complex (some memcpys and special casing of short messages), as libsodium has done for its crypto_secretbox_easy[1], you can avoid the need for this padding.
Personally, I think djb should have avoided exposing this implementation detail in the interface. To simplify the implementation, he could have used the first output block (stream position 0) of Salsa20 only for the Poly1305 key, and then encrypted the message starting with stream position 1, instead of trying to use the first output block for both the Poly1305 key and the message. The implementation would have been only marginally more complex (not as complex as the libsodium implementation, as you wouldn't need special casing). Unfortunately this can't be changed now without breaking interoperability with messages encrypted with the current version of crypto_secretbox.
[1] https://github.com/jedisct1/libsodium/blob/17932c782e5900ac8...